Details

      Description

      This is a big query which basically does not work. One of the issues is the filter placement, where, instead of being executed as soon as the necessary variables are bound, the static optimizer moves the filters to the end.

      A much smaller example is needed as the first step of working on this issue.

      A specific defect is that the filter -exists-2 depending only on NgsExperimentGeneric_B could have been executed much sooner

      Original SPARQL

      base <https://test-s-jjc.syapse.com/>
      prefix owl: <http://www.w3.org/2002/07/owl#>
      prefix based: <https://test-s-jjc.syapse.com/bdm/api/appindividual/based:>
      prefix xsd: <http://www.w3.org/2001/XMLSchema#>
      prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
      prefix dc: <http://purl.org/dc/elements/1.1/>
      prefix sys: <https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:>
      prefix s: <https://test-s-jjc.syapse.com/bdm/api/>
      prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
      prefix bds: <http://www.bigdata.com/rdf/search#>
      prefix sysd: <https://test-s-jjc.syapse.com/bdm/api/appindividual/sysd:>
      prefix c2: <https://test-s-jjc.syapse.com/bdm/api/kbobject/c2:>
      prefix base: <https://test-s-jjc.syapse.com/bdm/api/kbobject/base:>
      prefix skos: <http://www.w3.org/2004/02/skos/core#>
      prefix syapse: <https://test-s-jjc.syapse.com/graph/syapse#>
      prefix c2d: <https://test-s-jjc.syapse.com/bdm/api/appindividual/c2d:>
      SELECT *
      FROM <https://test-s-jjc.syapse.com/graph/ontology/c2>
      FROM <https://test-s-jjc.syapse.com/graph/ontology/base>
      FROM <https://test-s-jjc.syapse.com/graph/django/south-park-university>
      FROM <https://test-s-jjc.syapse.com/graph/south-park-university/vocabulary>
      FROM <https://test-s-jjc.syapse.com/graph/south-park-university/abox>
      FROM <https://test-s-jjc.syapse.com/graph/ontology/sys>
      FROM <https://test-s-jjc.syapse.com/graph/vocabulary>
      FROM <https://test-s-jjc.syapse.com/graph/syapse>
      FROM NAMED <https://test-s-jjc.syapse.com/graph/ontology/c2>
      FROM NAMED <https://test-s-jjc.syapse.com/graph/ontology/base>
      FROM NAMED <https://test-s-jjc.syapse.com/graph/django/south-park-university>
      FROM NAMED <https://test-s-jjc.syapse.com/graph/south-park-university/vocabulary>
      FROM NAMED <https://test-s-jjc.syapse.com/graph/south-park-university/abox>
      FROM NAMED <https://test-s-jjc.syapse.com/graph/ontology/sys>
      FROM NAMED <https://test-s-jjc.syapse.com/graph/vocabulary>
      FROM NAMED <https://test-s-jjc.syapse.com/graph/syapse>
      
      WITH {
      SELECT DISTINCT $SIZE_LIMIT_EXCEEDED ?NgsMeasurement_C ?NgsMeasurement_A $j__1 $j__2
      WHERE {
      
        INCLUDE %__MainQuery
        OPTIONAL {
          ?NgsMeasurement_C sys:name $j__1
        }
        OPTIONAL {
          ?NgsMeasurement_A sys:name $j__2
        }
        { SELECT (COUNT(*) as $CHECK_LIMIT) {
            hint:SubQuery hint:runOnce true .
          INCLUDE %__MainQuery
        }}
        BIND ( ( $CHECK_LIMIT = 20000 ) AS $SIZE_LIMIT_EXCEEDED )
      }} AS %__FullQuery
      WITH {
      SELECT *
      WHERE {
      
        FILTER EXISTS {
          { ?NgsMeasurement_A sys:owner <https://test-s-jjc.syapse.com/bdm/api/syuser/7> }
          UNION
          { ?NgsMeasurement_A sys:assignedProject / syapse:isPrivate false .
          }
          UNION
          { ?NgsMeasurement_A sys:assignedProject / syapse:member <https://test-s-jjc.syapse.com/bdm/api/syuser/7> .
          }
      }
        { SELECT ?NgsMeasurement_A {
          { SELECT $j__6 {
            hint:SubQuery hint:runOnce true .
            $j__6 rdfs:subClassOf * c2:NgsMeasurement
          } }
          ?NgsMeasurement_A rdf:type $j__6 .
          hint:Prior hint:runLast true
        } }
        FILTER EXISTS {
          { ?NgsExperimentGeneric_B sys:owner <https://test-s-jjc.syapse.com/bdm/api/syuser/7> }
          UNION
          { ?NgsExperimentGeneric_B sys:assignedProject / syapse:isPrivate false .
          }
          UNION
          { ?NgsExperimentGeneric_B sys:assignedProject / syapse:member <https://test-s-jjc.syapse.com/bdm/api/syuser/7> .
          }
      }
        { SELECT ?NgsExperimentGeneric_B {
          { SELECT $j__7 {
            hint:SubQuery hint:runOnce true .
            $j__7 rdfs:subClassOf * c2:NgsExperimentGeneric
          } }
          ?NgsExperimentGeneric_B rdf:type $j__7 .
          hint:Prior hint:runLast true
        } }
        ?NgsExperimentGeneric_B c2:hasNgsMeasurement ?NgsMeasurement_C .
        FILTER EXISTS {
          { ?NgsMeasurement_C sys:owner <https://test-s-jjc.syapse.com/bdm/api/syuser/7> }
          UNION
          { ?NgsMeasurement_C sys:assignedProject / syapse:isPrivate false .
          }
          UNION
          { ?NgsMeasurement_C sys:assignedProject / syapse:member <https://test-s-jjc.syapse.com/bdm/api/syuser/7> .
          }
      }
        { SELECT ?NgsMeasurement_C {
          { SELECT $j__12 {
            hint:SubQuery hint:runOnce true .
            $j__12 rdfs:subClassOf * c2:NgsMeasurementQpcr
          } }
          ?NgsMeasurement_C rdf:type $j__12 .
          hint:Prior hint:runLast true
        } }
        { SELECT $j__8 {
          { SELECT $j__10 {
            hint:SubQuery hint:runOnce true .
            $j__10 rdfs:subClassOf * c2:NgsExperiment
          } }
          $j__8 rdf:type $j__10 .
          hint:Prior hint:runLast true
        } }
        $j__8 syapse:part? $j__9 .
        $j__9 $j__11 ?NgsMeasurement_C
        FILTER ( $j__8 != ?NgsMeasurement_C )
        { SELECT ?NgsExperimentGeneric_B {
          { SELECT $j__4 {
            hint:SubQuery hint:runOnce true .
            $j__4 rdfs:subClassOf * c2:NgsExperiment
          } }
          ?NgsExperimentGeneric_B rdf:type $j__4 .
          hint:Prior hint:runLast true
        } }
        ?NgsExperimentGeneric_B syapse:part? $j__3 .
        $j__3 $j__5 ?NgsMeasurement_A
        FILTER ( ?NgsExperimentGeneric_B != ?NgsMeasurement_A )
      }
      LIMIT 20000} AS %__MainQuery
      
      WHERE {
      
      { SELECT (COUNT(*) AS $S__COUNT)
        WHERE {
          INCLUDE %__FullQuery
        }
       }
       INCLUDE %__FullQuery
      
      }
      

      Optimized

      PREFIX owl: <http://www.w3.org/2002/07/owl#>
      PREFIX based: <https://test-s-jjc.syapse.com/bdm/api/appindividual/based:>
      PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
      PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
      PREFIX dc: <http://purl.org/dc/elements/1.1/>
      PREFIX sys: <https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:>
      PREFIX s: <https://test-s-jjc.syapse.com/bdm/api/>
      PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
      PREFIX bds: <http://www.bigdata.com/rdf/search#>
      PREFIX sysd: <https://test-s-jjc.syapse.com/bdm/api/appindividual/sysd:>
      PREFIX c2: <https://test-s-jjc.syapse.com/bdm/api/kbobject/c2:>
      PREFIX base: <https://test-s-jjc.syapse.com/bdm/api/kbobject/base:>
      PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
      PREFIX syapse: <https://test-s-jjc.syapse.com/graph/syapse#>
      PREFIX c2d: <https://test-s-jjc.syapse.com/bdm/api/appindividual/c2d:>
      PREFIX hint: <http://www.bigdata.com/queryHints#>
        defaultGraphs=DataSetSummary{ngraphs=8, nknown=8, nunknown=0, graphs=[TermId(1U)[https://test-s-jjc.syapse.com/graph/syapse], TermId(12U)[https://test-s-jjc.syapse.com/graph/vocabulary], TermId(19770U)[https://test-s-jjc.syapse.com/graph/django/south-park-university], TermId(21246U)[https://test-s-jjc.syapse.com/graph/ontology/base], TermId(21247U)[https://test-s-jjc.syapse.com/graph/ontology/sys], TermId(29521U)[https://test-s-jjc.syapse.com/graph/ontology/c2], TermId(32985U)[https://test-s-jjc.syapse.com/graph/south-park-university/abox], TermId(32986U)[https://test-s-jjc.syapse.com/graph/south-park-university/vocabulary]]}
        namedGraphs=DataSetSummary{ngraphs=8, nknown=8, nunknown=0, graphs=[TermId(1U)[https://test-s-jjc.syapse.com/graph/syapse], TermId(12U)[https://test-s-jjc.syapse.com/graph/vocabulary], TermId(19770U)[https://test-s-jjc.syapse.com/graph/django/south-park-university], TermId(21246U)[https://test-s-jjc.syapse.com/graph/ontology/base], TermId(21247U)[https://test-s-jjc.syapse.com/graph/ontology/sys], TermId(29521U)[https://test-s-jjc.syapse.com/graph/ontology/c2], TermId(32985U)[https://test-s-jjc.syapse.com/graph/south-park-university/abox], TermId(32986U)[https://test-s-jjc.syapse.com/graph/south-park-university/vocabulary]]}
      WITH {
        QueryType: SELECT
        SELECT ( VarNode(j__6) AS VarNode(j__6) )
          JoinGroupNode {
            ArbitraryLengthPathNode(left=VarNode(j__6), right=ConstantNode(TermId(29066U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/c2:NgsMeasurement])) {
              JoinGroupNode {
                StatementPatternNode(VarNode(-tVarLeft-7901efef-7abf-4dda-bb82-038f95c9c3bb)[anonymous], ConstantNode(Vocab(31)[http://www.w3.org/2000/01/rdf-schema#subClassOf]), VarNode(-tVarRight-db6eae2f-2f42-42df-afae-80588396a32c)[anonymous]) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=3
                  AST2BOpBase.originalIndex=POCS
              }
            }
          }
      } AS -subSelect-2 JOIN ON () DEPENDS ON ()
      WITH {
        QueryType: SELECT
        SELECT ( VarNode(j__7) AS VarNode(j__7) )
          JoinGroupNode {
            ArbitraryLengthPathNode(left=VarNode(j__7), right=ConstantNode(TermId(29040U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/c2:NgsExperimentGeneric])) {
              JoinGroupNode {
                StatementPatternNode(VarNode(-tVarLeft-ea8a51a5-bc53-4da6-a270-406f2522b06e)[anonymous], ConstantNode(Vocab(31)[http://www.w3.org/2000/01/rdf-schema#subClassOf]), VarNode(-tVarRight-5eb2d784-23ad-4487-a359-e1f4bb1a082f)[anonymous]) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=0
                  AST2BOpBase.originalIndex=POCS
              }
            }
          }
      } AS -subSelect-3 JOIN ON () DEPENDS ON ()
      WITH {
        QueryType: SELECT
        SELECT ( VarNode(j__12) AS VarNode(j__12) )
          JoinGroupNode {
            ArbitraryLengthPathNode(left=VarNode(j__12), right=ConstantNode(TermId(29069U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/c2:NgsMeasurementQpcr])) {
              JoinGroupNode {
                StatementPatternNode(VarNode(-tVarLeft-4f2e29a8-dc3c-4a2e-9a53-6b7083b60591)[anonymous], ConstantNode(Vocab(31)[http://www.w3.org/2000/01/rdf-schema#subClassOf]), VarNode(-tVarRight-b046f310-252e-4fd6-8dc9-5edf7fa2a18c)[anonymous]) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=0
                  AST2BOpBase.originalIndex=POCS
              }
            }
          }
      } AS -subSelect-4 JOIN ON () DEPENDS ON ()
      WITH {
        QueryType: SELECT
        SELECT ( VarNode(j__10) AS VarNode(j__10) )
          JoinGroupNode {
            ArbitraryLengthPathNode(left=VarNode(j__10), right=ConstantNode(TermId(29039U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/c2:NgsExperiment])) {
              JoinGroupNode {
                StatementPatternNode(VarNode(-tVarLeft-f1cd3090-947c-41dd-a2da-734e6caac65a)[anonymous], ConstantNode(Vocab(31)[http://www.w3.org/2000/01/rdf-schema#subClassOf]), VarNode(-tVarRight-df97912a-af9e-43ec-a4af-7352259080af)[anonymous]) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=1
                  AST2BOpBase.originalIndex=POCS
              }
            }
          }
      } AS -subSelect-5 JOIN ON () DEPENDS ON ()
      WITH {
        QueryType: SELECT
        SELECT ( VarNode(j__4) AS VarNode(j__4) )
          JoinGroupNode {
            ArbitraryLengthPathNode(left=VarNode(j__4), right=ConstantNode(TermId(29039U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/c2:NgsExperiment])) {
              JoinGroupNode {
                StatementPatternNode(VarNode(-tVarLeft-39c06046-4409-4c73-8b00-bce55102ecec)[anonymous], ConstantNode(Vocab(31)[http://www.w3.org/2000/01/rdf-schema#subClassOf]), VarNode(-tVarRight-2a0e563a-cb5e-4b69-a4e8-5191a6cc7c36)[anonymous]) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=1
                  AST2BOpBase.originalIndex=POCS
              }
            }
          }
      } AS -subSelect-6 JOIN ON () DEPENDS ON ()
      WITH {
        QueryType: SELECT
        SELECT VarNode(NgsMeasurement_A) VarNode(NgsExperimentGeneric_B) VarNode(NgsMeasurement_C) VarNode(j__8) VarNode(j__9) VarNode(j__11) VarNode(j__3) VarNode(j__5)
          JoinGroupNode {
            StatementPatternNode(VarNode(NgsExperimentGeneric_B), ConstantNode(TermId(28918U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/c2:hasNgsMeasurement]), VarNode(NgsMeasurement_C)) [scope=DEFAULT_CONTEXTS]
              AST2BOpBase.estimatedCardinality=2
              AST2BOpBase.originalIndex=POCS
            StatementPatternNode(VarNode(j__9), VarNode(j__11), VarNode(NgsMeasurement_C)) [scope=DEFAULT_CONTEXTS]
              AST2BOpBase.estimatedCardinality=75296
              AST2BOpBase.originalIndex=SPOC
            ArbitraryLengthPathNode(left=VarNode(j__8), right=VarNode(j__9)) {
              JoinGroupNode {
                StatementPatternNode(VarNode(-tVarLeft-f95ba0ac-5826-42fd-972a-6e1860fa5f6f)[anonymous], ConstantNode(TermId(25735U)[https://test-s-jjc.syapse.com/graph/syapse#part]), VarNode(-tVarRight-0c24a9b7-0d2b-4ccd-9a81-99dad41c12a3)[anonymous]) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=496
                  AST2BOpBase.originalIndex=POCS
              }
            }
            ArbitraryLengthPathNode(left=VarNode(NgsExperimentGeneric_B), right=VarNode(j__3)) {
              JoinGroupNode {
                StatementPatternNode(VarNode(-tVarLeft-4ca3c22d-42d2-455f-9d77-b036f39d32c0)[anonymous], ConstantNode(TermId(25735U)[https://test-s-jjc.syapse.com/graph/syapse#part]), VarNode(-tVarRight-62199333-8d56-4a3c-a0d5-6ed004c76ba7)[anonymous]) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=496
                  AST2BOpBase.originalIndex=POCS
              }
            }
            StatementPatternNode(VarNode(j__3), VarNode(j__5), VarNode(NgsMeasurement_A)) [scope=DEFAULT_CONTEXTS] [#filters=1]
              FILTER( FunctionNode(VarNode(NgsExperimentGeneric_B),VarNode(NgsMeasurement_A))[ FunctionNode.scalarVals=null, FunctionNode.functionURI=http://www.w3.org/2005/xpath-functions#not-equal-to, valueExpr=com.bigdata.rdf.internal.constraints.CompareBOp(NgsExperimentGeneric_B,NgsMeasurement_A)[ CompareBOp.op=NE]] )
              AST2BOpBase.estimatedCardinality=75296
              AST2BOpBase.originalIndex=SPOC
            QueryType: SELECT
            SELECT ( VarNode(NgsMeasurement_A) AS VarNode(NgsMeasurement_A) )
              JoinGroupNode {
                INCLUDE -subSelect-2 JOIN ON ()
                StatementPatternNode(VarNode(NgsMeasurement_A), ConstantNode(Vocab(14)[http://www.w3.org/1999/02/22-rdf-syntax-ns#type]), VarNode(j__6)) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=18510
                  AST2BOpBase.originalIndex=POCS
              }
            
            QueryType: SELECT
            SELECT ( VarNode(NgsExperimentGeneric_B) AS VarNode(NgsExperimentGeneric_B) )
              JoinGroupNode {
                INCLUDE -subSelect-3 JOIN ON ()
                StatementPatternNode(VarNode(NgsExperimentGeneric_B), ConstantNode(Vocab(14)[http://www.w3.org/1999/02/22-rdf-syntax-ns#type]), VarNode(j__7)) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=18510
                  AST2BOpBase.originalIndex=POCS
              }
            
            QueryType: SELECT
            SELECT ( VarNode(NgsMeasurement_C) AS VarNode(NgsMeasurement_C) )
              JoinGroupNode {
                INCLUDE -subSelect-4 JOIN ON ()
                StatementPatternNode(VarNode(NgsMeasurement_C), ConstantNode(Vocab(14)[http://www.w3.org/1999/02/22-rdf-syntax-ns#type]), VarNode(j__12)) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=18510
                  AST2BOpBase.originalIndex=POCS
              }
            
            QueryType: SELECT
            SELECT ( VarNode(j__8) AS VarNode(j__8) )
              JoinGroupNode {
                INCLUDE -subSelect-5 JOIN ON ()
                StatementPatternNode(VarNode(j__8), ConstantNode(Vocab(14)[http://www.w3.org/1999/02/22-rdf-syntax-ns#type]), VarNode(j__10)) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=18510
                  AST2BOpBase.originalIndex=POCS
              }
            
              FILTER( FunctionNode(VarNode(j__8),VarNode(NgsMeasurement_C))[ FunctionNode.scalarVals=null, FunctionNode.functionURI=http://www.w3.org/2005/xpath-functions#not-equal-to, valueExpr=com.bigdata.rdf.internal.constraints.CompareBOp(j__8,NgsMeasurement_C)[ CompareBOp.op=NE]] )
            QueryType: SELECT
            SELECT ( VarNode(NgsExperimentGeneric_B) AS VarNode(NgsExperimentGeneric_B) )
              JoinGroupNode {
                INCLUDE -subSelect-6 JOIN ON ()
                StatementPatternNode(VarNode(NgsExperimentGeneric_B), ConstantNode(Vocab(14)[http://www.w3.org/1999/02/22-rdf-syntax-ns#type]), VarNode(j__4)) [scope=DEFAULT_CONTEXTS]
                  AST2BOpBase.estimatedCardinality=18510
                  AST2BOpBase.originalIndex=POCS
              }
            
            QueryType: ASK
            SELECT VarNode(NgsMeasurement_A) VarNode(-exists-1)[anonymous]
              JoinGroupNode {
                UnionNode [joinVars=[]] [projectInVars=[]] {
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_A), ConstantNode(TermId(25690U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:owner]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=15
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_A), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-90a6fd1c-bb47-4719-8d0c-c97ebd518424)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-90a6fd1c-bb47-4719-8d0c-c97ebd518424)[anonymous], ConstantNode(TermId(19635U)[https://test-s-jjc.syapse.com/graph/syapse#isPrivate]), ConstantNode(XSDBoolean(false))) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=7
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_A), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-1dc909bf-e2fa-432c-972f-3eadc5e696c8)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-1dc909bf-e2fa-432c-972f-3eadc5e696c8)[anonymous], ConstantNode(TermId(19636U)[https://test-s-jjc.syapse.com/graph/syapse#member]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=2
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                } JOIN ON ()
              }
            @askVar=-exists-1
            QueryType: ASK
            SELECT VarNode(NgsExperimentGeneric_B) VarNode(-exists-2)[anonymous]
              JoinGroupNode {
                UnionNode [joinVars=[]] [projectInVars=[]] {
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsExperimentGeneric_B), ConstantNode(TermId(25690U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:owner]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=15
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsExperimentGeneric_B), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-628bd213-824b-4961-966d-de414e9c5077)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-628bd213-824b-4961-966d-de414e9c5077)[anonymous], ConstantNode(TermId(19635U)[https://test-s-jjc.syapse.com/graph/syapse#isPrivate]), ConstantNode(XSDBoolean(false))) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=7
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsExperimentGeneric_B), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-88856abc-dd54-47a6-b65c-8a062eefdfda)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-88856abc-dd54-47a6-b65c-8a062eefdfda)[anonymous], ConstantNode(TermId(19636U)[https://test-s-jjc.syapse.com/graph/syapse#member]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=2
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                } JOIN ON ()
              }
            @askVar=-exists-2
            QueryType: ASK
            SELECT VarNode(NgsMeasurement_C) VarNode(-exists-3)[anonymous]
              JoinGroupNode {
                UnionNode [joinVars=[]] [projectInVars=[]] {
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_C), ConstantNode(TermId(25690U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:owner]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=15
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_C), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-a91e558f-e1cd-43bf-9a8c-549caee68c65)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-a91e558f-e1cd-43bf-9a8c-549caee68c65)[anonymous], ConstantNode(TermId(19635U)[https://test-s-jjc.syapse.com/graph/syapse#isPrivate]), ConstantNode(XSDBoolean(false))) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=7
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_C), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-657cb018-f0f2-4458-a316-c16b1fc8224e)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-657cb018-f0f2-4458-a316-c16b1fc8224e)[anonymous], ConstantNode(TermId(19636U)[https://test-s-jjc.syapse.com/graph/syapse#member]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=2
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                } JOIN ON ()
              }
            @askVar=-exists-3
            FILTER( ExistsNode(VarNode(-exists-1))[ FunctionNode.scalarVals=null, FunctionNode.functionURI=http://www.bigdata.com/sparql-1.1-undefined-functionsexists, graphPattern=
              JoinGroupNode {
                UnionNode [joinVars=[]] [projectInVars=[]] {
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_A), ConstantNode(TermId(25690U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:owner]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=15
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_A), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-90a6fd1c-bb47-4719-8d0c-c97ebd518424)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-90a6fd1c-bb47-4719-8d0c-c97ebd518424)[anonymous], ConstantNode(TermId(19635U)[https://test-s-jjc.syapse.com/graph/syapse#isPrivate]), ConstantNode(XSDBoolean(false))) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=7
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_A), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-1dc909bf-e2fa-432c-972f-3eadc5e696c8)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-1dc909bf-e2fa-432c-972f-3eadc5e696c8)[anonymous], ConstantNode(TermId(19636U)[https://test-s-jjc.syapse.com/graph/syapse#member]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=2
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                } JOIN ON ()
              }, valueExpr=-exists-1] )
            FILTER( ExistsNode(VarNode(-exists-2))[ FunctionNode.scalarVals=null, FunctionNode.functionURI=http://www.bigdata.com/sparql-1.1-undefined-functionsexists, graphPattern=
              JoinGroupNode {
                UnionNode [joinVars=[]] [projectInVars=[]] {
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsExperimentGeneric_B), ConstantNode(TermId(25690U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:owner]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=15
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsExperimentGeneric_B), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-628bd213-824b-4961-966d-de414e9c5077)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-628bd213-824b-4961-966d-de414e9c5077)[anonymous], ConstantNode(TermId(19635U)[https://test-s-jjc.syapse.com/graph/syapse#isPrivate]), ConstantNode(XSDBoolean(false))) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=7
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsExperimentGeneric_B), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-88856abc-dd54-47a6-b65c-8a062eefdfda)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-88856abc-dd54-47a6-b65c-8a062eefdfda)[anonymous], ConstantNode(TermId(19636U)[https://test-s-jjc.syapse.com/graph/syapse#member]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=2
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                } JOIN ON ()
              }, valueExpr=-exists-2] )
            FILTER( ExistsNode(VarNode(-exists-3))[ FunctionNode.scalarVals=null, FunctionNode.functionURI=http://www.bigdata.com/sparql-1.1-undefined-functionsexists, graphPattern=
              JoinGroupNode {
                UnionNode [joinVars=[]] [projectInVars=[]] {
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_C), ConstantNode(TermId(25690U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:owner]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=15
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_C), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-a91e558f-e1cd-43bf-9a8c-549caee68c65)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-a91e558f-e1cd-43bf-9a8c-549caee68c65)[anonymous], ConstantNode(TermId(19635U)[https://test-s-jjc.syapse.com/graph/syapse#isPrivate]), ConstantNode(XSDBoolean(false))) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=7
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                  JoinGroupNode [joinVars=[]] [projectInVars=[]] {
                    StatementPatternNode(VarNode(NgsMeasurement_C), ConstantNode(TermId(25684U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:assignedProject]), VarNode(--pp-anon-657cb018-f0f2-4458-a316-c16b1fc8224e)[anonymous]) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=387
                      AST2BOpBase.originalIndex=POCS
                    StatementPatternNode(VarNode(--pp-anon-657cb018-f0f2-4458-a316-c16b1fc8224e)[anonymous], ConstantNode(TermId(19636U)[https://test-s-jjc.syapse.com/graph/syapse#member]), ConstantNode(TermId(19768U)[https://test-s-jjc.syapse.com/bdm/api/syuser/7])) [scope=DEFAULT_CONTEXTS]
                      AST2BOpBase.estimatedCardinality=2
                      AST2BOpBase.originalIndex=POCS
                  } JOIN ON ()
                } JOIN ON ()
              }, valueExpr=-exists-3] )
          }
        slice(limit=20000)
      } AS %__MainQuery JOIN ON () DEPENDS ON (-subSelect-2,-subSelect-3,-subSelect-4,-subSelect-5,-subSelect-6)
      WITH {
        QueryType: SELECT
        SELECT ( com.bigdata.rdf.sparql.ast.FunctionNode(VarNode(*))[ FunctionNode.scalarVals=null, FunctionNode.functionURI=http://www.w3.org/2006/sparql-functions#count, valueExpr=com.bigdata.bop.rdf.aggregate.COUNT(*)] AS VarNode(CHECK_LIMIT) )
          JoinGroupNode {
            INCLUDE %__MainQuery JOIN ON ()
          }
      } AS -subSelect-1 JOIN ON () DEPENDS ON (%__MainQuery)
      WITH {
        QueryType: SELECT
        SELECT DISTINCT ( VarNode(SIZE_LIMIT_EXCEEDED) AS VarNode(SIZE_LIMIT_EXCEEDED) ) ( VarNode(NgsMeasurement_C) AS VarNode(NgsMeasurement_C) ) ( VarNode(NgsMeasurement_A) AS VarNode(NgsMeasurement_A) ) ( VarNode(j__1) AS VarNode(j__1) ) ( VarNode(j__2) AS VarNode(j__2) )
          JoinGroupNode {
            INCLUDE %__MainQuery JOIN ON ()
            INCLUDE -subSelect-1 JOIN ON ()
            StatementPatternNode(VarNode(NgsMeasurement_C), ConstantNode(TermId(25689U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:name]), VarNode(j__1)) [scope=DEFAULT_CONTEXTS] [optional]
              AST2BOpBase.estimatedCardinality=354
              AST2BOpBase.originalIndex=POCS
            StatementPatternNode(VarNode(NgsMeasurement_A), ConstantNode(TermId(25689U)[https://test-s-jjc.syapse.com/bdm/api/kbobject/sys:name]), VarNode(j__2)) [scope=DEFAULT_CONTEXTS] [optional]
              AST2BOpBase.estimatedCardinality=354
              AST2BOpBase.originalIndex=POCS
            ( com.bigdata.rdf.sparql.ast.FunctionNode(VarNode(CHECK_LIMIT),ConstantNode(XSDInteger(20000)))[ FunctionNode.scalarVals=null, FunctionNode.functionURI=http://www.w3.org/2005/xpath-functions#equal-to, valueExpr=com.bigdata.rdf.internal.constraints.CompareBOp(CHECK_LIMIT,XSDInteger(20000))[ CompareBOp.op=EQ]] AS VarNode(SIZE_LIMIT_EXCEEDED) )
          }
      } AS %__FullQuery JOIN ON () DEPENDS ON (%__MainQuery,-subSelect-1)
      WITH {
        QueryType: SELECT
        SELECT ( com.bigdata.rdf.sparql.ast.FunctionNode(VarNode(*))[ FunctionNode.scalarVals=null, FunctionNode.functionURI=http://www.w3.org/2006/sparql-functions#count, valueExpr=com.bigdata.bop.rdf.aggregate.COUNT(*)] AS VarNode(S__COUNT) )
          JoinGroupNode {
            INCLUDE %__FullQuery JOIN ON ()
          }
      } AS -subSelect-7 JOIN ON () DEPENDS ON (%__FullQuery)
      QueryType: SELECT
      includeInferred=true
      SELECT VarNode(S__COUNT) VarNode(SIZE_LIMIT_EXCEEDED) VarNode(NgsMeasurement_C) VarNode(NgsMeasurement_A) VarNode(j__1) VarNode(j__2)
        JoinGroupNode {
          INCLUDE %__FullQuery JOIN ON ()
          INCLUDE -subSelect-7 JOIN ON ()
        }
      

        Issue Links

          Activity

          beebs Brad Bebee created issue -
          Hide
          bryanthompson bryanthompson added a comment -

          Michael,

          Would you please take a look at this ticket and see whether there is enough information to make progress against the ticket or whether it duplicates some of the other tickets related to FILTER placement?

          Thanks,
          Bryan

          Show
          bryanthompson bryanthompson added a comment - Michael, Would you please take a look at this ticket and see whether there is enough information to make progress against the ticket or whether it duplicates some of the other tickets related to FILTER placement? Thanks, Bryan
          beebs Brad Bebee made changes -
          Field Original Value New Value
          Workflow Trac Import v2 [ 12854 ] Trac Import v3 [ 13395 ]
          beebs Brad Bebee made changes -
          Workflow Trac Import v3 [ 13395 ] Trac Import v4 [ 14724 ]
          beebs Brad Bebee made changes -
          Workflow Trac Import v4 [ 14724 ] Trac Import v5 [ 16114 ]
          Hide
          michaelschmidt michaelschmidt added a comment -

          Issue has been fixed in the master as part of join refactoring. FILTER (NOT) EXISTS was treated quite different from normal filters and was not attached as soon as possible. The new logics now performs more precise placement.

          Please find below some generic comments on the join order refactoring which included this fix.

          1. Key changes:
            1. Implemented new optimizer (ASTJoinGroupOptimizer), which puts the constructs into the right order
              1. Addressing mainly two correctness problems
                1. Reordering of non-reorderable constructs (OPTIONAL problematics)
                2. Proper handling of nodes that require variables to be bound (FILTER NOT EXISTS, BIND, certain SERVICEs)
            2. Approaches to the two problems
              1. Join order optimization now takes “partitions” (in form of OPTIONALs into account), reordering across partitions only where valid
              2. New Interface IVariableBindingRequirements with central method getRequiredVariables(), which determines the variables that must be bound prior to executing a mode
            3. Optimizations:
              1. Proper treatment of FILTER NOT EXISTS queries, now correct + more precise placement (as for other FILTERs)
              2. Also more precise placement of FILTER expressions that cannot be attached to a single join group node, namely
                1. … as early as possible
                2. … correctly placed in case triple patterns are reordered by the static optimizer
              3. Proper treatment of complex SERVICEs requiring incoming bound variables -> placed at first possible position
              4. Proper treatment of BIND and VALUES clauses -> placed at first
            4. Reusable components for FILTER placement, extraction of interesting variable sets from join groups, etc. -> clear the way for accelerated implementation of other rewriting heuristics
            5. Two “modes” for optimizer:
              1. Assert valid order & optimize
              2. Assert valid order only
            6. Test cases for optimizer itself at query and AST level
          1. Refactoring changes
            1. Interface IVariableBindingRequirements, with the central method getRequiredBound() and associated methods at ServiceFactory (with slightly different signature for extracting this information from service nodes)
              1. Implementation of the interface in various classes based on StaticAnalysis utility methods
              2. For services, the interface implementation is (at the time being) the same everywhere (requiredBound = \emptyset, imposing no constraint), see FulltextSearchServiceFactory.getRequiredBound() for how an implementation could look like (and we may want to adjust other services to offer incoming bound variables in the future)
              3. The implementation of this interface amounts for most of the changes. Many of them invoke setting up a new empty hash set.
            2. The optimizer itself: ASTJoinGroupOrderOptimizer
              1. Closely related are ASTJoinGroupPartition(s), offering data structure and the key utility methods for placement of nodes within partitions based on heuristics and variable binding constraints
              2. IASTJoinGroupPartitionReorderer: interface for an inter-partition optimizer; currently, there’s only a simple implementation (TypeBasedASTJoinGroupPartitionReorderer, in large parts reflecting the behaviour of the old optimizer), but this is how I would envision to hook in the StaticAnalysis (the interface might need to be changed/extended, it’s a first step).
            3. Reusable utility classes (used by the optimizer)
              1. GroupNodeVarBindingInfo & GroupNodeVarBindingInfoMap -> summary container for looking up different aspects related to variables in the group node (independent from its position in a given join group), and an associated class easing construction of GroupNodeVarBindingInfo objects for a set of nodes
              2. ASTFilterPlacer: utility class that supports the precise and correct placement of FILTER expressions within a list of IGroupMemberNodes
                ASTJoinGroupFilterExistsInfo: class implementing functionality to identify and access FILTER [NOT] EXISTS nodes within a join group (which are translated as a hybrid of an ASK subquery and a FilterNode, thus requiring special handling)
              3. ASTTypeBasedNodeClassifier (&ASTTypeBasedNodeClassifierConstraint): supports, given a set of types as input, the partitioning of a node list into lists of nodes with the give type + rest, including additional constraints for membership to a certain partition
            4. Minor things
          2. ASTBottomUpOptimizer -> minor bugfix (as documented inline)
          3. ASTSparql11SubqueryOptimizer -> minor bugfix (inheritance of bindings clauses to subqueries)
          4. ASTStaticBindingsOptimizer -> minor bugfix (see in code documentation for details)
          5. DefaultOptimizerList -> hooked in new optimizer
          Show
          michaelschmidt michaelschmidt added a comment - Issue has been fixed in the master as part of join refactoring. FILTER (NOT) EXISTS was treated quite different from normal filters and was not attached as soon as possible. The new logics now performs more precise placement. Please find below some generic comments on the join order refactoring which included this fix. — Key changes: Implemented new optimizer (ASTJoinGroupOptimizer), which puts the constructs into the right order Addressing mainly two correctness problems Reordering of non-reorderable constructs (OPTIONAL problematics) Proper handling of nodes that require variables to be bound (FILTER NOT EXISTS, BIND, certain SERVICEs) Approaches to the two problems Join order optimization now takes “partitions” (in form of OPTIONALs into account), reordering across partitions only where valid New Interface IVariableBindingRequirements with central method getRequiredVariables(), which determines the variables that must be bound prior to executing a mode Optimizations: Proper treatment of FILTER NOT EXISTS queries, now correct + more precise placement (as for other FILTERs) Also more precise placement of FILTER expressions that cannot be attached to a single join group node, namely … as early as possible … correctly placed in case triple patterns are reordered by the static optimizer Proper treatment of complex SERVICEs requiring incoming bound variables -> placed at first possible position Proper treatment of BIND and VALUES clauses -> placed at first Reusable components for FILTER placement, extraction of interesting variable sets from join groups, etc. -> clear the way for accelerated implementation of other rewriting heuristics Two “modes” for optimizer: Assert valid order & optimize Assert valid order only Test cases for optimizer itself at query and AST level Refactoring changes Interface IVariableBindingRequirements, with the central method getRequiredBound() and associated methods at ServiceFactory (with slightly different signature for extracting this information from service nodes) Implementation of the interface in various classes based on StaticAnalysis utility methods For services, the interface implementation is (at the time being) the same everywhere (requiredBound = \emptyset, imposing no constraint), see FulltextSearchServiceFactory.getRequiredBound() for how an implementation could look like (and we may want to adjust other services to offer incoming bound variables in the future) The implementation of this interface amounts for most of the changes. Many of them invoke setting up a new empty hash set. The optimizer itself: ASTJoinGroupOrderOptimizer Closely related are ASTJoinGroupPartition(s), offering data structure and the key utility methods for placement of nodes within partitions based on heuristics and variable binding constraints IASTJoinGroupPartitionReorderer: interface for an inter-partition optimizer; currently, there’s only a simple implementation (TypeBasedASTJoinGroupPartitionReorderer, in large parts reflecting the behaviour of the old optimizer), but this is how I would envision to hook in the StaticAnalysis (the interface might need to be changed/extended, it’s a first step). Reusable utility classes (used by the optimizer) GroupNodeVarBindingInfo & GroupNodeVarBindingInfoMap -> summary container for looking up different aspects related to variables in the group node (independent from its position in a given join group), and an associated class easing construction of GroupNodeVarBindingInfo objects for a set of nodes ASTFilterPlacer: utility class that supports the precise and correct placement of FILTER expressions within a list of IGroupMemberNodes ASTJoinGroupFilterExistsInfo: class implementing functionality to identify and access FILTER [NOT] EXISTS nodes within a join group (which are translated as a hybrid of an ASK subquery and a FilterNode, thus requiring special handling) ASTTypeBasedNodeClassifier (&ASTTypeBasedNodeClassifierConstraint): supports, given a set of types as input, the partitioning of a node list into lists of nodes with the give type + rest, including additional constraints for membership to a certain partition Minor things ASTBottomUpOptimizer -> minor bugfix (as documented inline) ASTSparql11SubqueryOptimizer -> minor bugfix (inheritance of bindings clauses to subqueries) ASTStaticBindingsOptimizer -> minor bugfix (see in code documentation for details) DefaultOptimizerList -> hooked in new optimizer
          michaelschmidt michaelschmidt made changes -
          Status Accepted [ 10101 ] In Progress [ 3 ]
          Priority Highest [ 1 ]
          michaelschmidt michaelschmidt made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          jjc Jeremy Carroll made changes -
          Link This issue relates to BLZG-1366 [ BLZG-1366 ]
          bryanthompson bryanthompson made changes -
          Status Resolved [ 5 ] In Review [ 10100 ]
          bryanthompson bryanthompson made changes -
          Resolution Done [ 10000 ]
          Status In Review [ 10100 ] Done [ 10000 ]
          beebs Brad Bebee made changes -
          Workflow Trac Import v5 [ 16114 ] Trac Import v6 [ 18244 ]
          beebs Brad Bebee made changes -
          Fix Version/s BLAZEGRAPH_RELEASE_1_5_2 [ 10164 ]
          beebs Brad Bebee made changes -
          Workflow Trac Import v6 [ 18244 ] Trac Import v7 [ 19641 ]
          beebs Brad Bebee made changes -
          Workflow Trac Import v7 [ 19641 ] Trac Import v8 [ 21263 ]

            People

            • Assignee:
              michaelschmidt michaelschmidt
              Reporter:
              jeremycarroll jeremycarroll
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: