Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-9159

PipelinedHashIndexAndSolutionSetJoinOp messes up SERVICE + MINUS

    Details

      Description

      When running this query with LIMIT, MINUS does not seem to apply properly:

      SELECT DISTINCT ?item WHERE {
        hint:Query hint:optimizer "None".
        SERVICE wikibase:mwapi {
          bd:serviceParam wikibase:api "Search";
                          wikibase:endpoint "www.wikidata.org";
                          mwapi:srsearch "zika haswbstatement:P31=Q13442814".
          ?title wikibase:apiOutput mwapi:title.
        }
        BIND(IRI(CONCAT(STR(wd:), ?title)) AS ?item)
        MINUS { ?item wdt:P921 wd:Q202864. }
      } 
      # will work fine without LIMIT
      LIMIT 10000
      

      The difference is that without LIMIT, the query uses JVMSolutionSetHashJoinOp to perform the join, while with LIMIT it uses PipelinedHashIndexAndSolutionSetJoinOp (per usePipelinedHashJoin in AST2Bop.addSubgroup). That last one seems to not do the MINUS correctly.

      See more discussion in: https://github.com/blazegraph/database/issues/107

        Activity

        Hide
        stasmalyshev stasmalyshev added a comment -
        Show
        stasmalyshev stasmalyshev added a comment - Pull request: https://github.com/blazegraph/database/pull/113
        Show
        beebs Brad Bebee added a comment - https://github.com/blazegraph/bigdata/pull/523
        Hide
        beebs Brad Bebee added a comment -

        stasmalyshev One test failure here. Try:

        cd bigdata-rdf-test
        mvn -Dtest=TestPipelinedHashJoin#testPipelinedHashIncomingBindingsBufferThreshold test
        

        bryanthompson

        Show
        beebs Brad Bebee added a comment - stasmalyshev One test failure here. Try: cd bigdata-rdf-test mvn -Dtest=TestPipelinedHashJoin#testPipelinedHashIncomingBindingsBufferThreshold test bryanthompson
        Hide
        stasmalyshev stasmalyshev added a comment -

        Will take a look tomorrow, if the proposed fix fails the tests and I can't figure it out tomorrow, we can bump this one from 2.1.5...

        Show
        stasmalyshev stasmalyshev added a comment - Will take a look tomorrow, if the proposed fix fails the tests and I can't figure it out tomorrow, we can bump this one from 2.1.5...
        Hide
        stasmalyshev stasmalyshev added a comment -

        Seems not to fail locally for me, but I notice it says it's only a partial fix. So I propose to postpone this one till next release.

        Show
        stasmalyshev stasmalyshev added a comment - Seems not to fail locally for me, but I notice it says it's only a partial fix. So I propose to postpone this one till next release.
        Hide
        stasmalyshev stasmalyshev added a comment -

        Here's the log of the CI run (we've finally made it run): https://integration.wikimedia.org/ci/job/wikidata-query-blazegraph-maven-java8-docker/6/console and it seems to be all OK. Is there something we're doing wrong maybe?

        Show
        stasmalyshev stasmalyshev added a comment - Here's the log of the CI run (we've finally made it run): https://integration.wikimedia.org/ci/job/wikidata-query-blazegraph-maven-java8-docker/6/console and it seems to be all OK. Is there something we're doing wrong maybe?
        Hide
        beebs Brad Bebee added a comment -
        Show
        beebs Brad Bebee added a comment - bryanthompson
        Hide
        beebs Brad Bebee added a comment -

        bryanthompson Michael Schmidt Applied Michael's fix with test cases.

        Show
        beebs Brad Bebee added a comment - bryanthompson Michael Schmidt Applied Michael's fix with test cases.

          People

          • Assignee:
            beebs Brad Bebee
            Reporter:
            stasmalyshev stasmalyshev
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: