Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-2027

HTreeDistinctBindingSetsOp fails on query with named solution set and SELECT *

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Done
    • Affects Version/s: BLAZEGRAPH_2_1_2
    • Fix Version/s: BLAZEGRAPH_2_2_0
    • Component/s: Workbench
    • Labels:
      None

      Description

      The following query:

      base <http://localhost:8000/>
      DELETE {
        GRAPH </graph/aboxes/test0_abox> {
          ?record ?p ?o
        }
      }
      WHERE {
        { SELECT * { INCLUDE %SelectAndModifyOperation } OFFSET 0 LIMIT 1 }
        ?record ?p ?o.
      }
      

      runs normally without analytic mode.

      Enabling analytic mode, by using the

      -Dcom.bigdata.rdf.sparql.ast.QueryHints.analytic=true

      setting, makes the query fail with:

      ERROR: SPARQL-UPDATE: updateStr=base 
      DELETE {
        GRAPH  {
          ?record ?p ?o
        }
      }
      WHERE {
        { SELECT * { INCLUDE %SelectAndModifyOperation } OFFSET 0 LIMIT 1 }
        ?record ?p ?o.
      }
      
      java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.IllegalArgumentException
      	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
      	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
      	at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281)
      	at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlUpdate(QueryServlet.java:448)
      	at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:229)
      	at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:269)
      	at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:192)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
      	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
      	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
      	at org.eclipse.jetty.server.Server.handle(Server.java:497)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
      	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.util.concurrent.ExecutionException: org.openrdf.query.UpdateExecutionException: java.lang.IllegalArgumentException
      	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
      	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
      	at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:552)
      	at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlUpdateTask.call(QueryServlet.java:460)
      	at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	... 1 more
      Caused by: org.openrdf.query.UpdateExecutionException: java.lang.IllegalArgumentException
      	at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1080)
      	at com.bigdata.rdf.sail.BigdataSailUpdate.execute2(BigdataSailUpdate.java:152)
      	at com.bigdata.rdf.sail.webapp.BigdataRDFContext$UpdateTask.doQuery(BigdataRDFContext.java:1934)
      	at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.innerCall(BigdataRDFContext.java:1536)
      	at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:1501)
      	at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:714)
      	... 4 more
      Caused by: java.lang.IllegalArgumentException
      	at com.bigdata.bop.solutions.HTreeDistinctBindingSetsOp.(HTreeDistinctBindingSetsOp.java:112)
      	at com.bigdata.bop.solutions.HTreeDistinctBindingSetsOp.(HTreeDistinctBindingSetsOp.java:118)
      	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.addDistinctProjectionOp(AST2BOpUtility.java:5861)
      	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.addNamedSubquery(AST2BOpUtility.java:904)
      	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.addNamedSubqueries(AST2BOpUtility.java:875)
      	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.convertQueryBaseWithScopedVars(AST2BOpUtility.java:447)
      	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.convert(AST2BOpUtility.java:286)
      	at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.optimizeQuery(ASTEvalHelper.java:426)
      	at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.evaluateTupleQuery(ASTEvalHelper.java:212)
      	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertDeleteInsert(AST2BOpUpdate.java:672)
      	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertUpdateSwitch(AST2BOpUpdate.java:443)
      	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUpdate.convertUpdate(AST2BOpUpdate.java:293)
      	at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.executeUpdate(ASTEvalHelper.java:1072)
      	... 9 more
      

      Specifically, the issue is in HTreeDistinctBindingSetsOp which seems to require non-empty vars; comparing it to it's non-analytic mode counterpart JVMDistinctBindingSetsOp, which only checks that vars are not null, but could be empty.

        Activity

        Hide
        beebs Brad Bebee added a comment -
        Show
        beebs Brad Bebee added a comment - Adding Michael Schmidt thompsonbry
        Hide
        michaelschmidt michaelschmidt added a comment -

        Edgar, thanks for reporting. I gave it a quick try but was not able to reproduce it. Could you please provide the following information:

        1.) Which release did you test?
        2.) Is it possible to reproduce the failure with an example not using a named solution set (e.g., by just injecting the value via BIND or VALUES in the subquery)? If so, could you please sketch a toy example that allows to reproduce the failure? If not, could you please tell what's contained in the solution set)?

        Show
        michaelschmidt michaelschmidt added a comment - Edgar, thanks for reporting. I gave it a quick try but was not able to reproduce it. Could you please provide the following information: 1.) Which release did you test? 2.) Is it possible to reproduce the failure with an example not using a named solution set (e.g., by just injecting the value via BIND or VALUES in the subquery)? If so, could you please sketch a toy example that allows to reproduce the failure? If not, could you please tell what's contained in the solution set)?
        Hide
        edgarr Edgar R added a comment - - edited

        Michael, thank you for looking into this.

        1.) Which release did you test?

        I tested with v2.0.2 and with the latest master in https://github.com/blazegraph/database.
        I've tried to replicate by using the "Analytic" flag in the workbench and that
        didn't replicate the issue, but when enabling analytic mode globally using the system property
        `-Dcom.bigdata.rdf.sparql.ast.QueryHints.analytic=true` on blazegraph startup, the issue is present.
        I'm not sure if that's another bug, but the query exec seems to have two different code paths for both
        previous cases.

        2.) Is it possible to reproduce the failure with an example not using a named solution set (e.g., by just injecting the value via BIND or VALUES in the subquery)? If so, could you please sketch a toy example that allows to reproduce the failure? If not, could you please tell what's contained in the solution set)?

        I was not able to reproduce without using the named solution set. It seems this issue happens when
        the created solution set is empty.

        So, the recipe to replicate would be:
        1. Start Blazegraph with -Dcom.bigdata.rdf.sparql.ast.QueryHints.analytic=true
        2. Create an empty namespace, use that namespace
        3. Execute update:

        DROP SILENT SOLUTIONS %SelectAndModifyOperation;
        INSERT INTO %SelectAndModifyOperation
        SELECT ?record
        WHERE {
           ?record <urn:property> ?record
        }
        

        4. Execute update:

        base <http://localhost:8000/>
        DELETE {
          GRAPH </graph/aboxes/test0_abox> {
            ?record ?p ?o
          }
        }
        WHERE {
          { SELECT * { INCLUDE %SelectAndModifyOperation } OFFSET 0 LIMIT 1 }
          ?record ?p ?o.
        }
        
        Show
        edgarr Edgar R added a comment - - edited Michael, thank you for looking into this. 1.) Which release did you test? I tested with v2.0.2 and with the latest master in https://github.com/blazegraph/database . I've tried to replicate by using the "Analytic" flag in the workbench and that didn't replicate the issue, but when enabling analytic mode globally using the system property `-Dcom.bigdata.rdf.sparql.ast.QueryHints.analytic=true` on blazegraph startup, the issue is present. I'm not sure if that's another bug, but the query exec seems to have two different code paths for both previous cases. 2.) Is it possible to reproduce the failure with an example not using a named solution set (e.g., by just injecting the value via BIND or VALUES in the subquery)? If so, could you please sketch a toy example that allows to reproduce the failure? If not, could you please tell what's contained in the solution set)? I was not able to reproduce without using the named solution set. It seems this issue happens when the created solution set is empty. So, the recipe to replicate would be: 1. Start Blazegraph with -Dcom.bigdata.rdf.sparql.ast.QueryHints.analytic=true 2. Create an empty namespace, use that namespace 3. Execute update: DROP SILENT SOLUTIONS %SelectAndModifyOperation; INSERT INTO %SelectAndModifyOperation SELECT ?record WHERE { ?record <urn:property> ?record } 4. Execute update: base <http: //localhost:8000/> DELETE { GRAPH </graph/aboxes/test0_abox> { ?record ?p ?o } } WHERE { { SELECT * { INCLUDE %SelectAndModifyOperation } OFFSET 0 LIMIT 1 } ?record ?p ?o. }
        Hide
        edgarr Edgar R added a comment -

        Any updates on this issue? Thanks.

        Show
        edgarr Edgar R added a comment - Any updates on this issue? Thanks.
        Hide
        michaelschmidt michaelschmidt added a comment -

        Managed to reproduce with latest master. As you pointed out already, there are two problems:

        • The first (causing the issue) is that we do not allow for empty join variables in projection in HTreeDistinctBindingSetsOp. Given that the downstream operators (HTreeHashJoinUtility) support this case, from my side nothing speaks against removing this null check. The proposed fix is in https://github.com/SYSTAP/bigdata/pull/461, running through CI now.
        • The second problem is the different behavior w.r.t. analytics mode, i.e. what you observe seems to suggest that analytic mode via workbench for UPDATE queries is actually not working (properly). I added a follow-up ticket: BLZG-2030.

        I guess this can be safely merged down once CI is green and could also be included into our 2.2 branch. Brad Bebee

        Show
        michaelschmidt michaelschmidt added a comment - Managed to reproduce with latest master. As you pointed out already, there are two problems: The first (causing the issue) is that we do not allow for empty join variables in projection in HTreeDistinctBindingSetsOp. Given that the downstream operators (HTreeHashJoinUtility) support this case, from my side nothing speaks against removing this null check. The proposed fix is in https://github.com/SYSTAP/bigdata/pull/461 , running through CI now. The second problem is the different behavior w.r.t. analytics mode, i.e. what you observe seems to suggest that analytic mode via workbench for UPDATE queries is actually not working (properly). I added a follow-up ticket: BLZG-2030 . I guess this can be safely merged down once CI is green and could also be included into our 2.2 branch. Brad Bebee
        Hide
        michaelschmidt michaelschmidt added a comment -

        CI green, PR has been merged. Brad Bebee As per discussion with Bryan, please carry over to 2.2RC branch.

        Show
        michaelschmidt michaelschmidt added a comment - CI green, PR has been merged. Brad Bebee As per discussion with Bryan, please carry over to 2.2RC branch.
        Show
        beebs Brad Bebee added a comment - Patched into 2.2.0 RC branch: https://github.com/SYSTAP/bigdata/pull/455/commits/17d301f237adc8b8f31fc7f065c4744656decf50

          People

          • Assignee:
            michaelschmidt michaelschmidt
            Reporter:
            edgarr Edgar R
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: