Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-2082

one-argument substr produces NPE when combined with GROUP By

    Details

      Description

      The following query:

      SELECT (substr(str(?url), 3) as ?n2) WHERE {
        ?x wdt:P31 wd:Q5 .
        ?url schema:about ?x .
      } GROUP BY ?url
      LIMIT 10
      

      produces NPE when run. Adding second argument removes the NPE, so does removing GROUP BY. More complex query exhibiting same behavior:

      SELECT (concat("[[wikipedia:", substr(str(?link),31),  " | ", ?headLabel, "]]") as ?title) 
      WHERE {
        ?id wdt:P31 wd:Q35657 .
        ?id wdt:P6 ?head .
        ?head wdt:P102 ?party .
        SERVICE wikibase:label {
          bd:serviceParam wikibase:language "en" .
          ?head rdfs:label ?headLabel .
        }
        ?link schema:about ?head .
        ?link schema:isPartOf <https://en.wikipedia.org/> .
      } GROUP BY ?id ?head ?headLabel ?link 
      

      Again, using substr(..., 31, 1000) works.

        Activity

        Hide
        stasmalyshev stasmalyshev added a comment - - edited

        The full trace:

        ERROR: SPARQL-QUERY: queryStr=SELECT (concat("[[wikipedia:", substr(str(?link),31),  " | ", ?headLabel, "]]") as ?title) 
        WHERE {
          ?id wdt:P31 wd:Q35657 .
          ?id wdt:P6 ?head .
          ?head wdt:P102 ?party .
          SERVICE wikibase:label {
            bd:serviceParam wikibase:language "en" .
            ?head rdfs:label ?headLabel .
          }
          ?link schema:about ?head .
          ?link schema:isPartOf <https://en.wikipedia.org/> .
        } GROUP BY ?id ?head ?headLabel ?link 
        
        java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.lang.NullPointerException
        	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        	at java.util.concurrent.FutureTask.get(FutureTask.java:188)
        	at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281)
        	at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:636)
        	at com.bigdata.rdf.sail.webapp.QueryServlet.doGet(QueryServlet.java:278)
        	at com.bigdata.rdf.sail.webapp.RESTServlet.doGet(RESTServlet.java:240)
        	at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doGet(MultiTenancyServlet.java:270)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
        	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
        	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
        	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
        	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
        	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
        	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        	at org.eclipse.jetty.server.Server.handle(Server.java:497)
        	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        	at java.lang.Thread.run(Thread.java:745)
        Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
        	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        	at java.util.concurrent.FutureTask.get(FutureTask.java:188)
        	at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:834)
        	at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:653)
        	at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        	... 1 more
        Caused by: java.lang.NullPointerException
        	at com.bigdata.bop.solutions.GroupByRewriter.rewrite2(GroupByRewriter.java:396)
        	at com.bigdata.bop.solutions.GroupByRewriter.rewrite(GroupByRewriter.java:373)
        	at com.bigdata.bop.solutions.GroupByRewriter.rewrite2(GroupByRewriter.java:413)
        	at com.bigdata.bop.solutions.GroupByRewriter.rewrite(GroupByRewriter.java:373)
        	at com.bigdata.bop.solutions.GroupByRewriter.rewrite2(GroupByRewriter.java:413)
        	at com.bigdata.bop.solutions.GroupByRewriter.rewrite(GroupByRewriter.java:373)
        	at com.bigdata.bop.solutions.GroupByRewriter.rewrite2(GroupByRewriter.java:413)
        	at com.bigdata.bop.solutions.GroupByRewriter.rewrite(GroupByRewriter.java:373)
        	at com.bigdata.bop.solutions.GroupByRewriter.<init>(GroupByRewriter.java:277)
        	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.addAggregation(AST2BOpUtility.java:4551)
        	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.convertQueryBaseWithScopedVars(AST2BOpUtility.java:515)
        	at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.convert(AST2BOpUtility.java:286)
        	at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.optimizeQuery(ASTEvalHelper.java:426)
        	at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.evaluateTupleQuery(ASTEvalHelper.java:211)
        	at com.bigdata.rdf.sail.BigdataSailTupleQuery.evaluate(BigdataSailTupleQuery.java:79)
        	at com.bigdata.rdf.sail.BigdataSailTupleQuery.evaluate(BigdataSailTupleQuery.java:61)
        	at org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:75)
        	at com.bigdata.rdf.sail.webapp.BigdataRDFContext$TupleQueryTask.doQuery(BigdataRDFContext.java:1679)
        	at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.innerCall(BigdataRDFContext.java:1536)
        	at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:1501)
        	at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:714)
        	... 4 more
        
        Show
        stasmalyshev stasmalyshev added a comment - - edited The full trace: ERROR: SPARQL-QUERY: queryStr=SELECT (concat("[[wikipedia:", substr(str(?link),31), " | ", ?headLabel, "]]") as ?title) WHERE { ?id wdt:P31 wd:Q35657 . ?id wdt:P6 ?head . ?head wdt:P102 ?party . SERVICE wikibase:label { bd:serviceParam wikibase:language "en" . ?head rdfs:label ?headLabel . } ?link schema:about ?head . ?link schema:isPartOf <https://en.wikipedia.org/> . } GROUP BY ?id ?head ?headLabel ?link java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281) at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:636) at com.bigdata.rdf.sail.webapp.QueryServlet.doGet(QueryServlet.java:278) at com.bigdata.rdf.sail.webapp.RESTServlet.doGet(RESTServlet.java:240) at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doGet(MultiTenancyServlet.java:270) at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:834) at com.bigdata.rdf.sail.webapp.QueryServlet$SparqlQueryTask.call(QueryServlet.java:653) at com.bigdata.rdf.task.ApiTaskForIndexManager.call(ApiTaskForIndexManager.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ... 1 more Caused by: java.lang.NullPointerException at com.bigdata.bop.solutions.GroupByRewriter.rewrite2(GroupByRewriter.java:396) at com.bigdata.bop.solutions.GroupByRewriter.rewrite(GroupByRewriter.java:373) at com.bigdata.bop.solutions.GroupByRewriter.rewrite2(GroupByRewriter.java:413) at com.bigdata.bop.solutions.GroupByRewriter.rewrite(GroupByRewriter.java:373) at com.bigdata.bop.solutions.GroupByRewriter.rewrite2(GroupByRewriter.java:413) at com.bigdata.bop.solutions.GroupByRewriter.rewrite(GroupByRewriter.java:373) at com.bigdata.bop.solutions.GroupByRewriter.rewrite2(GroupByRewriter.java:413) at com.bigdata.bop.solutions.GroupByRewriter.rewrite(GroupByRewriter.java:373) at com.bigdata.bop.solutions.GroupByRewriter.<init>(GroupByRewriter.java:277) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.addAggregation(AST2BOpUtility.java:4551) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.convertQueryBaseWithScopedVars(AST2BOpUtility.java:515) at com.bigdata.rdf.sparql.ast.eval.AST2BOpUtility.convert(AST2BOpUtility.java:286) at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.optimizeQuery(ASTEvalHelper.java:426) at com.bigdata.rdf.sparql.ast.eval.ASTEvalHelper.evaluateTupleQuery(ASTEvalHelper.java:211) at com.bigdata.rdf.sail.BigdataSailTupleQuery.evaluate(BigdataSailTupleQuery.java:79) at com.bigdata.rdf.sail.BigdataSailTupleQuery.evaluate(BigdataSailTupleQuery.java:61) at org.openrdf.repository.sail.SailTupleQuery.evaluate(SailTupleQuery.java:75) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$TupleQueryTask.doQuery(BigdataRDFContext.java:1679) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.innerCall(BigdataRDFContext.java:1536) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:1501) at com.bigdata.rdf.sail.webapp.BigdataRDFContext$AbstractQueryTask.call(BigdataRDFContext.java:714) ... 4 more
        Hide
        stasmalyshev stasmalyshev added a comment -
        Show
        stasmalyshev stasmalyshev added a comment - /cc michaelschmidt
        Hide
        bryanthompson bryanthompson added a comment -

        igorkim Please triage.

        michaelschmidt

        Show
        bryanthompson bryanthompson added a comment - igorkim Please triage. michaelschmidt
        Hide
        igorkim igorkim added a comment -

        NPE is thrown due to GroupByRewriter tries to check arity for all arguments of substr function, but third argument is omitted and thus defaults to null.
        SPARQL grammar allows 2-argument variant of substr function, so GroupByRewriter should be changed to allow null arguments.

        Show
        igorkim igorkim added a comment - NPE is thrown due to GroupByRewriter tries to check arity for all arguments of substr function, but third argument is omitted and thus defaults to null. SPARQL grammar allows 2-argument variant of substr function, so GroupByRewriter should be changed to allow null arguments.
        Hide
        bryanthompson bryanthompson added a comment -

        igorkim Can you propose a specific code change and new unit tests?

        The following block would appear to be implicated.

                /*
                 * Depth first recursion for each child argument.
                 */
                int index = 0;
        
                final Iterator<BOp> itr = expr.argIterator();
                
                while (itr.hasNext()) {
        
                    final IValueExpression<?> c = (IValueExpression<?>) itr.next();
        
                    // rewrite child.
                    final IValueExpression<?> newC = rewrite(c, f, aggExpr);
        
                    if (newC != c) {
        
                        // copy-on-write for parent iff child was rewritten.
                        expr = (IValueExpression<?>) ((BOpBase) expr).setArg(index,
                                newC);
        
                    }
        
                    index++;
                    
                }
        

        Personally, I suggest simply a special case if the child reference is null.

        Show
        bryanthompson bryanthompson added a comment - igorkim Can you propose a specific code change and new unit tests? The following block would appear to be implicated. /* * Depth first recursion for each child argument. */ int index = 0; final Iterator<BOp> itr = expr.argIterator(); while (itr.hasNext()) { final IValueExpression<?> c = (IValueExpression<?>) itr.next(); // rewrite child. final IValueExpression<?> newC = rewrite(c, f, aggExpr); if (newC != c) { // copy-on-write for parent iff child was rewritten. expr = (IValueExpression<?>) ((BOpBase) expr).setArg(index, newC); } index++; } Personally, I suggest simply a special case if the child reference is null.
        Hide
        michaelschmidt michaelschmidt added a comment -

        Proposed fix in https://github.com/blazegraph/bigdata/pull/495, running through CI now.

        Show
        michaelschmidt michaelschmidt added a comment - Proposed fix in https://github.com/blazegraph/bigdata/pull/495 , running through CI now.
        Hide
        beebs Brad Bebee added a comment -

        Cherry picked into 2.1.5.

        Show
        beebs Brad Bebee added a comment - Cherry picked into 2.1.5.
        Hide
        michaelschmidt michaelschmidt added a comment -
        Show
        michaelschmidt michaelschmidt added a comment - For the records: see https://github.com/blazegraph/bigdata/pull/495 for PR.

          People

          • Assignee:
            michaelschmidt michaelschmidt
            Reporter:
            stasmalyshev stasmalyshev
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: