Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-492

Query failure when comparing with non materialized value

    Details

      Description

      The following query fails when the query binding of arg is set to a value that is not available as a term in the database:

      SELECT ?subj WHERE {
        ?subj <os:prop> ?val .
        FILTER(STR(?val) != ?arg)}
      

      I discovered this because the logs of our CI are filled with com.bigdata.rdf.internal.NotMaterializedException reports. But on queries that compare for equality (=) instead of !=, and those return the correct value even after the ConditionalRoutingOp is shut down because of this exception.

      Other than the obvious query result failure, it actually worries me that queries will silently (only logging) ignore unexpected exceptions and return possibly incorrect results instead of reporting the unexpected failure as a SailException to the caller. Instead of shutting down this bop after catching an exception, the exception should have propagated to the thread that started the query in the first place.

      Example of logged stacktraces:

      WARN : 3309      com.bigdata.journal.Journal.executorService3 com.bigdata.util.concurrent.Haltable.logCause(Haltable.java:418): com.bigdata.util.concurrent.Haltable@de16483 : isFirstCause=true : java.util.concurrent.ExecutionException: java.lang.RuntimeException: com.bigdata.rdf.internal.NotMaterializedException: DummyIV
      java.util.concurrent.ExecutionException: java.lang.RuntimeException: com.bigdata.rdf.internal.NotMaterializedException: DummyIV
      	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
      	at java.util.concurrent.FutureTask.get(FutureTask.java:83)
      	at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTask.call(ChunkedRunningQuery.java:1106)
      	at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTaskWrapper.run(ChunkedRunningQuery.java:704)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      	at com.bigdata.concurrent.FutureTaskMon.run(FutureTaskMon.java:63)
      	at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkFutureTask.run(ChunkedRunningQuery.java:617)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      	at java.lang.Thread.run(Thread.java:662)
      Caused by: java.lang.RuntimeException: com.bigdata.rdf.internal.NotMaterializedException: DummyIV
      	at com.bigdata.rdf.internal.constraints.SPARQLConstraint.accept(SPARQLConstraint.java:128)
      	at com.bigdata.bop.bset.ConditionalRoutingOp$ConditionalRouteTask.call(ConditionalRoutingOp.java:168)
      	at com.bigdata.bop.bset.ConditionalRoutingOp$ConditionalRouteTask.call(ConditionalRoutingOp.java:1)
      	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      	at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTask.call(ChunkedRunningQuery.java:1105)
      	... 9 more
      Caused by: com.bigdata.rdf.internal.NotMaterializedException: DummyIV
      	at com.bigdata.rdf.internal.AbstractIV.getValue(AbstractIV.java:659)
      	at com.bigdata.rdf.internal.constraints.CompareBOp.accept(CompareBOp.java:154)
      	at com.bigdata.rdf.internal.constraints.XSDBooleanIVValueExpression.get(XSDBooleanIVValueExpression.java:60)
      	at com.bigdata.rdf.internal.constraints.XSDBooleanIVValueExpression.get(XSDBooleanIVValueExpression.java:1)
      	at com.bigdata.rdf.internal.constraints.SPARQLConstraint.accept(SPARQLConstraint.java:111)
      	... 14 more
      WARN : 3310      com.bigdata.journal.Journal.executorService3 com.bigdata.util.concurrent.Haltable.logCause(Haltable.java:418): com.bigdata.util.concurrent.Haltable@de16483 : isFirstCause=false : java.lang.InterruptedException
      java.lang.InterruptedException
      	at com.bigdata.util.concurrent.Haltable.cancel(Haltable.java:220)
      	at com.bigdata.bop.engine.AbstractRunningQuery.cancel(AbstractRunningQuery.java:1024)
      	at com.bigdata.bop.engine.AbstractRunningQuery.halt(AbstractRunningQuery.java:995)
      	at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTask.call(ChunkedRunningQuery.java:1108)
      	at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkTaskWrapper.run(ChunkedRunningQuery.java:704)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      	at com.bigdata.concurrent.FutureTaskMon.run(FutureTaskMon.java:63)
      	at com.bigdata.bop.engine.ChunkedRunningQuery$ChunkFutureTask.run(ChunkedRunningQuery.java:617)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      	at java.lang.Thread.run(Thread.java:662)
      

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        The QueryEngine notes the first cause of an error on a query. The logged messages indicate this first cause with

        <pre>
        isFirstCause=true
        </pre>

        For the stack trace above, it appears that the nested exception was caught by the SPARQLConstraint, which is providing the effective boolean value (EBV) of the value expression. The try/catch logic there is:

        	public boolean accept(final IBindingSet bs) {
        
        		try {
        
        			// evaluate the EBV operator
        			final XSDBooleanIV iv = get(0).get(bs);
        			
        			return iv.booleanValue();
        
        		} catch (Throwable t) {
        
        			if (InnerCause.isInnerCause(t, SparqlTypeErrorException.class)) {
        
        				// trap the type error and filter out the solution
        				if (log.isInfoEnabled())
        					log.info("discarding solution due to type error: " + bs
        							+ " : " + t);
        
        				return false;
        
        			}
        
        			throw new RuntimeException(t);
        
        		}
        
        	}
        

        That logic appears to be correct. The stack trace also shows that the exception was rethrown out of SPARQLConstraint and caught in the ChunkedRunningQuery. The relevant code block there is:

                public Void call() throws Exception {
                    if (log.isDebugEnabled())
                        log.debug("Running chunk: " + this);
        			try {
        	            ft.run(); // run
        				ft.get(); // verify success
        			} catch (Throwable t) {
        				halt(t);  // ensure query halts.
        				if (getCause() != null) {
        					// abnormal termination - wrap and rethrow.
        					throw new Exception(t);
        				}
        				// otherwise ignore exception (normal completion).
        			}
                    // Done.
                    return null;
                } // call()
        
            } // class ChunkTask
        

        The logic in halt(cause) should be such that getCause() will report non-null iff this is an abnormal termination (various exceptions whose root cause indicates that the query was interrupted are treated as normal termination. Such interrupts are generated when Sesame closes an iterator reading solutions from the query and do not represent an error in the query evaluation.)

        Running the unit test, it appears that the exception was trapped in the ChunkedRunningQuery and set on the query. However, it was not reported back to the thread draining the query solutions. I will look into why. I have filed a separate issue for this as the failure to report the error is a different problem from the NotMaterializedValue exception.

        The attachment has been incorporated into CI in the 1.0.0 release branch and the development branch as TestTicket355.

        A separate issue has been filed for the failure to report the error back to the thread draining the query solutions. See [1].

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/356

        Committed revision r4945.

        Show
        bryanthompson bryanthompson added a comment - The QueryEngine notes the first cause of an error on a query. The logged messages indicate this first cause with <pre> isFirstCause=true </pre> For the stack trace above, it appears that the nested exception was caught by the SPARQLConstraint, which is providing the effective boolean value (EBV) of the value expression. The try/catch logic there is: public boolean accept(final IBindingSet bs) { try { // evaluate the EBV operator final XSDBooleanIV iv = get(0).get(bs); return iv.booleanValue(); } catch (Throwable t) { if (InnerCause.isInnerCause(t, SparqlTypeErrorException.class)) { // trap the type error and filter out the solution if (log.isInfoEnabled()) log.info("discarding solution due to type error: " + bs + " : " + t); return false; } throw new RuntimeException(t); } } That logic appears to be correct. The stack trace also shows that the exception was rethrown out of SPARQLConstraint and caught in the ChunkedRunningQuery. The relevant code block there is: public Void call() throws Exception { if (log.isDebugEnabled()) log.debug("Running chunk: " + this); try { ft.run(); // run ft.get(); // verify success } catch (Throwable t) { halt(t); // ensure query halts. if (getCause() != null) { // abnormal termination - wrap and rethrow. throw new Exception(t); } // otherwise ignore exception (normal completion). } // Done. return null; } // call() } // class ChunkTask The logic in halt(cause) should be such that getCause() will report non-null iff this is an abnormal termination (various exceptions whose root cause indicates that the query was interrupted are treated as normal termination. Such interrupts are generated when Sesame closes an iterator reading solutions from the query and do not represent an error in the query evaluation.) Running the unit test, it appears that the exception was trapped in the ChunkedRunningQuery and set on the query. However, it was not reported back to the thread draining the query solutions. I will look into why. I have filed a separate issue for this as the failure to report the error is a different problem from the NotMaterializedValue exception. The attachment has been incorporated into CI in the 1.0.0 release branch and the development branch as TestTicket355. A separate issue has been filed for the failure to report the error back to the thread draining the query solutions. See [1] . [1] https://sourceforge.net/apps/trac/bigdata/ticket/356 Committed revision r4945.

          People

          • Assignee:
            mikepersonick mikepersonick
            Reporter:
            gjdev gjdev
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: