Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-774

AtomicRowFilter UnsupportedOperationException

    Details

      Description

      The following stack trace was produced when preparing 1.2.3 for a release while running the NanoSparqlServer test suite. This trace appears for several REST API methods that attempt to scan the GlobalRowStore in order to report the existing KBs. The root cause appears to be that the GRS was split due to the number of tests that were run and the consequent mutations on the GRS index. Once the GRS was split, the AtomicRowFilter hits the UnsupportedOperationException. This is apparently a long-standing problem, but we normally use the sparse index only for locatable resource metadata, and it is unusual to have so many distinct locatable resources. The large number of unit tests is driving the creation (and destruction) of those mutable resources with the result that an overflow on the DataService journal for the GRS eventually does an IndexSegment BUILD rather than just copying across the tuples in the index to the new live Journal. Once that BUILD is put into play, we wind up with a view of the GRS shard that is comprised of multiple resources (the live journal and an index segment). At that point, a GRS scan will hit this exception.

      Based on this analysis, we can write a unit test that would recreate this problem.

      test_describeDataSets01
      Caused by: java.lang.UnsupportedOperationException
      	at com.bigdata.sparse.AtomicRowFilter$Transformerator$1.getSourceIndex(AtomicRowFilter.java:215)
      	at com.bigdata.btree.ResultSet.copyTuple(ResultSet.java:577)
      	at com.bigdata.btree.ResultSet.<init>(ResultSet.java:1083)
      	at com.bigdata.service.DataService$RangeIteratorTask.doTask(DataService.java:1795)
      	at com.bigdata.service.DataService$RangeIteratorTask.doTask(DataService.java:1741)
      	at com.bigdata.journal.AbstractTask.call2(AbstractTask.java:1899)
      	at com.bigdata.journal.AbstractTask.call(AbstractTask.java:1788)
      	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      	... 1 more
      

      The problem with the UnsupportedOperationException in getSourceIndex() is a blocker.

      This attribute is probably not used, and we might be able to substitute a ZERO (0) for this specific method, but it needs to be tracked down.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        I have a unit test that replicates the problem.

        Show
        bryanthompson bryanthompson added a comment - I have a unit test that replicates the problem.
        Hide
        bryanthompson bryanthompson added a comment -

        Bug fix. I modified the getSourceIndex() to return ZERO (0) for the AtomicRowFilter. The GRS does not support blobs so the source index attribute can not be used.

        The underlying reason why we can not expose the source index attribute is that each logical row could be pulling tuples from more than one source index (e.g., the live journal, a historical journal, and an index segment).

        The fix causes the unit test to pass.

        Committed revision r7181.

        Show
        bryanthompson bryanthompson added a comment - Bug fix. I modified the getSourceIndex() to return ZERO (0) for the AtomicRowFilter. The GRS does not support blobs so the source index attribute can not be used. The underlying reason why we can not expose the source index attribute is that each logical row could be pulling tuples from more than one source index (e.g., the live journal, a historical journal, and an index segment). The fix causes the unit test to pass. Committed revision r7181.

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: