Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-332

CI : com.bigdata.resources.TestBuildTask2.test_builds()

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Affects Version/s: QUADS_QUERY_BRANCH
    • Fix Version/s: None
    • Component/s: B+Tree

      Description

      com.bigdata.resources.TestBuildTask2.test_builds() occasionally fails. A sample stack trace is inline below. This test should be watched to determine if the test can fail with other stack traces as well.

      {
      java.lang.IllegalArgumentException
      at com.bigdata.btree.AbstractBTree.readNodeOrLeaf(AbstractBTree.java:3819)
      at com.bigdata.btree.IndexSegment.readNodeOrLeaf(IndexSegment.java:564)
      at com.bigdata.btree.IndexSegment.readLeaf(IndexSegment.java:511)
      at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.last(IndexSegment.java:1372)
      at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.<init>(IndexSegment.java:1310)
      at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:1245)
      at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:55)
      at com.bigdata.btree.AbstractBTreeTupleCursor.lastPosition(AbstractBTreeTupleCursor.java:513)
      at com.bigdata.btree.AbstractBTreeTupleCursor.hasPrior(AbstractBTreeTupleCursor.java:1017)
      at com.bigdata.btree.view.FusedTupleCursor.hasPrior(FusedTupleCursor.java:300)
      at com.bigdata.btree.filter.Reverserator.hasNext(Reverserator.java:39)
      at com.bigdata.btree.AbstractBTreeTestCase.doEntryIteratorTest(AbstractBTreeTestCase.java:2162)
      at com.bigdata.btree.AbstractBTreeTestCase.assertSameBTree(AbstractBTreeTestCase.java:1991)
      at com.bigdata.resources.TestBuildTask2.test_builds(TestBuildTask2.java:513)
      }

        Activity

        Hide
        dmacgbr dmacgbr added a comment -

        This error has also been seen in the QUADS_QUERY_BRANCH, revision 3847, with the following stack trace:

        <testcase classname="com.bigdata.resources.TestBuildTask2" name="test_builds" time="1.371">

        <error type="java.lang.IllegalArgumentException">java.lang.IllegalArgumentException
        at com.bigdata.btree.AbstractBTree.readNodeOrLeaf(AbstractBTree.java:3778)
        at com.bigdata.btree.IndexSegment.readNodeOrLeaf(IndexSegment.java:564)
        at com.bigdata.btree.IndexSegment.readLeaf(IndexSegment.java:511)
        at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.last(IndexSegment.java:1372)
        at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.<init>(IndexSegment.java:1310)
        at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:1245)
        at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:55)
        at com.bigdata.btree.AbstractBTreeTupleCursor.lastPosition(AbstractBTreeTupleCursor.java:515)
        at com.bigdata.btree.AbstractBTreeTupleCursor.hasPrior(AbstractBTreeTupleCursor.java:1019)
        at com.bigdata.btree.view.FusedTupleCursor.hasPrior(FusedTupleCursor.java:300)
        at com.bigdata.btree.filter.Reverserator.hasNext(Reverserator.java:39)
        at com.bigdata.btree.AbstractBTreeTestCase.doEntryIteratorTest(AbstractBTreeTestCase.java:2162)
        at com.bigdata.btree.AbstractBTreeTestCase.assertSameBTree(AbstractBTreeTestCase.java:1991)
        at com.bigdata.resources.TestBuildTask2.test_builds(TestBuildTask2.java:513)
        </error>

        </testcase>

        Show
        dmacgbr dmacgbr added a comment - This error has also been seen in the QUADS_QUERY_BRANCH, revision 3847, with the following stack trace: <testcase classname="com.bigdata.resources.TestBuildTask2" name="test_builds" time="1.371"> <error type="java.lang.IllegalArgumentException">java.lang.IllegalArgumentException at com.bigdata.btree.AbstractBTree.readNodeOrLeaf(AbstractBTree.java:3778) at com.bigdata.btree.IndexSegment.readNodeOrLeaf(IndexSegment.java:564) at com.bigdata.btree.IndexSegment.readLeaf(IndexSegment.java:511) at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.last(IndexSegment.java:1372) at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.<init>(IndexSegment.java:1310) at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:1245) at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:55) at com.bigdata.btree.AbstractBTreeTupleCursor.lastPosition(AbstractBTreeTupleCursor.java:515) at com.bigdata.btree.AbstractBTreeTupleCursor.hasPrior(AbstractBTreeTupleCursor.java:1019) at com.bigdata.btree.view.FusedTupleCursor.hasPrior(FusedTupleCursor.java:300) at com.bigdata.btree.filter.Reverserator.hasNext(Reverserator.java:39) at com.bigdata.btree.AbstractBTreeTestCase.doEntryIteratorTest(AbstractBTreeTestCase.java:2162) at com.bigdata.btree.AbstractBTreeTestCase.assertSameBTree(AbstractBTreeTestCase.java:1991) at com.bigdata.resources.TestBuildTask2.test_builds(TestBuildTask2.java:513) </error> </testcase>
        Hide
        bryanthompson bryanthompson added a comment -

        It is quite possible that this issue is related to

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/71 (Concurrency problem with unisolated btree and memoizer)
        [2] https://sourceforge.net/apps/trac/bigdata/ticket/201 (Hot spot in AbstractBTree#touch())
        [3] https://sourceforge.net/apps/trac/bigdata/ticket/284 (IndexOfOfBounds in Node#getChild())
        [4] https://sourceforge.net/apps/trac/bigdata/ticket/288 (Node already coded)

        Show
        bryanthompson bryanthompson added a comment - It is quite possible that this issue is related to [1] https://sourceforge.net/apps/trac/bigdata/ticket/71 (Concurrency problem with unisolated btree and memoizer) [2] https://sourceforge.net/apps/trac/bigdata/ticket/201 (Hot spot in AbstractBTree#touch()) [3] https://sourceforge.net/apps/trac/bigdata/ticket/284 (IndexOfOfBounds in Node#getChild()) [4] https://sourceforge.net/apps/trac/bigdata/ticket/288 (Node already coded)
        Hide
        bryanthompson bryanthompson added a comment -

        This may have been caused by [1]. See if it reoccurs now.

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/308

        Show
        bryanthompson bryanthompson added a comment - This may have been caused by [1] . See if it reoccurs now. [1] https://sourceforge.net/apps/trac/bigdata/ticket/308
        Hide
        bryanthompson bryanthompson added a comment -

        This test is continuing to fail on a stochastic basis. Here is a recent stack trace:

        java.lang.IllegalArgumentException
        	at com.bigdata.btree.AbstractBTree.readNodeOrLeaf(AbstractBTree.java:3757)
        	at com.bigdata.btree.IndexSegment.readNodeOrLeaf(IndexSegment.java:564)
        	at com.bigdata.btree.IndexSegment.readLeaf(IndexSegment.java:511)
        	at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.last(IndexSegment.java:1382)
        	at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.<init>(IndexSegment.java:1320)
        	at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:1255)
        	at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:55)
        	at com.bigdata.btree.AbstractBTreeTupleCursor.lastPosition(AbstractBTreeTupleCursor.java:515)
        	at com.bigdata.btree.AbstractBTreeTupleCursor.hasPrior(AbstractBTreeTupleCursor.java:1019)
        	at com.bigdata.btree.view.FusedTupleCursor.hasPrior(FusedTupleCursor.java:300)
        	at com.bigdata.btree.filter.Reverserator.hasNext(Reverserator.java:39)
        	at com.bigdata.btree.AbstractBTreeTestCase.doEntryIteratorTest(AbstractBTreeTestCase.java:2331)
        	at com.bigdata.btree.AbstractBTreeTestCase.assertSameBTree(AbstractBTreeTestCase.java:2153)
        	at com.bigdata.resources.TestBuildTask2.test_builds(TestBuildTask2.java:518)
        

        There is clearly something odd going on. The read is against an index segment and the address of the node/leaf to be read is ZERO (however expressed). A fused view is being used with an iterator. This might be an edge condition where the index segment is empty.

        Show
        bryanthompson bryanthompson added a comment - This test is continuing to fail on a stochastic basis. Here is a recent stack trace: java.lang.IllegalArgumentException at com.bigdata.btree.AbstractBTree.readNodeOrLeaf(AbstractBTree.java:3757) at com.bigdata.btree.IndexSegment.readNodeOrLeaf(IndexSegment.java:564) at com.bigdata.btree.IndexSegment.readLeaf(IndexSegment.java:511) at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.last(IndexSegment.java:1382) at com.bigdata.btree.IndexSegment$ImmutableLeafCursor.<init>(IndexSegment.java:1320) at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:1255) at com.bigdata.btree.IndexSegment.newLeafCursor(IndexSegment.java:55) at com.bigdata.btree.AbstractBTreeTupleCursor.lastPosition(AbstractBTreeTupleCursor.java:515) at com.bigdata.btree.AbstractBTreeTupleCursor.hasPrior(AbstractBTreeTupleCursor.java:1019) at com.bigdata.btree.view.FusedTupleCursor.hasPrior(FusedTupleCursor.java:300) at com.bigdata.btree.filter.Reverserator.hasNext(Reverserator.java:39) at com.bigdata.btree.AbstractBTreeTestCase.doEntryIteratorTest(AbstractBTreeTestCase.java:2331) at com.bigdata.btree.AbstractBTreeTestCase.assertSameBTree(AbstractBTreeTestCase.java:2153) at com.bigdata.resources.TestBuildTask2.test_builds(TestBuildTask2.java:518) There is clearly something odd going on. The read is against an index segment and the address of the node/leaf to be read is ZERO (however expressed). A fused view is being used with an iterator. This might be an edge condition where the index segment is empty.
        Hide
        bryanthompson bryanthompson added a comment -

        Ok. This appears to be related to a special case for an IndexSegment with no tuples. In this case, the root leaf is not actually written onto the IndexSegment file and nleaves will be zero. This is causing problems for several bits of code, including division by zero errors which expect nleaves to be GTE ONE (1). It is also causing a problem with the fused view iterator, which is trying to materialize the first leaf in the B+Tree.

        This could be a case of an optimization which is not worth while. It might be much easier to just generate the empty leaf rather than trying to handle an IndexSegmentStore with no leaves....

        Show
        bryanthompson bryanthompson added a comment - Ok. This appears to be related to a special case for an IndexSegment with no tuples. In this case, the root leaf is not actually written onto the IndexSegment file and nleaves will be zero. This is causing problems for several bits of code, including division by zero errors which expect nleaves to be GTE ONE (1). It is also causing a problem with the fused view iterator, which is trying to materialize the first leaf in the B+Tree. This could be a case of an optimization which is not worth while. It might be much easier to just generate the empty leaf rather than trying to handle an IndexSegmentStore with no leaves....
        Hide
        bryanthompson bryanthompson added a comment -

        I'm going to handle this by changing the IndexSegmentPlan and IndexSegmentBuilder to generate an IndexSegment with a single empty leaf.


        - IndexSegmentPlan
        - if ntuples==0, configure for an empty root leaf (versus no leaf).
        - IndexSegmentBuilder
        - verify handles case with empty root leaf.
        - IndexSegmentPlan
        - unit test for empty tree.
        - IndexSegmentBuilder
        - unit test for empty tree and compare with
        ground truth empty tree.
        - Update javadoc in relevant locations.
        - Remove added trace from TestBuildTask2

        Show
        bryanthompson bryanthompson added a comment - I'm going to handle this by changing the IndexSegmentPlan and IndexSegmentBuilder to generate an IndexSegment with a single empty leaf. - IndexSegmentPlan - if ntuples==0, configure for an empty root leaf (versus no leaf). - IndexSegmentBuilder - verify handles case with empty root leaf. - IndexSegmentPlan - unit test for empty tree. - IndexSegmentBuilder - unit test for empty tree and compare with ground truth empty tree. - Update javadoc in relevant locations. - Remove added trace from TestBuildTask2
        Hide
        bryanthompson bryanthompson added a comment -

        The IndexSegment has been modified per the notes above to generate an empty root leaf. This appears to fix the test, which was failing quite consistently.

        Committed revision 4596.

        Show
        bryanthompson bryanthompson added a comment - The IndexSegment has been modified per the notes above to generate an empty root leaf. This appears to fix the test, which was failing quite consistently. Committed revision 4596.

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: