Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-699

B+Tree branching factor and HTree addressBits are confused in their NodeSerializer implementations

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Affects Version/s: BIGDATA_RELEASE_1_2_1
    • Fix Version/s: None
    • Component/s: B+Tree

      Description

      The B+Tree and Htree have different versions of that class. At some point, changes were made to those NodeSerializer implementation such that the Btree was using the Htree parameter semantics (addressBits, which provides a branching factor of 2^addressBits) and the Htree was using the B+Tree semantics (branching factor is given directly). This results in oversizing of the initial capacity of the buffer used by the B+Tree NodeSerializer and, for larger branching factors, triggers the exception because the initial capacity value has wrapped and become negative.

      This exception was triggered using an override in the properties file for the SPO index:

      com.bigdata.namespace.kb.spo.SPO.com.bigdata.btree.BTree.branchingFactor=1081
      

      A sample stack trace is present below.

      Caused by: java.lang.IllegalArgumentException: initialCapacity
          at com.bigdata.io.ByteArrayBuffer.assertNonNegative(ByteArrayBuffer.java:133) ~[na:na]
          at com.bigdata.io.ByteArrayBuffer.<init>(ByteArrayBuffer.java:161) ~[na:na]
          at com.bigdata.io.DataOutputBuffer.<init>(DataOutputBuffer.java:73) ~[na:na]
          at com.bigdata.btree.NodeSerializer.allocWriteBuffer(NodeSerializer.java:308) ~[bigdata.osgi-1.2.1.6390.mike.jar:na]
          at com.bigdata.btree.NodeSerializer.<init>(NodeSerializer.java:272) ~[bigdata.osgi-1.2.1.6390.mike.jar:na]
          at com.bigdata.btree.AbstractBTree.<init>(AbstractBTree.java:951) ~[bigdata.osgi-1.2.1.6390.mike.jar:na]
          at com.bigdata.btree.BTree.<init>(BTree.java:331) ~[bigdata.osgi-1.2.1.6390.mike.jar:na]
          ... 82 common frames omitted
      

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        The fix is to com.bigdata.btree.NodeSerializer, which should have read:

                        this.initialBufferCapacity = DEFAULT_BUFFER_CAPACITY_PER_ENTRY
                                * branchingFactor;
        

        Note that the parameter name was changed from addressBits to branchingFactor as well.

        com.bigdata.htree.NodeSerializer was also fixed. The parallel code in that constructor now reads as follows. Again, the parameter name was changed to reflect the semantics of the HTree class.

                        // The effective branching factor.
                        final int branchingFactor = (1 << addressBits);
                        
                        this.initialBufferCapacity = DEFAULT_BUFFER_CAPACITY_PER_ENTRY
                                * branchingFactor;
        

        A unit test was added for the branching factor (1081) that triggered the exception.

        Javadoc related to the branching factors and write retention queue capacity in IndexMetadata was updated and the maximum write retention queue was reduced to a more reasonable value.

        The minimum and maximum legal values for HTree were extracted into static constants and informative error messages are now provided when the addressBits parameter for the HTree is out of bounds.

        Committed revision r6551.

        Show
        bryanthompson bryanthompson added a comment - The fix is to com.bigdata.btree.NodeSerializer, which should have read: this.initialBufferCapacity = DEFAULT_BUFFER_CAPACITY_PER_ENTRY * branchingFactor; Note that the parameter name was changed from addressBits to branchingFactor as well. com.bigdata.htree.NodeSerializer was also fixed. The parallel code in that constructor now reads as follows. Again, the parameter name was changed to reflect the semantics of the HTree class. // The effective branching factor. final int branchingFactor = (1 << addressBits); this.initialBufferCapacity = DEFAULT_BUFFER_CAPACITY_PER_ENTRY * branchingFactor; A unit test was added for the branching factor (1081) that triggered the exception. Javadoc related to the branching factors and write retention queue capacity in IndexMetadata was updated and the maximum write retention queue was reduced to a more reasonable value. The minimum and maximum legal values for HTree were extracted into static constants and informative error messages are now provided when the addressBits parameter for the HTree is out of bounds. Committed revision r6551.

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: