Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-497

Truth maintenace in SAIL throws OutOfMemoryException

    Details

    • Type: Bug
    • Status: In Progress
    • Resolution: Unresolved
    • Affects Version/s: BIGDATA_RELEASE_1_0_0
    • Fix Version/s: None
    • Component/s: Bigdata SAIL
    • Labels:

      Description

      Truth maintenance in the SAIL can throw an OutOfMemoryException. This occurs when the application uses a pattern where removes and adds are tightly interwoven. For example, a loop where the inner operation is:

      remove(s,p,_);
      add(s,p,o);
      

        Activity

        beebs Brad Bebee created issue -
        Hide
        bryanthompson bryanthompson added a comment -

        This tight interweaving of removes and adds described above is NOT efficient. The correct way to do this is to batch all removes together followed by batching all asserts together. This can be 1000s of times more efficient than applying the remove/assert one subject/predicate at a time.

        I have assigned this issue to MikeP, who is working on a unit test for the problem. Once that is available, I will take the issue back to work on a fix. The problem appears to be related to large branching factors and retention queues used with the RWStore. Truth maintenance is performance against a TemporaryTripleStore, which is based by a TemporaryRawStore which is a kind of WORM. It appears that the large branching factor is driving up the memory demand. However, there is probably a memory leak which is preventing the timely finalization of the TemporaryTripleStores associated with the assert/retract pattern.

        Another memory inefficiency is the buffer capacity for the statement buffer associated with truth maintenance. Normally a large buffer capacity is used for the SAIL (the default is 10k and people may configured values as high as 1M for higher throughput when loading data). When the same large buffer capacity is applied to truth maintenance, especially for such small deltas, there is significant memory waste in the StatementBuffer's internal data structures.

        Show
        bryanthompson bryanthompson added a comment - This tight interweaving of removes and adds described above is NOT efficient. The correct way to do this is to batch all removes together followed by batching all asserts together. This can be 1000s of times more efficient than applying the remove/assert one subject/predicate at a time. I have assigned this issue to MikeP, who is working on a unit test for the problem. Once that is available, I will take the issue back to work on a fix. The problem appears to be related to large branching factors and retention queues used with the RWStore. Truth maintenance is performance against a TemporaryTripleStore, which is based by a TemporaryRawStore which is a kind of WORM. It appears that the large branching factor is driving up the memory demand. However, there is probably a memory leak which is preventing the timely finalization of the TemporaryTripleStores associated with the assert/retract pattern. Another memory inefficiency is the buffer capacity for the statement buffer associated with truth maintenance. Normally a large buffer capacity is used for the SAIL (the default is 10k and people may configured values as high as 1M for higher throughput when loading data). When the same large buffer capacity is applied to truth maintenance, especially for such small deltas, there is significant memory waste in the StatementBuffer's internal data structures.
        Hide
        bryanthompson bryanthompson added a comment -

        Mike, please attach the unit test when you get the chance.

        Show
        bryanthompson bryanthompson added a comment - Mike, please attach the unit test when you get the chance.
        Hide
        mikepersonick mikepersonick added a comment -

        Unit test checked in
        - bigdata-sails/src/test/com/bigdata/rdf/sail/contrib/TestOOM.java

        1_0 maintenance branch and terms branch

        Show
        mikepersonick mikepersonick added a comment - Unit test checked in - bigdata-sails/src/test/com/bigdata/rdf/sail/contrib/TestOOM.java 1_0 maintenance branch and terms branch
        beebs Brad Bebee made changes -
        Field Original Value New Value
        Workflow Trac Import v2 [ 12333 ] Trac Import v3 [ 13470 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v3 [ 13470 ] Trac Import v4 [ 14799 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v4 [ 14799 ] Trac Import v5 [ 16188 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v5 [ 16188 ] Trac Import v6 [ 18512 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v6 [ 18512 ] Trac Import v7 [ 19932 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v7 [ 19932 ] Trac Import v8 [ 21621 ]

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated: