Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Affects Version/s: BIGDATA_RELEASE_1_4_0
    • Fix Version/s: None
    • Component/s: blueprints

      Description

      Overhaul the connection management in BigdataGraph and BigdataGraphEmbedded to properly use and free bigdata resources.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        The getEdges() and getVertices() methods of the blueprints API return an Iterable. This interface has a single method returning a java.lang.Iterator. We could return a CloseableIterator that implements the Iterable interface. Invoking the close() method on that returned object would correctly close the connection and prevent resource leaks (connection objects that are not closed until collected by GC). However, existing code would not be aware of the CloseableIterator interface and would not invoke close(). Further, code that uses the following pattern would not close the iterator.

        for(Vertex a : graph.getVertices()) {...}
        

        This issue can be resolved by modifying the getEdges() and getVertices() methods to fully materialize the results in memory, then close the connection, and then finally return an Iterable to the caller over that materialized in-memory collection. Since this pattern is not very scalable and imposes a high latency to the first result if the cardinality of the edge or vertex list is high, I suggest that we also introduce bigdata specific methods into the interface that return ICloseableIterator. This methods would provide a streaming pattern. The caller would be responsible for invoking close() on the returned ICloseableIterator to avoid leaking connection resources.

        Show
        bryanthompson bryanthompson added a comment - The getEdges() and getVertices() methods of the blueprints API return an Iterable. This interface has a single method returning a java.lang.Iterator. We could return a CloseableIterator that implements the Iterable interface. Invoking the close() method on that returned object would correctly close the connection and prevent resource leaks (connection objects that are not closed until collected by GC). However, existing code would not be aware of the CloseableIterator interface and would not invoke close(). Further, code that uses the following pattern would not close the iterator. for(Vertex a : graph.getVertices()) {...} This issue can be resolved by modifying the getEdges() and getVertices() methods to fully materialize the results in memory, then close the connection, and then finally return an Iterable to the caller over that materialized in-memory collection. Since this pattern is not very scalable and imposes a high latency to the first result if the cardinality of the edge or vertex list is high, I suggest that we also introduce bigdata specific methods into the interface that return ICloseableIterator. This methods would provide a streaming pattern. The caller would be responsible for invoking close() on the returned ICloseableIterator to avoid leaking connection resources.
        Hide
        bryanthompson bryanthompson added a comment -

        These CI failures date back to yesterday's merge.

         com.bigdata.blueprints.TestBigdataGraphClient.testVertexTestSuite	0.11 sec	5
         com.bigdata.blueprints.TestBigdataGraphClient.testGraphQueryTestSuite	0.2 sec	5
         com.bigdata.blueprints.TestBigdataGraphClient.testEdgeSuite	2.3 sec	5
         com.bigdata.blueprints.TestBigdataGraphClient.testVertexQueryTestSuite	0.46 sec	5
         com.bigdata.blueprints.TestBigdataGraphClient.testGraphSuite	0.3 sec	5
         com.bigdata.blueprints.TestBigdataGraphClient.testVertexTestSuite	0.11 sec	5
         com.bigdata.blueprints.TestBigdataGraphClient.testGraphQueryTestSuite	0.2 sec	5
         com.bigdata.blueprints.TestBigdataGraphClient.testEdgeSuite	2.3 sec	5
         com.bigdata.blueprints.TestBigdataGraphClient.testVertexQueryTestSuite	0.46 sec	5
         com.bigdata.blueprints.TestBigdataGraphClient.testGraphSuite
        
        Show
        bryanthompson bryanthompson added a comment - These CI failures date back to yesterday's merge. com.bigdata.blueprints.TestBigdataGraphClient.testVertexTestSuite 0.11 sec 5 com.bigdata.blueprints.TestBigdataGraphClient.testGraphQueryTestSuite 0.2 sec 5 com.bigdata.blueprints.TestBigdataGraphClient.testEdgeSuite 2.3 sec 5 com.bigdata.blueprints.TestBigdataGraphClient.testVertexQueryTestSuite 0.46 sec 5 com.bigdata.blueprints.TestBigdataGraphClient.testGraphSuite 0.3 sec 5 com.bigdata.blueprints.TestBigdataGraphClient.testVertexTestSuite 0.11 sec 5 com.bigdata.blueprints.TestBigdataGraphClient.testGraphQueryTestSuite 0.2 sec 5 com.bigdata.blueprints.TestBigdataGraphClient.testEdgeSuite 2.3 sec 5 com.bigdata.blueprints.TestBigdataGraphClient.testVertexQueryTestSuite 0.46 sec 5 com.bigdata.blueprints.TestBigdataGraphClient.testGraphSuite

          People

          • Assignee:
            mikepersonick mikepersonick
            Reporter:
            mikepersonick mikepersonick
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: