Details

    • Type: Task
    • Status: In Progress
    • Resolution: Unresolved
    • Affects Version/s: BIGDATA_RELEASE_1_1_0
    • Fix Version/s: None
    • Component/s: CI, Project Management
    • Labels:
      None

      Description

      There are a lot of temporary journals showing up in CI. Track down tests which are creating these journals but failing to destroy them.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        I have disabled the embedded federation test suite for the sail again. Per the comment above, it is leaking threads which the JVM is unable to recover without a full GC. In fact, this is probably the root cause for [1]. I only identified this interaction by forcing a full GC on the JVM using an external utility (the profiler). It is not enough to request a GC from within the JVM as it is free to ignore that request.

        com.bigdata.rdf.sail.TestAll:

        //        suite.addTest(TestBigdataSailEmbeddedFederationWithQuads.suite());
        

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/196 (Journal leaks memory)

        Committed revision r6200.

        Show
        bryanthompson bryanthompson added a comment - I have disabled the embedded federation test suite for the sail again. Per the comment above, it is leaking threads which the JVM is unable to recover without a full GC. In fact, this is probably the root cause for [1] . I only identified this interaction by forcing a full GC on the JVM using an external utility (the profiler). It is not enough to request a GC from within the JVM as it is free to ignore that request. com.bigdata.rdf.sail.TestAll: // suite.addTest(TestBigdataSailEmbeddedFederationWithQuads.suite()); [1] https://sourceforge.net/apps/trac/bigdata/ticket/196 (Journal leaks memory) Committed revision r6200.
        Hide
        bryanthompson bryanthompson added a comment -

        Per [1], it appears that the root cause may be the failure of the JVM to force a full GC when there are active threads which could be collected. This needs to be investigated further. If so, it is a JVM bug/feature.

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/524 (Temporary Journals in CI)

        Show
        bryanthompson bryanthompson added a comment - Per [1] , it appears that the root cause may be the failure of the JVM to force a full GC when there are active threads which could be collected. This needs to be investigated further. If so, it is a JVM bug/feature. [1] https://sourceforge.net/apps/trac/bigdata/ticket/524 (Temporary Journals in CI)
        Hide
        bryanthompson bryanthompson added a comment -

        The above comment was for [1].

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/196 (Journal leaks memory)

        Show
        bryanthompson bryanthompson added a comment - The above comment was for [1] . [1] https://sourceforge.net/apps/trac/bigdata/ticket/196 (Journal leaks memory)
        Hide
        bryanthompson bryanthompson added a comment -

        Restored test suites per [1,2]. Now that the journal memory leak has been addressed, I want to see if we can enable the embedded federation test suite in CI.

        com.bigdata.rdf.sail.TestAll:
           suite.addTest(TestBigdataSailEmbeddedFederationWithQuads.suite());
        

        Committed revision r6824. (Dev branch)

        [1] http://sourceforge.net/apps/trac/bigdata/ticket/196 (Journal leaks memory).
        [2] http://sourceforge.net/apps/trac/bigdata/ticket/523 (Temporary journals in CI : blocking embedded federation CI)

        Show
        bryanthompson bryanthompson added a comment - Restored test suites per [1,2] . Now that the journal memory leak has been addressed, I want to see if we can enable the embedded federation test suite in CI. com.bigdata.rdf.sail.TestAll: suite.addTest(TestBigdataSailEmbeddedFederationWithQuads.suite()); Committed revision r6824. (Dev branch) [1] http://sourceforge.net/apps/trac/bigdata/ticket/196 (Journal leaks memory). [2] http://sourceforge.net/apps/trac/bigdata/ticket/523 (Temporary journals in CI : blocking embedded federation CI)
        Hide
        bryanthompson bryanthompson added a comment -

        Re-disabling in CI. It looks like the embedded federation is still leaking resources. It gets very slow during the embedded federation test suite run and I observe a LOT of threads when requesting a thread dump.

        com.bigdata.rdf.sail.TestAll:
           suite.addTest(TestBigdataSailEmbeddedFederationWithQuads.suite());
        

        Committed revision r6828.

        Show
        bryanthompson bryanthompson added a comment - Re-disabling in CI. It looks like the embedded federation is still leaking resources. It gets very slow during the embedded federation test suite run and I observe a LOT of threads when requesting a thread dump. com.bigdata.rdf.sail.TestAll: suite.addTest(TestBigdataSailEmbeddedFederationWithQuads.suite()); Committed revision r6828.

          People

          • Assignee:
            martyncutcher martyncutcher
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated: