Details

      Description

      Expose online backups for the Journal (vs HAJournal).

      I chased the code for the basic backup mechanism and this could be exposed without HA easily enough.

      Include an option to not gzip the file (easy) so the snapshots maybe immediately re-deployed if so desired.

      Note: Exposing the incremental tx log as well would be significantly trickier, but could be done (I've looked at this before
      - the snapshot manager and HALogNexus would both have to be pushed down to the Journal so this touches a lot of code). Without the TX logs the application is responsible for replaying any lost updates in the case of a failover.

        Activity

        beebs Brad Bebee created issue -
        Hide
        bryanthompson bryanthompson added a comment -

        Includes initial test suite which runs correctly for com.bigdata.journal.

        We still need a test for a concurrent writer while we are taking a snapshot.

        I touched the proxy test suite a bit for the journal package. It was failing to call super.setUp() and super.tearDown(). I also had to change WORMStrategy.getProperties() to not set the CREATE_TEMP_FILE property once I made this change (the TestRWJournal suite already had this change).

        Snapshot was not working (and was not tested) for WORMStrategy.

        AbstractJournal.snapshotAllocationData() was calling ByteBuffer.array() even when the array was not visible. I have replaced this with BytesUtil.toArray(ByteBuffer).

        writeOnStream() for the backing store now allows a null quorum (for non-HA invocations).

        Show
        bryanthompson bryanthompson added a comment - Includes initial test suite which runs correctly for com.bigdata.journal. We still need a test for a concurrent writer while we are taking a snapshot. I touched the proxy test suite a bit for the journal package. It was failing to call super.setUp() and super.tearDown(). I also had to change WORMStrategy.getProperties() to not set the CREATE_TEMP_FILE property once I made this change (the TestRWJournal suite already had this change). Snapshot was not working (and was not tested) for WORMStrategy. AbstractJournal.snapshotAllocationData() was calling ByteBuffer.array() even when the array was not visible. I have replaced this with BytesUtil.toArray(ByteBuffer). writeOnStream() for the backing store now allows a null quorum (for non-HA invocations).
        Hide
        bryanthompson bryanthompson added a comment -

        I've added a test for a concurrent writer (it passes).

        I've issued a pull request for this feature. It is just waiting on a good CI run for the branch.

        See https://github.com/SYSTAP/bigdata/pull/43

        Show
        bryanthompson bryanthompson added a comment - I've added a test for a concurrent writer (it passes). I've issued a pull request for this feature. It is just waiting on a good CI run for the branch. See https://github.com/SYSTAP/bigdata/pull/43
        Hide
        bryanthompson bryanthompson added a comment -

        I have updated the feature matrix to indicate support for online backups.

        Show
        bryanthompson bryanthompson added a comment - I have updated the feature matrix to indicate support for online backups.
        Hide
        bryanthompson bryanthompson added a comment -

        CI is clean.

        Merged pull request.

        Closed ticket.

        Show
        bryanthompson bryanthompson added a comment - CI is clean. Merged pull request. Closed ticket.
        bryanthompson bryanthompson made changes -
        Field Original Value New Value
        Fix Version/s BLAZEGRAPH_RELEASE_1_5_2 [ 10164 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v2 [ 13041 ] Trac Import v3 [ 14475 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v3 [ 14475 ] Trac Import v4 [ 15804 ]
        beebs Brad Bebee made changes -
        Status Closed - Won't Fix [ 6 ] Open [ 1 ]
        beebs Brad Bebee made changes -
        Status Open [ 1 ] Accepted [ 10101 ]
        beebs Brad Bebee made changes -
        Status Accepted [ 10101 ] In Progress [ 3 ]
        Priority Highest [ 1 ]
        beebs Brad Bebee made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        beebs Brad Bebee made changes -
        Status Resolved [ 5 ] In Review [ 10100 ]
        beebs Brad Bebee made changes -
        Status In Review [ 10100 ] Done [ 10000 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v4 [ 15804 ] Trac Import v5 [ 16046 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v5 [ 16046 ] Trac Import v6 [ 18349 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v6 [ 18349 ] Trac Import v7 [ 19748 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v7 [ 19748 ] Trac Import v8 [ 21371 ]

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: