Details

      Description

      Expose online backups for the Journal (vs HAJournal).

      I chased the code for the basic backup mechanism and this could be exposed without HA easily enough.

      Include an option to not gzip the file (easy) so the snapshots maybe immediately re-deployed if so desired.

      Note: Exposing the incremental tx log as well would be significantly trickier, but could be done (I've looked at this before
      - the snapshot manager and HALogNexus would both have to be pushed down to the Journal so this touches a lot of code). Without the TX logs the application is responsible for replaying any lost updates in the case of a failover.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        Includes initial test suite which runs correctly for com.bigdata.journal.

        We still need a test for a concurrent writer while we are taking a snapshot.

        I touched the proxy test suite a bit for the journal package. It was failing to call super.setUp() and super.tearDown(). I also had to change WORMStrategy.getProperties() to not set the CREATE_TEMP_FILE property once I made this change (the TestRWJournal suite already had this change).

        Snapshot was not working (and was not tested) for WORMStrategy.

        AbstractJournal.snapshotAllocationData() was calling ByteBuffer.array() even when the array was not visible. I have replaced this with BytesUtil.toArray(ByteBuffer).

        writeOnStream() for the backing store now allows a null quorum (for non-HA invocations).

        Show
        bryanthompson bryanthompson added a comment - Includes initial test suite which runs correctly for com.bigdata.journal. We still need a test for a concurrent writer while we are taking a snapshot. I touched the proxy test suite a bit for the journal package. It was failing to call super.setUp() and super.tearDown(). I also had to change WORMStrategy.getProperties() to not set the CREATE_TEMP_FILE property once I made this change (the TestRWJournal suite already had this change). Snapshot was not working (and was not tested) for WORMStrategy. AbstractJournal.snapshotAllocationData() was calling ByteBuffer.array() even when the array was not visible. I have replaced this with BytesUtil.toArray(ByteBuffer). writeOnStream() for the backing store now allows a null quorum (for non-HA invocations).
        Hide
        bryanthompson bryanthompson added a comment -

        I've added a test for a concurrent writer (it passes).

        I've issued a pull request for this feature. It is just waiting on a good CI run for the branch.

        See https://github.com/SYSTAP/bigdata/pull/43

        Show
        bryanthompson bryanthompson added a comment - I've added a test for a concurrent writer (it passes). I've issued a pull request for this feature. It is just waiting on a good CI run for the branch. See https://github.com/SYSTAP/bigdata/pull/43
        Hide
        bryanthompson bryanthompson added a comment -

        I have updated the feature matrix to indicate support for online backups.

        Show
        bryanthompson bryanthompson added a comment - I have updated the feature matrix to indicate support for online backups.
        Hide
        bryanthompson bryanthompson added a comment -

        CI is clean.

        Merged pull request.

        Closed ticket.

        Show
        bryanthompson bryanthompson added a comment - CI is clean. Merged pull request. Closed ticket.

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: