Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-829

Offsite replication for HA replication cluster

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Open
    • Resolution: Unresolved
    • Affects Version/s: BIGDATA_RELEASE_1_2_3
    • Fix Version/s: None
    • Component/s: HAJournalServer
    • Labels:
      None

      Description

      Support off-site replication with an active/inactive model.

      The best approach to offsite replication is to replicate the HALog files. Each HALog file contains the write set for a single commit point. These files are dense and the writes on them are sequential. The data are compressed on the disk and on the wire. (In contrast, the writes on the backing Journal are random IOs and could put significantly more pressure on a block level file system replication solution.)

      There are two alternatives for replication of the HALog files.

      1. Monitor the file system and replicate files from the HALog directory structure to an ingest directory on the failover cluster.

      2. Monitor the write replication pipeline and directly replicate the writes over a secure tunnel to an ingest directory on the failover cluster.

      The failover cluster needs to apply the HALog files as they arrive, which is why it makes sense to have an "ingest" or "incoming" directory. The files found in that directory can be applied by the "leader" on the failover site and replicated across the replication cluster on the failover site.

      A globally atomic decision needs to be made concerning which site is active and which site is read-only.

      This solution could support read on the inactive cluster.

      This solution could support multiple sites (more than two).

      New code would have to be written for:


      - Offsite replication of the HALog files;
      - Replay of the replicated HALog files on the failover site(s); and
      - Test suites.

        Attachments

          Activity

            People

            Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: