Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-744

Compress write cache blocks for replication and in HALogs

    Details

      Description

      It would be advantageous to compress the write cache blocks before they are replicated and before they are written to the HALog files. The leader would do the compression in the WriteCacheService.WriteTask.call() thread, so this would not add latency to writes. The receiver would immediately replicate the buffer. A receiver that only needs to log the buffer would log the compressed buffer. If the receiver needs to write the buffer onto the backing store, then it would decompress the buffer into a 2nd buffer
      - that 2nd buffer could be allocated and held by the HAJournalServer in its life cycle.

      We need to add fields for the compressionScheme (None, Zip, etc) and the wireByteLength to the IHAWriteMessage interface. For non-compressed data, the wireByteLength and the data byte length would be the same. The compressionScheme should default to None and the wireByteLength should default to the actual byte length for backwards compatibility.

      The HAWriteMessages are written onto the HALog. The messages themselves would not be compressed. However, the replicated write cache blocks WOULD be compressed. This should present a substantial space savings for the HALog files.

        Activity

          People

          • Assignee:
            martyncutcher martyncutcher
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: