Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-762

Accumulating HALog files cause latency for HA commit.

    XMLWordPrintable

    Details

      Description

      We are observing an increasing delay over time as the #of HALog files grows in the file system. Right now it is scanning an increasing number of small files in the HALog directory, which is causing performance to degrade.

      What we need to do is track the HALog files in a recursive directory structure (to avoid limits on the size of a directory) and use an in-memory index to determine in a low-latency fashion which HALog files (if any) may be removed at each commit point.

      Another optimization would be to notice whether any of the key conditions for allowing a release of HALogs or snapshots has changed since the last commit (e.g., have we taken a new snapshot or has a snapshot been expired).

        Attachments

          Activity

            People

            Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: