Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-864

Incremental or asynchronous purge of HALog files

    XMLWordPrintable

    Details

      Description

      A large latency was observed on the leader (bigdata15) and the first follower (bigdata16) following a restart of all services during the first commit. The services had approximately 150,000 HALog files each. Most of these HALog files had been rendered unnecessary by a recent snapshot. Therefore, they were being purged during the commit. This protocol should be modified to either purge the HALogs in an asynchronous thread (with an invariant monitor) or to purge them in blocks of a few hundred to a thousand (each directory contains 1000 HALog files) at a time.

      See https://sourceforge.net/apps/trac/bigdata/ticket/738#comment:39 for context.

        Attachments

          Activity

            People

            Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: