Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-372

Expose performance counters for the standalone database.

    Details

    • Type: New Feature
    • Status: Done
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Journal

      Description

      Expose performance counters for the standalone database. Add options to the Journal to enable: (a) performance counter collection every 60 seconds; and (b) an httpd service which may be used to query the last 60 minutes worth of performance counter data. Provide a version of the Excel worksheet which may be used to query the httpd service and demonstrates how to extract various performance counters of interest. When performance counter collection is enabled, log the counters against a log4j logger with a plain format and a FileAppender or RollingDailyFileAppender at the INFO level. Ensure that counters are reported from the Journal for the backing store, the indices, and the operating system (Unix or Windows). Gracefully handle cases where pidstat is not available / found since it is much less common to find sysstat installed and the database should start regardless. Likewise, a failure to start typeperf or vmstat should not prevent correct functioning of the database. Provide online documentation for the performance counters and a guide to diagnosing performance problems.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        I have taken this up as part of [1]. Note that OSX has a (very) different vmstat so a different hook will have to be written to extract performance counters from OSX. Since we have historically only reported performance counters for cluster installations, this was not an issue as long as nobody was deploying clusters on OSX servers. However, it is an issue now that performance counters will be reported for the standalone database as well.

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/206 (report top-N queries).
        [2] https://sourceforge.net/apps/trac/bigdata/ticket/225 (OSX performance counters).

        Show
        bryanthompson bryanthompson added a comment - I have taken this up as part of [1] . Note that OSX has a (very) different vmstat so a different hook will have to be written to extract performance counters from OSX. Since we have historically only reported performance counters for cluster installations, this was not an issue as long as nobody was deploying clusters on OSX servers. However, it is an issue now that performance counters will be reported for the standalone database as well. [1] https://sourceforge.net/apps/trac/bigdata/ticket/206 (report top-N queries). [2] https://sourceforge.net/apps/trac/bigdata/ticket/225 (OSX performance counters).
        Hide
        bryanthompson bryanthompson added a comment -

        Added https://sourceforge.net/apps/trac/bigdata/ticket/269 (Refactor performance counters using access interface) to document some of the changes being made in support of this ticket and additional testing which will have to be performed against scale-out in order to validate that performance counters are still correctly reported for scale-out.

        Show
        bryanthompson bryanthompson added a comment - Added https://sourceforge.net/apps/trac/bigdata/ticket/269 (Refactor performance counters using access interface) to document some of the changes being made in support of this ticket and additional testing which will have to be performed against scale-out in order to validate that performance counters are still correctly reported for scale-out.
        Hide
        bryanthompson bryanthompson added a comment -

        The performance counters are now collected and exposed. Martyn is taking over on [1] to migrate the code the to Servlet API with embedded server support using jetty.

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/200 (Raise NanoSparqlServer to front line class).

        Show
        bryanthompson bryanthompson added a comment - The performance counters are now collected and exposed. Martyn is taking over on [1] to migrate the code the to Servlet API with embedded server support using jetty. [1] https://sourceforge.net/apps/trac/bigdata/ticket/200 (Raise NanoSparqlServer to front line class).
        Hide
        bryanthompson bryanthompson added a comment -

        I've added performance counters for the indices to standalone.

        Show
        bryanthompson bryanthompson added a comment - I've added performance counters for the indices to standalone.
        Hide
        bryanthompson bryanthompson added a comment -

        The last change is in the LARGE_LITERALS_REFACTOR branch. It will come back to the QUADS_QUERY_BRANCH in short order. I'm closing out this issue.

        Show
        bryanthompson bryanthompson added a comment - The last change is in the LARGE_LITERALS_REFACTOR branch. It will come back to the QUADS_QUERY_BRANCH in short order. I'm closing out this issue.

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: