Details

    • Type: New Feature
    • Status: Closed - Won't Fix
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Bigdata Federation
    • Labels:
      None

      Description

      Bigdata has extensive performance counters which are self-reported by the various services using NanoHTTP and also aggregated to the load balancer. In turn, the load balancer provides a NanoHTTP interface which may be used to create interesting views, and in some cases, visualizations, of the aggregated performance counters.

      Historically, we have used Excel worksheets with tabs dedicated to the different performance counters (CPU Utilization, IO Wait, toldTriplesLoadedPerSec, etc). Each tab has one or more http queries which draw down the relevant performance counters, populate a table in Excel, and then update a graph linked to that table. Those worksheets are in src/resources/analysis.

      This works but is platform specific. In some cases it is also specific to the name of a monitored job or the namespace of a triple store. A more flexible solution could be easily assembled using a web application which issues these queries and then paints the page in the browser. A little logic in the web application would make it possible to monitor specific jobs or namespaces in the database.

      Create a web application per the description above to provide more flexible performance monitoring.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        Martyn, I'd like to bring this issue to your attention. The purpose is to expose interesting views of the performance counters. This is related to updating the performance counter http end point to use the Servlet API, but goes further to provide a UI. It would be nice to do this with the easy start issue so we have a place where people can go in the UI to observe metrics on the database. Bryan

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/165

        Show
        bryanthompson bryanthompson added a comment - Martyn, I'd like to bring this issue to your attention. The purpose is to expose interesting views of the performance counters. This is related to updating the performance counter http end point to use the Servlet API, but goes further to provide a UI. It would be nice to do this with the easy start issue so we have a place where people can go in the UI to observe metrics on the database. Bryan [1] https://sourceforge.net/apps/trac/bigdata/ticket/165
        Hide
        bryanthompson bryanthompson added a comment -

        This is being closed in favor of the ganglia integration [1]. Ganglia already provides a monitoring and graphing ecosystem. Bigdata now has a native ganglia integration which allows bigdata services to operate as a ganglia peer.

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/441 (Ganglia Integration)

        Show
        bryanthompson bryanthompson added a comment - This is being closed in favor of the ganglia integration [1] . Ganglia already provides a monitoring and graphing ecosystem. Bigdata now has a native ganglia integration which allows bigdata services to operate as a ganglia peer. [1] https://sourceforge.net/apps/trac/bigdata/ticket/441 (Ganglia Integration)

          People

          • Assignee:
            Unassigned
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: