Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-472

Add an "EXPLAIN" option to the NanoSparqlServer

    Details

      Description

      Add an "EXPLAIN" option to the NanoSparqlServer which returns an (X)HTML page containing the query plan, followed by detailed performance counters for the evaluation of the query.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        This information is already available from a variety of sources.

        The detailed query evaluation statistics:

        log4j.logger.com.bigdata.engine.bop.QueryLog=INFO
        

        The details on the Sesame operator tree and on the generated bigdata operator plan.

        log4j.logger.com.bigdata.rdf.sail.BigdataEvaluationStrategyImpl3=INFO
        

        The details on the translation from the Sesame Operator Tree:

        log4j.logger.com.bigdata.rdf.sail.sop.SOp2BOpUtility=ALL
        

        The details on the generation of the bigdata operator plan.

        log4j.logger.com.bigdata.rdf.sail.Rule2BOpUtility=INFO
        

        For this issue, it should be synthethized and prettied up in a single user-accessible (X)HTML page.

        Show
        bryanthompson bryanthompson added a comment - This information is already available from a variety of sources. The detailed query evaluation statistics: log4j.logger.com.bigdata.engine.bop.QueryLog=INFO The details on the Sesame operator tree and on the generated bigdata operator plan. log4j.logger.com.bigdata.rdf.sail.BigdataEvaluationStrategyImpl3=INFO The details on the translation from the Sesame Operator Tree: log4j.logger.com.bigdata.rdf.sail.sop.SOp2BOpUtility=ALL The details on the generation of the bigdata operator plan. log4j.logger.com.bigdata.rdf.sail.Rule2BOpUtility=INFO For this issue, it should be synthethized and prettied up in a single user-accessible (X)HTML page.
        Hide
        bryanthompson bryanthompson added a comment -

        Hacked in a rough XHTML table rendering of the query stats. There is now an URL query parameter "showRunningQueryStats" which may be used to enable this for the currently running queries. I need to debug and pretty up the HTML rendering and then I can focus on providing an "explain" option for a query based on the same code.

        Show
        bryanthompson bryanthompson added a comment - Hacked in a rough XHTML table rendering of the query stats. There is now an URL query parameter "showRunningQueryStats" which may be used to enable this for the currently running queries. I need to debug and pretty up the HTML rendering and then I can focus on providing an "explain" option for a query based on the same code.
        Hide
        bryanthompson bryanthompson added a comment -

        The "status" page with the URL query parameter "showRunningQueryStats" now displays a per-query table with a summary of the performance counters for each currently running query. If the URL query parameter "showRunningQueryDetailStats" is given instead then the page will have details for each bop in the query plan for each running query. The HTML of the status page passes validation.

        The next step is to integrate this into the NanoSparqlServer as a new REST request which runs the query and provides the detailed performance counters as part of an explanation instead of the query results.

        The status page should probably be refactored somewhat further to provide the table rather than the existing view when the table is requested. I'll return to this once I have the EXPLAIN request working.

        Committed revision r4713.

        Show
        bryanthompson bryanthompson added a comment - The "status" page with the URL query parameter "showRunningQueryStats" now displays a per-query table with a summary of the performance counters for each currently running query. If the URL query parameter "showRunningQueryDetailStats" is given instead then the page will have details for each bop in the query plan for each running query. The HTML of the status page passes validation. The next step is to integrate this into the NanoSparqlServer as a new REST request which runs the query and provides the detailed performance counters as part of an explanation instead of the query results. The status page should probably be refactored somewhat further to provide the table rather than the existing view when the table is requested. I'll return to this once I have the EXPLAIN request working. Committed revision r4713.
        Hide
        bryanthompson bryanthompson added a comment -

        For example, the following URL will display a summary table (assuming that the NanoSparqlServer is running at that port on the local host). You can view the contents of the table or select the table and copy-and-paste it into a worksheet.

        http://localhost:88/status?showRunningQueries&showRunningQueryStats
        
        Show
        bryanthompson bryanthompson added a comment - For example, the following URL will display a summary table (assuming that the NanoSparqlServer is running at that port on the local host). You can view the contents of the table or select the table and copy-and-paste it into a worksheet. http://localhost:88/status?showRunningQueries&showRunningQueryStats
        Hide
        bryanthompson bryanthompson added a comment -

        The syntax for requesting the query statistics has been simplified:

        A one line summary for each running query in order by descending elapsed run time.

        http://localhost:88/status?showQueries
        

        A one line summary followed by the per-operator query evaluation statistics for each running query in order by descending elapsed run time

        http://localhost:88/status?showQueries=details
        
        Show
        bryanthompson bryanthompson added a comment - The syntax for requesting the query statistics has been simplified: A one line summary for each running query in order by descending elapsed run time. http://localhost:88/status?showQueries A one line summary followed by the per-operator query evaluation statistics for each running query in order by descending elapsed run time http://localhost:88/status?showQueries=details
        Hide
        bryanthompson bryanthompson added a comment -

        Added an "explain" option to the GET method. Just add the following query parameter to the URL. The query will be executed normally, but an HTML document will be returned containing a summary of the query evaluation statistics.

        &explain=
        

        I will put in some other tweaks over time, such as control over whether or not the detail rows are shown as well, but the core feature is now implemented.

        I still need to document this feature on the NanoSparqlServer page.

        Committed revision r4717.

        Show
        bryanthompson bryanthompson added a comment - Added an "explain" option to the GET method. Just add the following query parameter to the URL. The query will be executed normally, but an HTML document will be returned containing a summary of the query evaluation statistics. &explain= I will put in some other tweaks over time, such as control over whether or not the detail rows are shown as well, but the core feature is now implemented. I still need to document this feature on the NanoSparqlServer page. Committed revision r4717.
        Hide
        bryanthompson bryanthompson added a comment -

        Added the original SPARQL and parsed SPARQL operator tree in their own sections of the page.

        Modified to stream the explanation and status page results rather than buffering in memory.

        Removed the XMLBuilder and HTMLBuilder constructor variants which did not require the caller to provide the target on which to write the data. The XMLBuilder#toString() assumed that the Writer was a StringWriter. That assumption was not compatible with the use of the class when building a servlet response.

        Documented the feature at [1] and [2].

        [1] https://sourceforge.net/apps/mediawiki/bigdata/index.php?title=NanoSparqlServer#QUERY
        [2] https://sourceforge.net/apps/mediawiki/bigdata/index.php?title=NanoSparqlServer#STATUS

        Committed revision r4726.

        This issue is closed.

        Show
        bryanthompson bryanthompson added a comment - Added the original SPARQL and parsed SPARQL operator tree in their own sections of the page. Modified to stream the explanation and status page results rather than buffering in memory. Removed the XMLBuilder and HTMLBuilder constructor variants which did not require the caller to provide the target on which to write the data. The XMLBuilder#toString() assumed that the Writer was a StringWriter. That assumption was not compatible with the use of the class when building a servlet response. Documented the feature at [1] and [2] . [1] https://sourceforge.net/apps/mediawiki/bigdata/index.php?title=NanoSparqlServer#QUERY [2] https://sourceforge.net/apps/mediawiki/bigdata/index.php?title=NanoSparqlServer#STATUS Committed revision r4726. This issue is closed.

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: