Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-988

Settable timeout on SPARQL Query (REST API)

    Details

      Description

      Allow the http client to specify a timeout on query. We support this internally, but the client is not able to specify this on a per-query basis through the REST API. The openrdf API has a mechanism for this as well. We do already support placing a timeout on the entire webapp through web.xml, but that does not support use cases where only queries from some end points should have a timeout applied.

      I think that the timeout should be communicated through either an http header or perhaps the Request-URI, but it should be something that can be imposed by an HTTP proxy. This will make it possible to define a proxy that automatically imposes a timeout for HTTP connections arriving on some ports or URLs while not imposing that timeout on other connections. This could support a variety of use cases, including long running administrative queries, forcing a timeout on some servers in an HA replication cluster in order to bound their load, etc.

      Integrate support for this into the BigdataSailRemoteRepository through TupleQuery, GraphQuery, and BooleanQuery.setMaxQueryTime(). Be careful about the units (milliseconds vs ...).

      This needs to be handled in the QueryServlet, which needs to look at the HTTP header and then set the timeout on the TupleQuery, GraphQuery, or BooleanQuery. The internal APIs will then set it on the underlying IRunningQuery.

        Issue Links

          Activity

          Hide
          bryanthompson bryanthompson added a comment -

          Note: org.openrdf.query.Query.setMaxQuery() accepts an int32 value which is seconds.

          The bigdata interfaces allow the control of the query timeout in milliseconds.

          We should expose the means to set a sub-second timeout on the BigdataSailRemoteRepository. E.g., setMaxQueryMillis(). The openrdf setMaxQuery() should just turn the seconds into milliseconds and then invoke setMaxQueryMillis(). The HTTP header should specify the millis (as a long value).

          Here is my proposed change to BigdataRDFContext.newQuery()

                  private AbstractQuery newQuery(final BigdataSailRepositoryConnection cxn) {
          
                      long queryTimeoutMillis = getConfig().queryTimeout;
          
                      {
                          final String s = req.getHeader("BIGDATA_TIMEOUT_MILLIS");
                          if (s != null) {
                              long tmp = Long.valueOf(s);
                              if (queryTimeoutMillis == 0/* noLimit */
                                      || tmp < queryTimeoutMillis/* shorterLimit */) {
                                  queryTimeoutMillis = tmp;
                              }
                          }
                      }
          
                      if (queryTimeoutMillis > 0) {
          
                          final QueryRoot originalQuery = astContainer.getOriginalAST();
          
                          originalQuery.setTimeout(queryTimeoutMillis);
          
                      }
          

          The name of the HTTP header should be extracted into a constant.

          The value of the HTTP header is decimal OCTETs. I think that this means 8-bit ASCII digits. That should be verified.

          Show
          bryanthompson bryanthompson added a comment - Note: org.openrdf.query.Query.setMaxQuery() accepts an int32 value which is seconds . The bigdata interfaces allow the control of the query timeout in milliseconds . We should expose the means to set a sub-second timeout on the BigdataSailRemoteRepository. E.g., setMaxQueryMillis(). The openrdf setMaxQuery() should just turn the seconds into milliseconds and then invoke setMaxQueryMillis(). The HTTP header should specify the millis (as a long value). Here is my proposed change to BigdataRDFContext.newQuery() private AbstractQuery newQuery(final BigdataSailRepositoryConnection cxn) { long queryTimeoutMillis = getConfig().queryTimeout; { final String s = req.getHeader("BIGDATA_TIMEOUT_MILLIS"); if (s != null) { long tmp = Long.valueOf(s); if (queryTimeoutMillis == 0/* noLimit */ || tmp < queryTimeoutMillis/* shorterLimit */) { queryTimeoutMillis = tmp; } } } if (queryTimeoutMillis > 0) { final QueryRoot originalQuery = astContainer.getOriginalAST(); originalQuery.setTimeout(queryTimeoutMillis); } The name of the HTTP header should be extracted into a constant. The value of the HTTP header is decimal OCTETs. I think that this means 8-bit ASCII digits. That should be verified.
          Hide
          bryanthompson bryanthompson added a comment -

          Added support for setMaxQuery(). This does not yet allow people to (easily) set query timeouts of less than one second (which is the API granularity for openrdf). However, the timeouts ARE communicated in milliseconds using an HTTP header.

          Committed revision r8336.

          Show
          bryanthompson bryanthompson added a comment - Added support for setMaxQuery(). This does not yet allow people to (easily) set query timeouts of less than one second (which is the API granularity for openrdf). However, the timeouts ARE communicated in milliseconds using an HTTP header. Committed revision r8336.
          Hide
          bryanthompson bryanthompson added a comment -

          renamed the http header to X-BIGDATA-MAX-QUERY-MILLIS.

          Committed revision r8337.

          Show
          bryanthompson bryanthompson added a comment - renamed the http header to X-BIGDATA-MAX-QUERY-MILLIS. Committed revision r8337.

            People

            • Assignee:
              bryanthompson bryanthompson
              Reporter:
              bryanthompson bryanthompson
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: