Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-1960

Propagate maxParallel hint to non-single threaded operators

    Details

      Description

      Currently, the maxParallel query hint is only propagated to join nodes (see PipelineMaxParallelHint). We want this hint to apply to all operators that are not single-threaded by design (i.e., not an instance of the marker interface ISingleThreadedOp).

      This requires changes to (a) the query hint and (b) a review of how we propagate the query hint, we need some test cases (similiar to TestQueryHints by idea) to verify that the query hint comes through for the different operators. In particular, we also want the query hint to apply to the chunked materialization operator following the final project (cf. BLZG-1958).

        Activity

        Hide
        michaelschmidt michaelschmidt added a comment -

        Initial implementation in PR https://github.com/SYSTAP/bigdata/pull/426 (please do not yet merge). Need to write a couple of couple of test cases, but we could give it a try (manual tests showed that this fixes the issue, and it should be sufficient for our WatDiv tests).

        Show
        michaelschmidt michaelschmidt added a comment - Initial implementation in PR https://github.com/SYSTAP/bigdata/pull/426 (please do not yet merge). Need to write a couple of couple of test cases, but we could give it a try (manual tests showed that this fixes the issue, and it should be sufficient for our WatDiv tests).
        Hide
        michaelschmidt michaelschmidt added a comment -

        I added a variety of tests, looks like the behavior is exactly what's expected, the hint was propagated successfully in all cases and not propagated to the single threaded operators. Will merge down once CI is green, we can then redeploy master on the workstation and measure the impact on the WatDiv query.

        Show
        michaelschmidt michaelschmidt added a comment - I added a variety of tests, looks like the behavior is exactly what's expected, the hint was propagated successfully in all cases and not propagated to the single threaded operators. Will merge down once CI is green, we can then redeploy master on the workstation and measure the impact on the WatDiv query.
        Hide
        bryanthompson bryanthompson added a comment -

        Excellent.

        Show
        bryanthompson bryanthompson added a comment - Excellent.
        Hide
        michaelschmidt michaelschmidt added a comment -

        CI is green, merged down to master.

        Alexandre Riazanov please deploy latest master on the workstation and give it a try

        Show
        michaelschmidt michaelschmidt added a comment - CI is green, merged down to master. Alexandre Riazanov please deploy latest master on the workstation and give it a try
        Show
        alexr Alexandre Riazanov added a comment - The results for WatDiv C3 are here https://docs.google.com/spreadsheets/d/1QYkDKqcCXgk0PuYS2ZoB_T6fYHwrpsoZIoxdU35fZpk/edit#gid=2118609979 A bit more shape now. Michael Schmidt bryanthompson

          People

          • Assignee:
            michaelschmidt michaelschmidt
            Reporter:
            michaelschmidt michaelschmidt
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: