Affects Version/s: None
Fix Version/s: BLAZEGRAPH_2_2_0
Run our usual baseline benchmarks once 2.2.0 is ready to go.
Note: Please conduct a parameterized test with maxParallel = 5 (default today), 1, 10, 15, and 20 on our benchmarks. It may be useful to test on a subset of the benchmarks to identify a suitable maxParallel value.
The only benchmark that uses SPARQL UPDATE is BSBM EXPLORE + UPDATE.
Single query at a time benchmarks could potentially benefit from increased inter-query parallelism either in joins or in materialization (for high cardinality queries that have a high solution production rate).
Note: Please verify that the maxParallel hint is respected in SPARQL UPDATE. If it is not, then see
Expose maxParallelDefault for global override
Note: Using the native heap for intermediate solutions (
BLZG-1963) or for final solutions (as a native buffer ring buffer) will reduce heap pressure and could easily change the "good" default for maxParallel, as could the specifics of the hardware platform. The "good" value for chunkSize could also change. Also note that for at least some queries, the system throughput on a single query appears to be relatively robust to changes in maxParallel. Maybe what we should look for are conditions under which we clearly should increase parallelism in materialization or in JOINS. maxParallel could also be adaptive in terms of the total heap pressure and total number of concurrent operators running on the database. A heavy query (either joins that can benefit or materialization that could benefit) could be allowed to spend more of the "maxParallel" global budget but there would be a global budget (effectively the maximum number of threads). Also note that we can look for operators that are bottlenecks and then dynamically allocate more resources to queries for those operators. Bottlenecks can be observed by looking at the input queues for operators. The producer will block if the consumer queue is full.