Type: New Feature
Affects Version/s: BIGDATA_RELEASE_1_3_3
Fix Version/s: None
Component/s: Query Plan Generator
The QUADS mode default graph access path does not support the analytic query mode. The QUADS mode default graph access path is used when the KB is in QUADS mode and the query must read on the RDF merge of the named graphs (which is what bigdata considers to be the default graph).
This is currently achieved by stripping off the context position of the SPOC tuple and then imposing a DISTINCT SPO filter on the underlying quads-mode index access path. That DISTINCT SPO filter is implemented by com.bigdata.bop.ap.filter.DistinctFilter and is backed by a JVM HashMap.
A native (aka analytic) mode DISTINCT SPO filter exists, but it is not used when pipeline joins are in use because it lacks access to the IRunningQuery during query plan generation and hence can not use the IMemoryManager for the IRunningQuery as the backing store for the DISTINCT SPO filter.
The choice of the DISTINCT SPO filter is made by
One possible approach would be to add a method to BOpFilterBase to initialize the filter with the IRunningQuery object when the query is submitted to the QueryEngine for evaluation. This would provide the access required to use the same IMemoryManager as the IRunningQuery.
The NativeDistinctFilter has some notes that it can not be used with the PipelineJoin operator because it would allocate one IMemoryManager per as-bound predicate evaluation which implies that the scope of the DISTINCT semantics might not be correctly imposed across all ISPO objects flowing into that filter from the associated access path. I will analyze this under a debugger and then see whether we can repurpose the NativeDistinctFilter by associating it at runtime with the IRunningQuery.
Ticket BLZG-888 is closely related.
@see http://trac.bigdata.com/ticket/807 (native distinct in quad mode (insert/delete))