Type: New Feature
Affects Version/s: BIGDATA_RELEASE_1_3_1
Fix Version/s: None
The property path operation (ArbitraryLengthPathOp) can become CPU bound and non-responsive. It can also drive the heap heavily leading to high GC overhead. This occurs when the property path is fully materialized in cache for queries such as
We need to review the code for sources of unnecessary heap pressure, refactor the code to derive two versions of the operator (one for the native heap and one for the JVM heap), and look at ways to relieve the CPU pressure, e.g., by yielding occasionally.
Two other comments:
- There may be cases (such as the above) where the GASService could provide an alternative execution strategy (however note that the GASService does not support the native heap at this time)
- We are re-planning the sub-query at each invocation. It would be nice if we could avoid that overhead.
Note: It is possible that the root cause is:
BLZG-1061Property path operator should output solutions incrementally