Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-339

Optimize mapping of binding sets across shards

    Details

      Description

      A basic algorithm has been implemented to map the binding sets across shards. This is essentially a nested index subquery using range iterator against the remote metadata service.

      A variety of special cases exist where we can do substantially better than this basic algorithm, including when the target predicate will not get inherit any bindings from the intermediate results (this is true for the first join in a query), when the query optimizer knows that target predicate will be fully bound (this is true for many query plans
      - they tend to wind up doing point tests after several joins), etc.

      A number of such variations have been identified. They should be implemented together with unit tests which focus on their different edge conditions and condition logic to select the best approach based on a mixture of annotations in the query plan and metadata about the target scale-out index.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        MapBindingSetsOverShardsBuffer had a broken assumption which I have just fixed and will commit as soon as I have a coherent change set. It was written under the assumption that the IKeyOrder of the next predicate against which a join would be performed (or on which data would be written) is known in advance. In fact, this is not true. The IKeyOrder must be selected dynamically based on the actual bindings which will be imposed on the predicate when we read (or write) on it. I have not reviewed the various optimizations which I had noted earlier in the light of this change in assumptions. However, MapBindingSetsOverShardsBuffer is basically a custom JOIN against the metadata service and those optimizations are basically special cases for that JOIN.

        Show
        bryanthompson bryanthompson added a comment - MapBindingSetsOverShardsBuffer had a broken assumption which I have just fixed and will commit as soon as I have a coherent change set. It was written under the assumption that the IKeyOrder of the next predicate against which a join would be performed (or on which data would be written) is known in advance. In fact, this is not true. The IKeyOrder must be selected dynamically based on the actual bindings which will be imposed on the predicate when we read (or write) on it. I have not reviewed the various optimizations which I had noted earlier in the light of this change in assumptions. However, MapBindingSetsOverShardsBuffer is basically a custom JOIN against the metadata service and those optimizations are basically special cases for that JOIN.
        Hide
        bryanthompson bryanthompson added a comment -

        Closed. Not relevant to the new architecture.

        Show
        bryanthompson bryanthompson added a comment - Closed. Not relevant to the new architecture.

          People

          • Assignee:
            Unassigned
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: