Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-96

Dynamically increase RTO sampling limit.

    Details

      Description

      Dynamically increase the sampling limit as necessary to address significant variability in cardinality estimates or underflow in cardinality estimates.

      Do not arbitrarily chose among paths spanning the same vertices when we have a cardinality underflow. Instead, deepen the samples until we have enough information to make a choice? This concept should be tested as it might lead to a lot more work and cost might reflect tuples read which do not join which could help break ties for paths with cardinality underflow.

      This implies that we need to retain the distribution of the estimated costs for each vertex and cutoff join (but not the sampled solution sets) so we can assess the variability in the estimates as we resample. Since the sample size is increasing with each sample taken, the estimates should rapidly converge. Depending on the nature of the estimate, we may be able to know that it will converge to a lower value (from an upper bound) or that it will converge to a higher value (from an underflow).

      If a path segment is a clear winner and leads into a cardinality underflow, then we could execute that join path segment in its entirety in order to obtain an accurate estimate.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        Added logic to resample paths if there is a cardinality estimate underflow until the no path is experiencing underflow.

        Show
        bryanthompson bryanthompson added a comment - Added logic to resample paths if there is a cardinality estimate underflow until the no path is experiencing underflow.
        Hide
        bryanthompson bryanthompson added a comment -

        The RTO implementation does this. What is does not do (yet) is assign a different LIMIT to different paths based on which paths are experiencing cardinality underflow. There are some notes about this in the JGraph code.

        Show
        bryanthompson bryanthompson added a comment - The RTO implementation does this. What is does not do (yet) is assign a different LIMIT to different paths based on which paths are experiencing cardinality underflow. There are some notes about this in the JGraph code.

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: