Details

      Description

      The HandleChunkBuffer class provides a (significant) throughput oriented optimization. However, that optimization can cause the order of the solutions to be changed. This means the HandleChunkBuffer class breaks the semantics for ORDER BY and SLICE, both of which should preserve order. It also breaks the semantics for ORDER_BY + DISTINCT. This problem only appears in cases where the size of the output chunks was such that a (sufficiently) full chunk would be output ahead of smaller chunks in an internal buffer.

      This is a long standing issue. It was identified while working on cutoff join evaluation for the RTO (BLZG-265).

      Note: Practically speaking, this issue only effects the output chunking of an operator and only can cause reordering when some small output chunks are followed by full chunks. In fact, this case will never arise with either ORDER_BY. The ORDER_BY operator will flush all outputs at once and will emit full sized chunks. However, it could arise with ORDER_BY + DISTINCT since the DISTINCT operator can filter solutions and then reduce input chunks having duplicate solutions from full chunks to small chunks. SLICE never reduces the number of solutions and so ORDER_BY + SLICE would never show this problem. However, ORDER_BY+DISTINCT+SLICE could show the problem.

      The problem would also be manifest as an inability to reliably preserve the implicit order of an index scan through a query. However, bigdata does not really support this. In order to do this reliably, a query hint would need to be developed to instruct the query generator to create an order preserving plan. This is certainly possible and it could reuse much of the code that allows the RTO (BLZG-265) to perform cutoff evaluation of a join.

        Activity

        beebs Brad Bebee created issue -
        beebs Brad Bebee made changes -
        Field Original Value New Value
        Workflow Trac Import v2 [ 12715 ] Trac Import v3 [ 14234 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v3 [ 14234 ] Trac Import v4 [ 15563 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v4 [ 15563 ] Trac Import v5 [ 16949 ]
        beebs Brad Bebee made changes -
        Labels Issue_patch_20150625
        beebs Brad Bebee made changes -
        Status Closed - Won't Fix [ 6 ] Open [ 1 ]
        beebs Brad Bebee made changes -
        Status Open [ 1 ] Accepted [ 10101 ]
        beebs Brad Bebee made changes -
        Status Accepted [ 10101 ] In Progress [ 3 ]
        beebs Brad Bebee made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        beebs Brad Bebee made changes -
        Status Resolved [ 5 ] In Review [ 10100 ]
        beebs Brad Bebee made changes -
        Resolution Fixed [ 1 ] Done [ 10000 ]
        Status In Review [ 10100 ] Done [ 10000 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v5 [ 16949 ] Trac Import v6 [ 18144 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v6 [ 18144 ] Trac Import v7 [ 19541 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v7 [ 19541 ] Trac Import v8 [ 21163 ]

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: