Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-1063

Invalidate BTree objects if error occurs during eviction

    Details

    • Type: New Feature
    • Status: Done
    • Resolution: Done
    • Affects Version/s: BIGDATA_RELEASE_1_3_1
    • Fix Version/s: None
    • Component/s: B+Tree

      Description

      The unisolated mutable BTree should be invalidated if an error occurs during an eviction in order to prevent partially evicted nodes or leaves from being later flushed to the disk.

      See BLZG-933 (do not drive evictions from read-only operations on unisolated indices). The changes for this ticket were made in the same change set as the changes for BLZG-933.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        We have added a volatile Throwable error field to AbstractBTree and AbstractHTree. This field is set if an error occurs during processing that would render the unisolated view of an index object as unusable. The primary use case (and the only case that is currently trapped) is an eviction of a dirty node or leaf that fails, e.g., due to a storage error, an out of memory error, an interrupt, etc. In these cases the index object becomes unusable and will refused some operations (notably you can not write on it).

        This only effects unisolated index views.

        We might extend both the set of conditions that invalidate the unisolated index object over time and also check the error field more frequently. For example, perhaps in getRoot() we can fail if the error field is set.

        Change set is not yet committed. We are testing locally for correctness and performance regressions.

        Show
        bryanthompson bryanthompson added a comment - We have added a volatile Throwable error field to AbstractBTree and AbstractHTree. This field is set if an error occurs during processing that would render the unisolated view of an index object as unusable. The primary use case (and the only case that is currently trapped) is an eviction of a dirty node or leaf that fails, e.g., due to a storage error, an out of memory error, an interrupt, etc. In these cases the index object becomes unusable and will refused some operations (notably you can not write on it). This only effects unisolated index views. We might extend both the set of conditions that invalidate the unisolated index object over time and also check the error field more frequently. For example, perhaps in getRoot() we can fail if the error field is set. Change set is not yet committed. We are testing locally for correctness and performance regressions.
        Hide
        bryanthompson bryanthompson added a comment -

        Committing to CI a fix for BLZG-933 (reads must not drive evictions for the unisolated view of the BTree and HTree) and BLZG-1063 (Unisolated BTree and HTree should be marked as invalid after error during eviction of a dirty leaf or node). See the tickets for more detail about this change set.

        I have run much of the total test suite locally, including the HA test suite.

        We are testing for performance regressions with this change set.

        Committed revision r8639.

        Show
        bryanthompson bryanthompson added a comment - Committing to CI a fix for BLZG-933 (reads must not drive evictions for the unisolated view of the BTree and HTree) and BLZG-1063 (Unisolated BTree and HTree should be marked as invalid after error during eviction of a dirty leaf or node). See the tickets for more detail about this change set. I have run much of the total test suite locally, including the HA test suite. We are testing for performance regressions with this change set. Committed revision r8639.
        Hide
        bryanthompson bryanthompson added a comment -

        LUBM U50 on bigdata11 does not show any performance regression for load or query

        1.3.0 (r8639)		
        bigdata11		
        1.7.0_25		
        1500m (load + query)		
        Load: 6890949 stmts added in 94.621 secs, rate= 72823, commitLatency=0ms		
        Closure: ClosureStats{mutationCount=1699274, elapsed=51856ms, rate=32769}		
        Wrote: 578551808 bytes.		
        Total elapsed=153767ms		
        		
        		
        		
        		
        		
        		
             [java] BIGDATA_SPARQL_ENDPOINT	#trials=10	#parallel=1
             [java] query	Time	Result#
             [java] query1	117	4
             [java] query3	53	6
             [java] query4	99	34
             [java] query5	88	719
             [java] query7	44	61
             [java] query8	355	6463
             [java] query10	36	0
             [java] query11	45	0
             [java] query12	48	0
             [java] query13	43	0
             [java] query14	2467	393730
             [java] query6	3093	430114
             [java] query2	869	130
             [java] query9	5205	8627
             [java] Total	12562	
        		
             [java] BIGDATA_SPARQL_ENDPOINT	#trials=10	#parallel=1
             [java] query	Time	Result#
             [java] query1	58	4
             [java] query3	37	6
             [java] query4	51	34
             [java] query5	39	719
             [java] query7	35	61
             [java] query8	176	6463
             [java] query10	39	0
             [java] query11	42	0
             [java] query12	46	0
             [java] query13	41	0
             [java] query14	2202	393730
             [java] query6	2206	430114
             [java] query2	706	130
             [java] query9	4522	8627
             [java] Total	10200	
        		
             [java] BIGDATA_SPARQL_ENDPOINT	#trials=10	#parallel=1
             [java] query	Time	Result#
             [java] query1	56	4
             [java] query3	36	6
             [java] query4	44	34
             [java] query5	37	719
             [java] query7	39	61
             [java] query8	164	6463
             [java] query10	40	0
             [java] query11	40	0
             [java] query12	43	0
             [java] query13	40	0
             [java] query14	1647	393730
             [java] query6	1877	430114
             [java] query2	615	130
             [java] query9	3978	8627
             [java] Total	8656	
        
        Show
        bryanthompson bryanthompson added a comment - LUBM U50 on bigdata11 does not show any performance regression for load or query 1.3.0 (r8639) bigdata11 1.7.0_25 1500m (load + query) Load: 6890949 stmts added in 94.621 secs, rate= 72823, commitLatency=0ms Closure: ClosureStats{mutationCount=1699274, elapsed=51856ms, rate=32769} Wrote: 578551808 bytes. Total elapsed=153767ms [java] BIGDATA_SPARQL_ENDPOINT #trials=10 #parallel=1 [java] query Time Result# [java] query1 117 4 [java] query3 53 6 [java] query4 99 34 [java] query5 88 719 [java] query7 44 61 [java] query8 355 6463 [java] query10 36 0 [java] query11 45 0 [java] query12 48 0 [java] query13 43 0 [java] query14 2467 393730 [java] query6 3093 430114 [java] query2 869 130 [java] query9 5205 8627 [java] Total 12562 [java] BIGDATA_SPARQL_ENDPOINT #trials=10 #parallel=1 [java] query Time Result# [java] query1 58 4 [java] query3 37 6 [java] query4 51 34 [java] query5 39 719 [java] query7 35 61 [java] query8 176 6463 [java] query10 39 0 [java] query11 42 0 [java] query12 46 0 [java] query13 41 0 [java] query14 2202 393730 [java] query6 2206 430114 [java] query2 706 130 [java] query9 4522 8627 [java] Total 10200 [java] BIGDATA_SPARQL_ENDPOINT #trials=10 #parallel=1 [java] query Time Result# [java] query1 56 4 [java] query3 36 6 [java] query4 44 34 [java] query5 37 719 [java] query7 39 61 [java] query8 164 6463 [java] query10 40 0 [java] query11 40 0 [java] query12 43 0 [java] query13 40 0 [java] query14 1647 393730 [java] query6 1877 430114 [java] query2 615 130 [java] query9 3978 8627 [java] Total 8656

          People

          • Assignee:
            martyncutcher martyncutcher
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: