Details

      Description

      The goal I am trying to achieve is as follows:

      a) I generate some turtle for a single named graph: it may have errors in it
      b) I upload it to bigdata in a temporary named graph
      c) if I get a 200 I atomically delete the current target named graph and replace it with the contents of the temporary graph deleting the temporary graph in the process

      Here is the query for (c)

      # Atomic update of uploaded graph
      DELETE {
          GRAPH <eg:a> {
             ?olds ?oldp ?oldo
          }
          GRAPH <eg:A> {
             ?news ?newp ?newo
          }
      }
      INSERT {
          GRAPH <eg:a> {
              ?news ?newp ?newo
          }
      }
      WHERE {
          {
              GRAPH <eg:a> {
                  ?olds ?oldp ?oldo
              }
          } UNION {
              GRAPH <eg:A> {
                 ?news ?newp ?newo
              }
          }
      }
      

      In the test mode, I am using another update to generate test data

      prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
      INSERT DATA
      { 
       GRAPH <eg:a> {
         [ a "Blank" ] .
         <eg:b> rdf:type <eg:c> ; rdf:value []
       }
       GRAPH <eg:A> {
         [ a "Blankx" ] .
         <eg:B> rdf:type <eg:C> ; rdf:value []
       }
      }
      

      and I am using the following query to see what is going on:

      SELECT * { 
      GRAPH ?g {?s ?p ?o } } LIMIT 10
      

      With some playing around with these queries it is fairly easy to run the atomic update query and then the last query and still have a ground triple in graph <eg:A> which should have been completed deleted.

        Activity

        beebs Brad Bebee created issue -
        Hide
        jeremycarroll jeremycarroll added a comment -

        This appears to have improved fairly recently (last few weeks), and the very small test data does not exhibit the problem, whereas larger named graphs do still exhibit this issue

        Show
        jeremycarroll jeremycarroll added a comment - This appears to have improved fairly recently (last few weeks), and the very small test data does not exhibit the problem, whereas larger named graphs do still exhibit this issue
        Hide
        jeremycarroll jeremycarroll added a comment -

        Adding a

            hint:Query hint:chunkSize 2 .
        

        to the update query (the atomic renaming) reproduces the issue with the trivial test data

        Show
        jeremycarroll jeremycarroll added a comment - Adding a hint:Query hint:chunkSize 2 . to the update query (the atomic renaming) reproduces the issue with the trivial test data
        Hide
        jeremycarroll jeremycarroll added a comment -

        bigdata-sails/src/test/com/bigdata/rdf/sail/webapp/NamedGraphUpdateTest.java
        in commit r7818 pretty convincingly illustrates the issue.
        The query hint seems to be a red herring, and the small data sizes are OK to exercise the problem with enough attempts.

        Show
        jeremycarroll jeremycarroll added a comment - bigdata-sails/src/test/com/bigdata/rdf/sail/webapp/NamedGraphUpdateTest.java in commit r7818 pretty convincingly illustrates the issue. The query hint seems to be a red herring, and the small data sizes are OK to exercise the problem with enough attempts.
        Hide
        jeremycarroll jeremycarroll added a comment -

        Here is a non-heisen recipe:

        1) Execute following updates to initial with two named graphs each containing the same triple.

        DROP ALL;
        prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
        INSERT DATA
        { 
         GRAPH <eg:a> {
           <eg:b> rdf:type <eg:c> 
         }
         GRAPH <eg:A> {
           <eg:b> rdf:type <eg:c> 
         }
        }
        

        2) Execute following update which should delete both triples and add a third

        DELETE {
            GRAPH <eg:a> {
               ?olds ?oldp ?oldo
            }
            GRAPH <eg:A> {
               ?olds ?oldp ?oldo
            }
        }
        INSERT {
            GRAPH <eg:C> {
               ?olds ?oldp ?oldo
            }
        }
        WHERE {
                GRAPH <eg:a> {
                    ?olds ?oldp ?oldo
                }
        }
        

        3) Examine store to find that only one of the two triples was deleted, and there are still two triples in the store
        (it appears that the triple in eg:A does not get deleted, otoh I was expecting a heisen bug as to which one ...)

        Show
        jeremycarroll jeremycarroll added a comment - Here is a non-heisen recipe: 1) Execute following updates to initial with two named graphs each containing the same triple. DROP ALL; prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> INSERT DATA { GRAPH <eg:a> { <eg:b> rdf:type <eg:c> } GRAPH <eg:A> { <eg:b> rdf:type <eg:c> } } 2) Execute following update which should delete both triples and add a third DELETE { GRAPH <eg:a> { ?olds ?oldp ?oldo } GRAPH <eg:A> { ?olds ?oldp ?oldo } } INSERT { GRAPH <eg:C> { ?olds ?oldp ?oldo } } WHERE { GRAPH <eg:a> { ?olds ?oldp ?oldo } } 3) Examine store to find that only one of the two triples was deleted, and there are still two triples in the store (it appears that the triple in eg:A does not get deleted, otoh I was expecting a heisen bug as to which one ...)
        Hide
        jeremycarroll jeremycarroll added a comment -

        The insert clause is not needed to illustrate the issue

        Show
        jeremycarroll jeremycarroll added a comment - The insert clause is not needed to illustrate the issue
        Hide
        jeremycarroll jeremycarroll added a comment -

        com.bigdata.rdf.sparql.ast.eval.ASTConstructIterator.addStatementToBuffer(BigdataStatement)

        has the following incorrect code:

                if (filter != null) {
        
                    /*
                     * Impose a DISTINCT SPO filter on the generated statements in the
                     * constructed graph.
                     * 
                     * @see <a
                     * href="https://sourceforge.net/apps/trac/bigdata/ticket/579">
                     * CONSTRUCT should apply DISTINCT (s,p,o) filter </a>
                     */
        
                    if (filter.isValid(stmt)) {
        
                        buffer.add(stmt);
                        
                    }
        

        the solution will be to:

        a) when creating the construct code in the ASTBOpUpdate signal eityher using an attribute or a subclass or a flag that we want it created differently

        b) when creating the filter above make it a distinct c-spo

        Show
        jeremycarroll jeremycarroll added a comment - com.bigdata.rdf.sparql.ast.eval.ASTConstructIterator.addStatementToBuffer(BigdataStatement) has the following incorrect code: if (filter != null) { /* * Impose a DISTINCT SPO filter on the generated statements in the * constructed graph. * * @see <a * href="https://sourceforge.net/apps/trac/bigdata/ticket/579"> * CONSTRUCT should apply DISTINCT (s,p,o) filter </a> */ if (filter.isValid(stmt)) { buffer.add(stmt); } the solution will be to: a) when creating the construct code in the ASTBOpUpdate signal eityher using an attribute or a subclass or a flag that we want it created differently b) when creating the filter above make it a distinct c-spo
        Hide
        jeremycarroll jeremycarroll added a comment -

        Fixed in r7825

        Show
        jeremycarroll jeremycarroll added a comment - Fixed in r7825
        beebs Brad Bebee made changes -
        Field Original Value New Value
        Workflow Trac Import v2 [ 12721 ] Trac Import v3 [ 14239 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v3 [ 14239 ] Trac Import v4 [ 15568 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v4 [ 15568 ] Trac Import v5 [ 16954 ]
        beebs Brad Bebee made changes -
        Labels Issue_patch_20150625
        beebs Brad Bebee made changes -
        Status Closed - Won't Fix [ 6 ] Open [ 1 ]
        beebs Brad Bebee made changes -
        Status Open [ 1 ] Accepted [ 10101 ]
        beebs Brad Bebee made changes -
        Status Accepted [ 10101 ] In Progress [ 3 ]
        beebs Brad Bebee made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        beebs Brad Bebee made changes -
        Status Resolved [ 5 ] In Review [ 10100 ]
        beebs Brad Bebee made changes -
        Resolution Fixed [ 1 ] Done [ 10000 ]
        Status In Review [ 10100 ] Done [ 10000 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v5 [ 16954 ] Trac Import v6 [ 18149 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v6 [ 18149 ] Trac Import v7 [ 19546 ]
        beebs Brad Bebee made changes -
        Workflow Trac Import v7 [ 19546 ] Trac Import v8 [ 21168 ]

          People

          • Assignee:
            jeremycarroll jeremycarroll
            Reporter:
            jeremycarroll jeremycarroll
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: