Details

      Description

      The goal I am trying to achieve is as follows:

      a) I generate some turtle for a single named graph: it may have errors in it
      b) I upload it to bigdata in a temporary named graph
      c) if I get a 200 I atomically delete the current target named graph and replace it with the contents of the temporary graph deleting the temporary graph in the process

      Here is the query for (c)

      # Atomic update of uploaded graph
      DELETE {
          GRAPH <eg:a> {
             ?olds ?oldp ?oldo
          }
          GRAPH <eg:A> {
             ?news ?newp ?newo
          }
      }
      INSERT {
          GRAPH <eg:a> {
              ?news ?newp ?newo
          }
      }
      WHERE {
          {
              GRAPH <eg:a> {
                  ?olds ?oldp ?oldo
              }
          } UNION {
              GRAPH <eg:A> {
                 ?news ?newp ?newo
              }
          }
      }
      

      In the test mode, I am using another update to generate test data

      prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
      INSERT DATA
      { 
       GRAPH <eg:a> {
         [ a "Blank" ] .
         <eg:b> rdf:type <eg:c> ; rdf:value []
       }
       GRAPH <eg:A> {
         [ a "Blankx" ] .
         <eg:B> rdf:type <eg:C> ; rdf:value []
       }
      }
      

      and I am using the following query to see what is going on:

      SELECT * { 
      GRAPH ?g {?s ?p ?o } } LIMIT 10
      

      With some playing around with these queries it is fairly easy to run the atomic update query and then the last query and still have a ground triple in graph <eg:A> which should have been completed deleted.

        Activity

        Hide
        jeremycarroll jeremycarroll added a comment -

        This appears to have improved fairly recently (last few weeks), and the very small test data does not exhibit the problem, whereas larger named graphs do still exhibit this issue

        Show
        jeremycarroll jeremycarroll added a comment - This appears to have improved fairly recently (last few weeks), and the very small test data does not exhibit the problem, whereas larger named graphs do still exhibit this issue
        Hide
        jeremycarroll jeremycarroll added a comment -

        Adding a

            hint:Query hint:chunkSize 2 .
        

        to the update query (the atomic renaming) reproduces the issue with the trivial test data

        Show
        jeremycarroll jeremycarroll added a comment - Adding a hint:Query hint:chunkSize 2 . to the update query (the atomic renaming) reproduces the issue with the trivial test data
        Hide
        jeremycarroll jeremycarroll added a comment -

        bigdata-sails/src/test/com/bigdata/rdf/sail/webapp/NamedGraphUpdateTest.java
        in commit r7818 pretty convincingly illustrates the issue.
        The query hint seems to be a red herring, and the small data sizes are OK to exercise the problem with enough attempts.

        Show
        jeremycarroll jeremycarroll added a comment - bigdata-sails/src/test/com/bigdata/rdf/sail/webapp/NamedGraphUpdateTest.java in commit r7818 pretty convincingly illustrates the issue. The query hint seems to be a red herring, and the small data sizes are OK to exercise the problem with enough attempts.
        Hide
        jeremycarroll jeremycarroll added a comment -

        Here is a non-heisen recipe:

        1) Execute following updates to initial with two named graphs each containing the same triple.

        DROP ALL;
        prefix rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
        INSERT DATA
        { 
         GRAPH <eg:a> {
           <eg:b> rdf:type <eg:c> 
         }
         GRAPH <eg:A> {
           <eg:b> rdf:type <eg:c> 
         }
        }
        

        2) Execute following update which should delete both triples and add a third

        DELETE {
            GRAPH <eg:a> {
               ?olds ?oldp ?oldo
            }
            GRAPH <eg:A> {
               ?olds ?oldp ?oldo
            }
        }
        INSERT {
            GRAPH <eg:C> {
               ?olds ?oldp ?oldo
            }
        }
        WHERE {
                GRAPH <eg:a> {
                    ?olds ?oldp ?oldo
                }
        }
        

        3) Examine store to find that only one of the two triples was deleted, and there are still two triples in the store
        (it appears that the triple in eg:A does not get deleted, otoh I was expecting a heisen bug as to which one ...)

        Show
        jeremycarroll jeremycarroll added a comment - Here is a non-heisen recipe: 1) Execute following updates to initial with two named graphs each containing the same triple. DROP ALL; prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> INSERT DATA { GRAPH <eg:a> { <eg:b> rdf:type <eg:c> } GRAPH <eg:A> { <eg:b> rdf:type <eg:c> } } 2) Execute following update which should delete both triples and add a third DELETE { GRAPH <eg:a> { ?olds ?oldp ?oldo } GRAPH <eg:A> { ?olds ?oldp ?oldo } } INSERT { GRAPH <eg:C> { ?olds ?oldp ?oldo } } WHERE { GRAPH <eg:a> { ?olds ?oldp ?oldo } } 3) Examine store to find that only one of the two triples was deleted, and there are still two triples in the store (it appears that the triple in eg:A does not get deleted, otoh I was expecting a heisen bug as to which one ...)
        Hide
        jeremycarroll jeremycarroll added a comment -

        The insert clause is not needed to illustrate the issue

        Show
        jeremycarroll jeremycarroll added a comment - The insert clause is not needed to illustrate the issue
        Hide
        jeremycarroll jeremycarroll added a comment -

        com.bigdata.rdf.sparql.ast.eval.ASTConstructIterator.addStatementToBuffer(BigdataStatement)

        has the following incorrect code:

                if (filter != null) {
        
                    /*
                     * Impose a DISTINCT SPO filter on the generated statements in the
                     * constructed graph.
                     * 
                     * @see <a
                     * href="https://sourceforge.net/apps/trac/bigdata/ticket/579">
                     * CONSTRUCT should apply DISTINCT (s,p,o) filter </a>
                     */
        
                    if (filter.isValid(stmt)) {
        
                        buffer.add(stmt);
                        
                    }
        

        the solution will be to:

        a) when creating the construct code in the ASTBOpUpdate signal eityher using an attribute or a subclass or a flag that we want it created differently

        b) when creating the filter above make it a distinct c-spo

        Show
        jeremycarroll jeremycarroll added a comment - com.bigdata.rdf.sparql.ast.eval.ASTConstructIterator.addStatementToBuffer(BigdataStatement) has the following incorrect code: if (filter != null) { /* * Impose a DISTINCT SPO filter on the generated statements in the * constructed graph. * * @see <a * href="https://sourceforge.net/apps/trac/bigdata/ticket/579"> * CONSTRUCT should apply DISTINCT (s,p,o) filter </a> */ if (filter.isValid(stmt)) { buffer.add(stmt); } the solution will be to: a) when creating the construct code in the ASTBOpUpdate signal eityher using an attribute or a subclass or a flag that we want it created differently b) when creating the filter above make it a distinct c-spo
        Hide
        jeremycarroll jeremycarroll added a comment -

        Fixed in r7825

        Show
        jeremycarroll jeremycarroll added a comment - Fixed in r7825

          People

          • Assignee:
            jeremycarroll jeremycarroll
            Reporter:
            jeremycarroll jeremycarroll
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: