Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-1014

Create/Destroy of KB followed by Create with different Vocabulary causes runtime exception

    Details

    • Type: Bug
    • Status: Accepted
    • Resolution: Unresolved
    • Affects Version/s: BIGDATA_RELEASE_1_3_1
    • Fix Version/s: None
    • Component/s: NanoSparqlServer
    • Labels:
      None

      Description

      Create/Destroy of KB followed by Create with different Vocablary causes runtime exception. Maybe we are not removing the namespace from the static canonicalizing value factory?

      java.lang.RuntimeException: java.lang.IllegalStateException: Already assigned: old=Vocab(-55), new=Vocab(62)
      	at com.bigdata.rdf.store.AbstractTripleStore.create(AbstractTripleStore.java:1722)
      	at com.bigdata.rdf.sail.BigdataSail.createLTS(BigdataSail.java:755)
      	at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doCreateNamespace(MultiTenancyServlet.java:361)
      	at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:116)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:534)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:475)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:929)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
      	at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114)
      	at org.eclipse.jetty.server.Server.handle(Server.java:352)
      	at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596)
      	at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1051)
      	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:599)
      	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:212)
      	at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:426)
      	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:508)
      	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34)
      	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451)
      	at java.lang.Thread.run(Thread.java:724)
      Caused by: java.lang.IllegalStateException: Already assigned: old=Vocab(-55), new=Vocab(62)
      	at com.bigdata.rdf.model.BigdataValueImpl.setIV(BigdataValueImpl.java:135)
      	at com.bigdata.rdf.vocab.BaseVocabulary.generateIVs(BaseVocabulary.java:311)
      	at com.bigdata.rdf.vocab.BaseVocabulary.init(BaseVocabulary.java:180)
      	at com.bigdata.rdf.vocab.BaseVocabulary.init(BaseVocabulary.java:142)
      	at com.bigdata.rdf.store.AbstractTripleStore.create(AbstractTripleStore.java:1580)
      	... 24 more
      

      We need to write a test for this. Perhaps Toby can verify the problem from the new workspace?

        Activity

        Hide
        tobycraig tobycraig added a comment -

        From the new workbench you can't get create a namespace with a different vocabulary. Here's the XML that is POSTed to /bigdata/namespace:

        <?xml version="1.0" encoding="UTF-8" standalone="no"?>
        <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
        <properties>
        <entry key="com.bigdata.rdf.sail.namespace">

        {namespace}</entry>
        </properties>

        with {namespace}

        being whatever the user has typed in. (Currently, it's not validated; it'd be easy to add a regex for this though.)

        Show
        tobycraig tobycraig added a comment - From the new workbench you can't get create a namespace with a different vocabulary. Here's the XML that is POSTed to /bigdata/namespace: <?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd"> <properties> <entry key="com.bigdata.rdf.sail.namespace"> {namespace}</entry> </properties> with {namespace} being whatever the user has typed in. (Currently, it's not validated; it'd be easy to add a regex for this though.)
        Hide
        bryanthompson bryanthompson added a comment -

        I will look at this as part of the group commit support since I need to write more tests around the create/destroy of namespaces anyway.

        Show
        bryanthompson bryanthompson added a comment - I will look at this as part of the group commit support since I need to write more tests around the create/destroy of namespaces anyway.
        Hide
        bryanthompson bryanthompson added a comment -

        I've replicated this issue in TestConcurrentKBCreate.test_CreateDestroy_ticket_948() in the SF NSS_GROUP_COMMIT branch. The problem may be a failure to completely clear the resourceCache on the DefaultResourceLocator for related items or possibly the failure to clear the propertyCache on that class at all.

        Note: The problem requires a conflict in the order in which the vocabulary IVs are assigned. For example, the two classes BSBMVocabulary and RDFSVocabulary demonstrate this conflict. A namespace created with one of these classes can not be deleted and then recreated with the other.

        resourceCache during delete of the initial KB instance.

        {
        NT{name=kb.spo,timestamp=0}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@ad99f2d{key=NT{name=kb.spo,timestamp=0},val=SPORelation{timestamp=0, namespace=kb.spo, container=kb, indexManager=com.bigdata.journal.Journal@def310b}},
        NT{name=kb,timestamp=1424875434743}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@65fa6817{key=NT{name=kb,timestamp=1424875434743},val=null},
        NT{name=kb,timestamp=-1}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@3cf52e45{key=NT{name=kb,timestamp=-1},val=null},
        NT{name=kb,timestamp=0}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@4b0d347{key=NT{name=kb,timestamp=0},val=LocalTripleStore{timestamp=0, namespace=kb, container=null, indexManager=com.bigdata.journal.Journal@def310b}},
        NT{name=kb.lex,timestamp=0}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@74fedcb8{key=NT{name=kb.lex,timestamp=0},val=LexiconRelation{timestamp=0, namespace=kb.lex, container=kb, indexManager=com.bigdata.journal.Journal@def310b}}}
        

        propertyCache:

        {
        NT{name=kb,timestamp=1424875434639}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@4a35e00c{key=NT{name=kb,timestamp=1424875434639},val={awt.toolkit=sun.lwawt.macosx.LWCToolkit, com.bigdata.journal.AbstractJournal.bufferMode=Disk, com.bigdata.journal.AbstractJournal.createTempFile=true, com.bigdata.journal.AbstractJournal.deleteOnExit=true, com.bigdata.journal.Journal.collectPlatformStatistics=false, com.bigdata.journal.Journal.collectQueueStatistics=false, com.bigdata.journal.Journal.httpdPort=-1, com.bigdata.rdf.sail.namespace=kb, com.bigdata.rdf.sail.truthMaintenance=false, com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms, com.bigdata.rdf.store.AbstractTripleStore.quads=true, com.bigdata.rdf.store.AbstractTripleStore.quadsMode=true, com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false, com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=com.bigdata.rdf.vocab.BSBMVocabulary, com.bigdata.rdf.store.axioms=com.bigdata.rdf.axioms.NoAxioms@170859e4, com.bigdata.rdf.store.vocabulary=com.bigdata.rdf.vocab.BSBMVocabulary@2545938c, com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore, com.bigdata.relation.container=kb, com.bigdata.relation.namespace=kb, com.bigdata.search.FullTextIndex.fieldsEnabled=false, ...}}}
        

        The discard() call then proceeds to remove the resourceCache entries for

        NT{name=kb.lex,timestamp=0}
        NT{name=kb.lex,timestamp=-1} // aka READ_COMMITTED
        NT{name=kb.spo,timestamp=0}
        NT{name=kb.spo,timestamp=-1}
        NT{name=kb,timestamp=0}
        NT{name=kb,timestamp=-1}
        

        this would leave the NT{name=kb,timestamp=1424875434743} entry expect that it is cleared by a timeout expiring hard references from the associated hard reference queue and hence was already gone by this time under the debugger.

        I have verified that a re-create with the same properties is not a problem and now have a regression test for this.

        I have verified that the problem is stable across a re-open of the Journal
        - you still can not re-create the namespace with a different Vocabulary class.

        I have verified that the problem is stable across a JVM instance. If you shutdown a NSS process and then restart that process, you still can not re-create the namespace with a different Vocabulary.

        Note: The problem appears to be related to the relative order in which the XSD URIs are declared in the various Vocabulary class implementations. Those URIs are coined by the BigdataValueFactory when it is initialized. There may be some cycle connecting the BigdataValueFactory initialization, the automatic de-serialization of the Vocabulary with atomic row store operations on the GRS for the namespace declaration, and the BigdataValueFactoryImpl's per-namespace ValueFactory cache.

        Show
        bryanthompson bryanthompson added a comment - I've replicated this issue in TestConcurrentKBCreate.test_CreateDestroy_ticket_948() in the SF NSS_GROUP_COMMIT branch. The problem may be a failure to completely clear the resourceCache on the DefaultResourceLocator for related items or possibly the failure to clear the propertyCache on that class at all. Note: The problem requires a conflict in the order in which the vocabulary IVs are assigned. For example, the two classes BSBMVocabulary and RDFSVocabulary demonstrate this conflict. A namespace created with one of these classes can not be deleted and then recreated with the other. resourceCache during delete of the initial KB instance. { NT{name=kb.spo,timestamp=0}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@ad99f2d{key=NT{name=kb.spo,timestamp=0},val=SPORelation{timestamp=0, namespace=kb.spo, container=kb, indexManager=com.bigdata.journal.Journal@def310b}}, NT{name=kb,timestamp=1424875434743}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@65fa6817{key=NT{name=kb,timestamp=1424875434743},val=null}, NT{name=kb,timestamp=-1}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@3cf52e45{key=NT{name=kb,timestamp=-1},val=null}, NT{name=kb,timestamp=0}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@4b0d347{key=NT{name=kb,timestamp=0},val=LocalTripleStore{timestamp=0, namespace=kb, container=null, indexManager=com.bigdata.journal.Journal@def310b}}, NT{name=kb.lex,timestamp=0}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@74fedcb8{key=NT{name=kb.lex,timestamp=0},val=LexiconRelation{timestamp=0, namespace=kb.lex, container=kb, indexManager=com.bigdata.journal.Journal@def310b}}} propertyCache: { NT{name=kb,timestamp=1424875434639}=com.bigdata.cache.ConcurrentWeakValueCache$WeakRef@4a35e00c{key=NT{name=kb,timestamp=1424875434639},val={awt.toolkit=sun.lwawt.macosx.LWCToolkit, com.bigdata.journal.AbstractJournal.bufferMode=Disk, com.bigdata.journal.AbstractJournal.createTempFile=true, com.bigdata.journal.AbstractJournal.deleteOnExit=true, com.bigdata.journal.Journal.collectPlatformStatistics=false, com.bigdata.journal.Journal.collectQueueStatistics=false, com.bigdata.journal.Journal.httpdPort=-1, com.bigdata.rdf.sail.namespace=kb, com.bigdata.rdf.sail.truthMaintenance=false, com.bigdata.rdf.store.AbstractTripleStore.axiomsClass=com.bigdata.rdf.axioms.NoAxioms, com.bigdata.rdf.store.AbstractTripleStore.quads=true, com.bigdata.rdf.store.AbstractTripleStore.quadsMode=true, com.bigdata.rdf.store.AbstractTripleStore.statementIdentifiers=false, com.bigdata.rdf.store.AbstractTripleStore.vocabularyClass=com.bigdata.rdf.vocab.BSBMVocabulary, com.bigdata.rdf.store.axioms=com.bigdata.rdf.axioms.NoAxioms@170859e4, com.bigdata.rdf.store.vocabulary=com.bigdata.rdf.vocab.BSBMVocabulary@2545938c, com.bigdata.relation.class=com.bigdata.rdf.store.LocalTripleStore, com.bigdata.relation.container=kb, com.bigdata.relation.namespace=kb, com.bigdata.search.FullTextIndex.fieldsEnabled=false, ...}}} The discard() call then proceeds to remove the resourceCache entries for NT{name=kb.lex,timestamp=0} NT{name=kb.lex,timestamp=-1} // aka READ_COMMITTED NT{name=kb.spo,timestamp=0} NT{name=kb.spo,timestamp=-1} NT{name=kb,timestamp=0} NT{name=kb,timestamp=-1} this would leave the NT{name=kb,timestamp=1424875434743 } entry expect that it is cleared by a timeout expiring hard references from the associated hard reference queue and hence was already gone by this time under the debugger. I have verified that a re-create with the same properties is not a problem and now have a regression test for this. I have verified that the problem is stable across a re-open of the Journal - you still can not re-create the namespace with a different Vocabulary class. I have verified that the problem is stable across a JVM instance. If you shutdown a NSS process and then restart that process, you still can not re-create the namespace with a different Vocabulary. Note: The problem appears to be related to the relative order in which the XSD URIs are declared in the various Vocabulary class implementations. Those URIs are coined by the BigdataValueFactory when it is initialized. There may be some cycle connecting the BigdataValueFactory initialization, the automatic de-serialization of the Vocabulary with atomic row store operations on the GRS for the namespace declaration, and the BigdataValueFactoryImpl's per-namespace ValueFactory cache.
        Hide
        bryanthompson bryanthompson added a comment -

        I have conditionally disabled the two tests that are failing. This ticket will not be part of the 1.5.1 release.

        Show
        bryanthompson bryanthompson added a comment - I have conditionally disabled the two tests that are failing. This ticket will not be part of the 1.5.1 release.

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated: