Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-2077

NotMaterializedException when comparing dates in Wikidata Query Service

    Details

    • Type: Bug
    • Status: Done
    • Priority: Medium
    • Resolution: Done
    • Affects Version/s: BLAZEGRAPH_2_1_1
    • Fix Version/s: None
    • Component/s: Wikidata Query Service
    • Labels:
      None

      Description

      When running this query in WDQS:

      # People who died in the last 7 days
      SELECT ?person ?deathDate {
        ?person wdt:P31 wd:Q5 .
        ?person wdt:P570 ?deathDate .
        FILTER ( ?deathDate >= (now()-"P7D"^^xsd:duration) )
      }
      

      I get and exception:

      ERROR: SPARQL-QUERY: queryStr=# People who died in the last 7 days
      SELECT ?person ?deathDate {
      #  BIND( wd:Q15992661	as ?person)
        ?person wdt:P31 wd:Q5 .
        ?person wdt:P570 ?deathDate .
      #  FILTER(isBlank(?deathDate))
        FILTER ( ?deathDate >= (now()-"P7D"^^xsd:duration) )
      } LIMIT 100
      java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: org.openrdf.query.QueryEvaluationException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.Exception: task=ChunkTask{query=ada79a3a-1d7a-4b25-b3c1-2765d5f18e09,bopId=3,partitionId=-1,sinkId=7,altSinkId=null}, cause=java.util.concurrent.ExecutionException: java.lang.RuntimeException: com.bigdata.rdf.internal.NotMaterializedException
      

      (full exception below)

      1. TestTicket2077.java
        4 kB
        igorkim
      2. TestTicket2077.n3
        0.2 kB
        igorkim

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        igorkim I believe that mikepersonick ran into a similar problem and used a similar solution for the IPV4 inline literal datatype.

        Show
        bryanthompson bryanthompson added a comment - igorkim I believe that mikepersonick ran into a similar problem and used a similar solution for the IPV4 inline literal datatype.
        Hide
        igorkim igorkim added a comment -

        stasmalyshev, did you intend to support xsd:date+xsd:duration? XML Schema Part 2: Datatypes Second Edition defines that xsd:date might be used to add a duration, but does not define datatype of the result, thought example given '2000-01-12' + 'PT33H' = '2000-01-13' implies that datatype should be xsd:date. The same is stated in XQuery 1.0 and XPath 2.0 Functions and Operators (Second Edition), which also says that the time components from the resulting xs:dateTime should be discarded. In this case the code should be:

            @SuppressWarnings({"rawtypes", "unchecked"})
            private IV datePlusDuration(LiteralExtensionIV iv, Duration d, BigdataValueFactory vf) {
                BigdataURI dataType;
                if (iv.hasValue() && iv.getValue() instanceof BigdataLiteral) {
                    // Use dataType of cached value
                    dataType = ((BigdataLiteral)iv.getValue()).getDatatype();
                } else if (iv.getExtensionIV().hasValue()) {
                    // Use dataType from extension IV
                    dataType = (BigdataURI)iv.getExtensionIV().getValue();
                } else {
                    // Could not identify datatype of the result
                    throw new SparqlTypeErrorException();
                }
                long ts = iv.getDelegate().longValue();
                WikibaseDate newdate = WikibaseDate.fromSecondsSinceEpoch(ts).addDuration(d);
                if (XMLSchema.DATE.equals(dataType)) {
                    newdate = new WikibaseDate(newdate.year(), newdate.month(), newdate.day(), 0, 0, 0);
                }
                LiteralExtensionIV result = new LiteralExtensionIV(new XSDNumericIV(newdate.secondsSinceEpoch()), iv.getExtensionIV());
        		result.setValue(safeAsValue(result, vf, dataType));
                return result;
            }
        

        Note that in this case FILTER ( now() < (?deathDate+"P1Y"^^xsd:duration) ) would discard all the results for dataType(?deathDate)==xsd:date, as xsd:date is not comparable with xsd:dateType (dateType(now()).

        You might also want to propagate timezone of the original date/dateTime to the result, which is currently is not handled by this dates extension.

        Show
        igorkim igorkim added a comment - stasmalyshev , did you intend to support xsd:date+xsd:duration? XML Schema Part 2: Datatypes Second Edition defines that xsd:date might be used to add a duration, but does not define datatype of the result, thought example given '2000-01-12' + 'PT33H' = '2000-01-13' implies that datatype should be xsd:date. The same is stated in XQuery 1.0 and XPath 2.0 Functions and Operators (Second Edition) , which also says that the time components from the resulting xs:dateTime should be discarded. In this case the code should be: @SuppressWarnings({ "rawtypes" , "unchecked" }) private IV datePlusDuration(LiteralExtensionIV iv, Duration d, BigdataValueFactory vf) { BigdataURI dataType; if (iv.hasValue() && iv.getValue() instanceof BigdataLiteral) { // Use dataType of cached value dataType = ((BigdataLiteral)iv.getValue()).getDatatype(); } else if (iv.getExtensionIV().hasValue()) { // Use dataType from extension IV dataType = (BigdataURI)iv.getExtensionIV().getValue(); } else { // Could not identify datatype of the result throw new SparqlTypeErrorException(); } long ts = iv.getDelegate().longValue(); WikibaseDate newdate = WikibaseDate.fromSecondsSinceEpoch(ts).addDuration(d); if (XMLSchema.DATE.equals(dataType)) { newdate = new WikibaseDate(newdate.year(), newdate.month(), newdate.day(), 0, 0, 0); } LiteralExtensionIV result = new LiteralExtensionIV( new XSDNumericIV(newdate.secondsSinceEpoch()), iv.getExtensionIV()); result.setValue(safeAsValue(result, vf, dataType)); return result; } Note that in this case FILTER ( now() < (?deathDate+"P1Y"^^xsd:duration) ) would discard all the results for dataType(?deathDate)==xsd:date, as xsd:date is not comparable with xsd:dateType (dateType(now()). You might also want to propagate timezone of the original date/dateTime to the result, which is currently is not handled by this dates extension.
        Hide
        stasmalyshev stasmalyshev added a comment -

        Fortunately, Wikidata data does not use timezones. We do not use xsd:date in the dataset itself right now, even though technically most of our dateTime values are actually dates, and that's why we support it. So, I think adding date support would be good. Thanks for the advice.

        Show
        stasmalyshev stasmalyshev added a comment - Fortunately, Wikidata data does not use timezones. We do not use xsd:date in the dataset itself right now, even though technically most of our dateTime values are actually dates, and that's why we support it. So, I think adding date support would be good. Thanks for the advice.
        Hide
        stasmalyshev stasmalyshev added a comment -

        I checked and xsd:date +/- xsd:duration now returns xsd:date so this seems to be correct. Comparisons, however, do not work between xsd:date and xsd:datetime, because comparisons don't seem to use extensions, but instead they seem to go this route:

        1. In CompareBOp, if both types are extensions, and they have the same type, then they are compared with compareTo.
        2. compareTo uses delegate IVs as way to compare (in LiteralExtensionIV._compareTo) but again only if datatypes are equal
        3. Fortunately, WikibaseDate delegates are longs suitable for such comparison, but that prevents comparing xsd:date to xsd:dateTime if both are extensions.
        4. If the types do not match, the code goes to QueryEvaluationUtils.compareLiterals, however this one does not seem to support dates.

        So I wonder whether it's possible to do it and I'm still doing something wrong, or it's impossible right now?

        Comparison, of course, can be rewritten as (?x - ?y) < 0 etc. but this is not nice I think.

        Show
        stasmalyshev stasmalyshev added a comment - I checked and xsd:date +/- xsd:duration now returns xsd:date so this seems to be correct. Comparisons, however, do not work between xsd:date and xsd:datetime, because comparisons don't seem to use extensions, but instead they seem to go this route: 1. In CompareBOp, if both types are extensions, and they have the same type, then they are compared with compareTo. 2. compareTo uses delegate IVs as way to compare (in LiteralExtensionIV._compareTo) but again only if datatypes are equal 3. Fortunately, WikibaseDate delegates are longs suitable for such comparison, but that prevents comparing xsd:date to xsd:dateTime if both are extensions. 4. If the types do not match, the code goes to QueryEvaluationUtils.compareLiterals, however this one does not seem to support dates. So I wonder whether it's possible to do it and I'm still doing something wrong, or it's impossible right now? Comparison, of course, can be rewritten as (?x - ?y) < 0 etc. but this is not nice I think.
        Hide
        igorkim igorkim added a comment -

        stasmalyshev, The reason why comparison of date and dateTime is not defined in https://www.w3.org/TR/xpath20/#mapping
        is that date represents a period of time rather than instant dateTime (see https://www.w3.org/TR/xmlschema-2/#date):
        date consists of top-open intervals of exactly one day in length on the timelines of dateTime,
        So a result of comparison will be ambiguous if dateTime is between date start and end.

        Although, it might be defined as a comparison between start point of date (0:00:00.000 of specified date) and provided dateTime, but that would be a strained interpretation of W3C documents. If such comparison needs to be allowed, CompareBOp.compareLiterals should be changed to provide date IV to dateTime IV comparison, as Sesame implementation of QueryEvaluationUtil.compareLiterals does not allow mixing datatypes.

        Show
        igorkim igorkim added a comment - stasmalyshev , The reason why comparison of date and dateTime is not defined in https://www.w3.org/TR/xpath20/#mapping is that date represents a period of time rather than instant dateTime (see https://www.w3.org/TR/xmlschema-2/#date): date consists of top-open intervals of exactly one day in length on the timelines of dateTime, So a result of comparison will be ambiguous if dateTime is between date start and end. Although, it might be defined as a comparison between start point of date (0:00:00.000 of specified date) and provided dateTime, but that would be a strained interpretation of W3C documents. If such comparison needs to be allowed, CompareBOp.compareLiterals should be changed to provide date IV to dateTime IV comparison, as Sesame implementation of QueryEvaluationUtil.compareLiterals does not allow mixing datatypes.

          People

          • Assignee:
            igorkim igorkim
            Reporter:
            stasmalyshev stasmalyshev
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: