Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-1965

External Full Text Search Does Not Work if text field is not present.

    Details

      Description

      When using the default dynamic schema in SOLR, the ExternalFullTextSearch component fails as the test field is not found.

      6061679 INFO  (qtp110456297-51) [   x:freebase] o.a.s.c.S.Request [collection1]  webapp=/solr path=/select params={q=Fuel&fl=uri_s,label_t&wt=json} status=400 QTime=1
      6566146 ERROR (qtp110456297-27) [   x:freebase] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: undefined field text
      	at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1300)
      	at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:452)
      	at org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74)
      	at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:169)
      	at org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:206)
      	at org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:372)
      	at org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742)
      	at org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:530)
      	at org.apache.solr.parser.QueryParser.Term(QueryParser.java:315)
      	at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:186)
      	at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107)
      	at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96)
      
      

        Activity

        Hide
        beebs Brad Bebee added a comment - - edited

        The issue appears to be that by default the SolrFulltextSearchImpl:99 does not append a field to the query. In SOLR 6.1.0, the default field is one is not specified is "text", but this is not present by default in the standard SOLR dynamic typing schema.

              request.param("q",  query.getQuery());
        

        The fix is to add a parameter as fieldToSearch, which defaults to text. This preserves the legacy behavior.

              request.param("q", query.getFieldToSearch() + ":" + query.getQuery());
        
        PREFIX fts: <http://www.bigdata.com/rdf/fts#>
        SELECT ?res ?score ?snippet WHERE {
          ?res fts:search "Truck" .
          ?res fts:endpoint "http://localhost:8983/solr/collection1/select" .
          ?res fts:endpointType  "SOLR" .
          ?res fts:timeout "100000" .
          ?res fts:score ?score .
          ?res fts:snippet ?snippet . 
          ?res fts:params "fl=id,label_t" .
          ?res fts:searchField "id" .
          ?res fts:fieldToSearch "label_t" .
          ?res fts:snippetField "label_t" .
          ?res fts:searchResultType "URI" .
        }
        
        Show
        beebs Brad Bebee added a comment - - edited The issue appears to be that by default the SolrFulltextSearchImpl:99 does not append a field to the query. In SOLR 6.1.0, the default field is one is not specified is "text", but this is not present by default in the standard SOLR dynamic typing schema. request.param( "q" , query.getQuery()); The fix is to add a parameter as fieldToSearch, which defaults to text. This preserves the legacy behavior. request.param( "q" , query.getFieldToSearch() + ":" + query.getQuery()); PREFIX fts: <http://www.bigdata.com/rdf/fts#> SELECT ?res ?score ?snippet WHERE { ?res fts:search "Truck" . ?res fts:endpoint "http://localhost:8983/solr/collection1/select" . ?res fts:endpointType "SOLR" . ?res fts:timeout "100000" . ?res fts:score ?score . ?res fts:snippet ?snippet . ?res fts:params "fl=id,label_t" . ?res fts:searchField "id" . ?res fts:fieldToSearch "label_t" . ?res fts:snippetField "label_t" . ?res fts:searchResultType "URI" . }
        Hide
        beebs Brad Bebee added a comment -

        Michael Schmidt says:

        request.param("q", query.getFieldToSearch() + ":" + query.getQuery());
        

        somewhat suggests that instead of searching for "Alice" over field "myField" I could as well have searched for "myField:Alice" in the old implementation? Or do I miss something here

        Show
        beebs Brad Bebee added a comment - Michael Schmidt says: request.param( "q" , query.getFieldToSearch() + ":" + query.getQuery()); somewhat suggests that instead of searching for "Alice" over field "myField" I could as well have searched for "myField:Alice" in the old implementation? Or do I miss something here
        Hide
        beebs Brad Bebee added a comment -

        Yes, I agree that would be a work-around. The issue from my side is that with a naive setup, the first configuration would fail and look like an error.

        Show
        beebs Brad Bebee added a comment - Yes, I agree that would be a work-around. The issue from my side is that with a naive setup, the first configuration would fail and look like an error.
        Hide
        michaelschmidt michaelschmidt added a comment -

        OK, understand, would have helpded to look at the backing ticket . Anyway, the change makes sense, feel free to merge down.

        Show
        michaelschmidt michaelschmidt added a comment - OK, understand, would have helpded to look at the backing ticket . Anyway, the change makes sense, feel free to merge down.
        Show
        beebs Brad Bebee added a comment - https://github.com/SYSTAP/bigdata/pull/427

          People

          • Assignee:
            beebs Brad Bebee
            Reporter:
            beebs Brad Bebee
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: