Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-445

Adjust defaults for the free text search engine

    Details

      Description

      The default MIN_RELEVANCE of ZERO (0.0) is causing some queries to slam the database which would otherwise run very quickly with a default more typical of full text search systems, e.g., .25 or .5.

      The defaults need to be reviewed and the parameterization of the defaults needs to be improved.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        I am adding new constants for the defaults to the BD interface where the various search magic predicates are declared. It turns out that there are two different code paths in BigdataEvaluationStrategyImpl3 for search. One where there are no overrides for the search and one where there are.

        I am driving the defaults into both locations so they can be controlled from the constants on the BD interface. Then we can choose the defaults that make the best sense and impose those changes in a single location (the BD interface).

        I am also updating the TestSearchQuery class to use the defaults when it runs the query using the low level API in order to verify that the search overrides were correctly applied.

        Show
        bryanthompson bryanthompson added a comment - I am adding new constants for the defaults to the BD interface where the various search magic predicates are declared. It turns out that there are two different code paths in BigdataEvaluationStrategyImpl3 for search. One where there are no overrides for the search and one where there are. I am driving the defaults into both locations so they can be controlled from the constants on the BD interface. Then we can choose the defaults that make the best sense and impose those changes in a single location (the BD interface). I am also updating the TestSearchQuery class to use the defaults when it runs the query using the low level API in order to verify that the search overrides were correctly applied.
        Hide
        bryanthompson bryanthompson added a comment -

        Note: I am going to have to reconcile the changes to the eval strategy impl.

        minRelevance of zero is not really an error. There are times when it makes sense. You only get a "hit" if there is a keyword in common. So, no matter how low the value of minRelevance which is specfified, you are only going to get "hits" -- documents which do not share "any" keywords will not be part of the result even when minRelevance is zero.

        However, a default minRelevance of zero is not a good idea as it can drag in too much which is useless.

        Show
        bryanthompson bryanthompson added a comment - Note: I am going to have to reconcile the changes to the eval strategy impl. minRelevance of zero is not really an error. There are times when it makes sense. You only get a "hit" if there is a keyword in common. So, no matter how low the value of minRelevance which is specfified, you are only going to get "hits" -- documents which do not share "any" keywords will not be part of the result even when minRelevance is zero. However, a default minRelevance of zero is not a good idea as it can drag in too much which is useless.
        Hide
        bryanthompson bryanthompson added a comment -

        Code complete and unit tests are passing. I'll commit this once I finish benchmarking for [1] and reconcile the conflicts in BigdataEvaluationStrategyImpl3 (which are minor).

        [1] https://sourceforge.net/apps/trac/bigdata/ticket/23

        Show
        bryanthompson bryanthompson added a comment - Code complete and unit tests are passing. I'll commit this once I finish benchmarking for [1] and reconcile the conflicts in BigdataEvaluationStrategyImpl3 (which are minor). [1] https://sourceforge.net/apps/trac/bigdata/ticket/23
        Hide
        bryanthompson bryanthompson added a comment -

        Committed revision r4530.

        Show
        bryanthompson bryanthompson added a comment - Committed revision r4530.

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: