Full-Text Search with XQFT

contains text compares tokens, not characters, so "abc" contains text "a" is false. Options follow the terms: any, all, phrase; using stemming, wildcards or case sensitive; distance, window and ordered filters; ftand, ftor, ftnot. let score binds a relevance from 0 to 1. Run with basex -c "SET STEMMING true" -c "CREATE DB booknest booknest-catalog.xml" -c "CREATE INDEX fulltext" -c "RUN ft.xq":

ft.xq: stemming, scores and proximity

(: Full-text search over the catalog's summaries :)
for $b in //book
let score $s := $b/summary contains text { "keeping", "trek", "thrive" } any using stemming
where $s > 0
order by $s descending
return $b/title || " (" || round($s, 2) || ")",
//book[summary contains text { "recipes", "stories" } all words
       distance at most 6 words]/@id/string(),
ft:mark(//summary[. contains text "thrive" using stemming])

Output

Gardens in Glass (0.19)
Small Steps to Big Summits (0.13)
b3
<summary>Designing and keeping terrariums that <mark>thrive</mark> for years.</summary>

Stemming matched "trek" to "trekking"; ft:mark() highlights hits. With -V, BaseX 693,830 reports apply full-text index: the predicates became inverted-index lookups, as in a search engine.

XQFT

XQuery Full Text (XQFT) is an extension to XQuery used to query strings for words and phrases, via the contains text expression. BaseX 693,830 is a program that supports XQFT.

The table below shows the boolean result of evaluating a contains text expression for a range of full text options: occurrence counts, any/all/phrase matching, Boolean combinators (ftand, ftor, ftnot, not in), word/sentence/paragraph distance and window constraints, position constraints (at start, at end, entire content), case sensitivity, stemming, stop words, and wildcards.

Expression	Result
"abc" contains text "a"	false
"a bc" contains text "Bc"	true
"ab...bc!" contains text "ab,bc"	true
"ab...bc..de!" contains text "ab,de"	false
"aa aa bb cc" contains text "aa" occurs exactly 2 times	true
"aa aa bb cc" contains text "aa" occurs at least 2 times	true
"aa aa bb cc" contains text "aa" occurs at most 2 times	true
"aa aa bb cc" contains text "aa" occurs from 2 to 3 times	true
"aa bb cc" contains text {"aa","ac"} any	true
"aa bb cc" contains text {"aa","ac"} all	false
"aa bb cc" contains text {"aa ab","ac"} any word	true
"aa bb cc" contains text {"bb aa","cc"} all words	true
"aa bb cc" contains text {"aa bb","cc"} phrase	true
"aa bb cc" contains text {"aa","cc"} phrase	false
"aa bb cc" contains text {"aa bb","cc"} ftor {"ab"}	true
"aa bb cc" contains text {"aa bb","cc"} ftand {"ab"}	false
"aa bb cc" contains text ftnot {"aa","cc"}	false
"aa bb cc" contains text "aa" not in "aa bb"	false
"a b c d e" contains text {"a","c","e"} all ordered distance at most 1 words	true
"a b c d e" contains text {"a","c","e"} all distance at most 1 paragraphs	false
"a b c d e" contains text {"a","e"} all window 4 words	false
"a b c d e" contains text {"a","e"} all window 5 words	true
"a b c d e" contains text {"a","e"} all window 1 sentences	true
"a b c! d e." contains text {"a","e"} all words same sentence	false
"a b c! d e." contains text {"a","e"} all words same paragraph	true
"a b c! d e." contains text {"b","a"} at start	true
"a b c! d e." contains text {"b","a"} at end	false
"a b c! d e." contains text {"a b c! d e."} entire content	true
"a b c! d e." contains text {"a","b","c","d","e"} entire content	false
"Hello World" contains text "world" using case sensitive	false
"cry" contains text "crying" using stemming	true
"a b c d e" contains text "a x c y e" using stop words("b","d","x","y")	true
"regular expression" contains text "r.?.+.*.{1,10}" using wildcards	true
'angry' contains text 'furious' using thesaurus default	true
'a <b>x y z</b> c' contains text 'a c' without content b	true

Boolean results of "contains text" expressions under various XQuery Full Text options.

A node-matching example: //p[.//text() contains text 'a c'] selects every p element whose text content matches the phrase.

score

The score clause of a for or let binds a relevance score, between 0 and 1, for how well an item matches a full text condition.

ch05-xqft-score.xq

for $x score $i in ("a","a b","a b c")
[. contains text "a"]

return $x || $i || '&#10;'

Output

a1
a b0.6
a b c0.4

ch05-xqft-letscore.xq

let score $s := "a b c d" contains text "a b"
return $s

Output

0.62011

MENU

XQFT

score