Apache Lucene Informacijos paieška
•
0 likes•628 views
Lucene Informacijos paieška
![Sukaposiu gabalais
(tokenization)
Dokumentas #1:
"The quick brown fox jumps over the lazy dog.”
Word the quick brown fox
Documents 1 1 1 1
Offsets 1[0] 1[4] 1[10] 1[16]](https://image.slidesharecdn.com/lucenepresentation-vjug-140428001228-phpapp02/85/Apache-Lucene-Informacijos-paieska-9-320.jpg)
![Sukaposiu gabalais
(tokenization) #2
Dokumentas #2:
"I saw a brown fox yesterday. It ran away quickly.”
Word the quick brown fox
Documents 1 1 1, 2 1, 2
Offsets 1[0] 1[4] 1[10], 2[8] 1[16], 2[13]](https://image.slidesharecdn.com/lucenepresentation-vjug-140428001228-phpapp02/85/Apache-Lucene-Informacijos-paieska-10-320.jpg)
![Užklausos
● field: foo bar
● field: +foo -bar
● field: “foo bar”
● field: +“foo bar” AND blah
● field: f?o bar*
● field: foo~ bar~0.8
● date_field: [2000 TO 2001]
● field: (foo AND bar) OR bob
http://searchhub.org/dev/2011/12/28/why-not-and-or-and-not/](https://image.slidesharecdn.com/lucenepresentation-vjug-140428001228-phpapp02/85/Apache-Lucene-Informacijos-paieska-22-320.jpg)
Apache Lucene Informacijos paieška
- 1. Informacijos paieška Trumpas įvadas į Apache Lucene Mindaugas Žakšauskas Vilnius Kaunas Java User Group
- 2. assert this != null
- 3. Kalbėsiu apie ● Probleminę sritį ● Informacijos paieškos teoriją ● Lucene ● Solr (jei liks laiko) ● ?
- 4. Who are we?
- 9. Sukaposiu gabalais (tokenization) Dokumentas #1: "The quick brown fox jumps over the lazy dog.” Word the quick brown fox Documents 1 1 1 1 Offsets 1[0] 1[4] 1[10] 1[16]
- 10. Sukaposiu gabalais (tokenization) #2 Dokumentas #2: "I saw a brown fox yesterday. It ran away quickly.” Word the quick brown fox Documents 1 1 1, 2 1, 2 Offsets 1[0] 1[4] 1[10], 2[8] 1[16], 2[13]
- 11. Stopwords Anglų: - artikeliai a, the Lietuvių: - prielinksniai: į, nuo - ištiktukai: oi! - ?
- 13. Quick| ⇒ quick quick|ly ⇒ quick Stemming, lowercasing
- 14. Apache Lucene ● ACID (+2 phase commit) ● NoSQL (rimtai!) ● Concurrency ● Java (.NET, Python, Ruby) ● Bendruomenė ● Plačiai naudojama
- 15. Ne caro laikų Lucene! Indeksavimo greitis, Lucene v4
- 17. Sinonimai ● Vanduo - H2 O ● Reikia spec. žodyno (SynonymMap) quick brown fox ⇒ "quick", "fast", "brown", "fox"
- 18. Fonetinis kodavimas http://en.wikipedia.org/wiki/Metaphone 0BFHJKLMNPRSTWXY Stephen Smith ⇒STFN SM0
- 22. Užklausos ● field: foo bar ● field: +foo -bar ● field: “foo bar” ● field: +“foo bar” AND blah ● field: f?o bar* ● field: foo~ bar~0.8 ● date_field: [2000 TO 2001] ● field: (foo AND bar) OR bob http://searchhub.org/dev/2011/12/28/why-not-and-or-and-not/
- 24. Finite state transducer mop, moth, pop, star, stop, top 10 million Wikipedia index - 69Mb