�ݺ�ߣ

Beauty of IR
Venkatesh Vinayakarao
An IR enthusiast!

Disclaimer
Most examples and discussions in this talk revolve
around well known search engines. This is just to get
a good learning experience. Please keep in mind
that IR is beyond search engines.
25+ slides of interesting discussion ahead… 
2/2014 Venkatesh Vinayakarao 2

Quiz
1. Explain any two challenges in Query Intent
Understanding using some examples and discuss
why is it a hard problem?
2. How are “Tiles” as discussed in the class used in
search engines? What purpose do they solve?
3. Search Engines have no UI related design
concerns. True/False?

About Me
BE Computer Science
(Y2K)
MS (IT)
IT Service Industry
Start Up
Nokia
Yahoo
Microsoft (Bing)
PhD
Let me learn
everything all
over again!

Our Agenda: The Beauty of IR!
Crawling Content Processing Indexing
Me!
Query (Intent)
Understanding
Ranking User Interface
Offline Horror!
Online Terror!
How to process
Korean queries for
local listings?

Crawling
 How frequently should we crawl?
 Fresh & Super-Fresh! How to crawl cricket scores? Are we
even crawling here?
 How to avoid 404 - Page not found?
 How much time did it take google to show your first
personal page?

Content Processing
 Good Read: https://getlisted.org/static/resources/local-search-data-providers.html

Content Processing
 Query: “Schools in Delhi”
 Answer: “Delhi Public School”
 Good or Bad?
 Query: “Schools in Hyderabad”
 Answer: “Delhi Public School”
 Good or Bad?
 Query: “Hotels in Bombay”
 Answer: “Grand Hyatt, Mumbai”
 Good or Bad?
 How to get same results for both Mumbai and Bombay?
 Query: “Maruti Car service in delhi”
 Answer: “Rana Motors Private Limited”.
 What happened?

Content Processing & Indexing
 A real example:
http://www.yelp.com/dataset_challenge/
Enriched Business
• Category Synonyms (for eg., auto service & car service are replaceable at times)
• User’s query forms (for eg., McDonalds is commonly queried as McD)

Derived Values & Indexing
 Given a location, how will you find all businesses
within 1km radius?
 Query: schools near govindpuri delhi

Query Understanding Challenge
Need a team of 3 people and one laptop.
Volunteers?

Rules
 I will give an entity name.
 You will have to frame at least three different
(dissimilar) queries (and as many as you can) that
give same document as the correct result at first
place.
 At the end, you should submit:
 Query, Max. no. of top n correct results that you
maintained to be same.
 You will have 5 minutes.

Questions
 Tom Cruise
 Aishwarya Rai
 Tom Hanks
 Srikanta Bedathur
 Venkatesh Vinayakarao
 Pankaj Jalote
 Amir Khan
 Andre Agassi
 Manmohan Singh

Query Understanding
 Query: Michael Jordon
 Which MJ to return? The basketball player or actor?
 Factors
 User profile
 Query context (session details, browser data, links, etc)
 …
 Query: Delhi School
 What does user want? “Delhi Public School” or
“Schools in Delhi” or “some Indian school in US”?
 Query: “IR”
 Predict top three results

Ok! I give up!!
 A frustrated search user: “please show me some t-
shirt brands”

More fun with auto completion

System Overview (Simplified)
Front-end Front-end Front-end Front-end
Query Understanding, Query Classifiers
Web Answer Local Answer
Finance
Answer
Tech Answer &
Many more
KB
Index Serve Crawled
Content
Crawler
Web
Expanded Query
User Query

Ranking & Relevance
 How do we know if the document is relevant (in
web search context)?
 Popularity of url
 Domain score (is it ac.in or .edu?)
 TF, IDF
 Entity, Chain entity?
 Trust Factor (Wikipedia?)
 Inlinks/Outlinks
 Position of query terms
 Sequence of query terms
 … and 1000 of such things

Are current search engines good at
relevance & ranking?
Bing Google
Query1: Vegetarian hotels in south delhi
Query2: South Indian hotels in south delhi

…More examples
Query3: South Indian restaurants in south delhi
What’s the difference between query2 and query3? Should search engines give different
results?

How far for a coffee?
Google: Just one word
(iiitd) missing. So
what?
Let’s make the query as “coffee shops
near iiitd delhi”.
“Coffee shops near me” gives results
from Janakpuri, Gurgaon, CP & Kamla
Nagar.

Why is it hard?
 What makes Ranking & Relevance hard?

User Interface
 Is UI important for search engine?
 Maps in local results
 Live sport score cards
 Finance tickers
 Filters
 Search Operators
 Entity Infoboxes
 What impact does these
make?

Our Agenda: The Beauty of IR!
Crawling Content Processing Indexing
Me!
Query (Intent)
Understanding
Ranking User Interface
Offline Horror!
Online Terror!
How to process
Korean queries for
local listings?

Evaluation
 Various evaluation methods
 Precision/Recall
 Mean Avg Precision
 Mean Reciprocal Rank
 If first relevant doc is at kth position, RR = 1/k.
 NDCG
 Non-Boolean/Graded relevance scores
 DCG = r1 + r2/log22 + r3/log23 + … rn/log2n

NDCG - Example
i
Ground Truth Ranking Function1 Ranking Function2
Document
Order
ri
Document
Order
ri
Document
Order
ri
1 d4 2 d3 2 d3 2
2 d3 2 d4 2 d2 1
3 d2 1 d2 1 d4 2
4 d1 0 d1 0 d1 0
NDCGGT=1.00 NDCGRF1=1.00 NDCGRF2=0.9203
4 documents: d1, d2, d3, d4
Taken from http://www.stanford.edu/class/cs276/handouts/EvaluationNew.ppt

Are we done?
 Q & A

�ݺ�ߣ

Beauty ofir

Recommended

More Related Content

Featured (20)

Beauty ofir