Facebook Spotlight - Facebook Mobile Hack Hong Kong
Download as PPTX, PDF
0 likes1,956 views
Facebook Spotlight is the hack that we built in the Facebook Mobile Hack Hong Kong 2012. The idea is to crawl facebook's data, index it and provide full text search. All these happen in javascript and made possible by HTML5 powerful features.
Facebook Spotlight - Facebook Mobile Hack Hong Kong
- 1. Search my social circle STANLEY . F STANLEY . N KEUNG . H STEVE . N Facebook Mobile Hack Hong Kong - 22nd March 2012
- 2. Current facebook search bar can only search people, pages, events, etc page 2
- 3. But can we search by user profile details? page 3
- 4. Can we search by the comments? page 4
- 5. Can we search by the event details? page 5
- 6. Build a search engine for my social circle! ONLY in JAVASCRIPT!
- 7. Our Idea Crawl data from my social circle provided by Facebook Graph API Build inverted index from the content page 7
- 8. Why Javascript? We love Javascript! To demonstrate how powerful Javascript can be For a facebook application, it should be more ethical to process users data in client side page 8
- 9. Benefits of Client Side Indexing Computation power and storage is now off- loaded to client side browsers facebook does not need to manage the huge amount of index data The search result is very personalized! You are searching for your own data and that is what you are interested in! page 9
- 10. Demo
- 12. Auto-complete
- 13. Query: baby search all objects containing baby
- 14. We are searching facebook objects based on the content!
- 15. Query: birthday @photos search all photos object containing birthday
- 16. Query: single @profile search all profile object containing single
- 17. Challenges Indexing performance Persistent storage for index Implementing inverted index algorithm page 17
- 18. Web Workers Web DB page 18
- 19. Indexing Performance For each user, we do a batch API call to Graph API already However, it still takes on average 3.8 seconds for the data to return Most of my friends has around 300 friends So it takes 19 minutes to complete the crawling!?!?!? page 19
- 20. Web Workers come to save me! Time to index 100 facebook users 400 350 300 Time (seconds) 250 200 150 100 50 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Number of web workers page 20
- 21. Web Workers Web Workers, forked by main thread Crawl data and build sub-index Main thread merge all sub-indices page 21
- 22. Web Workers Web workers has much limitation, e.g. you cannot access the DOM You also cannot import the facebook SDK or jQuery library. It DOES NOT WORK! Luckily, you can still use the most fundamental function - xmlhttprequest So we have to use xmlhttprequest to simulate the FB.api function in batch mode page 22
- 23. Web DB Web DB provides us a solution for permanent offline storage My friends list has 556 users Generating a search index having ~18,000 English keywords Only takes less than a second to save and load from the Web DB page 23
- 24. Problems still need to tackle Asian language (Chinese, Japanese, etc) tokenization Search relevancy improvement Need a more standardized query language page 24
- 25. Inverted Indexing concept page 25
- 26. Conclusion Client side full text search is made possible with HTML5s web workers and storage! How to put it forward? What if we are not using facebooks data? Index content of viewed websites by users? Index RSS feeds? How to play with the indexed data? page 26
- 27. Credits Javscript full text search engine from TheLadders.com iQuery Masonry Backbone.js page 27
- 28. Stanley Fok Stanley Ng Steve Ng Hon Happy hacking!