Genn.ai introduction for Buzzwords
3 likes825 views
We are making a realtime marketing system for general marketplace type web sites with Kafka + Strom. http://genn.ai/
![http://genn.ai/
Calculated values
?Recording behaviors
?Useful values for each sites
? Ex.)Hotel reservation site,
22
{
"_id": 154,
"largeAreaCntAcl": {
"Tokyo": 3,
"Berlin": 1
},
"largeAreaCntPre": {
"Tokyo": 4,
"Berlin": 3
},
"rateAvgAcl": 4200,
"rateAvgPre": 6200,
"rateCntAcl": [
4, 1, 0, 0, 0
],
"rateCntPre": [
4, 1, 2, 0, 0
],
"lastUpdate": "2013-04-16T04:04:37.432Z"
}](https://image.slidesharecdn.com/buzzwordskafka-edition-key-130615014148-phpapp01/85/Genn-ai-introduction-for-Buzzwords-22-320.jpg)
![http://genn.ai/
Calculated values
?Recording behaviors
?Useful values for each sites
? Ex.)Hotel reservation site,
? Where is the favorite place?
23
{
"_id": 154,
"largeAreaCntAcl": {
"Tokyo": 3,
"Berlin": 1
},
"largeAreaCntPre": {
"Tokyo": 4,
"Berlin": 3
},
"rateAvgAcl": 4200,
"rateAvgPre": 6200,
"rateCntAcl": [
4, 1, 0, 0, 0
],
"rateCntPre": [
4, 1, 2, 0, 0
],
"lastUpdate": "2013-04-16T04:04:37.432Z"
}](https://image.slidesharecdn.com/buzzwordskafka-edition-key-130615014148-phpapp01/85/Genn-ai-introduction-for-Buzzwords-23-320.jpg)
![http://genn.ai/
Calculated values
?Recording behaviors
?Useful values for each sites
? Ex.)Hotel reservation site,
? Where is the favorite place?
? Where is the this searching place?
24
{
"_id": 154,
"largeAreaCntAcl": {
"Tokyo": 3,
"Berlin": 1
},
"largeAreaCntPre": {
"Tokyo": 4,
"Berlin": 3
},
"rateAvgAcl": 4200,
"rateAvgPre": 6200,
"rateCntAcl": [
4, 1, 0, 0, 0
],
"rateCntPre": [
4, 1, 2, 0, 0
],
"lastUpdate": "2013-04-16T04:04:37.432Z"
}](https://image.slidesharecdn.com/buzzwordskafka-edition-key-130615014148-phpapp01/85/Genn-ai-introduction-for-Buzzwords-24-320.jpg)
![http://genn.ai/
Calculated values
?Recording behaviors
?Useful values for each sites
? Ex.)Hotel reservation site,
? Where is the favorite place?
? Where is the this searching place?
? What price range has he booked?
25
{
"_id": 154,
"largeAreaCntAcl": {
"Tokyo": 3,
"Berlin": 1
},
"largeAreaCntPre": {
"Tokyo": 4,
"Berlin": 3
},
"rateAvgAcl": 4200,
"rateAvgPre": 6200,
"rateCntAcl": [
4, 1, 0, 0, 0
],
"rateCntPre": [
4, 1, 2, 0, 0
],
"lastUpdate": "2013-04-16T04:04:37.432Z"
}
? etc..](https://image.slidesharecdn.com/buzzwordskafka-edition-key-130615014148-phpapp01/85/Genn-ai-introduction-for-Buzzwords-25-320.jpg)
![http://genn.ai/
Counting the frequency
?This function is under the e?ect of the Time
shift function.
35
{
"_id": 154,
"largeAreaCntAcl": {
"Tokyo": 3,
"Berlin": 1
},
"largeAreaCntPre": {
"Tokyo": 4,
"Berlin": 3
},
"rateAvgAcl": 4200,
"rateAvgPre": 6200,
"rateCntAcl": [
4, 1, 0, 0, 0
],
"rateCntPre": [
4, 1, 2, 0, 0
],
"lastUpdate": "2013-04-16T04:04:37.432Z"
}](https://image.slidesharecdn.com/buzzwordskafka-edition-key-130615014148-phpapp01/85/Genn-ai-introduction-for-Buzzwords-35-320.jpg)
![[@IndeedEng] Logrepo: Enabling Data-Driven Decisions](https://cdn.slidesharecdn.com/ss_thumbnails/indeedenglogrepo-140210123935-phpapp01-thumbnail.jpg?width=560&fit=bounds)
More Related Content
Similar to Genn.ai introduction for Buzzwords (20)
Recently uploaded (20)
Genn.ai introduction for Buzzwords
- 1. Genn.ai (Ô´ÄÚ) Attached realtime marketing platform 1
- 2. http://genn.ai/ Who am I? ?Takeshi NAKANO ?Senior Researcher / Architect. ?Co-authered Getting started with Solr in Japanese. ?Co-authered Hadoop Hacks in Japanese. 2 ?Writing Getting started with Kafka in Japanese.
- 3. http://genn.ai/ 3 Company Name Recruit Holdings Co., Ltd Founded March 31, 1960 (incorporated: August 26, 1963£© ¡ùgovernance changed from Recruit Co., Ltd since October 1st, 2012 Headquarters Tokyo, Japan Number of Group Employees 22,368 £¨as of the end of March,2012£© Number of Group Companies 102 £¨Consolidated Subsidiaries/as of the end of June,2012£© Capital 3,002.64 million yen £¨since March 1st, 1995£© Consolidated Sales 806,661 million yen £¨from April 1st 2011 to March 31st 2012£© Consolidated Ordinary Income 117,617 million yen £¨ from April 1st 2011 to March 31st 2012)
- 4. http://genn.ai/ 4 Recruit Holdings Headquarters Function Global Business Business R&D Recruit Career Co., Ltd Recruit Jobs Co., Ltd Recruit Sumai Company Ltd Recruit Marketing Partners Co., Ltd Recruit Lifestyle Co., Ltd Recruit Administration Co., Ltd Recruit Technologies Co., Ltd Recruit Communications Co., Ltd Operating Companies Domain Recruit Staf?ng Co., Ltd STAFF SERVICE HOLDINGS CO., LTD
- 5. http://genn.ai/ What is our challenge? ?Clients can get the information about their customers checking their items on our web sites. Providing a printed report monthly. ????¨Œ Send them more detailed report in real time. ?In real time fashion, the value of the information will be dramatically increased! 5
- 6. http://genn.ai/ Agenda ? What is genn.ai? ? Overview ? How to gather clicks ? 0) Injecting our javascripts 6 ? Genn.ai core details ? Structure ? 1) Setting up ?lters ? 2) Filtering customers ? 3) Visualizing them ? Future ? Dealing with historical streams ? Genn.ai wants friends.
- 7. http://genn.ai/ Overview of using Genn.ai 7 Customer A Customer B Customer C Realtime Analysis Platform Genn.ai ¤¸¤ã¤é¤ó Web Sites Client (Hotels) Marketer Analyst
- 8. http://genn.ai/ Demo videos! ?They are based on a hotel reservation site like Expedia.com. ?An animation and a screencapture what the marketting sta? see using Genn.ai. At this moment .. ?What prefectures are all my users searching for? (0:26) ?Comparing the number of searches in today s and yesterday s regarding places. (2:50) 8
- 9. http://genn.ai/ Overview of using Genn.ai 9 Customer A Customer B Customer C Realtime Analysis Platform Genn.ai ¤¸¤ã¤é¤ó Web Sites Client (Hotels) Marketer Analyst
- 10. http://genn.ai/ Overview of using Genn.ai 10 Customer A Customer B Customer C ¤¸¤ã¤é¤ó Web Sites Realtime Analysis Platform Genn.ai Client (Hotels) Marketer Analyst
- 11. http://genn.ai/ How to gain access to Genn.ai 11 Customer A Customer B Customer C App (ex.Tomcat) Apache ?Genn.ai will be able to be attached!
- 12. http://genn.ai/ How to gain access to Genn.ai ?A custom apache module for embeding our code on returning html sources from App. 12 Customer B Apache ?Genn.ai will be able to be attached! mod Genn.ai App (ex.Tomcat)
- 13. http://genn.ai/ How to gain access to Genn.ai 13 ?The embedded values and js on a html.
- 15. http://genn.ai/ Steps for getting results ?There are 3 steps! (after getting clicks) 15 1 2 3 Pouring clicks into Kafka Recording them as behaviors Visualizing them 0 gather clicks
- 16. Backend for Genn.ai http://genn.ai/ Structure (Pouring into Kafka) 16 Storm Topology (Filter) Kafka kakan.puTomcat iPad (APP) MongoDB 321 Genn.ai Front (Based on Finagle) Query genn.ai/web)Click Streams Frontend for Genn.ai Visualizer for Genn.ai Zoomdata
- 17. Backend for Genn.ai http://genn.ai/ Structure (Recording as behaviors) 17 Storm Topology (Filter) Kafka kakan.puTomcat iPad (APP) MongoDB Genn.ai Front (Based on Finagle) Query genn.ai/web)Click Streams 321 Visualizer for Genn.ai Frontend for Genn.ai Zoomdata
- 18. Backend for Genn.ai Visualizer for Genn.ai http://genn.ai/ Structure (Visualizing customers) 18 Storm Topology (Filter) Kafka kakan.puTomcat iPad (APP) MongoDB Genn.ai Front (Based on Finagle) Query genn.ai/web)Click Streams 321 Frontend for Genn.ai Zoomdata
- 19. http://genn.ai/ How is Genn.ai composed? ?Genn.ai Frontend! : Gathering clicks. ?Genn.ai Backend! : Recording behaviors. ?Genn.ai Visualizer! : Visualizing customers¡¯ attrs. ¡ü ?Kafka : Used for message bus ?MongoDB : Storing Additional tables ?Storm : Processing click streams 19
- 20. http://genn.ai/ ?Details of recording behaviors. 20
- 21. Backend for Genn.ai http://genn.ai/ Structure (Recording behaviors) 21 Storm Topology (Filter) Kafka kakan.puTomcat iPad (APP) MongoDB Genn.ai Front (Based on Finagle) Query genn.ai/web)Click Streams 321 Visualizer for Genn.ai Zoomdata Frontend for Genn.ai
- 22. http://genn.ai/ Calculated values ?Recording behaviors ?Useful values for each sites ? Ex.)Hotel reservation site, 22 { "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" }
- 23. http://genn.ai/ Calculated values ?Recording behaviors ?Useful values for each sites ? Ex.)Hotel reservation site, ? Where is the favorite place? 23 { "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" }
- 24. http://genn.ai/ Calculated values ?Recording behaviors ?Useful values for each sites ? Ex.)Hotel reservation site, ? Where is the favorite place? ? Where is the this searching place? 24 { "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" }
- 25. http://genn.ai/ Calculated values ?Recording behaviors ?Useful values for each sites ? Ex.)Hotel reservation site, ? Where is the favorite place? ? Where is the this searching place? ? What price range has he booked? 25 { "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" } ? etc..
- 26. http://genn.ai/ Functions for recording behaviors ?Time shift function ?Time sensitive counter (old item will be expired and deleted) ?Facet ( like Solr ) ?Categorized counter ?Range Facet ( like Solr ) ?Dynamic categorized counter 26 321
- 27. http://genn.ai/ Functions for recording behaviors ?Time shift function ?Time sensitive counter (old item will be expired and deleted) ?Facet ( like Solr ) ?Categorized counter ?Range Facet ( like Solr ) ?Dynamic categorized counter 27 321
- 28. http://genn.ai/ Time Shift function? ?We make 3 min time window and now start. ( no clicks here ) 28 t Now
- 29. http://genn.ai/ Time Shift function? ?In our 3 min time window, 1.5 min are captured. 2 clicks have been recorded. 29 1.5 mins Now Time window ¡ø Click ¡ø Click 2 clicks are captured. t
- 30. http://genn.ai/ Time Shift function? ?There are 3 clicks in the past. ?The length of the capturing window becomes max. 30 3 mins Now Time window ¡ø Click ¡ø Click ¡ø Click t
- 31. http://genn.ai/ Time Shift function? ?There are 4 clicks in the past. ?Ignore the ?rst 2 clicks 31 ¡ø Click ¡ø Click Now Time window ¡ø Click ¡ø Click 1.5 min 3 mins t
- 32. http://genn.ai/ Time Shift function? ?There are 4 clicks in the past. ?Ignore the ?rst 3 clicks 32 ¡ø Click ¡ø Click Now Time window ¡ø Click ¡ø Click 3 mins3 mins t
- 33. http://genn.ai/ Functions for recording behaviors ?Time shift function ?Time sensitive counter (old item will be expired and deleted) ?Facet ( like Solr ) ?Categorized counter ?Range Facet ( like Solr ) ?Dynamic categorized counter 33 321
- 34. http://genn.ai/ Functions for recording behaviors ?Time shift function ?Time sensitive counter (old item will be expired and deleted) ?Facet ( like Solr ) ?Categorized counter ?Range Facet ( like Solr ) ?Dynamic categorized counter 34 321
- 35. http://genn.ai/ Counting the frequency ?This function is under the e?ect of the Time shift function. 35 { "_id": 154, "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 }, "largeAreaCntPre": { "Tokyo": 4, "Berlin": 3 }, "rateAvgAcl": 4200, "rateAvgPre": 6200, "rateCntAcl": [ 4, 1, 0, 0, 0 ], "rateCntPre": [ 4, 1, 2, 0, 0 ], "lastUpdate": "2013-04-16T04:04:37.432Z" }
- 36. http://genn.ai/ Counting the frequency ?This function is under the e?ect of the Time shift function. ?You have booked 1 stay in Berlin and 3 stays in Tokyo in this 3 weeks. 36 ¡ø L.A. ¡ø S.F. t Now Time window ¡ø Berlin ¡ø Tokyo 1.5 week 3 weeks ¡ø Tokyo "largeAreaCntAcl": { "Tokyo": 3, "Berlin": 1 } ¡ø Tokyo
- 38. Backend for Genn.ai http://genn.ai/ Visualizing the customers 38 Storm Topology (Filter) Kafka kakan.puTomcat iPad (APP) MongoDB Genn.ai Front (Based on Finagle) Query genn.ai/web)Click Streams 321 Visualizer for Genn.ai
- 39. Backend for Genn.ai http://genn.ai/ Visualizing the customers 39 Storm Topology (Filter) Kafka kakan.puTomcat iPad (APP) MongoDB Genn.ai Front (Based on Finagle) Query genn.ai/web)Click Streams 321 Visualizer for Genn.ai
- 40. http://genn.ai/ Kafka consumer ?Kakan.pu ?The program for relaying clicks from Kafka to the other systems. ?The enriching function (joining the MongoDB data) ?The converting function (for following systems) 40
- 41. Backend for Genn.ai http://genn.ai/ Visualizing the customers 41 Storm Topology (Filter) Kafka kakan.puTomcat iPad (APP) MongoDB Genn.ai Front (Based on Finagle) Query genn.ai/web)Click Streams 321 Visualizer for Genn.ai Zoomdata
- 42. http://genn.ai/ Visualizing tool ?Zoomdata ?A product containing a set of server app and ipad app for visualizing event data. 42 321
- 43. http://genn.ai/ Zoomdata ?Zoomdata has the functions like.. ?WebAPI for streaming data. ?Own event processor for vizualizing. 43 321 ?The bar and bubble graph on default setting. ?Customizable visualizing templates.
- 45. http://genn.ai/ Genn.ai Evolution ?Adding functions on Genn.ai core. ?Genn.ai should have ?ltering / aggrigation functions for federating with other systems. ?Historical rewinding function will be implemented with the Hadoop cluster. 45
- 46. http://genn.ai/ Future plan ?Genn.ai wants friends! ? We are looking for co-challenging partners for tuning and improving Genn.ai. ? You can use Genn.ai on your web sites for totally free (infrastructure etc..) if we could share the dream. ? Genn.ai codes will be opensourced in the future. ?Please feel free to contact us! ? http://genn.ai/ 46
- 48. 48