Index Activiti Data on Elasticsearch
2 likes2,607 views
The document discusses the integration of Activiti data into Elasticsearch to enhance analytics and reduce MySQL workload. It highlights lessons learned, such as the effectiveness of full-text search and the need for future improvements using technologies like Apache Spark. The presenters, Silvio dos Passos Neto and Mike Dias, emphasize the importance of eliminating the business-IT divide in data management.
![{
"?lter":{
"nested":{
"path":"variables",
"?lter":{
"bool":{
"must":[
{ “term": { "name":"passport" }},
{ “term": { "text":"1234" }}
]
}
}
}
}
}](https://image.slidesharecdn.com/activitielasticsearchv2-150611110031-lva1-app6892/85/Index-Activiti-Data-on-Elasticsearch-38-320.jpg)
Index Activiti Data on Elasticsearch
- 1. Index Activiti data on Elasticsearch Activiti User Day Paris 2015
- 2. Silvio dos Passos Neto CTO at iColabora @silvioneto
- 3. “Don’t ?bridge ?the ? business-?‐IT ?divide. ? Obliterate ?it!” ?(2003) Smith & Fingar
- 6. ?
- 7. ?
- 11. @mike_dias
- 13. ID_ NAME_ VALUE_* … ACT_HI_VARINST
- 14. User form Process Instance ID_ NAME_ VALUE_* … 1 client_name Jonh … 2 client_tel 123456 … 3 due_date 01/06/2015 … 4 demand_desc I have a problem… … ACT_HI_VARINST
- 15. User form Process Instance Process Instance User form ID_ NAME_ VALUE_* … 1 client_name Jonh … 2 client_tel 123456 … 3 due_date 01/06/2015 … 4 demand_desc I have a problem… … 5 client_name Bob … 6 client_tel 654321 … 7 due_date 10/06/2015 … 8 demand_desc My internet conn… … ACT_HI_VARINST
- 16. 85 ?elds x ~1000 Process per day = ~85.000 variables per day
- 17. ~15 million variables in 9 months
- 19. The Tool
- 21. Built on top of
- 22. Analytics
- 23. Distributed
- 25. Historic Data
- 27. P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P CPU 1 CPU 2 CPU 3 CPU 4 Process Lake
- 28. P P P CPU 1 V V V V V V V V Variables T T T Tasks P Process { }P V V V V V V V V T T T JSON REST API
- 29. Real-Time Data
- 30. E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E Engine Events E E E E E E E E E E E E
- 31. E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E Engine Events E E E E E E E E E E E E E EE E EE { } JSON REST API Listeners
- 33. Search
- 35. Search results
- 36. Compare
- 37. SELECT * FROM ACT_HI_VARINST WHERE NAME_ = 'passport' AND TEXT_ = '1234'
- 38. { "?lter":{ "nested":{ "path":"variables", "?lter":{ "bool":{ "must":[ { “term": { "name":"passport" }}, { “term": { "text":"1234" }} ] } } } } }
- 39. Response Time 0 secs 45 secs 90 secs 135 secs 180 secs MySQL Elasticsearch 0,08 secs 161 secs
- 40. Response Time 0 secs 45 secs 90 secs 135 secs 180 secs MySQL Elasticsearch 0,08 secs 161 secs CENSURED
- 41. Lessons learned
- 42. Full text search is a helpful feature
- 44. ES is great for analytics
- 45. Next steps
- 46. Apache Spark Lightning-Fast Cluster Computing
- 48. Open source