Apache Zeppelin ??
- 1. Introduction to Zeppelin Lee moon soo, NFLabs moon@apache.org
- 2. me ? Lee moon soo, aka. ‘moon’ ? moon@apache.org Committer and PPMC of Zeppelin ? moon@n?abs.com Co-founder of NFLabs
- 3. Agenda ? Why do you like Zeppelin? ? Why your project likes Zeppelin? ? Roadmap ? QnA
- 7. What gets measured, gets managed - Peter Drucker A price of light is less than the cost of darkness - Arthur C. Nielsen , Founder of ACNielsen War is ninety percent information - Napoleon Bonaparate In God we trust, all others must bring data - W. Edwards Deming Data = Understanding
- 8. For data analysis, we need tool
- 9. But couldn’t ?nd one i like ImpalaDrill Hive tajoPig Cloudera-ML MLLib MRQL
- 10. Decided to make one Really good one
- 11. Good analytics environment Analytical language Many Libraries Interactive Visualization Sharing
- 12. The First attempt 2012~2013
- 13. It’s got graphic REPL, deployment, search, import tool But failed, because
- 14. It wasn’t widely used It wasn’t opensource
- 15. Second attempt 2013~2014 Opensourced graphic REPL from commercial product The ?rst version of Zeppelin
- 16. Second attempt 2013~2014 Not widely used It was slow, dif?cult to use, …
- 17. Third attempt 2014~ After few weeks of study, decided to rewrite Zeppelin Graphic REPl -> Notebook with Apache Spark integration
- 18. Third attempt 2014~ Next week, beautifulized
- 19. Why do you like Zeppelin?
- 20. Web Based web framework d3.js Visualization Language Package management bower Build
- 21. Notebook Code Result Code Result ??? ?? ????? ??? ??? ???? ???? ???? ??? Notebook
- 23. Pivot Pivot
- 27. Why do your project likes Zeppelin?
- 29. Zeppelin Interpreter Architecture Classloader InterpreterGroup Interpreter Interpreter Server Client … HTTP Rest / Websocket Classloader InterpreterGroup Spark SparkSQL Dep
- 30. Classloader InterpreterGroup Interpreter Interpreter Server Client … HTTP Rest / Websocket InterpreterGroup Interpreter Interpreter … Seprate JVM process Thrift Zeppelin Interpreter Architecture
- 31. public abstract void open(); public abstract void close(); public abstract InterpreterResult interpret(String st, InterpreterContext context); public abstract void cancel(InterpreterContext context); public abstract int getProgress(InterpreterContext context); public abstract List<String> completion(String buf, int cursor); public abstract FormType getFormType(); public Scheduler getScheduler(); Implementing new Interpreter Must have Good to have More controls
- 32. Roadmap ? Integration with more distributed processing framework ? Flink, Ignite, Tajo, etc.. ? Output message streaming ? Ability to create rich GUI
- 34. QnA