Next Generation Cassandra
1 like824 views
Acunu is developing an enterprise Cassandra appliance called Castle that aims to simplify Cassandra deployment and management. Castle includes a storage engine optimized for large disks and workloads, and allows for high density on commodity hardware. It also features fast disk rebuilds through its shared memory architecture. Acunu provides a web UI called the Control Center to configure, monitor, and troubleshoot Castle without deep Cassandra expertise. Acunu performs extensive automated testing of Castle to ensure reliability.
Next Generation Cassandra
- 1. Next Generation Cassandra Tom Wilkie Founder & VP Engineering, Acunu
- 3. Cassandra needs a team Installing, con鍖guring and deploying Automating management operations Monitoring and alerting $ $ $ $ Testing and 鍖xing bugs Tuning the garbage collector Tweaking the storage engine 3
- 4. Acunu: your enterprise Cassandra Appliance Data driven applications Web UI Apache Cassandra Control Center Acunu Storage Engine Con鍖gured and tuned OS Commodity Hardware 4
- 5. Storage Engine aka Castle
- 6. Castle: Built for Big Data Storage engine optimized for large slow disks, many cores, Big Data workloads Enterprise density on commodity hardware Lightning disk rebuilds:10x faster than RAID Shared memory interface Castle keys Userspace Acunu Kernel userspace interface values In-kernel async, shared memory ring workloads shared buffers kernelspace Streaming interface interface range key buffered key buffered queries insert value insert get value get Doubling Arrays Opensource (GPLv2, MIT doubling array mapping layer for user libraries) insert Bloom 鍖lters queues key get arrays x range arrays queries management http://goo.gl/gzihe key insert merges http://bitbucket.org/acunu Arrays mapping layer modlist btree key Version tree Loadable Kernel Module, insert btree key get btree targeting CentOSs 2.6.18 range queries value arrays 6 Cache apping & http://www.acunu.com/ ng layer "Extent" layer prefetc extent block extent cache
- 8. Rebuild time 5 4 Rebuild Time (Hours) 3 2 1 0 RAID10, 8 Disks RAID5, 8 Disks RDA, 8 Disks RDA, 15 Disks 8
- 10. Control Center The new UI has been critical in helping us work out what is wrong in our code The new UI has been critical in helping us work out what is wrong in our code - Matt Miller, TellyBug
- 11. Breaking System
- 12. 13,000 tests per day 1.1 million tests per release 3.5k machine days of testing for each release 12
- 13. Short, unit tests Distributed tests Data integrity tests Tests for common, long running operations (bootstrap, decommission, backup, restore etc) Failure simulation Resource usage tests (checking memory and cpu consumption) Long running performance tests 13
- 18. Turnkey Cassandra No need to master the internals No tuning required No need for a committer on staff