Making Cassandra more capable, faster, and more reliable (at ApacheCon@Home 2...Scalar, Inc.
?
The document presents advancements in making Cassandra more efficient and reliable through scalar DB, which adds ACID transaction support without modifying Cassandra's core. It discusses new methods for commit log synchronization to enhance durability and performance, particularly with the introduction of a group commit log sync mode. The performance and scalability of scalar DB on Cassandra are benchmarked, highlighting significant improvements in transaction management and reliability verified through extensive testing.
29回勉強会資料「PostgreSQLのリカバリ超入門」
See also http://www.interdb.jp/pgsql (Coming soon!)
初心者向け。PostgreSQLのWAL、CHECKPOINT、 オンラインバックアップの仕組み解説。
これを見たら、次は→ http://www.slideshare.net/satock/29shikumi-backup
In the first half, we give an introduction to modern serialization systems, Protocol Buffers, Apache Thrift and Apache Avro. Which one does meet your needs?
In the second half, we show an example of data ingestion system architecture using Apache Avro.
Scalar DB: Universal Transaction ManagerScalar, Inc.
?
Scalar DB is a universal transaction manager that provides database-agnostic ACID transactions across various databases such as Cassandra, DynamoDB, and PostgreSQL, enabling seamless database migration and enhanced transaction capabilities. It operates using a two-phase commit protocol and offers flexibility with a non-invasive, scalable architecture, while also facing challenges like slower performance and lack of native SQL support. The system has been extensively tested for consistency and scalability, with future expansions planned for additional interfaces and database adaptors.
The document details the updates and architecture of Contrail 3.x, including its integration with virtual and physical networks, health check mechanisms, and network management. Discussed are the features such as load balancing, policy enforcement, and orchestration for VMs, containers, and bare metal services. The content also covers best practices for connecting private cloud setups to co-location solutions and AWS through secure tunneling methods.
Making Cassandra more capable, faster, and more reliable (at ApacheCon@Home 2...Scalar, Inc.
?
The document presents advancements in making Cassandra more efficient and reliable through scalar DB, which adds ACID transaction support without modifying Cassandra's core. It discusses new methods for commit log synchronization to enhance durability and performance, particularly with the introduction of a group commit log sync mode. The performance and scalability of scalar DB on Cassandra are benchmarked, highlighting significant improvements in transaction management and reliability verified through extensive testing.
29回勉強会資料「PostgreSQLのリカバリ超入門」
See also http://www.interdb.jp/pgsql (Coming soon!)
初心者向け。PostgreSQLのWAL、CHECKPOINT、 オンラインバックアップの仕組み解説。
これを見たら、次は→ http://www.slideshare.net/satock/29shikumi-backup
In the first half, we give an introduction to modern serialization systems, Protocol Buffers, Apache Thrift and Apache Avro. Which one does meet your needs?
In the second half, we show an example of data ingestion system architecture using Apache Avro.
Scalar DB: Universal Transaction ManagerScalar, Inc.
?
Scalar DB is a universal transaction manager that provides database-agnostic ACID transactions across various databases such as Cassandra, DynamoDB, and PostgreSQL, enabling seamless database migration and enhanced transaction capabilities. It operates using a two-phase commit protocol and offers flexibility with a non-invasive, scalable architecture, while also facing challenges like slower performance and lack of native SQL support. The system has been extensively tested for consistency and scalability, with future expansions planned for additional interfaces and database adaptors.
The document details the updates and architecture of Contrail 3.x, including its integration with virtual and physical networks, health check mechanisms, and network management. Discussed are the features such as load balancing, policy enforcement, and orchestration for VMs, containers, and bare metal services. The content also covers best practices for connecting private cloud setups to co-location solutions and AWS through secure tunneling methods.
Jubatus is a distributed online machine learning framework that is distributed, fault tolerant, and allows for fixed time computation. It combines a machine learning model with a feature extractor. Jubatus uses a shared-everything architecture that allows it to be fast and fault tolerant. The architecture allows clients to access Jubatus through a single RPC interface even as the number of Jubatus servers scales out dynamically. Jubatus supports various machine learning algorithms including classification, recommendation, anomaly detection, clustering, and regression.
Jubatus is an open source machine learning framework that allows for distributed, online machine learning. It features algorithms like classification, recommendation, anomaly detection, and clustering. The architecture uses a feature extractor to transform data into feature vectors which are then used to train machine learning models. Models are combined with feature extractors and accessed via client libraries using an RPC interface, enabling applications in languages like Ruby, Python, Perl, and JavaScript.
The document describes SkipGraph, a distributed hash table based on SkipLists. SkipGraph uses a SkipList data structure to store key-value pairs and distribute them across nodes through consistent hashing of keys to nodes. SkipGraph assigns each node a membership vector that describes which keys it stores, allowing efficient lookups of O(log n) by routing queries through nodes with overlapping membership vectors.
21. Early Lock Release
Group Commit等で、何をCommitするかという内容を決めてディスクに書き出す順序まで確定(pre-
commit)した後なら、ロックを手放してしまってもトランザクションの性質は変わらないよという提案
仮に手放したロックを握った別のトランザクションがやってきても、手放した側のトランザクションを追い抜く事はな
い（ディスクに書き出す順序は既に確定したので）
ディスク動作を待つ間にデータのロックを手放して良いので、より多くのトランザクションがロックを握って進行でき
るようになる
詳細な証明は結構ゴツいので論文参照
実地でどの程度使われてるかは未調査。PostgreSQLは未実装らしい。
Unlockcommit
Disk Write
Unlockcommit
Disk Write
