Prisoner's Dilemma and Service-oriented Architectures
Download as PPTX, PDF
2 likes573 views
Moving to a service-oriented architecture solves problems like fate sharing, coupling, and scaling. It also brings the complexity of a partially up/partially down system, with nodes regularly getting sick or otherwise partitioned. Learn what the Prisoners Dilemma teaches us about building a partition tolerant system. Hear how the CAP theorem (consistency, availability, and partition tolerance) helps the architect make intelligent trade-offs. See real examples from Google, Amazon, and others of building highly available distributed systems.
Prisoner's Dilemma and Service-oriented Architectures
- 1. The Prisoners Dilemma & SOAs Lessons from Amazon, Google, and Lucidchart By Derrick Isaacson
- 7. Can I get that without the bacon? - no one ever http://www.food.com/photo-finder/all/bacon?photog=1072593
- 9. https://qzprod.files.wordpress.com/2013/03/costco-retail-web.jpg?w=1600 Wow, that was a cheap trip to Costco - no one ever http://www.someecards.com/usercards/viewcard/MjAxMi03YWZiMjJiMTg3NDFhYTUy
- 10. I cant remember if that getter function takes 100ns or 100ms. - no one ever Should I try to abstract away this service request as a remote procedure call? 6 orders of magnitude difference!
- 11. My front-side bus only fails for 1 second every 17 minutes! - no one ever 99.9% availability
- 12. Our internet only supports .NET. - no one ever Do your clients rely on an SDK?
- 13. Distributed System Architectures Does it have to be Service-oriented?
- 17. Distributed File System mount -t nfs -o proto=tcp,port=2049 nfs-server:/ /mnt
- 18. Distributed Data Stores Replicated MySQL Mongo S3 RDS BigTable Cassandra
- 19. P2P
- 20. Streaming Media
- 21. Service-oriented Architecture Attempt Social Bookmarking App
- 23. GET /profiles/123 GET /users/123 Calculate something GET /users/123/permissions If user cant view profile send 403 POST /eventFeed {new profile view} GET /users/123/friends GET /bookmarks?userId=123 GET /catalog/books?ids=1,3,10 Calculate something else GET /bookmarks/trending Send HTML
- 24. Failure calling a service?
- 25. Simple SOA Availability <98.7% 99.5% 99.8% 99.6% .995 * .998 * .998 * .996 = 0.987
- 26. Early days Lucidchart by Status Code 96.5% 2xx or 3xx
- 27. Early days Lucidchart 1 second Latencies 10.8% > 1s
- 28. What Happened?!? How can a SOA make my app better!
- 29. "A distributed system is at best a necessary evil, evil because of the extra complexity... or perhaps better put, a sensible engineering decision given the trade-offs involved." -David Cheriton, Distributed Systems Lecture Notes, ch. 1
- 32. The CAP Theorem1 Safety nothing bad ever happens Liveness good things happen Unreliability network dis-connectivity, crash failures, message loss, Byzantine failures, slowdown, etc. Consistency every response sent to a client is correct Availability every request gets a response Partition tolerance operating in the face of arbitrary failures
- 35. Assumption: Failures Happen Availability Consistency
- 36. GET /profiles/123 GET /users/123 Calculate something GET /users/123/permissions If user cant view profile send 403 POST /eventFeed {new profile view} GET /users/123/friends GET /bookmarks?userId=123 GET /catalog/books?ids=1,3,10 Calculate something else GET /bookmarks/trending Send response
- 37. ResponseHandler<User> handler = new ResponseHandler<User>() { public User handleResponse( final HttpResponse response) { int status = response.getStatusLine().getStatusCode(); if (status >= 200 && status < 300) { HttpEntity entity = response.getEntity(); return entity != null ? Parser.parse(entity) : null; } else { } } }; HttpGet userGet = new HttpGet("http://example.com/users/123"); User user = httpclient.execute(userGet, handler); https://hc.apache.org/httpcomponents-client-4.3.x/examples.html Works great to calculate a user!
- 38. Best Effort Availability - Guaranteed consistency
- 39. Best Effort Consistency - Guaranteed availability
- 41. WOW I really regret sacrificing consistency for availability -said no amazon ever Thats $74 Billion
- 42. Google File System: relaxed consistency model Throughput Latency
- 43. Hang Consistency! Add: Caching Timeouts Retries Guessing Anything!
- 44. Anti-entropy Added energy to combat failure
- 46. Tip 2: HTTP Caching as Fallback
- 47. Tip 3: Retries Exponential backoffs & max retries
- 48. Tip 3: HTTP Caching Technologies Apache HttpComponents HttpClient Cache Ehcache Redis Memcached CloudFront Akamai Berkeley DB AWS SNS (for notifying caches components of changes)
- 49. Segmenting CAP A is highly available and B is highly consistent
- 50. Segmenting Consistency and Availability 1. Data Partitioning Shopping Cart Warehouse Inventory DB
- 52. Segmenting 3. Functional partitioning User Service, Document Snapshots Document Service
- 54. http://www.slashgear.com/google-data-center-hd-photos-hit-where-the-internet-lives-gallery-17252451/
- 55. Data Driven Design 1 Timeouts & Retries
- 56. Final Tip: Data driven design
- 57. Max I/O wait time = # of threads * (CONNECT_TIMEOUT + READ_TIMEOUT) 9 front end servers received 1900 requests in 60 seconds and 300 for Flickr resources (16%). 35 requests per server per minute Max 100 threads, => 6,000 thread seconds in one minute Goal: ensure < 10% of thread seconds spent blocked on Flickr I/O 600 < 35 requests * (CONNECT_TIMEOUT + READ_TIMEOUT) CONNECT_TIMEOUT + READ_TIMEOUT < 17 seconds TCP Connect Send Request Block on socket read Read response CONNECT_TIMEOU T READ_TIMEOUT
- 58. Data Driven Design 2 Caching
- 61. Best Effort Consistency System 99.9% 99.5% 99.8% 99.6%
- 62. Wow, my pizza has too many toppings - no one ever http://upload.wikimedia.org/wikipedia/commons/6/60/Pizza_Hut_Meat_Lover's_pizza_3.JPG
- 63. WOW My system has too much availability. -no one ever
- 65. References 1. Perspectives on the CAP Theorem 2. Bacon Ipsum 3. Brewers Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web 4. The Google File System 5. Big Table 6. Amazon Architecture References 7. Apache HttpComponents 8. Apache HttpClient Cache 9. Ehcache
Editor's Notes
- #2: Effective SOA: Lessons from Amazon, Google, and Lucidchart It has been observed that "A distributed system is at best a necessary evil, evil because of the extra complexity." Multiple nodes computing on inconsistent state with regular communication failures present entirely different challenges than those computer science students face in the classroom writing DFS algorithms. The past 30 years have seen some interesting theories and architectures to deal with these complexities in what we now call "cloud computing". Some researchers worked on "distributed memory" and others built "remote procedure calls". More commercially successful architectures of late have popularized ideas like the CAP theorem, distributed caches, and REST. Using examples from companies like Amazon and Google this presentation walks through some practical tips to evolve your service-oriented architecture. Google's Chubby service demonstrates how you can take advantage of CAP's "best effort availability" options and Amazon's "best effort consistency" services show the other end of the spectrum. Practical lessons learned from Lucidchart's forays into SOA share insight through quantitative analyses on how to make your system highly available. Bio: Derrick Isaacson is the Director of Engineering for Lucid Software Inc (lucidchart.com). He has a BS in EE from BYU and an MS in CS from Stanford. He's developed big services at Amazon, web platforms for Microsoft, and graphical apps at Lucidchart. Derrick has two patent applications at Microsoft and Domo. For fun he cycles, backpacks, and takes his son out in their truck.
- #4: http://en.wikipedia.org/wiki/Prisoner%27s_dilemma http://www.acting-man.com/?p=34313
- #11: Multiple nodes computing on inconsistent state with regular communication failures present entirely different challenges than those computer science students face in the classroom writing DFS algorithms.
- #12: Multiple nodes computing on inconsistent state with regular communication failures present entirely different challenges than those computer science students face in the classroom writing DFS algorithms.
- #13: Multiple nodes computing on inconsistent state with regular communication failures present entirely different challenges than those computer science students face in the classroom writing DFS algorithms.
- #16: Idea: Nothings more familiar to programmers than reading from and writing to memory? We access variables all day long. Why not make distributed state access look like simple memory access? We can use modern operating systems support for virtual memory to swap in memory that is located on another machine. Problem: How often do you go to access a variable and cant because a section of memory is down? How do you provide a mutex to parallel threads of execution? How can the distributed memory layer be efficient when it has no knowledge of the application?
- #17: Idea: Next to memory access, nothings more familiar to programmers than functional calls. Can we make distributed state transfer look like a simple procedure call? SOAP! Problems: How often do you retry a method call because the JVM failed to invoke it the first time? Why does incrementing a value take 100 milliseconds? Why does your internet only support .NET and PHP (stub compiler/SDK)?
- #18: Idea: Easy network file sharing. NFS, AFS, GFS Works great for files.
- #19: Idea: Easy network file sharing. NFS, AFS, GFS Works great for files.
- #20: Idea: How could you steal bandwidth from universities and avoid infringement lawsuits at the same time? Problems: Mooching resources is a great business model but a terrible architecture if thats not what youre going for.
- #21: Idea: I have so much state I dont want to transfer it all in a single response. http://www.therufus.com/wp-content/uploads/2014/03/221-Sherlock-Holmes-Poster.jpg
- #26: Whats the availability of the overall system if a single response for service A is calculated by making 4 total requests to services B, C, and D? If the average availabilities of those 3 components are as given, and the random values are modeled as IID, what is the maximum percentage of requests is service A able to calculate correctly? .995 * .998 * .998 * .996 = 0.987 IID is a bad assumption for nearly any distributed system, but it illustrates the effective of naively distributing computation. When crash failures originating at service A are included, the total availability is < 98.7%! Thats an average of 19 minutes of downtime per day!
- #31: Conjecture made by UC Berkely computer scientist, Eric Brewer, in 2000 Gilbert & Lynch published a formal proof
- #33: Conjecture made by UC Berkely computer scientist in 2000 Gilbert & Lynch published a formal proof
- #35: We want the user to 1) get a response (available) and 2) have it be consistent with the view of other nodes. From that end user definition, a slow, error status, or non-existent response are all incorrect.
- #36: In order to model partition tolerance, the network will be allowed to lose arbitrarily many messages sent from one node to another. Gilbert & Lynch http://lpd.epfl.ch/sgilbert/pubs/BrewersConjecture-SigAct.pdf It becomes a fundamental tradeoff between availability and consistency.
- #37: It turns out the usual approach to implementing a computation like this errs on the side of consistency. If a single service request fails, this calculation hangs or returns an error to the user.
- #38: It drops below the SLA for a consistent, available response perhaps 5 of every 1000 requests.
- #41: For the checkout process you always want to honor requests to add items to a shopping cart because it's revenue producing. In this case you choose high availability. Errors are hidden from the customer and sorted out later. http://highscalability.com/amazon-architecture
- #43: Web crawler, Big Table our access patterns are highly stylized GFS has a relaxed consistency model that supports our highly distributed applications well but remains relatively simple and e鍖cient to implement. Record appends append-at-least-once semantics preserves each writers output. Readers deal with the occasional padding and duplicates as follows. Each record prepared by the writer contains extra information like checksums so that its validity can be verified. A reader can identify and discard extra padding and record fragments using the checksums. If it cannot tolerate the occasional duplicates (e.g., if they would trigger non-idempotent operations), it can filter them out using unique identifiers in the records, which are often needed anyway to name corresponding application entities such as web documents.
- #52: The Amazon Dynamo and Yahoo PNUTS data stores support high read availability while limiting write availability in the face of partitions. For example, Dynamo has a configurable number of replicas on which the data must be stored before a write is confirmed to the client.
- #54: Network partitions are less frequent at the leaves of a geographically hierarchical system.
- #55: The CAP theorem appears to have implications for scalability. Intuitively, we think of a system as scalable if it can grow efficiently, using new resources efficiently to handle more load. In order to efficiently use new resources, there must be coordination among those resources; Gilbert & Lynch
- #60: http://en.wikipedia.org/wiki/Prisoner%27s_dilemma http://www.acting-man.com/?p=34313
- #64: Wow, that guy must be an engineer said no one ever