際際滷

際際滷Share a Scribd company logo

How we build caibangzi.com

R
Robin Lu

The document summarizes how caibangzi.com, a personal investment management tool and community, was built and evolved over time to handle increasing traffic demands. It started on a single server in 2007 but faced challenges with performance and bandwidth as daily pageviews reached 1 million. Key steps taken included upgrading hardware, separating application and database servers, adding a load balancer, implementing distributed computation solutions, and upgrading to Rails 2.0. Even at peak loads of 1 million daily pageviews and 160 requests per second, CPU usage never exceeded 30% due to these architectural improvements.

1 of 24
Download to read offline
How we built Caibangzi.com Robin Lu (caibangzi.com)
Introduction Personal Investment Management. Tool + Community Founders: We are engineers! Technology: Ruby On Rails MySQL鐚Linux鐚Lighttpd ...
Topics Problems we are facing Architecture evolution of caibangzi Maintenance Q&A
Peaks! High Peak Heavy Computation Visiting Peak, Computation Peak and Cache expiration Peak are in the same time. :...(
Evolution - Start Launched at March, 2007 Hardware: 1 x 1U server 1xAMD Opteron 250, 2G Memory, 250G SATA, RAID 1 Bandwidth Shared less than 3Mbps
Evolution - Start Apache 2 + mod_proxy_balancer mongrel x 8 All in memcached caibangzi one box rails 1.1.6 MySQL 5
First Challenge May, 2007 Extremely time consuming actions blocked the site. Performance tuning Optimize Rails cache
Second Challenge August, 2007 We were running out of bandwidth Solution: Money upgrade to 5Mbps
Server upgrading High growth rate after August, 2007 Hardware upgrading in Sept. 2007 still 1 x 1U 2 x AMD 2214 2.2GHz 4G RAM
Until Sept. 2007 Apache 2 + mod_proxy_balancer mongrel x 20 Still in memcached caibangzi one box rails 1.1.6 MySQL 5
Something happened in Oct. 2007 Stock Index reached 6000 Daily pv reached 1 million Bigger challenge came
Several Changes We Made Upgrade bandwidth to 10Mbps Separate APP server and DB server Load Balancer Distributed Computation
2 Boxes Web Server event mongrel x 30 AP memcached caibangzi P rails MySQL 5 Tianyuanji a
Load balancer Run with 2 mongrels behind balancer
Load balancer Slow actions block the others with balancer: by request Apach e by traffic HAproxy round-robin Nginx Lighttpd 1.4 Note: Nginx fixed it with fair balancer later
Caibangzi Choice of Balancer Lighttpd 1.5 beta proxy-core.balancer = "sqf" http://redmine.lighttpd. net/issues/show/1362 For Rails application, choice of balancer is sometimes more important than choice of web server
More on Web Server disable keep-alive serve static files with web server instead of mongrel tune configurations according to your web server documentations
Distributed solution BackgrounDRB based on Distributed Ruby (DRB) use ruby thread high overhead during context switching very slow when concurrent connections higher than 10
Distributed solution FastBackground our own distributed computation solution based on Eventmachine simple, but fast and works for us
Rails upgrading Early of 2008, we upgraded to Rails 2.0 expired several plugins refactory to be more RESTful cookie session multi view => wap.caibangzi.com
Open API API Based on OAuth integrate caibangzi into your own sites Other APPs support several F8 clones xiaonei.com, hainei.com, UCHome
Maintenance Munin easy to use and extend Monit Crontab
With all that.... we handled: pv/day 1M rails req/day 1.8M peak rails req/sec 160 total req/day 5M peak req/sec 430 Its the fact. Not the capacity. Our CPU usage never goes higher than 30%
Q&A

More Related Content

How we build caibangzi.com

  • 1. How we built Caibangzi.com Robin Lu (caibangzi.com)
  • 2. Introduction Personal Investment Management. Tool + Community Founders: We are engineers! Technology: Ruby On Rails MySQL鐚Linux鐚Lighttpd ...
  • 3. Topics Problems we are facing Architecture evolution of caibangzi Maintenance Q&A
  • 4. Peaks! High Peak Heavy Computation Visiting Peak, Computation Peak and Cache expiration Peak are in the same time. :...(
  • 5. Evolution - Start Launched at March, 2007 Hardware: 1 x 1U server 1xAMD Opteron 250, 2G Memory, 250G SATA, RAID 1 Bandwidth Shared less than 3Mbps
  • 6. Evolution - Start Apache 2 + mod_proxy_balancer mongrel x 8 All in memcached caibangzi one box rails 1.1.6 MySQL 5
  • 7. First Challenge May, 2007 Extremely time consuming actions blocked the site. Performance tuning Optimize Rails cache
  • 8. Second Challenge August, 2007 We were running out of bandwidth Solution: Money upgrade to 5Mbps
  • 9. Server upgrading High growth rate after August, 2007 Hardware upgrading in Sept. 2007 still 1 x 1U 2 x AMD 2214 2.2GHz 4G RAM
  • 10. Until Sept. 2007 Apache 2 + mod_proxy_balancer mongrel x 20 Still in memcached caibangzi one box rails 1.1.6 MySQL 5
  • 11. Something happened in Oct. 2007 Stock Index reached 6000 Daily pv reached 1 million Bigger challenge came
  • 12. Several Changes We Made Upgrade bandwidth to 10Mbps Separate APP server and DB server Load Balancer Distributed Computation
  • 13. 2 Boxes Web Server event mongrel x 30 AP memcached caibangzi P rails MySQL 5 Tianyuanji a
  • 14. Load balancer Run with 2 mongrels behind balancer
  • 15. Load balancer Slow actions block the others with balancer: by request Apach e by traffic HAproxy round-robin Nginx Lighttpd 1.4 Note: Nginx fixed it with fair balancer later
  • 16. Caibangzi Choice of Balancer Lighttpd 1.5 beta proxy-core.balancer = "sqf" http://redmine.lighttpd. net/issues/show/1362 For Rails application, choice of balancer is sometimes more important than choice of web server
  • 17. More on Web Server disable keep-alive serve static files with web server instead of mongrel tune configurations according to your web server documentations
  • 18. Distributed solution BackgrounDRB based on Distributed Ruby (DRB) use ruby thread high overhead during context switching very slow when concurrent connections higher than 10
  • 19. Distributed solution FastBackground our own distributed computation solution based on Eventmachine simple, but fast and works for us
  • 20. Rails upgrading Early of 2008, we upgraded to Rails 2.0 expired several plugins refactory to be more RESTful cookie session multi view => wap.caibangzi.com
  • 21. Open API API Based on OAuth integrate caibangzi into your own sites Other APPs support several F8 clones xiaonei.com, hainei.com, UCHome
  • 22. Maintenance Munin easy to use and extend Monit Crontab
  • 23. With all that.... we handled: pv/day 1M rails req/day 1.8M peak rails req/sec 160 total req/day 5M peak req/sec 430 Its the fact. Not the capacity. Our CPU usage never goes higher than 30%
  • 24. Q&A