闯补惫补贰测别网站架构解密

Apr 1, 20119 likes888 views

JavaEye is a popular Chinese technical blog founded in 2006 that uses a Ruby on Rails stack. It originally used a single server configuration but later scaled to separate web and database servers. It employs various caching techniques including memcached, cache_fu, and custom cache plugins to improve performance, reducing SQL queries by up to 96%. The site also added search, PDF generation, social media integration and other features over time to enhance the user experience.

JavaEye

? 1U
?1 Web Server, 1 DB Server
? 2.6
? 3

Web Server

? AMD Opteron 2.4GHz *2
? 8G
? 146G SCSI

DB Server

? AMD Opteron 2.0GHz *2
? 4G
? 73G SCSI

IT

CSDN 260

JavaEye 74

ChinaUnix 73

Cnblogs 55

ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 260 3.5X
JavaEye 74

ChinaUnix 73

Cnblogs 55

ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server 260 3.5X
JavaEye 74

ChinaUnix 73

Cnblogs 55

ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74

ChinaUnix 73

Cnblogs 55

ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server
ChinaUnix 73

Cnblogs 55

ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73

Cnblogs 55

ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server
Cnblogs 55

ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55

ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55 4+ Server
ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55 4+ Server ASP.net
ITPUB 38

51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55 4+ Server ASP.net
ITPUB 38 PHP
51CTO 23

infoq 8

0 75 150 225 300

IT

CSDN 30+ Server ASP.net 260 3.5X
JavaEye 74 2 Server ruby
ChinaUnix 73 4+ Server PHP
Cnblogs 55 4+ Server ASP.net
ITPUB 38 PHP
51CTO 23

infoq 8 Java
0 75 150 225 300

2006.09

? lighttpd
? ruby 1.8.4, rails 1.1.2, fastcgi
? mysql5.0

Single Server

lighttpd

fastcgi(ruby)

mysql

Web Server DB Server

lighttpd

fastcgi(ruby) mysql

2007.02

? posts
? posts select count 30
0.1

posts

? posts(id, post_text_id,...) 50MB
? post_texts(id, body) 2GB

2007.03

?
? memcached CachedModel
?
? 240 sql query/s 140 sql query/s
? memcached 75%

Web Server DB Server

lighttpd

fastcgi(ruby) mysql

memcached

2008.01

? JavaEye
? cache_fu
? 84%
? sql query 50 /s

Web Server DB Server

lighttpd SearchServer
(lucene)
fastcgi(ruby) mysql

memcached

Web Server DB Server

lighttpd SearchServer
(lucene)
fastcgi(ruby) mysql

PDF Creating
memcached (ruby, C)

Web Server DB Server RSS/API

fastcgi(ruby)
lighttpd SearchServer
(lucene)
fastcgi(ruby) mysql

PDF Creating
memcached (ruby, C)
twitter sync

Web Server DB Server RSS/API

fastcgi(ruby)
lighttpd SearchServer
analytics (lucene)
fastcgi(ruby) mysql
PDF Creating
memcached (ruby, C)
twitter sync
DNS Server/Email Server/
NFS Server DNS Server/Email Server

Backgroud Server
Web Server fastcgi(ruby)
IM Server
lighttpd PDF Creating
SearchServer
fastcgi(ruby)
DB Server

memcached
mysql

JavaEye

? CachedModel+ QueryCache
? cache_fu
? cache_plugin
? cache_money*

?
?
?
? page cache
? frgment cache
? REST

CachedModel

? , Hibernate
?
? select * from post_texts where id = ?
? 75%

cache_fu

? cache_fu AR

?
? cache_fu memcached
84%

cache plugin

? Rails Cache 60
? n:1
1:n
? memcached 96%

JavaEye

? ruby db Web Server load
DB
? ruby

JavaEye

? memcached 96%
? cache get : sql query = 4 : 1

fastcgi json protocol SearchServer
(ruby) (java)
socket
long lived processes long lived threads

The document discusses JavaEye, a popular Chinese technical blog. It describes the evolution of JavaEye's architecture over time from a single server running Ruby on Rails to a more complex distributed system with front-end web servers, backend application servers, a search server, database servers, and caching. It highlights the technologies used at each stage, including Lighttpd, Memcached, and search tools like Ferret and Lucene, and how caching improved performance by reducing SQL queries.

Lindsay distributed geventzmqRobin Xiao

ZeroMQ provides a simple way to build distributed systems without relying on heavyweight message brokers. It uses a messaging model where messages are atomic and can be routed between applications. ZeroMQ avoids many of the complexities of traditional message queues and makes it easy to implement common distributed patterns like request-reply, publish-subscribe, and routing in a variety of programming languages using simple socket-like APIs.

ZeroMQ in PHPJosé Lorenzo Rodríguez Urdaneta

?mq is an intelligent socket library for messaging that allows for highly flexible messaging patterns between applications. It is fast, fault tolerant, and language agnostic. Common patterns supported by ?mq include request-reply, publish-subscribe, and push-pull. Devices can be used to route messages and act as brokers or balancers to solve problems like load balancing. ?mq offers advantages over traditional solutions when advanced network interactions or high performance is required.

Build reliable, traceable, distributed systems with ZeroMQRobin Xiao

ZeroMQ is used to build a distributed system with reliable and traceable communication. It allows exposing code over RPC with minimal modification. Exceptions are properly propagated across services. The system is language agnostic and brokerless. Introspection allows viewing methods and signatures without opening code. Streaming APIs allow continuous updates without timeouts. Tracing helps profile nested calls and identify performance bottlenecks. Security is not implemented but could use SSL or authentication layers.

Software Architecture over ZeroMQpieterh

The document contains a series of one-line statements by Pieter Hintjens about software architecture and distributed systems. Some of the key points made include that software needs to be designed to connect to other code and last for decades; distributed systems live or die by their communication protocols; and that ?MQ provides a framework for building distributed protocols and applications. The statements cover a range of topics from software design, distributed systems, security and ZeroMQ in particular.

Redis & ZeroMQ: How to scale your applicationrjsmelo

Overview of ZeroMQpieterh

?MQ was created to address the need for cheaper and more efficient connectivity between distributed systems as hardware capabilities increased. It is an intelligent socket library that provides various connection patterns for messaging between processes. ?MQ aims to be fast, small, multiplatform, and multilingual. It uses common patterns like request-reply and publish-subscribe to allow processes to communicate simply and efficiently over in-process, inter-process, and network connections. The ?MQ community continues to enhance and support the open source library.

Build your own private openstack cloudNUTC, imac

This document provides instructions for configuring OpenStack Nova compute on a controller and compute server. It discusses: 1. Configuring the MySQL database, RabbitMQ, and Keystone service credentials for Nova on the controller. 2. Installing and configuring the Nova packages, including API, scheduler, and conductor services on the controller and nova-compute on the compute server. 3. Configuring Nova to use the MySQL database, RabbitMQ for messaging, and Glance for images. 4. Starting the Nova services and cleaning the SQLite database file.

Cld006 azure v_net___express_route_最新情報Tech Summit 2016

Microsoft Tech Summit discussed Azure networking features including ExpressRoute UltraPerformance Gateway, which provides higher performance private connections between Azure and on-premises networks. It is currently available in two US regions with connections from Chicago and Dallas. Application Gateway was highlighted as providing web application firewall capabilities and URL routing. IPv6 support was also expanded on Azure load balancers and virtual networks.

Sprint 143ManageIQ

ProxySQL & PXC(Query routing and Failover Test)YoungHeon (Roy) Kim

This document describes setting up and testing ProxySQL for query routing and high availability with Percona XtraDB Cluster (PXC). It includes instructions for installing and configuring ProxySQL, adding backend PXC servers, creating query rules for routing, and testing read/write splitting and failover through sysbench tests. Failover is demonstrated by stopping one PXC node, and ProxySQL is shown routing queries to the remaining nodes and marking the failed node as offline in its status.

狈辞厂蚕尝を知るSadayuki Furuhashi

The document contains charts and graphs showing performance test results for different key-value store systems. A line graph shows the number of requests per second for Viver, Runes and V-Field systems with varying request sizes. Another set of line graphs show the latency percentage for different request sizes on two systems. The document also includes URLs and diagrams showing the architecture of different systems.

Cld006 azure v_net___express_route_最新情報Tech Summit 2016

OSMC 2008 | Monitoring MySQL by Geert VanderkelenNETWAYS

Monitoring MySQL has a long history within Nagios. Several plugins are available already. In addition to that, there are probably lots of plugins that have been developed by the community. We take a look at some of these and discuss what kind of additional useful information could be pulled out of a MySQL Server for monitoring it even better. A simple example on how to write such plugins will be shown, also using NDB API for monitoring MySQL Cluster. Now that MySQL Enterprise Monitor (MEM) is available, we'll go through the possibilities for combining the two platforms. We will also discuss the NDOUtils for storing configuration and event data using MySQL. This talk starts with a brief overview of MySQL itself: some history, where it's heading too, and why it is so successful.

MySQL Performance Schema in 20 MinutesSveta Smirnova

My talk for "MySQL, MariaDB and Friends" devroom at Fosdem on February 2, 2019 Born in 2010 in MySQL 5.5.3 as "a feature for monitoring server execution at a low level," grown in 5.6 times with performance fixes and DBA-faced features, in MySQL 5.7 Performance Schema is a mature tool, used by humans and more and more monitoring products. It becomes more popular over the years. In this talk I will give an overview of Performance Schema, focusing on its tuning, performance, and usability. Performance Schema helps to troubleshoot query performance, complicated locking issues, memory leaks, resource usage, problematic behavior, caused by inappropriate settings and much more. It comes with hundreds of options which allow precisely tune what to instrument. More than 100 consumers store collected data. Performance Schema is a potent tool. And very complicated at the same time. It does not affect performance in most cases and can slow down server dramatically if configured without care. It collects a lot of data, and sometimes this data is hard to read. This talk will start from the introduction of how Performance Schema designed, and you will understand why it slowdowns server in some cases and does not affect your queries in others. Then we will discuss which information you can retrieve from Performance Schema and how to do it effectively. I will cover its companion sys schema and graphical monitoring tools.

Introduction to QtWebKitAriya Hidayat

QtWebKit is a web browser engine that can be used to embed web content in applications. It is based on the WebKit open source project, which powers browsers like Safari and Chrome. QtWebKit provides APIs to display web pages, run JavaScript, and integrate web content with native applications. It supports features like SVG, CSS3 animations, and device access. QtWebKit is widely used due to its cross-platform support, extensive test suite, and active open source community.

分散碍别测-惫补濒耻别ストア办耻尘辞蹿蝉の思想と设计Sadayuki Furuhashi

丸レクセミナー2010 分散Key-valueストア「kumofs」について紹介いたします。近年では、スケーラビリティを重視した分散データストアが数多く登場してきました。その中でkumofsは、特に可用性の高さや動的な運用のしやすさを重視しており、また、高い単体性能を誇ります。このセッションではkumofsの特性についてご紹介すると共に、稼働中のサーバの追加やサーバ故障時の復旧方法などのデモを交えながら、kumofsの運用について具体的に紹介いたします。

Inside Architecture of Neutronmarkmcclain

- Neutron provides network abstraction and connectivity as a service for OpenStack. It uses a pluggable architecture with separate components for networking services like L2, L3, VPN, firewall, and load balancing. - The Neutron server exposes a REST API and uses plugins to interface with networking devices and agents. L2 agents connect virtual ports and isolate tenant traffic. L3 agents route between networks using Linux network namespaces. - When a VM boots, Neutron creates a port, notifies the DHCP agent, wires the port, and boots the instance while isolating network resources for each tenant. Additional services like load balancing are implemented through plugins and dedicated agents.

ずばり動く！kumofs とずばり動かないケースSadayuki Furuhashi

Enhancing MySQL SecurityVinicius M Grippa

Security is always a challenge when we come to data but regulations like GDPR brings a new layer on top of it with rules more and more restrictive to access and manipulate data. These slides cover security best practices, traditional and new features available for MySQL including features coming with the new MySQL 8. SO security SSL ACL TDE Audit Plugin MySQL 8 features (undo, redo and binlog encryption) New caching_sha2_password Roles Password Management FIPS mode

TYCS Training ProgramAmol Nimgulkar :Computer Science Researcher (Compilers & Dynamic Languages)

Hardware accelerated switching with Linux @ SWLUG Talks May 2014Nat Morris

DCSF 19 Online Feature Extraction and Event Generation for Computer-Animal In...Docker, Inc.

This talk will present an architecture developed to investigate the interaction with and between animals. The architecture allows online processing of multimedia streams and the generation, storing and visualizing of events using feature extraction. It allows biologists to analyze the events by monitoring live or by replaying streams through a web interface Docker swarm is the central component of the architecture and serves as infrastructure for stream processing, event generation, event processing and visualization. The main entry point for users is a web interface that spins up one container per user and allows independent replay of streams. This talk will focus on the architecture and on technical details concerning its implementation as well as how docker is utilized to process, store and visualize events. Some time will be spent explaining details about custom made docker solutions.

Percona Live 2019 - MySQL SecurityVinicius M Grippa

NoSQL and MySQL: News about JSONMario Beck

Midwest PHP Presentation - New MSQL FeaturesDave Stokes

MySQL Day Paris 2018 - Introduction & The State of the DolphinOlivier DASINI

DataSnapFederico Russo

Datasnap is a remoting framework that was formerly known as Midas. It has replaced the DCOM foundation with Indy components and is based on dbExpress. It allows for HTTP Basic authentication and authorization in Delphi XE3 but creates a thread per connection. It builds a remoting framework on top of a SQL framework. Testing showed that Datasnap performance decreases with increased threads while Node.js performance increases with more threads due to its asynchronous model. For small to medium projects, Datasnap may be suitable, but for large REST applications an alternative technology may need to be considered.

闯补惫补贰测别网站架构解密

1. JavaEye Robbin Fan

2. ? JavaEye ? JavaEye ? JavaEye ?

3. JavaEye

4. JavaEye ? 1U ?1 Web Server, 1 DB Server ? 2.6 ? 3

5. Web Server ? AMD Opteron 2.4GHz *2 ? 8G ? 146G SCSI

6. DB Server ? AMD Opteron 2.0GHz *2 ? 4G ? 73G SCSI

8. 150 /

9. JavaEye 10 20 RSS RSS 20 100

11. ruby ......

12. Google Adplanner Data for JavaEye

13. Google Adplanner Data for JavaEye

14. IT CSDN 260 JavaEye 74 ChinaUnix 73 Cnblogs 55 ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

15. IT CSDN 260 3.5X JavaEye 74 ChinaUnix 73 Cnblogs 55 ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

16. IT CSDN 30+ Server 260 3.5X JavaEye 74 ChinaUnix 73 Cnblogs 55 ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

17. IT CSDN 30+ Server ASP.net 260 3.5X JavaEye 74 ChinaUnix 73 Cnblogs 55 ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

18. IT CSDN 30+ Server ASP.net 260 3.5X JavaEye 74 2 Server ChinaUnix 73 Cnblogs 55 ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

19. IT CSDN 30+ Server ASP.net 260 3.5X JavaEye 74 2 Server ruby ChinaUnix 73 Cnblogs 55 ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

20. IT CSDN 30+ Server ASP.net 260 3.5X JavaEye 74 2 Server ruby ChinaUnix 73 4+ Server Cnblogs 55 ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

21. IT CSDN 30+ Server ASP.net 260 3.5X JavaEye 74 2 Server ruby ChinaUnix 73 4+ Server PHP Cnblogs 55 ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

22. IT CSDN 30+ Server ASP.net 260 3.5X JavaEye 74 2 Server ruby ChinaUnix 73 4+ Server PHP Cnblogs 55 4+ Server ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

23. IT CSDN 30+ Server ASP.net 260 3.5X JavaEye 74 2 Server ruby ChinaUnix 73 4+ Server PHP Cnblogs 55 4+ Server ASP.net ITPUB 38 51CTO 23 infoq 8 0 75 150 225 300

24. IT CSDN 30+ Server ASP.net 260 3.5X JavaEye 74 2 Server ruby ChinaUnix 73 4+ Server PHP Cnblogs 55 4+ Server ASP.net ITPUB 38 PHP 51CTO 23 infoq 8 0 75 150 225 300

25. IT CSDN 30+ Server ASP.net 260 3.5X JavaEye 74 2 Server ruby ChinaUnix 73 4+ Server PHP Cnblogs 55 4+ Server ASP.net ITPUB 38 PHP 51CTO 23 infoq 8 Java 0 75 150 225 300

26. JavaEye

27. 2006.09 ?1 ? ?

28. 2006.09 ? lighttpd ? ruby 1.8.4, rails 1.1.2, fastcgi ? mysql5.0

29. Single Server lighttpd fastcgi(ruby) mysql

30. 2007.01 ? 2 ? web DB ? IO

31. Web Server DB Server lighttpd fastcgi(ruby) mysql

32. 2007.02 ? posts ? posts select count 30 0.1

33. posts ? posts(id, ..., body) ? 2GB

34. posts ? posts(id, post_text_id,...) 50MB ? post_texts(id, body) 2GB

35. 2007.03 ? ? memcached CachedModel ? ? 240 sql query/s 140 sql query/s ? memcached 75%

36. Web Server DB Server lighttpd fastcgi(ruby) mysql memcached

37. 2007.05 ? ?

38. 2007.09 ? ? ruby ferret ?

39. 2008.01 ? JavaEye ? cache_fu ? 84% ? sql query 50 /s

40. 2008.05 ? rmmseg-cpp

41. 2008.10 ? cache plugin ? 96%

42. 2008.10 ? ferret ? Java lucene ? C/S

43. Web Server DB Server lighttpd SearchServer (lucene) fastcgi(ruby) mysql memcached

44. 2008.11 ? PDF

45. Web Server DB Server lighttpd SearchServer (lucene) fastcgi(ruby) mysql PDF Creating memcached (ruby, C)

46. 2009.03 ? SNS feed ? twitter ? API

47. Web Server DB Server RSS/API fastcgi(ruby) lighttpd SearchServer (lucene) fastcgi(ruby) mysql PDF Creating memcached (ruby, C) twitter sync

48. 2009.03 ? Google Analytics ?

49. Web Server DB Server RSS/API fastcgi(ruby) lighttpd SearchServer analytics (lucene) fastcgi(ruby) mysql PDF Creating memcached (ruby, C) twitter sync DNS Server/Email Server/ NFS Server DNS Server/Email Server

50. 2009.12 ? Web IM ? ?

51. Backgroud Server Web Server fastcgi(ruby) IM Server lighttpd PDF Creating SearchServer fastcgi(ruby) DB Server memcached mysql

52. JavaEye ? CachedModel+ QueryCache ? cache_fu ? cache_plugin ? cache_money*

53. ? ? ? ? page cache ? frgment cache ? REST

54. ? ? n ? ORM ? join n+1 SQL

55. ? Web ? IO ? ? n+1 SQL

58. CachedModel ? , Hibernate ? ? select * from post_texts where id = ? ? 75%

61. CachedModel ? AR ?nd_by_sql ? ?

62. cache_fu ? cache_fu AR ? ? cache_fu memcached 84%

63. cache plugin ? Rails Cache 60 ? n:1 1:n ? memcached 96%

64. memcached

65. memcached