�ݺ�ߣ

Jitsi Meet
Our tale of blood, sweat, tears and love.
Open Tech Will Save Us 2020

"Calm waters" by Abevil is licensed under CC BY-NC-ND 2.0
It’s calm before the storm

Jitsi Meet: our tale of blood, sweat, tears and love

📍
📍
📍
📍
📍
📍
📍

Getting Ready
"hope" by siette is licensed under CC BY-NC-ND 2.0

The Italian Flood
"Mexico City - Tourist Bus Ride" by CarlosVanVegas is licensed under CC BY 2.0

🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥

🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥
🔥

What went wrong?
"Collapse" by Jon Scally is licensed under CC BY-NC-ND 2.0

Panic Mode ON
😱😱😱🙀🙀🙀

Tens of millions of 
dropped packets

Users getting 504s 
left and right

Tons of ⭐ reviews on 
mobile stores

Hunders of support emails 
and forum posts

ulimit -n
Too small! We used prlimit to raise it in production.

Ephemeral port exhaustion
Add more IP addresses to our load balancers and HAProxys.

Inter-server traﬃc
Having so many servers was killing us, 
we made each JVB connect only to its shard.

This lasted for days
Standup turned into a Situation Room.

Reacting to Feedback
"File:Feedback Terminal With A Touchscreen.jpg" by Littokki is licensed under CC BY-SA 4.0

We’ll always have Paris
"File:Eiffel Tower - Tour Eiffel (34377053374).jpg" by Nan Palmero from San Antonio, TX, USA is licensed under CC BY 2.0

📍
📍
📍
📍
📍
📍
📍
📍

AWS Global Accelerator
Instrumental to balance the load across regions.

The numbers (so far)
"XYZ" by Marina Lewandowska is licensed under CC BY-NC-ND 4.0

1000s of servers
Instead of 10s

Optimize our use of resources
Optimize for transit cost by moving JVBs out of AWS

Move away from 
HAProxy stick tables
Split-brains are unavoidable with the current scheme.

Address more feedback
Keep it coming!

Jitsi Meet: our tale of blood, sweat, tears and love