狠狠撸

狠狠撸Share a Scribd company logo
background jobで?
気をつけないといかんところ
@threetreeslight
Repro Inc.
shinjukurb #37
May 25, 2016
※注意※
こういうときどうしました?
話です。sidekiq寄りです。
@treetreeslight
Repro
What’s Repro?
マーケティングツール
カジュアルな改善アクション
献立
? 优先顺位の制御
? 并列実行数?リソースの制御
? 実行顺序の制御
? 信頼性の担保
优先顺位の制御
background jobにまわしてユーザー体
験改善しまくりヒャッハー!
重要じゃないjobが積まれてす
ぎて消化しきれない、、、
さらに
? SidekiqなんでQueueで分けるが、、、
? でも共通処理として定義されたjob、呼び出
し元によって優先順位かわるんだけどどうす
る?
そんなQueueのPrioritize
どうします?
例えば
? 突っ込むQueueを動的に変える
? sidekiq起動条件変えてプロセスレベルで割
る
? こういうのを使う
? https://github.com/socialpandas/
sidekiq-priority
并列実行数?リソースの制御
jobで並列化しまくれば無限に
スケール!!
connection数, deadlock,
OOM…
そんなときどうやって
並列実行数を制御します?
例えば
? サービス全体で並列実行数を制御するのであれば
? redisでsemaphoreを管理したり
? こういうやつも(時間があったら読む)
? https://github.com/dv/redis-semaphore
? https://github.com/brainopia/sidekiq-
limit_fetch
? プロセス毎の同時実行数は管理したいのであれば
? プロセス内にsemaphore作ってQueueのpopをblock
して積み直しになるのかな?
実行顺序の制御
処理を高速化するためにjobを分割
して並列実行するべさ!
箩辞产间の依存関係どうしよう
そんなときどうやって
実行順序を制御します?
例えば
? 複数のjob完了を待って発火するもの
? 特定のjob郡が完了していることを待つwaiting jobづく
り
? 逐次にjobが発火するstyle
? continuation passing styleで次のjob渡す
依存関係の定義が自明で集約されており、全体感がつかみやすい
ものが良い。
例えば
? background jobにworkflowでかいけつする
? https://github.com/chaps-io/gush
? https://github.com/thebadmonkeydev/
sidekiq-workflow
? そもそも別の仕組み使う
? rundeck
? http://rundeck.org/
? rukawa
? https://github.com/joker1007/rukawa
信頼性の担保
background job
だいぶこなれてきた!!
process downや
中断に強くしなければ
そんなときどうやって
job実行の信頼性
を担保しますか?
例えば
? sidekiq使ってたらsidekiq proにしちゃう
? やっているのは別のqueueにコピーしてるだけ(多分)
? Proのお話はsumyappさんのslide参照が良さそう
? http://www.slideshare.net/SumiKoichiro/
sidekiq-pro1-rb-29th-shinjukurb
? 処理の冪等性担保する
? けどPush Notificationとかできないので、jobの中で
途中実行できるようにする
? 依存関係のあるJobはどう途中再開させるか?
? daemon作って監視させるしか無いか?
そんなつらみを乗り越えたい
そんなアタナに
WE ARE ?
HIRING NOW!!!
https://www.wantedly.com/companies/repro/projects
Ad

Recommended

今さらアシ?ャイル巡业滨苍福冈尝罢「社内勉强会の运営をアシ?ャイルにやってみた」
今さらアシ?ャイル巡业滨苍福冈尝罢「社内勉强会の运営をアシ?ャイルにやってみた」
kei eguchi
?
顿辞肠办别谤颈锄别して?大変だった话、幸せになった话
顿辞肠办别谤颈锄别して?大変だった话、幸せになった话
Akira Miki
?
performance hack 101
performance hack 101
Akira Miki
?
The bottleneck is you
The bottleneck is you
Akira Miki
?
QA teamを組成した話
QA teamを組成した話
Akira Miki
?
今だから言えるやらないほうが良かったこと
今だから言えるやらないほうが良かったこと
Akira Miki
?
AWS Auroraよもやま話
AWS Auroraよもやま話
Akira Miki
?
1秒でも早く础耻迟辞厂肠补濒别
1秒でも早く础耻迟辞厂肠补濒别
Akira Miki
?
2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing
Search Engine Journal
?
Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
Chiara Aliotta
?
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
OECD Directorate for Financial and Enterprise Affairs
?
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
SocialHRCamp
?
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
?
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
?
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
?
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
?
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
?
Skeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
?
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
?
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
?
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
?
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
?
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
?
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
?
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
?
Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
?
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
?
How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
?

More Related Content

Featured (20)

2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing
Search Engine Journal
?
Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
Chiara Aliotta
?
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
OECD Directorate for Financial and Enterprise Affairs
?
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
SocialHRCamp
?
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
?
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
?
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
?
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
?
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
?
Skeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
?
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
?
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
?
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
?
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
?
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
?
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
?
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
?
Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
?
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
?
How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
?
2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing
Search Engine Journal
?
Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
Chiara Aliotta
?
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
SocialHRCamp
?
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
?
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
?
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
?
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
?
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
?
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
?
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
?
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
?
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
?
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
?
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
?
Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
?
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
?

background jobで?気をつけないといかんところ