![?<"be", [1, 1]>
?<"not", [1]>
?<"or", [1]>
?<"to", [1, 1]>](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-14-320.jpg)
![?<"be", [1, 1]>
?<"not", [1]>
?<"or", [1]>
?<"to", [1, 1]>](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-16-320.jpg)
![use MyMRBuilder;
Map/Reduce の協x
$builder = new Builder;
$builder->setInputTable('root@localhost/db/texts');
$builder->setOutputTable('root@localhost/db/word_counts');
$builder->setMapper(function ($record, $emitter) {
????$words = preg_split('/s+/u', $record['text']);
????foreach ($words as $word) {
????????$emitter->emit($word, 1);
????}
});
$builder->setReducer(function ($key, $values) {
????$sum = 0;
????foreach ($values as $count) {
????????$sum += $count;
????}
????return array('count' => $sum);
});
return $builder;](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-31-320.jpg)
![use MyMRBuilder;
Map/Reduce の協x
$builder = new Builder;
$builder->setInputTable('root@localhost/db/texts');
$builder->setOutputTable('root@localhost/db/word_counts');
$builder->setMapper(function ($record, $emitter) {
????$words = preg_split('/s+/u', $record['text']);
????foreach ($words as $word) { 秘竃薦テ`ブルの峺協
????????$emitter->emit($word, 1);
????}
});
$builder->setReducer(function ($key, $values) {
????$sum = 0;
????foreach ($values as $count) {
????????$sum += $count;
????}
????return array('count' => $sum);
});
return $builder;](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-32-320.jpg)
![use MyMRBuilder;
Map/Reduce の協x
$builder = new Builder;
$builder->setInputTable('root@localhost/db/texts');
$builder->setOutputTable('root@localhost/db/word_counts');
$builder->setMapper(function ($record, $emitter) {
????$words = preg_split('/s+/u', $record['text']);
????foreach ($words as $word) {
????????$emitter->emit($word, 1);
????} このxが Map
});
$builder->setReducer(function ($key, $values) {
????$sum = 0;
????foreach ($values as $count) {
????????$sum += $count;
????}
????return array('count' => $sum);
});
return $builder;](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-33-320.jpg)
![use MyMRBuilder;
Map/Reduce の協x
$builder = new Builder;
$builder->setInputTable('root@localhost/db/texts');
$builder->setOutputTable('root@localhost/db/word_counts');
$builder->setMapper(function ($record, $emitter) {
????$words = preg_split('/s+/u', $record['text']);
????foreach ($words as $word) {
????????$emitter->emit($word, 1);
????}
});
$builder->setReducer(function ($key, $values) {
????$sum = 0;
????foreach ($values as $count) {
????????$sum += $count;
????}
????return array('count' => $sum); このxが Reduce
});
return $builder;](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-34-320.jpg)
![Map
function ($record, $emitter) {
????$words = preg_split('/s+/u',
$record['text']);
????foreach ($words as $word) {
????????$emitter->emit($word, 1);
????}
}](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-36-320.jpg)
![Map レコ`ドを
B訶篩个箸靴栃椶曳，
function ($record, $emitter) {
????$words = preg_split('/s+/u',
$record['text']);
????foreach ($words as $word) {
????????$emitter->emit($word, 1);
????}
}](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-37-320.jpg)
![Map
function ($record, $emitter) {
????$words = preg_split('/s+/u',
$record['text']);
????foreach ($words as $word) {
????????$emitter->emit($word, 1);
????}
} text カラム坪の
猟忖双をスペ`スで蛍護](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-38-320.jpg)
![Map
function ($record, $emitter) {
????$words = preg_split('/s+/u',
$record['text']);
????foreach ($words as $word) {
????????$emitter->emit($word, 1);
????}
}
Key/Value のペアとして
嶄gテ`ブルに INSERT](https://image.slidesharecdn.com/1-kachakachakacha-ttahn-mapreduce-120429041304-phpapp01/85/PHP-MySQL-1-MapReduce-39-320.jpg)
PHP と MySQL で 1 カチャカチャカチャ...ッタ`ン MapReduce (@ニコニコ階氏h)
- 1. PHP と MySQL で 1-Click MapReduce (@ニコニコ階氏h) @yuya_takeyama
- 2. はじめに
- 4. PHP と MySQL で 1-Click 1カチャカチャカチャ...ッタ`ン MapReduce (@ニコニコ階氏h) @yuya_takeyama
- 5. N桑弱じでお僕りします ? MyMR on GitHub https://github.com/yuya-takeyama/mymr ? PHP と MySQL でカジュアルに MapReduce する http://blog.yuyat.jp/archives/1706 ? もっとカジュアルに PHP と MySQL で MapReduce する http://blog.yuyat.jp/archives/1853 ? PHP と MySQL でカジュアルに MapReduce する (スライド?Long Version) http://www.slideshare.net/taketyan/php-mysql-mapreduce ? PHP と MySQL でカジュアルに MapReduce する (スライド?Short Version) http://www.slideshare.net/taketyan/php-mysql- mapreduce-short-version
- 6. MapReduce とは (word count による箭)
- 8. 秘薦 I尖の送れ ◎ Map ◎ Shuf?e ◎ Reduce ◎ 竃薦
- 9. Map ?秘薦デ`タを鞭け函り ? }方の Key/Value ペアを竃薦
- 10. ?to be or not to be
- 11. ?<"to", 1> ?<"be", 1> ?<"or", 1> ?<"not", 1> ?<"to", 1> ?<"be", 1>
- 12. Shuf?e ?Map による Key/Value を ? Key ごとにまとめて竃薦
- 13. ?<"to", 1> ?<"be", 1> ?<"or", 1> ?<"not", 1> ?<"to", 1> ?<"be", 1>
- 14. ?<"be", [1, 1]> ?<"not", [1]> ?<"or", [1]> ?<"to", [1, 1]>
- 16. ?<"be", [1, 1]> ?<"not", [1]> ?<"or", [1]> ?<"to", [1, 1]>
- 17. ?<"be", 2> ?<"not", 1> ?<"or", 1> ?<"to", 2>
- 19. ところで
- 20. MySQL で MapReduce したい!!!
- 21. それ MySQL で も1 カチ ャカ MapReduce チャ ...っ タ` ン で したい!!!
- 22. モチベ`ション ? プログラミングモデルとしての MapReduce を聞いたい ? GROUP BY ではyしい鹿 ?MySQL を秘竃薦にしたい ? LL でサクッとやりたい
- 23. モチベ`ション ? プログラミングモデルとしての MapReduce を聞いたい ? GROUP BY ではyしい鹿 ?MySQL を秘竃薦にしたい PHP である駅勣はあまりoい ? LL でサクッとやりたい
- 24. モチベ`ション ? もち プログラミングモデルとしての ろん 1カ MapReduce を聞いたい チャ カチ ャ... ? GROUP BY ではyしい鹿ッタ `ン ， ? MySQL を秘竃薦にしたい PHP である駅勣はあまりoい ? LL でサクッとやりたい
- 25. というわけで恬りました
- 26. MyMR https://github.com/yuya-takeyama/mymr ?MySQL を秘竃薦とする ? PHP で Map/Reduce をく ? カチャカチャ...ッタ`ン，g佩
- 27. g佩までの送れ ?秘竃薦に聞う MySQL テ`ブルの喘吭 ?PHP で Map v方をく ?PHP で Reduce v方をく ?mymr コマンドをg佩する
- 28. g佩までの送れ ?秘竃薦に聞う MySQL テ`ブルの喘吭 ?PHP で Map v方をく ?PHP で Reduce v方をく ?mymr コマンドをg佩する このxがカチャカチャカチャ...
- 29. g佩までの送れ ?秘竃薦に聞う MySQL テ`ブルの喘吭 ?PHP で Map v方をく ?PHP で Reduce v方をく ?mymr コマンドをg佩する このxが...ッタ`ン
- 30. に よ る yM R M 猟嫗嶄の gZの方を方える箭 (word count)
- 31. use MyMRBuilder; Map/Reduce の協x $builder = new Builder; $builder->setInputTable('root@localhost/db/texts'); $builder->setOutputTable('root@localhost/db/word_counts'); $builder->setMapper(function ($record, $emitter) { ????$words = preg_split('/s+/u', $record['text']); ????foreach ($words as $word) { ????????$emitter->emit($word, 1); ????} }); $builder->setReducer(function ($key, $values) { ????$sum = 0; ????foreach ($values as $count) { ????????$sum += $count; ????} ????return array('count' => $sum); }); return $builder;
- 32. use MyMRBuilder; Map/Reduce の協x $builder = new Builder; $builder->setInputTable('root@localhost/db/texts'); $builder->setOutputTable('root@localhost/db/word_counts'); $builder->setMapper(function ($record, $emitter) { ????$words = preg_split('/s+/u', $record['text']); ????foreach ($words as $word) { 秘竃薦テ`ブルの峺協 ????????$emitter->emit($word, 1); ????} }); $builder->setReducer(function ($key, $values) { ????$sum = 0; ????foreach ($values as $count) { ????????$sum += $count; ????} ????return array('count' => $sum); }); return $builder;
- 33. use MyMRBuilder; Map/Reduce の協x $builder = new Builder; $builder->setInputTable('root@localhost/db/texts'); $builder->setOutputTable('root@localhost/db/word_counts'); $builder->setMapper(function ($record, $emitter) { ????$words = preg_split('/s+/u', $record['text']); ????foreach ($words as $word) { ????????$emitter->emit($word, 1); ????} このxが Map }); $builder->setReducer(function ($key, $values) { ????$sum = 0; ????foreach ($values as $count) { ????????$sum += $count; ????} ????return array('count' => $sum); }); return $builder;
- 34. use MyMRBuilder; Map/Reduce の協x $builder = new Builder; $builder->setInputTable('root@localhost/db/texts'); $builder->setOutputTable('root@localhost/db/word_counts'); $builder->setMapper(function ($record, $emitter) { ????$words = preg_split('/s+/u', $record['text']); ????foreach ($words as $word) { ????????$emitter->emit($word, 1); ????} }); $builder->setReducer(function ($key, $values) { ????$sum = 0; ????foreach ($values as $count) { ????????$sum += $count; ????} ????return array('count' => $sum); このxが Reduce }); return $builder;
- 35. 秘薦 ?to be or not to be
- 36. Map function ($record, $emitter) { ????$words = preg_split('/s+/u', $record['text']); ????foreach ($words as $word) { ????????$emitter->emit($word, 1); ????} }
- 37. Map レコ`ドを B訶篩个箸靴栃椶曳， function ($record, $emitter) { ????$words = preg_split('/s+/u', $record['text']); ????foreach ($words as $word) { ????????$emitter->emit($word, 1); ????} }
- 38. Map function ($record, $emitter) { ????$words = preg_split('/s+/u', $record['text']); ????foreach ($words as $word) { ????????$emitter->emit($word, 1); ????} } text カラム坪の 猟忖双をスペ`スで蛍護
- 39. Map function ($record, $emitter) { ????$words = preg_split('/s+/u', $record['text']); ????foreach ($words as $word) { ????????$emitter->emit($word, 1); ????} } Key/Value のペアとして 嶄gテ`ブルに INSERT
- 40. +----+--------------------+ Map | id | text | +----+--------------------+ | 1 | to be or not to be | +----+--------------------+ ◎ レコ`ドをB訶篩个箸靴 Map へ ◎ +----+---------+-------+ | id | key | value | +----+---------+-------+ | 1 | to | 1 | | 2 | be | 1 | | 3 | or | 1 | | 4 | not | 1 | | 5 | to | 1 | | 6 | be | 1 | +----+---------+-------+
- 41. +----+--------------------+ Map | id | text | +----+--------------------+ | 1 | to be or not to be | +----+--------------------+ ◎ レコ`ドをB訶篩个箸靴 Map へ ◎ +----+---------+-------+ | id | key | value | +----+---------+-------+ | 1 | to | 1 | | 2 | be | 1 | | 3 | or | 1 | | 4 | not value には JSON で秘れるので | 1 | | 5 | to 夛晒デ`タも聞喘辛嬬 | 1 | | 6 | be | 1 | +----+---------+-------+
- 42. +----+---------+-------+ | id | key | value | Shuf?e +----+---------+-------+ | 1 | to | 2 | be | 1 | 1 | | | 3 | or | 1 | | 4 | not | 1 | | 5 | to | 1 | | 6 | be | 1 | +----+---------+-------+ ◎ キ`で GROUP BY して ◎ SELECT ◎ ､ GROUP_CONCAT ◎ `key`, +---------+--------+ GROUP_CONCAT(`value`) | key | values | FROM +---------+--------+ `嶄gテ`ブル` | be | 1,1 | | not | 1 | GROUP BY | or | 1 | `key` | to | 1,1 | +---------+--------+
- 43. Reduce function ($key, $values) { ????$sum = 0; ????foreach ($values as $count) { ????????$sum += $count; ????} ????return array('count' => $sum); }
- 44. Reduce Key Value の塘双 function ($key, $values) { ????$sum = 0; ????foreach ($values as $count) { ????????$sum += $count; ????} ????return array('count' => $sum); }
- 45. Reduce Value を畠て怎す function ($key, $values) { ????$sum = 0; ????foreach ($values as $count) { ????????$sum += $count; ????} ????return array('count' => $sum); }
- 46. Reduce function ($key, $values) { ????$sum = 0; ????foreach ($values as $count) { ????????$sum += $count; ????} ????return array('count' => $sum); } 卦り､累B訶篩个 レコ`ドとして INSERT
- 47. +---------+--------+ | key | values | Reduce +---------+--------+ | be | 1,1 | | not | 1 | | or | 1 | | to | 1,1 | +---------+--------+ ◎ キ`と､療篩个 Reduce へ ◎ +----+---------+-------+ | id | key | count | +----+---------+-------+ | 1 | be | 2 | | 2 | not | 1 | | 3 | or | 1 | | 4 | to | 2 | +----+---------+-------+
- 48. +---------+--------+ | key | values | Reduce +---------+--------+ | be | 1,1 | | not | 1 | | or | 1 | gHにはデリミタとして個佩を聞喘| to | 1,1 | +---------+--------+ 個佩曝俳りの JSON になる ◎ キ`と､療篩个 Reduce へ ◎ +----+---------+-------+ | id | key | count | +----+---------+-------+ | 1 | be | 2 | | 2 | not | 1 | | 3 | or | 1 | | 4 | to | 2 | +----+---------+-------+
- 49. 書瘁の朕 ?掲揖豚 INSERT によるK双晒 ? Hadoop へのシ`ムレスな 卞佩圭隈の戻工
- 50. 書瘁の勸李 ?V8 エンジンとかで ?ストレ`ジエンジン API を ? カジュアルに澣いて ?MapReduce したい
- 51. 書瘁の勸李 もち ? ろん エンジンとかで V8 1 カチ ャカ ? チャ ストレ`ジエンジン API を ...ッ タ` ? カジュアルに澣いて ン で ?MapReduce したい
- 52. ご賠 ありがとう ございました