CloudWatch Logsについて
?Download as PPTX, PDF?
10 likes?6,983 views
cloudpack night #10でのk燕Y創です
![AWS CLI
log groupの恬撹
$ aws logs create-log-group --log-group-name my-group
$ aws logs describe-log-groups | jq '.logGroups[] | select(.logGroupName == "my-group")'
{
"storedBytes": 0,
"arn": "arn:aws:logs:us-east-1:XXX:log-group:my-group:*",
"logGroupName": "my-group",
"creationTime": 1409629885412,
"metricFilterCount": 0
}](https://image.slidesharecdn.com/2014-09-02cloudwatchlogs-140903084207-phpapp02/85/CloudWatch-Logs-20-320.jpg)
![AWS CLI
log streamの恬撹
$ aws logs create-log-stream --log-group-name my-group --log-stream-name my-stream
$ aws logs describe-log-streams --log-group-name my-group
{
"logStreams": [
{
"creationTime": 1409630168044,
"logStreamName": "my-stream",
"arn": "arn:aws:logs:us-east-1:822997939312:log-group:my-group:log-stream:my-stream",
"storedBytes": 0
}
]
}](https://image.slidesharecdn.com/2014-09-02cloudwatchlogs-140903084207-phpapp02/85/CloudWatch-Logs-21-320.jpg)
![フィルタ`
$ aws logs put-metric-filter ?
> --log-group-name my-group ?
> --filter-name my-filter ?
> --filter-pattern '' ?
> --metric-transformations ?
> metricName=EventCount,metricNamespace=YourNamespace,metricValue=1
127.0.0.1 - frank [10/Oct/2000:13:25:15 -0700] "GET /apache_pb.gif HTTP/1.0" 200 1534
127.0.0.1 - frank [10/Oct/2000:13:35:22 -0700] "GET /apache_pb.gif HTTP/1.0" 500 5324
127.0.0.1 - frank [10/Oct/2000:13:50:35 -0700] "GET /apache_pb.gif HTTP/1.0" 200 4355
[ip, user, username, timestamp, request, status_code, bytes > 1000]
[ip, user, username, timestamp, request, status_code = 200, bytes]
[ip, user, username, timestamp, request, status_code = 4*, bytes]
[ip, user, username, timestamp, request = *html*, status_code = 4*, bytes]](https://image.slidesharecdn.com/2014-09-02cloudwatchlogs-140903084207-phpapp02/85/CloudWatch-Logs-34-320.jpg)
![フィルタ`
¢ 腎易曝俳りの夛晒猟忖双として盾
¢ ダブルクォ`ト(＾)、叔凄察([])はグル`プ晒
o シングルクォ`ト(｀)、噸宥の凄察()、嶄凄察{}はグル
`プ晒されない
¢ 仝...々でm輝に福待してくれる
o [..., status_code = 200, bytes]
挑硫のstatus_codeが200](https://image.slidesharecdn.com/2014-09-02cloudwatchlogs-140903084207-phpapp02/85/CloudWatch-Logs-35-320.jpg)
![フィルタ`
マッチングはCLIでテストできる
トの函誼~$ aws logs test-metric-filter ?
> --filter-pattern '[...]' ?
> --log-event-messages '127.0.0.1 - frank [10/Oct/2000:13:25:15 -0700] "GET /apache_pb.gif HTTP/1.0" 200 1534'
{
"matches": [
{
"eventNumber": 1,
"eventMessage": "127.0.0.1 - frank [10/Oct/2000:13:25:15 -0700] ?"GET /apache_pb.gif HTTP/1.0?" 200 1534",
"extractedValues": {
"$6": "200",
"$7": "1534",
"$4": "10/Oct/2000:13:25:15 -0700",
"$5": "GET /apache_pb.gif HTTP/1.0",
"$2": "-",
"$3": "frank",
"$1": "127.0.0.1"
}
}
]
}](https://image.slidesharecdn.com/2014-09-02cloudwatchlogs-140903084207-phpapp02/85/CloudWatch-Logs-36-320.jpg)
![Radiosonde / Meteorlog
Meteorlog
https://github.com/winebarrel/meteorlog
log_group "any-group" do
log_stream "my-stream"
# Please write the following if you do not want to manage log_streams
#any_log_streams
metric_filter "MyAppAccessCount" do
# see http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/FilterAndPatternSyntax.html
filter_pattern '[ip, user, username, timestamp, request, status_code, bytes > 1000]'
metric :name=>"EventCount", :namespace=>"YourNamespace", :value=>"1"
end
metric_filter "MyAppAccessCount2" do
metric :name=>"EventCount2", :namespace=>"YourNamespace2", :value=>"2"
endend](https://image.slidesharecdn.com/2014-09-02cloudwatchlogs-140903084207-phpapp02/85/CloudWatch-Logs-44-320.jpg)
![Radiosonde / Meteorlog
Radiosonde
https://github.com/winebarrel/radiosonde
alarm "alarm1" do
namespace "AWS/EC2"
metric_name "CPUUtilization"
dimensions "InstanceId"=>"i-XXXXXXXX"
period 300
statistic :average
threshold ">=", 50.0
evaluation_periods 1
actions_enabled true
alarm_actions []
ok_actions []
insufficient_data_actions ["arn:aws:sns:us-east-1:123456789012:my_topic"]end](https://image.slidesharecdn.com/2014-09-02cloudwatchlogs-140903084207-phpapp02/85/CloudWatch-Logs-45-320.jpg)
CloudWatch Logsについて
- 1. CloudWatch Logsを 噸宥に聞う クックパッド幄塀氏芙殫圻圷
- 2. お念lよ 殫圻圷 @sgwr_dts / http://so-wh.at/ ¢ Ruby / AWS ¢ https://bitbucket.org/winebarrel ¢ https://github.com/winebarrel ¢ 易署岬の圭から栖ました
- 7. CloudWatch Logs 喘議なログЪサ`ビス ¢ ログЪ ¢ グラフ晒 ¢ 宥岑
- 9. CloudWatch Logs ユ`スケ`ス ¢ ログを聞ったエラ`O
- 10. CloudWatch Logs ユ`スケ`ス ¢ ログを聞ったエラ`O ´ぐらい
- 11. CloudWatch Logs 裏虫なところ ¢ ログ函誼がストリ`ムg了
- 12. CloudWatch Logs 裏虫なところ ¢ ログ函誼がストリ`ムg了 ¢碧C嬬がない
- 13. CloudWatch Logs 9/2F壓、ログのЪ枠はus-east-1のみ ただしイベントの誘後はどこからでも辛嬬
- 14. CloudWatch Logs 光Nクライアント ¢ CloudWatch Logs Agenthttps://s3.amazonaws.com/aws-cloudwatch/ downloads/latest/awslogs-agent-setup.py ¢ AWS CLIhttp://docs.aws.amazon.com/cli/latest/reference/logs/index.html ¢ fluentdhttps://github.com/ryotarai/fluent-plugin-cloudwatch-logs ¢ APIhttp://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/Welcome.ht ml
- 15. 乙姻看顎沿と壊岳姻艶温馨について
- 16. log group / log stream
- 17. log group / log stream ¢ ログはlog groupごとにI尖 o 辛晒?グラフ晒?O o API貧ではstream阿縫蹈阿糧ゝ辰録苗 ¢ log groupにlog streamが奉する o プログラムはlog streamにイベントを誘げる o １スレッド★１ストリ`ム
- 18. 聞い圭
- 19. AWS CLI h廠篳のO協┐泙燭魯レデンシャルファイルをO協 $ export AWS_ACCESS_KEY_ID=... $ export SECRET_ACCESS_KEY=... $ export AWS_DEFAULT_REGION=us-east-1
- 20. AWS CLI log groupの恬撹 $ aws logs create-log-group --log-group-name my-group $ aws logs describe-log-groups | jq '.logGroups[] | select(.logGroupName == "my-group")' { "storedBytes": 0, "arn": "arn:aws:logs:us-east-1:XXX:log-group:my-group:*", "logGroupName": "my-group", "creationTime": 1409629885412, "metricFilterCount": 0 }
- 21. AWS CLI log streamの恬撹 $ aws logs create-log-stream --log-group-name my-group --log-stream-name my-stream $ aws logs describe-log-streams --log-group-name my-group { "logStreams": [ { "creationTime": 1409630168044, "logStreamName": "my-stream", "arn": "arn:aws:logs:us-east-1:822997939312:log-group:my-group:log-stream:my-stream", "storedBytes": 0 } ] }
- 22. AWS CLI イベントの誘後 $ aws logs put-log-events ? > --log-group-name my-group --log-stream-name my-stream ? > --log-events ? > timestamp=1409630448000,message="Test event 1" ? > timestamp=1409630448001,message="Test event 2" { "nextSequenceToken": "49540113588536892459580435126484523626194989597679702066" }
- 23. AWS CLI イベントの誘後╋指朕 $ aws logs put-log-events ? > --sequence-token 49540113588536892459580435126484523626194989597679702066 ? > --log-group-name my-group --log-stream-name my-stream ? > --log-events ? > timestamp=1409630448003,message="Test event 3" ? > timestamp=1409630448004,message="Test event 4" { "nextSequenceToken": "49540113588536892459580435126797635413475185494595751986" }
- 24. AWS CLI イベントの誘後 ¢ 1指のAPIコ`ルで}方のイベントを誘後辛嬬 ¢ timestampはミリ昼を根むUNIX time ¢ streamへの２指朕參週の誘後にはsequence tokenが駅勣 o }方のスレッドから揖じstreamの誘後はsequence tokenの揖豚が駅勣
- 25. AWS CLI イベントの函誼 $ aws logs get-log-events --log-group-name my-group --log-stream-name my-stream { "nextForwardToken": "f/31435809445027774223220423298428344103348277957912231937", "events": [ { "ingestionTime": 1409630516068, "timestamp": 1409630448000, "message": "Test event 1" }, { "ingestionTime": 1409630516068, "timestamp": 1409630448001, "message": "Test event 2" }, ´
- 26. AWS CLI コンソ`ル
- 27. CloudWatch Logs Agent 蝶芙のブログを歌孚してください。 http://dev.classmethod.jp/cloud/aws/amazon-cloudwatch- logs/
- 28. CloudWatch Logs Agent 粥乙艶稼岳宴旋なんですかねぇ´
- 29. 宴旋字嬬
- 30. グラフ晒 Metric Filterによりグラフ晒 http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CountingLogEventsExample.html log groupにフィルタを恬撹することで、ログ ┘ぅ戰鵐硲ごとにメトリクスをhできる
- 31. グラフ晒 $ aws logs put-metric-filter ? > --log-group-name my-group ? > --filter-name my-filter ? > --filter-pattern '' ? > --metric-transformations ? > metricName=EventCount,metricNamespace=YourNamespace,metricValue=1 metricName CloudWatchのメトリック兆 metricNamespace メトリックのネ`ムスペ`ス metricValue メトリックをhするときの｡￣1￣なら1イベントにつき￣1￣がhされる。猟忖双も辛 http://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_MetricTransformation.html
- 32. グラフ晒
- 33. フィルタ` http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/FilterAndPatternSyntax.html filter patterの峺協により蒙協の訳周をメトリッ クとしてhできる
- 34. フィルタ` $ aws logs put-metric-filter ? > --log-group-name my-group ? > --filter-name my-filter ? > --filter-pattern '' ? > --metric-transformations ? > metricName=EventCount,metricNamespace=YourNamespace,metricValue=1 127.0.0.1 - frank [10/Oct/2000:13:25:15 -0700] "GET /apache_pb.gif HTTP/1.0" 200 1534 127.0.0.1 - frank [10/Oct/2000:13:35:22 -0700] "GET /apache_pb.gif HTTP/1.0" 500 5324 127.0.0.1 - frank [10/Oct/2000:13:50:35 -0700] "GET /apache_pb.gif HTTP/1.0" 200 4355 [ip, user, username, timestamp, request, status_code, bytes > 1000] [ip, user, username, timestamp, request, status_code = 200, bytes] [ip, user, username, timestamp, request, status_code = 4*, bytes] [ip, user, username, timestamp, request = *html*, status_code = 4*, bytes]
- 35. フィルタ` ¢ 腎易曝俳りの夛晒猟忖双として盾 ¢ ダブルクォ`ト(＾)、叔凄察([])はグル`プ晒 o シングルクォ`ト(｀)、噸宥の凄察()、嶄凄察{}はグル `プ晒されない ¢ 仝...々でm輝に福待してくれる o [..., status_code = 200, bytes] 挑硫のstatus_codeが200
- 36. フィルタ` マッチングはCLIでテストできる トの函誼~$ aws logs test-metric-filter ? > --filter-pattern '[...]' ? > --log-event-messages '127.0.0.1 - frank [10/Oct/2000:13:25:15 -0700] "GET /apache_pb.gif HTTP/1.0" 200 1534' { "matches": [ { "eventNumber": 1, "eventMessage": "127.0.0.1 - frank [10/Oct/2000:13:25:15 -0700] ?"GET /apache_pb.gif HTTP/1.0?" 200 1534", "extractedValues": { "$6": "200", "$7": "1534", "$4": "10/Oct/2000:13:25:15 -0700", "$5": "GET /apache_pb.gif HTTP/1.0", "$2": "-", "$3": "frank", "$1": "127.0.0.1" } } ] }
- 37. 韓鉛顎艶稼岳糸との銭亊
- 38. fluent-plugin-cloudwatch-logs https://github.com/ryotarai/fluent-plugin-cloudwatch-logs ¢ @ryot_a_rai枠伏恬 ¢ Input / Output I圭 o Inputは畠隼編^してません
- 39. fluent-plugin-cloudwatch-logs <match cloudwatch_logs.**> type cloudwatch_logs log_group_name my-group log_stream_name my-stream auto_create_stream true message_keys message #use_tag_as_group false </match> $ echo '{"message":"item1 item2 item3"}' | fluent-cat cloudwatch_logs.test
- 40. fluent-plugin-cloudwatch-logs ¢ log group / log streamは徭嘛撹辛嬬 ¢ sequence tokenはメモリにh
- 41. fluent-plugin-cloudwatch-logs ¢ ＾message_keys￣を峺協しないとJSONとし て誘後される o フィルタ`の嘛を深えると峺協した圭がよい
- 42. 砿尖ツ`ル恬りました
- 43. Radiosonde / Meteorlog 仝AWSコ`ド晒々シリ`ズ ¢ Radiosonde o CloudWatchのAlartの砿尖 ¢ Meteorlog o CloudWatch Logsの砿尖
- 44. Radiosonde / Meteorlog Meteorlog https://github.com/winebarrel/meteorlog log_group "any-group" do log_stream "my-stream" # Please write the following if you do not want to manage log_streams #any_log_streams metric_filter "MyAppAccessCount" do # see http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/FilterAndPatternSyntax.html filter_pattern '[ip, user, username, timestamp, request, status_code, bytes > 1000]' metric :name=>"EventCount", :namespace=>"YourNamespace", :value=>"1" end metric_filter "MyAppAccessCount2" do metric :name=>"EventCount2", :namespace=>"YourNamespace2", :value=>"2" endend
- 45. Radiosonde / Meteorlog Radiosonde https://github.com/winebarrel/radiosonde alarm "alarm1" do namespace "AWS/EC2" metric_name "CPUUtilization" dimensions "InstanceId"=>"i-XXXXXXXX" period 300 statistic :average threshold ">=", 50.0 evaluation_periods 1 actions_enabled true alarm_actions [] ok_actions [] insufficient_data_actions ["arn:aws:sns:us-east-1:123456789012:my_topic"]end
- 46. Radiosonde / Meteorlog 宴旋ですよ
- 47. まとめ
- 48. まとめ ¢ CloudWatch 晦看乙壊は宴旋
- 49. まとめ ¢ CloudWatch 晦看乙壊は宴旋 ´そうな櫃あります
- 50. まとめ ¢ CloudWatch 晦看乙壊は宴旋 ´そうな櫃あります ¢碧C嬬圀しい