狠狠撸

? 2018 NHN FORWARD. All rights reserved.
Java ? ??,
Java ? ?? ???
???
??????????

CONTENTS
1. About
2. jvmtop
3. jvm-tools
4. jcmd
5. jhsdb
6. OpenJDK Tools
7. Conclusion

4 / 58
??? ???
??? ?? ??? ????
??? ??
? JVM ?? ??
? ?? ?? ??
? ?? ??

5 / 58
??? ???
??? ?? ??? ????
??? ??
? JVM ?? ??
? ??? ?? ??
? ?? ?? ??
? ?? ?? ??
? ????
? ????
? ?? ??
? ???? ??
? ?? ??? ??

6 / 58
??? ?? – JVM ?? ??
??? ?? ??
? JVM ? ??? ????? ???? ??
? ??? ?? ??: , ,
?? ?? ??
? JVM ?? ??? ???? ??? ???? ??
? ??? ?? : TDA, Thread Logic
? ? ?? : IBM Heap Analyzer, MAT

7 / 58
??? ?? – ?? ?? ??
?? ??
? ???? ???? ??? ???? ?? ?? ??
? APM : scouter, pinpoint
?? ??
? ?? ? ??? ???? ?? ??
? CPU, Memory, Coverage ? ?? ??
? Profiling : JProfiler, YourKit
? Object Layout :
? CPU ?? :

8 / 58
??? ?? – ?? ??
???? ??
? ??? ?? ?? ?? ??
? method ?? ?? :
?? ??
? ??? ???? ?? ??
? ?? ??? ?? : gatling, jmeter
? concurrent ?? ?? :

9 / 58
?? ??? ???
? jvmtop : CPU
? jvm-tools : Universal
? jcmd : Universal
? jhsdb : JVM analyze
? jmh : BMT
? jcstress : Concurrent
? jol : Java Object Layout

11 / 58
jvmtop??
? https://github.com/patric-r/jvmtop
? ?? ?? ????

12 / 58
jvmtop??
??
? JVM?? ???? CPU ???? ?????? ??
? ??? ?? ???
?? ??
? JVM? CPU ???? ?? ??? ?? ?? ?
? ?? ?? ?? CPU ???? ??? ?? ?? ?
? ????? CPU ???? ??? ?? ?
??
? Java 9 ??? JDK ?? ??
? ??? fork ?? ?? ?? ???,
??? git repository ? ?? ??

13 / 58
jvmtop ?? ?
?? ?? ??
? CPU ? ?? ???? ???? ?? ??
$ ./jvmtop.sh
JvmTop 0.8.0 alpha - 09:57:33, x86_64, 8 cpus, Mac OS X 10.12., load avg 9.57
http://code.google.com/p/jvmtop
PID MAIN-CLASS HPCUR HPMAX NHCUR NHMAX CPU GC VM USERNAME #T DL
49993 m.jvmtop.JvmTop 44m 3641m 16m n/a 1.47% 0.00% O8U15 smlee 20
13357 180611-1436.jar 214m 1024m 69m n/a 1.09% 0.00% O8U15 smlee 21
49826 CpuConsumerMain 83m 3641m 20m n/a 0.26% 0.00% O8U73 smlee 9
49827 mdline.Launcher 23m 622m 28m n/a 0.23% 0.00% O8U73 smlee 12

14 / 58
jvmtop ?? ?
<pid> ? ???? ??
? CPU ? ?? ???? ??? ?? ??
$ ./jvmtop.sh <pid>
PID 49826: com.nhnent.CpuConsumerMain
ARGS:
VMARGS: -XX:NativeMemoryTracking=detail -Dfile.encoding=UTF-8
VM: Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 1.8.0_73
UP: 0:46m #THR: 11 #THRPEAK: 15 #THRCREATED: 3365 USER: smlee
GC-Time: 0: 0m #GC-Runs: 5888 #TotalLoadedClasses: 2113
CPU: 0.23% GC: 0.00% HEAP: 259m /3641m NONHEAP: 23m / n/a
TID NAME STATE CPU TOTALCPU BLOCKEDBY
3344 RMI TCP Connection(4)-127.0.0. RUNNABLE 1.30% 0.01%
3371 JMX server connection timeout TIMED_WAITING 0.08% 0.00%
Note: Only top 10 threads (according cpu load) are shown!

15 / 58
jvmtop ?? ?
--profile <pid> ???? ??
$ ./jvmtop.sh --profile <pid>
Profiling PID 49826: com.nhnent.CpuConsumerMain
54.58% ( 1.36s) com.nhnent.cpu.InfiniteLoop.busyMethod()
21.51% ( 0.54s) ....fasterxml.jackson.databind.util.ClassUtil.getEnclosi()
8.46% ( 0.21s) ....fasterxml.jackson.databind.util.ClassUtil.hasEnclosi()
5.92% ( 0.15s) ....fasterxml.jackson.databind.cfg.MapperConfig.collectF()
3.29% ( 0.08s) ....fasterxml.jackson.databind.deser.DeserializerCache._()
3.13% ( 0.08s) ....fasterxml.jackson.databind.introspect.AnnotatedField()
3.10% ( 0.08s) ....fasterxml.jackson.databind.util.BeanUtil.okNameForMu()

16 / 58
Java Attach API
? ?? process ?? JVM ? ????
? ?? ??? ???
? ??? ???? ?? API
?? ??
? jdk/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.java
? jdk/hotspot/src/share/vm/services/diagnosticCommand.cpp
jvmtop ??

17 / 58
jvmtop ?? (Attach API ? ??)
jvmtop
JVM
1. SIGQUIT ??
2. AttachListener ??? ??
3. ?? ??
5. ?? ??
4. ??

20 / 58
jvm-tools?
? https://github.com/aragozin/jvm-tools
? ?? ?? ????

21 / 58
jvm-tools?
??
? ??? ???
?? ??
? ?? ??? ?? ??? ? ??
? git ?? ?? ?
? ???? sjk/target ????? jar ??? ???
?? ?? ??
? CPU ???? ?? ???? ????? ? ?
? flame ???? ??? ????? ? ?
$ mvn package -DskipTests

22 / 58
jvm-tools options
? ttop
? hh
? stcap, stcpy, ssa, frame
? mx
? jps
? gc

23 / 58
jvm-tools options
? ttop: thread top
? hh: heap histogram
? stcap, stcpy, ssa, frame: stack info
? mx: mbean
? jps: java processes
? gc: gc log

24 / 58
jvm-tools ?? ?
ttop(thread top) ?? ??
$ java -jar sjk-0.10.1.jar ttop -p <pid>
20XXXX-XXT15:10:39.580+0900 Process summary
process cpu=70.43%
application cpu=66.58% (user=65.00% sys=1.58%)
other: cpu=3.85%
thread count: 14
GC time=0.23% (young=0.23%, old=0.00%)
heap allocation rate 306mb/s
safe point rate: 2.3 (events/s) avg. safe point pause: 1.16ms
safe point sync time: 0.01% processing time: 0.26% (wallclock time)
[017398] user= 9.95% sys= 0.04% alloc= 49mb/s - pool-4344-thread-3
[017396] user= 4.92% sys= 0.02% alloc= 10b/s - pool-4344-thread-1
[017338] user= 0.38% sys= 0.13% alloc= 281kb/s - RMI TCP Connection(7)-127.0.0.1
[017400] user= 0.02% sys= 0.01% alloc= 3410b/s - JMX server connection timeout 17400
[000001] user= 0.01% sys= 0.01% alloc= 994b/s – main
[017339] user= 0.00% sys= 0.00% alloc= 0b/s - JMX server connection timeout 17339

26 / 58
$ java -jar sjk-0.10.1.jar stcap -p <pid> -o
dump.std -t 15s
Writing to /develop/javalib/jvmtool/dump.std
Collected 504
Collected 1000
Collected 1506
Collected 2001
…
$ java -jar sjk-0.10.1.jar flame -f dump.std -o
report.html
Input files
./dump.std
52050 samples processed
Generated /develop/javalib/jvmtool/report.html
(195423 bytes)
jvm-tools ?? ?
stcap ??
? ? ??? ???? ??? ????, flame graph? ??? ? ??
$ java -jar sjk-0.10.1.jar stcap -p <pid> -o dump.std -t 15s
Writing to /develop/javalib/jvmtool/dump.std
Collected 504
Collected 1000
Collected 1506
Collected 2001
…
$ java -jar sjk-0.10.1.jar flame -f dump.std -o report.html
Input files
./dump.std
52050 samples processed
Generated /develop/javalib/jvmtool/report.html (19543 bytes)

28 / 58
??
? ? ?? ?? ???? ??? ???? ?? ?? ??? ??? ? ???? ??
?? ??
? X ?: ?? ????? ??? ?? ??? ?????? ???
? Y ?: ??(stack)? ??? ???
? ??: ??? ?? ??
? ????
? X?? ??? ??? ???? ?? ??? ?? ??? ??? ?!
?? ??
? https://blog.codecentric.de/en/2017/09/jvm-fire-using-flame-graphs-analyse-performance/
? http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html
?? – flame ????

30 / 58
jcmd?
??
? JDK? ??
? Java 8 ?? ???? ?? ??
? Attach API ??
? jvm-tools ? ????? ?? ??
? ?? Java ?? ???? ??? ??
? jps, jmap, jstack, jinfo ?? ???? ??
???? ??? ?? ??? ????? ??? ?? ??!

31 / 58
?? ???
? jcmd
? jcmd <pid> help
? jcmd <pid> GC.class_stats
? jcmd <pid> help GC.class_stats
jcmd basic options

32 / 58
?? ???
? jcmd: Java ???? ??
? jcmd <pid> help: ?? ??? ??? ??
? jcmd <pid> GC.class_stats: GC.class_stats ??
? jcmd <pid> help GC.class_stats: GC.class_stats? ???
jcmd basic options

jcmd popular commands
?? ???? ???
? jcmd <pid> VM.version
? jcmd <pid> VM.system_properties
? jcmd <pid> VM.flags
? jcmd <pid> GC.class_histogram
? jcmd <pid> GC.heap_dump filename=dump.hprof
? jcmd <pid> GC.class_stats
? jcmd <pid> PerfCounter.print
? jcmd <pid> Thread.print

jcmd popular commands
?? ???? ???
? jcmd <pid> VM.version : JVM ?? ??
? jcmd <pid> VM.system_properties : ??? ??? ??
? jcmd <pid> VM.flags : VM ? ??/???? ??? ?? ?? ??
? jcmd <pid> GC.class_histogram : ??? ?? ????? ?. (jmap –histo <pid>? ??)
? jcmd <pid> GC.heap_dump filename=dump.hprof : ? ?? ?? (jmap ? ??? ??? ??)
? jcmd <pid> GC.class_stats : Java ???? ?? ??? ??
? jcmd <pid> PerfCounter.print : ?? ??? ? ??
? jcmd <pid> Thread.print : ??? ?? ??
https://dzone.com/articles/how-to-read-a-thread-dump ??

36 / 58
VM.log ??
? ????? gc log ? ???? ???? JVM ? ??? ??? ??
? jcmd ? ???? ???? JVM ? gc log ? ???? ? ? ??
VM.log ? ???? gc log ? ??? ?
? jcmd <pid> VM.log output="file=gc.log" what="gc=debug" decorators="time,level”
? <pid> ????? ???
? output=“file=gc.log”: gc.log?? ??? ????? ??? ??? ????
? what="gc=debug”: gc? ??? ??? ????
? decorators="time,level”: ??? ?? ?????? ????.
jcmd hidden function – VM.log

37 / 58
VM.log options
? output
? output_options
? what
? decorators
? disable
? list
? rotate

38 / 58
VM.log options
? output: ?? ?? ?? ??
? output_options: filecount (?? ??), filesize (?? ??)? ??
? what: ?? ?? ?? (trace, debug, info, warning, error)
? decorators: ?? ??? ??
? disable: ?? ??
? list: ?? ???
? rotate: ?? ???? ??
?? ??
? ?? ??, ?? ?????? ??, ?? ??? ???? list ?? ?? ? ???

39 / 58
VM.log ??? ????
? ?? ??
? output ??? ??? ????, JVM ? ??? ??? ??? ??? ??? ?? ??? ???? ?? ??
? ?? ?? ?? : pwdx <pid> (linux only)
? ?? ???? ???? ??? ???? ???? ??? ?? ??? ? ????,
???? ?? ?? ?? ???? ??? ?? ?
?? ??
? http://openjdk.java.net/jeps/158
$ jcmd <pid> VM.log list

40 / 58
VM.log ?? ???? ?
#!/bin/bash
pid=$1
echo "PID="$pid
jcmd $pid VM.log output="file=/develop/nhnforward/jcmd/gc.log" what="gc=debug" decorators="time,level"
sleep 3
cat gc.log
jcmd $pid VM.log list
jcmd $pid VM.log output="#2" what="all=off"

41 / 58
VM.log ?? ???? ?
#!/bin/bash
pid=$1
echo "PID="$pid
jcmd $pid VM.log output="file=/develop/nhnforward/jcmd/gc.log" what="gc=debug" decorators="time,level"
sleep 3
cat gc.log
jcmd $pid VM.log list
jcmd $pid VM.log output="#2" what="all=off"
logging ??
VM ? ?? ?? ??
VM ? ???? ?? GC ?? ??

43 / 58
NMT ??
? NMT == Native Memory Tracking
? HotSpot JVM ? ?? ??? ???? ???? ??
NMT ???
1. JVM ??? ??? ??? ??
? -XX:NativeMemoryTracking=summary ?? -XX:NativeMemoryTracking=detail
2. baseline ??
? jcmd <pid> VM.native_memory baseline
3. diff ? ??
? jcmd <pid> VM.native_memory detail.diff
jcmd hidden function – NMT

44 / 58
?? ??
? https://docs.oracle.com/javase/8/docs/technotes/guides/vm/nmt-8.html
? https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html
jcmd hidden function – NMT

46 / 58
jhsdb ??
? Java 9 ?? JDK? ???? ???? ?? ??
?
? ??
? core ?? ??
? crash ? JVM? ??? ??? ? ??
? ?? ??
? ?? ?? ?? ??????? ? ??? ?? ?? JVM? ???
? core ?? ??? gcore ???? core ??? ??? ? ???,
Heap ??? ?? ?? ? ? ?? ?? ??? ???.
??? ??? ?? ?? ??? ?
? https://docs.oracle.com/javase/9/tools/jhsdb.htm
jhsdb?

47 / 58
jhsdb ???
? sudo jhsdb [option] [--pid pid | --exe executable --core coredump]
? ????? root ???? ???? ?? ???
? JVM ? ? ?? ?? ??
? ?? ??? ??? ?? ?? ??
http://marxsoftware.blogspot.com/2017/06/jhsdb-jdk9.html
jhsdb usage

48 / 58
jhsdb ??
? jhsdb clhsdb
? jhsdb debugd
? jhsdb hsdb
jhsdb options

49 / 58
jhsdb ??
? jhsdb clhsdb: ????? ???
? jhsdb debugd: ?? ??? ?? ??
? jhsdb hsdb: ????? GUI ???
jhsdb options

52 / 58
jhsdb ?? ???
? jhsdb jstack
? jhsdb jmap
? jhsdb jinfo
? jhsdb jsnap
jhsdb other options

54 / 58
http://openjdk.java.net/projects/code-tools/
OpenJDK tools

55 / 58
JMH ??
? Java Microbenchmark Harness
? http://openjdk.java.net/projects/code-tools/jmh/
? ???? ????? ???? ?? ??
? ?? ??? ??? ??? ??
JMH? ???? ?? ??
? (?? ????? ?? ?? ??? ??? ??) ???? ????? ??
? JVM ???? ???? ??? ?? ??
JMH

56 / 58
JMH
JMH ?? ???? ??
? ?? ?? ??
? Mode.AverageTime
? Mode.SampleTime
? Mode.SingleShotTime
? ???
? Mode.Throughput
? ??
? Mode.All
?? ??
? ?? ??
? http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/

57 / 58
jcstress
jcstress ??
? Java Concurrency Stress tests
? https://wiki.openjdk.java.net/display/CodeTools/jcstress
?? ??
? ?? ??
? https://shipilev.net/talks/jvmls-July2013-jcstress.pdf
? ?? ??
? http://hg.openjdk.java.net/code-tools/jcstress/file/tip/jcstress-samples/src/main/java/org/openjdk/jcstress/samples

58 / 58
jol
jol ??
? Java Object Layout
? http://openjdk.java.net/projects/code-tools/jol/
jol ?? ??
? ?? ?? ??
? ?? ????, ?? ??, ?? ?, ?? ?? ?? (data on alignment losses)
? ?? ?? ??
? ?????? ?? ??? ???, ??, ?? ??? ???? ?? ?? ?

59 / 58
jol usage example
$ java -jar jol-cli/target/jol-cli.jar internals java.util.HashMap
Running 64-bit HotSpot VM.
Using compressed oop with 3-bit shift.
Using compressed klass with 3-bit shift.
Objects are 8 bytes aligned.
Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
java.util.HashMap object internals:
OFFSET SIZE TYPE DESCRIPTION VALUE
0 4 (object header) 05 00 00 00 (0000 0101 0000 0000 0000 0000 0000 0000)
4 4 (object header) 00 00 00 00 (0000 0000 0000 0000 0000 0000 0000 0000)
8 4 (object header) 8c 3b 00 f8 (1000 1100 0011 1011 0000 0000 1111 1000)
12 4 Set AbstractMap.keySet null
16 4 Collection AbstractMap.values null
20 4 int HashMap.size 0
24 4 int HashMap.modCount 0
28 4 int HashMap.threshold 0
32 4 float HashMap.loadFactor 0.75
36 4 Node[] HashMap.table null
40 4 Set HashMap.entrySet null
44 4 (loss due to the next object alignment)
Instance size: 48 bytes (reported by Instrumentation API)
Space losses: 0 bytes internal + 4 bytes external = 4 bytes total

61 / 58
??? ?? ??? ??? ???? ??? ?? ?? ?? ??
? ?? ??? ????? ????? ???
? ?? ?? ??(log)??? ????? ??
? ?
??? ??? ?

62 / 58
???? ??? ??? ?? ?? ???,
? ?? ?????,
? ??? ????,
? ? ??? ???? ??? ????,
? ? ??? ?? ? ??.
??? ?? ???? ???? ??? ??
???? ??? ? ?? ??

63 / 58
?? ???? ? ?? ??? ?? ?? ??
? jvmtop
? jcmd
???? ??? ? ?? ??

Q&AQ&A

THANK YOUTHANK YOU

狠狠撸

[2018] Java? ??, Java? ?? ???

Recommended

More Related Content

What's hot (20)

Similar to [2018] Java? ??, Java? ?? ??? (20)

More from NHN FORWARD (20)

[2018] Java? ??, Java? ?? ???