際際滷

際際滷Share a Scribd company logo

GDB? strace? Hang ?? Python Process ?? ???

?
?
Youngmin Koo
Youngmin Koo

Python ????? ????? ? ?? ?? ??????? ???? ?? ?? ???? ?? ?? PyCharm? ??? ????. PyCharm? JetBrains?? ?? GUI ???? ??? ? ?? Python IDE???. PyCharm?: ?? ????? ??? ??? (PyDev?) Python ????? ??? ? ????. ?? ????? ?? ?? Python ????? Attach? ???? ? ????. ?? ???? ??? ??? (PyDev?) Python ????? ??? ? ????. ??? ???? ???? ?? ????? ???? Attach?? ??? ???? ?? ????. ?? Python ??? ??? pdb? ????? ???? ?? ????? Attach?? ?? ???? ?? ????. ?? ????? ?? ?? ???? Python ????? ?? ????? ??? ?? ?? ?? ??? ?? ????? ?? ?? ?? ?? ?????. ? ? GDB? strace? ???? ???? ??? ????? ??? ? ????. ? ????? GDB? strace? ??? ????? ?? ??? ???? Python Process? Hang ?? ??? ??? ???? ???? ??? ????? ???.

1 of 50
Downloaded 32 times
GDB? strace? Hang ?? Python Process ?? ??? ??? <youngminz.kr@gmail.com>
whoami ? ??? / youngminz ? ???? ??? Python?? ?? ???? ?? ? ??? 2?? ??? ? ?? ????? ????? ??? ? (2018.01-) ? Python? Django, Linux? ??? ? ???? 5???? ????? ?? ? PyCon Korea 2014, 2016, 2017, 2018 ??
?? ??? ??? ? ?? ??? P2P ?? ??? ? ????? ??? ???? ? ?? ???? ?????? ??? ??? ????
???? ?? ? ? ???? ? 4.94% ???? 100??? ?? ?? 6?? ? ??? ? ? Case 1) 1?? ?, ＾?? ??? ??? ??? ?? ? ???￣ ? Case 2) 6?? ?, ＾?? ??? ??? ? ???, 1? ?? ? ???￣ ? Case 3) 6?? ?, ＾50??? ?? ???? 6??? ??? ???￣ ? ´´ ? ?? ?? ??? ??? ???? ???? ? ? ??????
??? ???: ???? >>> ?????? ??? ??? ????? ??? ????? ? ??? ??? ?? ??? ?? ???? ?? ???? ??? ? ?? ? ??? ??? ???? ? ??? ?????? ??? ??? ???? ? ??? ?? ??? ?? ?? ?????? ??? ????? ??? ? ??? ??? ??? ?? ??? ?????? ? ?? ??? ??? ??? ??? ?????? ? ´
??? ???: ?????? ? ??? ?? ????? ??? ??? ?? ?? ??? ? ???? ???? ?? ????? ? ?? ???? ??! ? ??? ? ???: Django Test Case ? ??? ?? ??? ???? ???? ? ?? ?? ??? ?? ?!
Test Case? ??? ? ???? ?? ??? ??? ? ?? ? ???? ?? ??? ???? ??? ? ?? ? ???? ??? ? ????? ??? ?? ? ?? ?? ??? ?? ??? [2] Django Documentation, https://docs.djangoproject.com/
Test Case ? ?? ??? ????? 2,000? ??? ??? ??? ?? ? ?? ?? ??? ? ?? ??? ?? ? ???? ???? ???? ? ???? ??! ? ??? Jenkins ???? ???? ???? ???
???? ? ??? ? ＾???, ???? ? ????? ?? ? ???? ??￣ ? ??? ?? ? ? ???? ?? ? 20?? ?? ? ???????????????? ? ???? ? ?? ??? ?? ?? ???? ?? ? ?? ? ??
?? ?? 1. ????? ? ????? ??? ????? ? ? 2. ??? ??? ?????? ?? 3. CPU ???? 0% ? ???? ??? ????? ???? ??!
????? ? ????? ???? ??? ?? ?? def add(x, y): value = (x - y) / 0 return value ? ??? ???? ???? ? ?? ???´ ? ??? ?? ?? ??
GDB? strace? Hang ?? Python Process ?? ???
Traceback / Call Stack Traceback (most recent call last): File "main.py", line 6, in <module> print(add(10, 5)) File "main.py", line 3, in add value = (x - y) / 0 ZeroDivisionError: division by zero def add(x, y): value = (x - y) / 0 return value print(add(10, 5)) ? ?? ??? ???? ??? ?? ?? ? ???? ?? ?? ?? ? ???? ??? ? ???? ? ?!! ??? ??
???: ?? ? ? ??? ?? ? Traceback? ?? ???? ? ??? ?? ?? ??? logger.debug ??? ?? ? ? ??? ??? ??? ? ?? ?? ?? ? ?? ????? ??? ??? <<< ??!
???: ?? ? ? segmentation fault (core dumped) ? ????? ?? ?? Traceback? ? ??? ?? ???? ? ????? ?? ?? ? ??? ??? ?? ?? ???? ? ? ??? ? ???? ?? ?????? ??? ???? ? ???? ??? ?
?? ?? Python ????? ?????? ?? ??
PyCharm ? ? ? ?? ? ? ???? ??? ??? ???? ? ???? ?? ?? Python Process? ??? ????? ? ???? ??? ??? ???? ? ??? ? ???? ?? ?? ????? ??? ??? ??
pdb: The Python Debugger ? ?? ? ? Python ?? ??? ? ?? ???? ??? ??? ??? ? ?? ? ??? ? ?? ?? ????? ??? ??? ??
GDB: The GNU Project Debugger ? ???? ?? ????? ??? ??!!! ? C / C++? ???? ? Python ????? ? ??? Java Virtual Machine / Go ??? ? ?? ????? ???? ???? ??? sudo ???? ?
?? ?? (?????) ? ?? ??? ? ???. ? ?????? ? ????. ? ????, ?????? ? ???.
?? ?? (CPython ??) ? Python? ?? ???? CPython? ? C? ?????. ? C ????? ???? ? GDB? ? ? ??. ? ????, CPython ???? ???? ? GDB? ? ? ??. ? + Python? ?????? ???? ??? ?? ? ? ??
hanging.py $ python3 hanging.py current pid 20457 import os, time print('current pid', os.getpid()) def hang(): time.sleep(30000) hang()
??? GDB ??? ? PID? ??? ???? ?? ????? ??: gdb Cp <pid> ? bt ??? C Level Call Stack? ??? ? C ?? ???? continue ? Ctrl + C? ?? ?? ? Ctrl + D? ??? ??? ?? ???? continue
hanging.py $ sudo gdb -p 20457 GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1 Copyright (C) 2016 Free Software Foundation, Inc. Attaching to process 20457 Reading symbols from /usr/bin/python3.5...(no debugging symbols found)...done. ... (gdb) bt
hanging.py (gdb) bt #0 0x00007f2214c025b3 in __select_nocancel () at ../sysdeps/u #1 0x00000000005fa30d in ?? () #2 0x000000000053b9c6 in PyEval_EvalFrameEx () #3 0x000000000053b7e4 in PyEval_EvalFrameEx () #4 0x0000000000540199 in ?? () #5 0x0000000000540e4f in PyEval_EvalCode () #6 0x000000000060c272 in ?? () #7 0x000000000060e71a in PyRun_FileExFlags () #8 0x000000000060ef0c in PyRun_SimpleFileExFlags () #9 0x000000000063fb26 in Py_Main () #10 0x00000000004cfeb1 in main () import os, time print('current pid', os.getpid()) def hang(): time.sleep(30000) hang()
C ?? ? ??? ??? ? ???? Python ?? ? ??? ? ??!
??? ?? ? ???? ??? ??? ?? C ????? ??? ?? ?? ? GDB? ??? ?? ? ???? ?? ??? ??? ?? ???? ??? ??? ? ? ? ?? ??? ??? ??? ?? ??? ??? ? ???? ???? ? ?? ?? (????) ? CPython ?? ?? ? ??? ?? ?? ? ? ??? ?!
?? (Ubuntu 16.04 ??) $ sudo apt-get install dpkg-dev $ sudo apt-get source python3 $ sudo apt-get install python3-dbg
hanging.py $ sudo gdb -p 20457 GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1 Copyright (C) 2016 Free Software Foundation, Inc. Attaching to process 20457 Reading symbols from /usr/bin/python3.5...Reading symbols from /usr/lib/debug/.build-id/59/a8ef36ca241df246.debug...done. ... (gdb) bt
hanging.py import os, time print('current pid', os.getpid()) def hang(): time.sleep(30000) hang()
GDB? strace? Hang ?? Python Process ?? ???
? ? ?? ? ? C Level? ??? Traceback? ? ? ?? ? ??? ??? ?? ??? ? Python Level? Traceback? ? ?? ????
?? ??: GDB Python Extension ? ??? C Level? Traceback? Python Level? ?? ? ? GDB 7??? ? ?? ??? ???? ??
py-bt: Pythonic Traceback (gdb) py-bt Traceback (most recent call first): File "hanging.py", line 4, in hang File "hanging.py", line 5, in <module> import os, time print('current pid', os.getpid()) def hang(): time.sleep(30000) hang()
?? ??: GDB Python Extension ? ??? C Level? Traceback? Python Level? ?? ? ? GDB 7??? ? ?? ??? ???? ?? ? ?? ??? ??: ? py-bt / py-bt-full / py-down / py-list / py-locals / py-print / py-up
?? ?? ??
??? HTTP ?? import requests response = requests.get(｀http://private-api.example.com/¨) print(response) ??? ?? ?? ????, ??? ??? ?? ? ??
py-bt: Pythonic Traceback (gdb) py-bt Traceback (most recent call first): File "/urllib3/connection.py", line 81, increate_connection sock.connect(sa) ... File ￣/requests/api.py", line 67, in get return request('get', url, params=params, **kwargs) File "test.py", line 7, in <module> ? ? ??? ????? ??? ?? ?? ??
??? ??, strace ? ??? ? ??? ? ?? ??? ? ????? ?? ??? ?? ????? ? ? ?? ? ?? ?? ?? IP? TCP ??? ????? ? ?? ????? ???? ??? sudo ?? ?? ? sudo strace -p <pid>
strace Demo
strace $ sudo strace -p 10099 strace: Process 10099 attached connect(3, {sa_family=AF_INET, sin_port=htons(1245), sin_addr=inet_addr("124.124.124.124")}, 16) response = requests.get(｀http://private-api.example.com/¨)
strace $ sudo strace -p 10099 strace: Process 10099 attached connect(3, {sa_family=AF_INET, sin_port=htons(1245), sin_addr=inet_addr("124.124.124.124")}, 16) = -1 ETIMEDOUT (Connection timed out) response = requests.get(｀http://private-api.example.com/¨)
??: ｀???? 71? ??¨ ? ?????
HTTP ??? ??? ? ?? ? HTTP ??? Timeout? ????! ? Best Effort: ??? ??? ? ?? ? ??? ??? ?? ?? ?? ? DNS ??? ??? ?? ?? ? ??? ???? ??? Drop ? ?? ?? [3] Requests Documentation, http://docs.python-requests.org/en/master/user/quickstart/#timeouts
HTTP ??? ??? ? ?? ? Requests ??? ???, ? If no timeout is specified explicitly, requests do not time out. ? Nearly all production code should use this parameter in nearly all requests. ? ????, ? Timeout? ????? ???? ???, requests? ???? ?? ? ?. ? ???? ???? ??? ? ???? ? ????? ??? ?. [3] Requests Documentation, http://docs.python-requests.org/en/master/user/quickstart/#timeouts
HTTP ??? ??? ? ?? ? ??????? timeout ????? ?? ??? ?? ??? ?? ? Response = requests.get(´, timeout=5) [3] Requests Documentation, http://docs.python-requests.org/en/master/user/quickstart/#timeouts
?? ?? / ?? ? ?? ??? ??? API ??? ??? ???? ?? ? timeout? ???? ?? ??? ???? ??? ?? ? API ??? ??? ??? ?? ?? ?? ? Redis ??? ?? ??? ??? ?? ??? ? ???? Lint ??????? ?? ??? ?? ?? ? ? ?? ??? ??? ? ??? ??? ???? ???!!!
GDB / strace ? Use Case ? Segmentation Fault (core dumped) / Hang / Deadlock ? GDB ? ????? Traceback? ?? ??? ?? ? strace ? ? ????? ?? ?? ??? ?? ??? ??? ? ??
We are hiring!
Reference ? http://tinyurl.com/PyCon-US-2011-GdbPython ? https://wiki.python.org/moin/DebuggingWithGdb ? http://www.aboutlinux.info/2006/05/strace-very-powerful- troubleshooting.html ? https://www.youtube.com/watch?v=rB9rPdMRxIA ? https://www.youtube.com/watch?v=X6O5vyAO9_c

More Related Content

GDB? strace? Hang ?? Python Process ?? ???

  • 1. GDB? strace? Hang ?? Python Process ?? ??? ??? <youngminz.kr@gmail.com>
  • 2. whoami ? ??? / youngminz ? ???? ??? Python?? ?? ???? ?? ? ??? 2?? ??? ? ?? ????? ????? ??? ? (2018.01-) ? Python? Django, Linux? ??? ? ???? 5???? ????? ?? ? PyCon Korea 2014, 2016, 2017, 2018 ??
  • 3. ?? ??? ??? ? ?? ??? P2P ?? ??? ? ????? ??? ???? ? ?? ???? ?????? ??? ??? ????
  • 4. ???? ?? ? ? ???? ? 4.94% ???? 100??? ?? ?? 6?? ? ??? ? ? Case 1) 1?? ?, ＾?? ??? ??? ??? ?? ? ???￣ ? Case 2) 6?? ?, ＾?? ??? ??? ? ???, 1? ?? ? ???￣ ? Case 3) 6?? ?, ＾50??? ?? ???? 6??? ??? ???￣ ? ´´ ? ?? ?? ??? ??? ???? ???? ? ? ??????
  • 5. ??? ???: ???? >>> ?????? ??? ??? ????? ??? ????? ? ??? ??? ?? ??? ?? ???? ?? ???? ??? ? ?? ? ??? ??? ???? ? ??? ?????? ??? ??? ???? ? ??? ?? ??? ?? ?? ?????? ??? ????? ??? ? ??? ??? ??? ?? ??? ?????? ? ?? ??? ??? ??? ??? ?????? ? ´
  • 6. ??? ???: ?????? ? ??? ?? ????? ??? ??? ?? ?? ??? ? ???? ???? ?? ????? ? ?? ???? ??! ? ??? ? ???: Django Test Case ? ??? ?? ??? ???? ???? ? ?? ?? ??? ?? ?!
  • 7. Test Case? ??? ? ???? ?? ??? ??? ? ?? ? ???? ?? ??? ???? ??? ? ?? ? ???? ??? ? ????? ??? ?? ? ?? ?? ??? ?? ??? [2] Django Documentation, https://docs.djangoproject.com/
  • 8. Test Case ? ?? ??? ????? 2,000? ??? ??? ??? ?? ? ?? ?? ??? ? ?? ??? ?? ? ???? ???? ???? ? ???? ??! ? ??? Jenkins ???? ???? ???? ???
  • 9. ???? ? ??? ? ＾???, ???? ? ????? ?? ? ???? ??￣ ? ??? ?? ? ? ???? ?? ? 20?? ?? ? ???????????????? ? ???? ? ?? ??? ?? ?? ???? ?? ? ?? ? ??
  • 10. ?? ?? 1. ????? ? ????? ??? ????? ? ? 2. ??? ??? ?????? ?? 3. CPU ???? 0% ? ???? ??? ????? ???? ??!
  • 11. ????? ? ????? ???? ??? ?? ?? def add(x, y): value = (x - y) / 0 return value ? ??? ???? ???? ? ?? ???´ ? ??? ?? ?? ??
  • 13. Traceback / Call Stack Traceback (most recent call last): File "main.py", line 6, in <module> print(add(10, 5)) File "main.py", line 3, in add value = (x - y) / 0 ZeroDivisionError: division by zero def add(x, y): value = (x - y) / 0 return value print(add(10, 5)) ? ?? ??? ???? ??? ?? ?? ? ???? ?? ?? ?? ? ???? ??? ? ???? ? ?!! ??? ??
  • 14. ???: ?? ? ? ??? ?? ? Traceback? ?? ???? ? ??? ?? ?? ??? logger.debug ??? ?? ? ? ??? ??? ??? ? ?? ?? ?? ? ?? ????? ??? ??? <<< ??!
  • 15. ???: ?? ? ? segmentation fault (core dumped) ? ????? ?? ?? Traceback? ? ??? ?? ???? ? ????? ?? ?? ? ??? ??? ?? ?? ???? ? ? ??? ? ???? ?? ?????? ??? ???? ? ???? ??? ?
  • 16. ?? ?? Python ????? ?????? ?? ??
  • 17. PyCharm ? ? ? ?? ? ? ???? ??? ??? ???? ? ???? ?? ?? Python Process? ??? ????? ? ???? ??? ??? ???? ? ??? ? ???? ?? ?? ????? ??? ??? ??
  • 18. pdb: The Python Debugger ? ?? ? ? Python ?? ??? ? ?? ???? ??? ??? ??? ? ?? ? ??? ? ?? ?? ????? ??? ??? ??
  • 19. GDB: The GNU Project Debugger ? ???? ?? ????? ??? ??!!! ? C / C++? ???? ? Python ????? ? ??? Java Virtual Machine / Go ??? ? ?? ????? ???? ???? ??? sudo ???? ?
  • 20. ?? ?? (?????) ? ?? ??? ? ???. ? ?????? ? ????. ? ????, ?????? ? ???.
  • 21. ?? ?? (CPython ??) ? Python? ?? ???? CPython? ? C? ?????. ? C ????? ???? ? GDB? ? ? ??. ? ????, CPython ???? ???? ? GDB? ? ? ??. ? + Python? ?????? ???? ??? ?? ? ? ??
  • 22. hanging.py $ python3 hanging.py current pid 20457 import os, time print('current pid', os.getpid()) def hang(): time.sleep(30000) hang()
  • 23. ??? GDB ??? ? PID? ??? ???? ?? ????? ??: gdb Cp <pid> ? bt ??? C Level Call Stack? ??? ? C ?? ???? continue ? Ctrl + C? ?? ?? ? Ctrl + D? ??? ??? ?? ???? continue
  • 24. hanging.py $ sudo gdb -p 20457 GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1 Copyright (C) 2016 Free Software Foundation, Inc. Attaching to process 20457 Reading symbols from /usr/bin/python3.5...(no debugging symbols found)...done. ... (gdb) bt
  • 25. hanging.py (gdb) bt #0 0x00007f2214c025b3 in __select_nocancel () at ../sysdeps/u #1 0x00000000005fa30d in ?? () #2 0x000000000053b9c6 in PyEval_EvalFrameEx () #3 0x000000000053b7e4 in PyEval_EvalFrameEx () #4 0x0000000000540199 in ?? () #5 0x0000000000540e4f in PyEval_EvalCode () #6 0x000000000060c272 in ?? () #7 0x000000000060e71a in PyRun_FileExFlags () #8 0x000000000060ef0c in PyRun_SimpleFileExFlags () #9 0x000000000063fb26 in Py_Main () #10 0x00000000004cfeb1 in main () import os, time print('current pid', os.getpid()) def hang(): time.sleep(30000) hang()
  • 26. C ?? ? ??? ??? ? ???? Python ?? ? ??? ? ??!
  • 27. ??? ?? ? ???? ??? ??? ?? C ????? ??? ?? ?? ? GDB? ??? ?? ? ???? ?? ??? ??? ?? ???? ??? ??? ? ? ? ?? ??? ??? ??? ?? ??? ??? ? ???? ???? ? ?? ?? (????) ? CPython ?? ?? ? ??? ?? ?? ? ? ??? ?!
  • 28. ?? (Ubuntu 16.04 ??) $ sudo apt-get install dpkg-dev $ sudo apt-get source python3 $ sudo apt-get install python3-dbg
  • 29. hanging.py $ sudo gdb -p 20457 GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1 Copyright (C) 2016 Free Software Foundation, Inc. Attaching to process 20457 Reading symbols from /usr/bin/python3.5...Reading symbols from /usr/lib/debug/.build-id/59/a8ef36ca241df246.debug...done. ... (gdb) bt
  • 30. hanging.py import os, time print('current pid', os.getpid()) def hang(): time.sleep(30000) hang()
  • 32. ? ? ?? ? ? C Level? ??? Traceback? ? ? ?? ? ??? ??? ?? ??? ? Python Level? Traceback? ? ?? ????
  • 33. ?? ??: GDB Python Extension ? ??? C Level? Traceback? Python Level? ?? ? ? GDB 7??? ? ?? ??? ???? ??
  • 34. py-bt: Pythonic Traceback (gdb) py-bt Traceback (most recent call first): File "hanging.py", line 4, in hang File "hanging.py", line 5, in <module> import os, time print('current pid', os.getpid()) def hang(): time.sleep(30000) hang()
  • 35. ?? ??: GDB Python Extension ? ??? C Level? Traceback? Python Level? ?? ? ? GDB 7??? ? ?? ??? ???? ?? ? ?? ??? ??: ? py-bt / py-bt-full / py-down / py-list / py-locals / py-print / py-up
  • 37. ??? HTTP ?? import requests response = requests.get(｀http://private-api.example.com/¨) print(response) ??? ?? ?? ????, ??? ??? ?? ? ??
  • 38. py-bt: Pythonic Traceback (gdb) py-bt Traceback (most recent call first): File "/urllib3/connection.py", line 81, increate_connection sock.connect(sa) ... File ￣/requests/api.py", line 67, in get return request('get', url, params=params, **kwargs) File "test.py", line 7, in <module> ? ? ??? ????? ??? ?? ?? ??
  • 39. ??? ??, strace ? ??? ? ??? ? ?? ??? ? ????? ?? ??? ?? ????? ? ? ?? ? ?? ?? ?? IP? TCP ??? ????? ? ?? ????? ???? ??? sudo ?? ?? ? sudo strace -p <pid>
  • 41. strace $ sudo strace -p 10099 strace: Process 10099 attached connect(3, {sa_family=AF_INET, sin_port=htons(1245), sin_addr=inet_addr("124.124.124.124")}, 16) response = requests.get(｀http://private-api.example.com/¨)
  • 42. strace $ sudo strace -p 10099 strace: Process 10099 attached connect(3, {sa_family=AF_INET, sin_port=htons(1245), sin_addr=inet_addr("124.124.124.124")}, 16) = -1 ETIMEDOUT (Connection timed out) response = requests.get(｀http://private-api.example.com/¨)
  • 43. ??: ｀???? 71? ??¨ ? ?????
  • 44. HTTP ??? ??? ? ?? ? HTTP ??? Timeout? ????! ? Best Effort: ??? ??? ? ?? ? ??? ??? ?? ?? ?? ? DNS ??? ??? ?? ?? ? ??? ???? ??? Drop ? ?? ?? [3] Requests Documentation, http://docs.python-requests.org/en/master/user/quickstart/#timeouts
  • 45. HTTP ??? ??? ? ?? ? Requests ??? ???, ? If no timeout is specified explicitly, requests do not time out. ? Nearly all production code should use this parameter in nearly all requests. ? ????, ? Timeout? ????? ???? ???, requests? ???? ?? ? ?. ? ???? ???? ??? ? ???? ? ????? ??? ?. [3] Requests Documentation, http://docs.python-requests.org/en/master/user/quickstart/#timeouts
  • 46. HTTP ??? ??? ? ?? ? ??????? timeout ????? ?? ??? ?? ??? ?? ? Response = requests.get(´, timeout=5) [3] Requests Documentation, http://docs.python-requests.org/en/master/user/quickstart/#timeouts
  • 47. ?? ?? / ?? ? ?? ??? ??? API ??? ??? ???? ?? ? timeout? ???? ?? ??? ???? ??? ?? ? API ??? ??? ??? ?? ?? ?? ? Redis ??? ?? ??? ??? ?? ??? ? ???? Lint ??????? ?? ??? ?? ?? ? ? ?? ??? ??? ? ??? ??? ???? ???!!!
  • 48. GDB / strace ? Use Case ? Segmentation Fault (core dumped) / Hang / Deadlock ? GDB ? ????? Traceback? ?? ??? ?? ? strace ? ? ????? ?? ?? ??? ?? ??? ??? ? ??
  • 50. Reference ? http://tinyurl.com/PyCon-US-2011-GdbPython ? https://wiki.python.org/moin/DebuggingWithGdb ? http://www.aboutlinux.info/2006/05/strace-very-powerful- troubleshooting.html ? https://www.youtube.com/watch?v=rB9rPdMRxIA ? https://www.youtube.com/watch?v=X6O5vyAO9_c