際際滷

際際滷Share a Scribd company logo

Seeing is Understanding: Debugging with the Multicore Visualizer

M
marckhouzam

This document discusses the challenges of debugging multicore systems and introduces the Multicore Visualizer tool. It provides an overview of the Multicore Visualizer, which presents a graphical view of application state on multicore systems to aid debugging. It demonstrates the tool's capabilities and discusses how the underlying visualization framework can be extended. The document concludes by describing ongoing work to further improve multicore debugging support.

1 of 35
Downloaded 36 times
SEEING IS UNDERSTANDING: DEBUGGING WITH THE MULTICORE VISUALIZER EclipseCon 2012 William R. Swanson Marc Khouzam
ABOUT US William R. Swanson Lead IDE Developer, Tilera Corporation CDT Committer, developed Visualizer framework and UI 22 years in parallel hardware, software & developer UI tools Marc Khouzam (@marckhouzam) Lead CDT developer, Ericsson CDT Committer, lead of Debug component (DSF-GDB) 15 years in Telecom: Multicore chips Multi CPU boards Multi board nodes Need better tools! EclipseCon 2012
AGENDA The Challenge of Multicore An Answer: Visualization The Multicore Visualizer & Framework (demo) Extending the Visualizer (demo) The CDT Multicore Debug Workgroup Current Projects, Future Plans After the talk: demo on 24-core machine. EclipseCon 2012
MULTICORE SYSTEMS Ambrics 336-core Am2045 Pluralitys 256-core Hypercore Tilera 100-core Tile-GX Intels 80-core Teraflop ClearSpeed 192-core CSX700 Coherent Logix 100-core HyperX AOCS 128-core ModemX Adapteva 64-core Epiphany 4 EclipseCon 2012
MULTICORE SCALABILITY Debug elements exploding - 100s of cores - 100s of processes - 1000s of threads Can be a literal embarrassment of riches... How to view and control on that scale? EclipseCon 2012
THE MULTICORE CHALLENGE Were in big data era, moving to big algorithm era lots of processes/threads, complex interactions ever-increasing need for a big picture overview CPU IN CPU IN OUT ... OUT CPU EclipseCon 2012
EXAMPLE: TILERA PROCESSOR Distinct pipeline stages or modes can run on different tiles, can also re-affinitize dynamically to help load-balancing. Data In Data Out Data In Data Out
EXAMPLE: TILERA PROCESSOR And this is ~100 cores! Imagine debugging CPUs / GPUs with 100s or 1000s of cores... where the app is NOT data-parallel or lock-step SIMD! Were going to need better tools to handle this!
CURRENT TOOLS DONT SCALE Command-line GDB multiple shells, even for only a few processes, or for multithreaded GDB, having to remember which thread youre on EclipseCon 2012
CURRENT TOOLS DONT SCALE Eclipses Debug View way better for managing large numbers of processes/threads but... for big apps, essentially a flat list too much repetitive detail cant easily see overall layout/behavior difficult to find and interact with important processes/threads... (like this one) -------------------------------------> EclipseCon 2012
THE BIG PICTURE: VISUALIZATION Need for new ways of looking at applications... One answer: visualization tools Visualization is the big picture: the important stuff made easily visible. What this means is that we shouldn't abbreviate the truth but rather get a new method of presentation. Edward Tufte EclipseCon 2012
A VISUALIZATION FRAMEWORK Theres also a need for a framework Rather than having everyone recreate the wheel, we need a platform to support visualization In the Eclipse spirit it should be flexible and extensible Should handle boilerplate code: - view/workbench interaction - selection - toolbars and context menus Visualizations should be able to focus on presentation. EclipseCon 2012
THE MULTICORE VISUALIZER New feature, currently optional Being added to CDT in Juno. EclipseCon 2012
THE MULTICORE VISUALIZER Visualizer presents an overview of application debug state Doesnt replace the Debug View, it augments it Analogy: adding pictures to a journal article. Pictures dont replace the text, instead they provide high-level context that makes it easier to parse the details. When in the Course of just powers from the When in the Course of are endowed by their human events it becomes consent of the governed, human events it becomes Creator with certain necessary for one people to That whenever any Form of necessary for one people to unalienable Rights, that dissolve the political bands Government becomes dissolve the political bands among these are Life, which have connected them destructive of these ends, it which have connected them Liberty and the pursuit of with another and to assume is the Right of the People to with another and to assume Happiness. That to among the powers of the alter or to abolish it, and to secure these rights, earth, the separate and institute new Government, Governments are instituted equal station to which the laying its foundation on among Men, deriving their Laws of Nature and of such principles and just powers from the Nature's God entitle them, a organizing its powers in consent of the governed, decent respect to the such form, as to them shall That whenever any Form of opinions of mankind seem most likely to effect Government becomes requires that they should their Safety and Happiness. destructive of these ends, it declare the causes which Prudence, indeed, will impel them to the dictate that Governments among the powers of the separation. long established should not earth, the separate and be changed for light and equal station to which the We hold these truths to be transient causes; and Laws of Nature and of self-evident, that all men accordingly all experience Nature's God entitle them, a are created equal, that they hath shewn that mankind decent respect to the are endowed by their are more disposed to opinions of mankind Creator with certain suffer, while evils are requires that they should is the Right of the People to unalienable Rights, that sufferable than to right declare the causes which alter or to abolish it, and to among these are Life, themselves by abolishing impel them to the institute new Government, Liberty and the pursuit of the forms to which they are separation. laying its foundation on Happiness. That to accustomed. But when a such principles and secure these rights, long train of abuses and We hold these truths to be organizing its powers in Governments are instituted usurpations, pursuing self-evident, that all men such form, as to them shall among Men, deriving their invariably the same Object are created equal, that they EclipseCon 2012
THE MULTICORE VISUALIZER Visualizer is interactive can select & interact with program elements (cores, processes, threads) Can still use Debug View to get more detail EclipseCon 2012
THE MULTICORE VISUALIZER Visualizer display scales to different cpu/core layouts: EclipseCon 2012
THE MULTICORE VISUALIZER Based on Grid View visualization in the Tilera IDE: EclipseCon 2012
MULTICORE VISUALIZER Demo EclipseCon 2012
MULTICORE VISUALIZER Demo: New Visualizer view in workbench Displays cpus, cores, processes & threads Sample program to debug: multi-process, multi-threaded View updates automatically as program layout changes Shows execution state of cpus, cores, processes, threads Click/drag selection, updating of Debug View Debug View state reflected in Visualizer Breakpoint handling (resume, step, etc.) Commands accessible from toolbars, context menu, and shortcuts Variables view (as usual) reflects current selection Displays crashed processes/threads in red Can click on problem thread(s) to select and view source Especially handy where there are multiple failures! EclipseCon 2012
VISUALIZER FRAMEWORK VisualizerView (Selection handling, Menus, etc.) IVisualizer MulticoreVisualizer Model Object (DSF state data, GraphicCanvas provided by GDB) EclipseCon 2012
ON BEYOND DEBUGGING Theres a framework because one view isnt sufficient Multicore Visualizer is currently aimed at debugging on homogenous multicore platforms; there are others Can extend multicore visualizer to suit your platform Also potential visualization uses beyond debugging Visualizations of selected data Use of overlays to select different kinds/views of data Static code analysis displays, etc. Framework includes examples you can build upon EclipseCon 2012
MULTIPLE VISUALIZERS Displayed visualizer is based on current selection. For multicore visualizer, the selection is DSF-GDB data Can also base visualizer on other information: Selected projects (example: display of warnings/errors) Selected text (program text, etc.) Anything thats selectable can potentially have a visualization associated with it. Visualizers report a weight indicating what they can handle The visualizer view selects the right visualizer for the current selection, based on visualizers reported weights EclipseCon 2012
MULTIPLE VISUALIZERS Demo: problem count visualizer source text analysis graph EclipseCon 2012
SEEING IS UNDERSTANDING Visualization is necessary as a big picture approach to large applications on multicore hardware You cant debug what you cant see What you can see, you can understand. What you understand, you can reason and feel sure about EclipseCon 2012
WE NEED NEW TOOLS The Multicore Visualizer is not an end, but a beginning. Visualizer exemplifies new kinds of tools and approaches needed for multicore and the big application era. These new tools are going to come from YOU the Eclipse developer community. The Visualizer framework provides a platform for development of more (and better!) visualizations. Lets make sure our favorite platform, Eclipse, is ready for the tasks ahead! EclipseCon 2012
IT TAKES MORE THAN ONE TOOL Visualizer is just one of the projects currently being worked on by the CDT Multicore Debug Workgroup EclipseCon 2012
MULTICORE DEBUG WORKGROUP Joint effort to bring multicore debugging to the CDT Visualizer, Pin&Clone, Multiprocess debug, etc Support for those that want to add new features Monthly conference calls (open to all interested and free ) http://wiki.eclipse.org/CDT/MultiCoreDebugWorkingGroup EclipseCon 2012
WORKGROUP PLANS Process/Thread/Core sets, as supported by GDB Global breakpoints OS Awareness Dynamic Printf Fully-integrated GDB console complete GDB console that can be used jointly with Eclipse Scalability and performance Ability for GDB to handle 100s or 1000s of threads/processes and more EclipseCon 2012
PTC SETS Process Thread Core (PTC) sets control groups of debug elements: Step threads numbered between 34 and 59 Step all threads running on core 2 Stop everything running on cores 5 to 7, preventing new threads from being started EclipseCon 2012
GLOBAL BREAKPOINTS Applies to every process Auto attach when hit Un-started or short lived process Kernel module required EclipseCon 2012
OS AWARENESS Sometimes examining OS Resources can help find bug Message Queues All Processes Loaded Kernel Modules Semaphores Sockets All Threads Process Groups Shared Memory Segments File Descriptors EclipseCon 2012
DYNAMIC-PRINTF Sometimes tracing is necessary DYNAMIC PRINTF COMPILED INSERTED AT RUNTIME DEBUGGER PRINTF WITH GDB EclipseCon 2012
SOME REFERENCES Multicore Debug workgroup, http://wiki.eclipse.org/CDT/MultiCoreDebugWorkingGroup CDT project, http://www.eclipse.org/cdt CDT Wiki, http://wiki.eclipse.org/CDT GDB, http://sourceware.org/gdb/ Multicore Visualizer, http://bugs.eclipse.org/335027 Contact: cdt-dev@eclipse.org EclipseCon 2012
Q&A Thanks for your time. Reminder: you can provide feedback: 1) go to www.eclipsecon.org 2) on Visualizer session, click EVALUATE 3) Vote ( ) and add comments! Enjoy the rest of EclipseCon! EclipseCon 2012
Multicore Visualizer Demo 際際滷s, Version 1.0, 3/22/2012 EclipseCon 2012

More Related Content

Seeing is Understanding: Debugging with the Multicore Visualizer

  • 1. SEEING IS UNDERSTANDING: DEBUGGING WITH THE MULTICORE VISUALIZER EclipseCon 2012 William R. Swanson Marc Khouzam
  • 2. ABOUT US William R. Swanson Lead IDE Developer, Tilera Corporation CDT Committer, developed Visualizer framework and UI 22 years in parallel hardware, software & developer UI tools Marc Khouzam (@marckhouzam) Lead CDT developer, Ericsson CDT Committer, lead of Debug component (DSF-GDB) 15 years in Telecom: Multicore chips Multi CPU boards Multi board nodes Need better tools! EclipseCon 2012
  • 3. AGENDA The Challenge of Multicore An Answer: Visualization The Multicore Visualizer & Framework (demo) Extending the Visualizer (demo) The CDT Multicore Debug Workgroup Current Projects, Future Plans After the talk: demo on 24-core machine. EclipseCon 2012
  • 4. MULTICORE SYSTEMS Ambrics 336-core Am2045 Pluralitys 256-core Hypercore Tilera 100-core Tile-GX Intels 80-core Teraflop ClearSpeed 192-core CSX700 Coherent Logix 100-core HyperX AOCS 128-core ModemX Adapteva 64-core Epiphany 4 EclipseCon 2012
  • 5. MULTICORE SCALABILITY Debug elements exploding - 100s of cores - 100s of processes - 1000s of threads Can be a literal embarrassment of riches... How to view and control on that scale? EclipseCon 2012
  • 6. THE MULTICORE CHALLENGE Were in big data era, moving to big algorithm era lots of processes/threads, complex interactions ever-increasing need for a big picture overview CPU IN CPU IN OUT ... OUT CPU EclipseCon 2012
  • 7. EXAMPLE: TILERA PROCESSOR Distinct pipeline stages or modes can run on different tiles, can also re-affinitize dynamically to help load-balancing. Data In Data Out Data In Data Out
  • 8. EXAMPLE: TILERA PROCESSOR And this is ~100 cores! Imagine debugging CPUs / GPUs with 100s or 1000s of cores... where the app is NOT data-parallel or lock-step SIMD! Were going to need better tools to handle this!
  • 9. CURRENT TOOLS DONT SCALE Command-line GDB multiple shells, even for only a few processes, or for multithreaded GDB, having to remember which thread youre on EclipseCon 2012
  • 10. CURRENT TOOLS DONT SCALE Eclipses Debug View way better for managing large numbers of processes/threads but... for big apps, essentially a flat list too much repetitive detail cant easily see overall layout/behavior difficult to find and interact with important processes/threads... (like this one) -------------------------------------> EclipseCon 2012
  • 11. THE BIG PICTURE: VISUALIZATION Need for new ways of looking at applications... One answer: visualization tools Visualization is the big picture: the important stuff made easily visible. What this means is that we shouldn't abbreviate the truth but rather get a new method of presentation. Edward Tufte EclipseCon 2012
  • 12. A VISUALIZATION FRAMEWORK Theres also a need for a framework Rather than having everyone recreate the wheel, we need a platform to support visualization In the Eclipse spirit it should be flexible and extensible Should handle boilerplate code: - view/workbench interaction - selection - toolbars and context menus Visualizations should be able to focus on presentation. EclipseCon 2012
  • 13. THE MULTICORE VISUALIZER New feature, currently optional Being added to CDT in Juno. EclipseCon 2012
  • 14. THE MULTICORE VISUALIZER Visualizer presents an overview of application debug state Doesnt replace the Debug View, it augments it Analogy: adding pictures to a journal article. Pictures dont replace the text, instead they provide high-level context that makes it easier to parse the details. When in the Course of just powers from the When in the Course of are endowed by their human events it becomes consent of the governed, human events it becomes Creator with certain necessary for one people to That whenever any Form of necessary for one people to unalienable Rights, that dissolve the political bands Government becomes dissolve the political bands among these are Life, which have connected them destructive of these ends, it which have connected them Liberty and the pursuit of with another and to assume is the Right of the People to with another and to assume Happiness. That to among the powers of the alter or to abolish it, and to secure these rights, earth, the separate and institute new Government, Governments are instituted equal station to which the laying its foundation on among Men, deriving their Laws of Nature and of such principles and just powers from the Nature's God entitle them, a organizing its powers in consent of the governed, decent respect to the such form, as to them shall That whenever any Form of opinions of mankind seem most likely to effect Government becomes requires that they should their Safety and Happiness. destructive of these ends, it declare the causes which Prudence, indeed, will impel them to the dictate that Governments among the powers of the separation. long established should not earth, the separate and be changed for light and equal station to which the We hold these truths to be transient causes; and Laws of Nature and of self-evident, that all men accordingly all experience Nature's God entitle them, a are created equal, that they hath shewn that mankind decent respect to the are endowed by their are more disposed to opinions of mankind Creator with certain suffer, while evils are requires that they should is the Right of the People to unalienable Rights, that sufferable than to right declare the causes which alter or to abolish it, and to among these are Life, themselves by abolishing impel them to the institute new Government, Liberty and the pursuit of the forms to which they are separation. laying its foundation on Happiness. That to accustomed. But when a such principles and secure these rights, long train of abuses and We hold these truths to be organizing its powers in Governments are instituted usurpations, pursuing self-evident, that all men such form, as to them shall among Men, deriving their invariably the same Object are created equal, that they EclipseCon 2012
  • 15. THE MULTICORE VISUALIZER Visualizer is interactive can select & interact with program elements (cores, processes, threads) Can still use Debug View to get more detail EclipseCon 2012
  • 16. THE MULTICORE VISUALIZER Visualizer display scales to different cpu/core layouts: EclipseCon 2012
  • 17. THE MULTICORE VISUALIZER Based on Grid View visualization in the Tilera IDE: EclipseCon 2012
  • 19. MULTICORE VISUALIZER Demo: New Visualizer view in workbench Displays cpus, cores, processes & threads Sample program to debug: multi-process, multi-threaded View updates automatically as program layout changes Shows execution state of cpus, cores, processes, threads Click/drag selection, updating of Debug View Debug View state reflected in Visualizer Breakpoint handling (resume, step, etc.) Commands accessible from toolbars, context menu, and shortcuts Variables view (as usual) reflects current selection Displays crashed processes/threads in red Can click on problem thread(s) to select and view source Especially handy where there are multiple failures! EclipseCon 2012
  • 20. VISUALIZER FRAMEWORK VisualizerView (Selection handling, Menus, etc.) IVisualizer MulticoreVisualizer Model Object (DSF state data, GraphicCanvas provided by GDB) EclipseCon 2012
  • 21. ON BEYOND DEBUGGING Theres a framework because one view isnt sufficient Multicore Visualizer is currently aimed at debugging on homogenous multicore platforms; there are others Can extend multicore visualizer to suit your platform Also potential visualization uses beyond debugging Visualizations of selected data Use of overlays to select different kinds/views of data Static code analysis displays, etc. Framework includes examples you can build upon EclipseCon 2012
  • 22. MULTIPLE VISUALIZERS Displayed visualizer is based on current selection. For multicore visualizer, the selection is DSF-GDB data Can also base visualizer on other information: Selected projects (example: display of warnings/errors) Selected text (program text, etc.) Anything thats selectable can potentially have a visualization associated with it. Visualizers report a weight indicating what they can handle The visualizer view selects the right visualizer for the current selection, based on visualizers reported weights EclipseCon 2012
  • 23. MULTIPLE VISUALIZERS Demo: problem count visualizer source text analysis graph EclipseCon 2012
  • 24. SEEING IS UNDERSTANDING Visualization is necessary as a big picture approach to large applications on multicore hardware You cant debug what you cant see What you can see, you can understand. What you understand, you can reason and feel sure about EclipseCon 2012
  • 25. WE NEED NEW TOOLS The Multicore Visualizer is not an end, but a beginning. Visualizer exemplifies new kinds of tools and approaches needed for multicore and the big application era. These new tools are going to come from YOU the Eclipse developer community. The Visualizer framework provides a platform for development of more (and better!) visualizations. Lets make sure our favorite platform, Eclipse, is ready for the tasks ahead! EclipseCon 2012
  • 26. IT TAKES MORE THAN ONE TOOL Visualizer is just one of the projects currently being worked on by the CDT Multicore Debug Workgroup EclipseCon 2012
  • 27. MULTICORE DEBUG WORKGROUP Joint effort to bring multicore debugging to the CDT Visualizer, Pin&Clone, Multiprocess debug, etc Support for those that want to add new features Monthly conference calls (open to all interested and free ) http://wiki.eclipse.org/CDT/MultiCoreDebugWorkingGroup EclipseCon 2012
  • 28. WORKGROUP PLANS Process/Thread/Core sets, as supported by GDB Global breakpoints OS Awareness Dynamic Printf Fully-integrated GDB console complete GDB console that can be used jointly with Eclipse Scalability and performance Ability for GDB to handle 100s or 1000s of threads/processes and more EclipseCon 2012
  • 29. PTC SETS Process Thread Core (PTC) sets control groups of debug elements: Step threads numbered between 34 and 59 Step all threads running on core 2 Stop everything running on cores 5 to 7, preventing new threads from being started EclipseCon 2012
  • 30. GLOBAL BREAKPOINTS Applies to every process Auto attach when hit Un-started or short lived process Kernel module required EclipseCon 2012
  • 31. OS AWARENESS Sometimes examining OS Resources can help find bug Message Queues All Processes Loaded Kernel Modules Semaphores Sockets All Threads Process Groups Shared Memory Segments File Descriptors EclipseCon 2012
  • 32. DYNAMIC-PRINTF Sometimes tracing is necessary DYNAMIC PRINTF COMPILED INSERTED AT RUNTIME DEBUGGER PRINTF WITH GDB EclipseCon 2012
  • 33. SOME REFERENCES Multicore Debug workgroup, http://wiki.eclipse.org/CDT/MultiCoreDebugWorkingGroup CDT project, http://www.eclipse.org/cdt CDT Wiki, http://wiki.eclipse.org/CDT GDB, http://sourceware.org/gdb/ Multicore Visualizer, http://bugs.eclipse.org/335027 Contact: cdt-dev@eclipse.org EclipseCon 2012
  • 34. Q&A Thanks for your time. Reminder: you can provide feedback: 1) go to www.eclipsecon.org 2) on Visualizer session, click EVALUATE 3) Vote ( ) and add comments! Enjoy the rest of EclipseCon! EclipseCon 2012
  • 35. Multicore Visualizer Demo 際際滷s, Version 1.0, 3/22/2012 EclipseCon 2012