GNU Toolchain is the de facto standard of IT industrial and has been improved by comprehensive open source contributions. In this session, it is expected to cover the mechanism of compiler driver, system interaction (take GNU/Linux for example), linker, C runtime library, and the related dynamic linker. Instead of analyzing the system design, the session is use case driven and illustrated progressively.
The document discusses how a "Hello World" program works behind the scenes. It covers topics like compilation, linking, executable file formats, loading programs into memory, and process creation. The key points are:
1) A C program is compiled into an object file, then linked with library files to create an executable. The linker resolves symbols and relocates addresses.
2) Executable files use formats like ELF that contain machine code, data, symbol tables, and sections. Object files have a similar format.
3) When a program runs, the OS loads pages of the executable into memory as needed and sets up the process with its own virtual address space.
4) System calls
Launch the First Process in Linux SystemJian-Hong Pan
?
The session: https://coscup.org/2022/en/session/AGCMDJ
After Linux kernel boots, it will try to launch first process “init” in User Space. Then, the system begins the featured journey of the Linux distribution.
This sharing takes Busybox as the example and shows that how does Linux kernel find the “init” which directs to the Busybox. And, what will Busybox do and how to get the console. Try to make it like a simple Linux system.
Before Linux kernel launches “init” process, the file system and storage corresponding drivers/modules must be loaded to find the “init”. Besides, to mount the root file system correctly, the kernel boot command must include the root device and file system format parameters.
On the other hand, the Busybox directed from “init” is a lightweight program, but has rich functions, just like a Swiss Army Knife. So, it is usually used on the simple environment, like embedded Linux system.
This sharing will have a demo on a virtual machine first, then on the Raspberry Pi.
Drafts:
* https://hackmd.io/@starnight/Busbox_as_the_init
* https://hackmd.io/@starnight/Build_Alpines_Root_Filesystem_Bootstrap
Relate idea: https://hackmd.io/@starnight/Systems_init_and_Containers_COMMAND_Dockerfiles_CMD
The document provides an overview of kernel crash dump analysis including:
- The tools and data required such as the crash utility, kernel symbol files, vmcore files
- How to install and use these components
- Basic crash commands to analyze system, memory, storage, and network subsystems
- How to dynamically load crash extension modules to add custom commands
Process Scheduler and Balancer in Linux KernelHaifeng Li
?
The document discusses process scheduling in the Linux kernel. It begins with an introduction to schedulers and outlines Linux scheduler history, including round-robin, O(N), O(1), and Completely Fair schedulers. It then provides more details on the Completely Fair Scheduler's use of virtual runtime to determine process execution order and maintain fairness. The document also briefly discusses the real-time scheduler and key data structures used for scheduling like task_struct and sched_domain. It concludes with an overview of the CFS load balancer's hierarchical design and methods used to balance tasks among CPU cores.
Introduce Brainf*ck, another Turing complete programming language. Then, try to implement the following from scratch: Interpreter, Compiler [x86_64 and ARM], and JIT Compiler.
The document summarizes the use of LLVM for code generation when recompiling Nintendo games as native games. LLVM provides a full compiler infrastructure that can be used to generate code for various platforms from a common intermediate representation (LLVM bitcode). The document discusses using LLVM for code generation from 6502 assembly to generate native code for emulation. Optimizations available through LLVM are also discussed.
Linux uses /proc/iomem as a "Rosetta Stone" to establish relationships between software and hardware. /proc/iomem maps physical memory addresses to devices, similar to how the Rosetta Stone helped map Egyptian hieroglyphs to Greek and decode ancient Egyptian texts. This virtual file allows the kernel to interface with devices by providing address translations between physical and virtual memory spaces.
It is the presentation file used by Jim Huang (jserv) at OSDC.tw 2009. New compiler technologies are invisible but highly integrated around our world, and we can enrich the experience via facilitating LLVM.
QEMU is an open source system emulator that uses just-in-time (JIT) compilation to achieve high performance system emulation. It works by translating target CPU instructions to simple host CPU micro-operations at runtime. These micro-operations are cached and chained together into basic blocks to reduce overhead. This approach avoids the performance issues of traditional emulators by removing interpretation overhead and leveraging CPU parallelism through pipelining of basic blocks.
LCU14-107: OP-TEE on ARMv8
---------------------------------------------------
Speaker: Jens Wiklander
Date: September 15, 2014
---------------------------------------------------
★ Session Summary ★
SWG is porting OP-TEE to ARMv8 using Fixed Virtual Platform. Initially OP-TEE is running secure world in aarch32 mode, but with the normal world code running in aarch64 mode. Since ARMv8 uses ARM Trusted Firmware we have patched it with an OP-TEE dispatcher to be able to communicate between secure and normal world.
---------------------------------------------------
★ Resources ★
Zerista: http://lcu14.zerista.com/event/member/137710
Google Event: https://plus.google.com/u/0/events/c0ef114n77bhgbns9vb85g9n6ak
Presentation: http://www.slideshare.net/linaroorg/lcu14-107-optee-on-ar-mv8
Video: https://www.youtube.com/watch?v=JViplz-ah9M&list=UUIVqQKxCyQLJS6xvSmfndLA
Etherpad: http://pad.linaro.org/p/lcu14-107
---------------------------------------------------
★ Event Details ★
Linaro Connect USA - #LCU14
September 15-19th, 2014
Hyatt Regency San Francisco Airport
---------------------------------------------------
http://www.linaro.org
http://connect.linaro.org
Troubleshooting Linux Kernel Modules And Device DriversSatpal Parmar
?
The document discusses various techniques for debugging Linux kernel modules and device drivers, including:
1) Using printk statements to output debug messages from kernel space.
2) Watching system calls with strace to debug interactions between user and kernel space.
3) Adding /proc file system entries and write functions to dynamically modify driver values at runtime.
4) Enabling source-level debugging with tools like kgdb to debug at the level of C source code.
QEMU is an emulator that uses dynamic translation to emulate one instruction set architecture (ISA) on another host ISA. It translates guest instructions to an intermediate representation (TCG IR) code, and then compiles the IR code to native host instructions. QEMU employs techniques like translation block caching and chaining to improve the performance of dynamic translation. It also uses helper functions to offload complex operations during translation to improve efficiency.
The document provides an overview of the initialization phase of the Linux kernel. It discusses how the kernel enables paging to transition from physical to virtual memory addresses. It then describes the various initialization functions that are called by start_kernel to initialize kernel features and architecture-specific code. Some key initialization tasks discussed include creating an identity page table, clearing BSS, and reserving BIOS memory.
Virtual File System in Linux Kernel
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
USENIX ATC 2017: Visualizing Performance with Flame GraphsBrendan Gregg
?
Talk by Brendan Gregg for USENIX ATC 2017.
"Flame graphs are a simple stack trace visualization that helps answer an everyday problem: how is software consuming resources, especially CPUs, and how did this change since the last software version? Flame graphs have been adopted by many languages, products, and companies, including Netflix, and have become a standard tool for performance analysis. They were published in "The Flame Graph" article in the June 2016 issue of Communications of the ACM, by their creator, Brendan Gregg.
This talk describes the background for this work, and the challenges encountered when profiling stack traces and resolving symbols for different languages, including for just-in-time compiler runtimes. Instructions will be included generating mixed-mode flame graphs on Linux, and examples from our use at Netflix with Java. Advanced flame graph types will be described, including differential, off-CPU, chain graphs, memory, and TCP events. Finally, future work and unsolved problems in this area will be discussed."
Understanding a kernel oops and a kernel panicJoseph Lu
?
This document discusses Linux kernel oops and kernel panics. It explains that a kernel oops occurs when there is an illegal instruction or illegal memory access in kernel space, and will kill the offending process to keep the system running. A kernel panic means the system must stop immediately. Kernel oops can be caused by illegal instructions, unrecognized system calls, undefined CPU instructions, unknown data aborts, or prefetch aborts. These result in a call to the arm_notify_die() function and generate an oops. Illegal instructions that handle interrupt vectors can cause a panic directly. A kernel panic performs further actions like console output and stopping other CPUs before restarting or halting the system. Methods to capture crash
The document discusses the bootloaders for the BeagleBone Black system. It describes the memory organization and booting process, including the roles of the X-Loader and U-Boot bootloaders. The X-Loader is described as the first stage bootloader that is derived from U-Boot and runs in internal SRAM. It loads the second stage U-Boot bootloader. U-Boot is then described as the universal bootloader that can be ported to different boards with minimal changes and is responsible for loading the Linux kernel from external DDR memory.
The document discusses Virtio, an interface for virtualized I/O devices. It introduces Virtio's architecture, which involves Virtqueues and Vrings to facilitate communication between guest drivers and hypervisor-level device emulators. It outlines the five Virtio APIs for adding, kicking, getting buffers and enabling/disabling callbacks. It also provides an overview of steps for adding new Virtio devices and drivers.
Linux executables are in ELF format. The document discusses Linux executable formats, compiling C programs in Linux using GCC, and executing programs. It also covers libraries in Linux including static and shared libraries, error handling using errno and assertions, signals and signal handling, process monitoring and /proc filesystem, and managing command line arguments using getopt_long.
This document discusses using the GNU Debugger (GDB) to debug programs. It begins with an introduction to GDB and why it is useful. Examples are then provided of using GDB for interactive debugging, examining core dumps, patching binaries, and advanced tricks. A real-world case study demonstrates using GDB to debug a crash in the GNU C library by examining assembly code and source-level debugging with debug symbols. The document concludes by mentioning another case study involving hijacking file descriptors in GDB.
The document summarizes the use of LLVM for code generation when recompiling Nintendo games as native games. LLVM provides a full compiler infrastructure that can be used to generate code for various platforms from a common intermediate representation (LLVM bitcode). The document discusses using LLVM for code generation from 6502 assembly to generate native code for emulation. Optimizations available through LLVM are also discussed.
Linux uses /proc/iomem as a "Rosetta Stone" to establish relationships between software and hardware. /proc/iomem maps physical memory addresses to devices, similar to how the Rosetta Stone helped map Egyptian hieroglyphs to Greek and decode ancient Egyptian texts. This virtual file allows the kernel to interface with devices by providing address translations between physical and virtual memory spaces.
It is the presentation file used by Jim Huang (jserv) at OSDC.tw 2009. New compiler technologies are invisible but highly integrated around our world, and we can enrich the experience via facilitating LLVM.
QEMU is an open source system emulator that uses just-in-time (JIT) compilation to achieve high performance system emulation. It works by translating target CPU instructions to simple host CPU micro-operations at runtime. These micro-operations are cached and chained together into basic blocks to reduce overhead. This approach avoids the performance issues of traditional emulators by removing interpretation overhead and leveraging CPU parallelism through pipelining of basic blocks.
LCU14-107: OP-TEE on ARMv8
---------------------------------------------------
Speaker: Jens Wiklander
Date: September 15, 2014
---------------------------------------------------
★ Session Summary ★
SWG is porting OP-TEE to ARMv8 using Fixed Virtual Platform. Initially OP-TEE is running secure world in aarch32 mode, but with the normal world code running in aarch64 mode. Since ARMv8 uses ARM Trusted Firmware we have patched it with an OP-TEE dispatcher to be able to communicate between secure and normal world.
---------------------------------------------------
★ Resources ★
Zerista: http://lcu14.zerista.com/event/member/137710
Google Event: https://plus.google.com/u/0/events/c0ef114n77bhgbns9vb85g9n6ak
Presentation: http://www.slideshare.net/linaroorg/lcu14-107-optee-on-ar-mv8
Video: https://www.youtube.com/watch?v=JViplz-ah9M&list=UUIVqQKxCyQLJS6xvSmfndLA
Etherpad: http://pad.linaro.org/p/lcu14-107
---------------------------------------------------
★ Event Details ★
Linaro Connect USA - #LCU14
September 15-19th, 2014
Hyatt Regency San Francisco Airport
---------------------------------------------------
http://www.linaro.org
http://connect.linaro.org
Troubleshooting Linux Kernel Modules And Device DriversSatpal Parmar
?
The document discusses various techniques for debugging Linux kernel modules and device drivers, including:
1) Using printk statements to output debug messages from kernel space.
2) Watching system calls with strace to debug interactions between user and kernel space.
3) Adding /proc file system entries and write functions to dynamically modify driver values at runtime.
4) Enabling source-level debugging with tools like kgdb to debug at the level of C source code.
QEMU is an emulator that uses dynamic translation to emulate one instruction set architecture (ISA) on another host ISA. It translates guest instructions to an intermediate representation (TCG IR) code, and then compiles the IR code to native host instructions. QEMU employs techniques like translation block caching and chaining to improve the performance of dynamic translation. It also uses helper functions to offload complex operations during translation to improve efficiency.
The document provides an overview of the initialization phase of the Linux kernel. It discusses how the kernel enables paging to transition from physical to virtual memory addresses. It then describes the various initialization functions that are called by start_kernel to initialize kernel features and architecture-specific code. Some key initialization tasks discussed include creating an identity page table, clearing BSS, and reserving BIOS memory.
Virtual File System in Linux Kernel
Note: When you view the the slide deck via web browser, the screenshots may be blurred. You can download and view them offline (Screenshots are clear).
USENIX ATC 2017: Visualizing Performance with Flame GraphsBrendan Gregg
?
Talk by Brendan Gregg for USENIX ATC 2017.
"Flame graphs are a simple stack trace visualization that helps answer an everyday problem: how is software consuming resources, especially CPUs, and how did this change since the last software version? Flame graphs have been adopted by many languages, products, and companies, including Netflix, and have become a standard tool for performance analysis. They were published in "The Flame Graph" article in the June 2016 issue of Communications of the ACM, by their creator, Brendan Gregg.
This talk describes the background for this work, and the challenges encountered when profiling stack traces and resolving symbols for different languages, including for just-in-time compiler runtimes. Instructions will be included generating mixed-mode flame graphs on Linux, and examples from our use at Netflix with Java. Advanced flame graph types will be described, including differential, off-CPU, chain graphs, memory, and TCP events. Finally, future work and unsolved problems in this area will be discussed."
Understanding a kernel oops and a kernel panicJoseph Lu
?
This document discusses Linux kernel oops and kernel panics. It explains that a kernel oops occurs when there is an illegal instruction or illegal memory access in kernel space, and will kill the offending process to keep the system running. A kernel panic means the system must stop immediately. Kernel oops can be caused by illegal instructions, unrecognized system calls, undefined CPU instructions, unknown data aborts, or prefetch aborts. These result in a call to the arm_notify_die() function and generate an oops. Illegal instructions that handle interrupt vectors can cause a panic directly. A kernel panic performs further actions like console output and stopping other CPUs before restarting or halting the system. Methods to capture crash
The document discusses the bootloaders for the BeagleBone Black system. It describes the memory organization and booting process, including the roles of the X-Loader and U-Boot bootloaders. The X-Loader is described as the first stage bootloader that is derived from U-Boot and runs in internal SRAM. It loads the second stage U-Boot bootloader. U-Boot is then described as the universal bootloader that can be ported to different boards with minimal changes and is responsible for loading the Linux kernel from external DDR memory.
The document discusses Virtio, an interface for virtualized I/O devices. It introduces Virtio's architecture, which involves Virtqueues and Vrings to facilitate communication between guest drivers and hypervisor-level device emulators. It outlines the five Virtio APIs for adding, kicking, getting buffers and enabling/disabling callbacks. It also provides an overview of steps for adding new Virtio devices and drivers.
Linux executables are in ELF format. The document discusses Linux executable formats, compiling C programs in Linux using GCC, and executing programs. It also covers libraries in Linux including static and shared libraries, error handling using errno and assertions, signals and signal handling, process monitoring and /proc filesystem, and managing command line arguments using getopt_long.
This document discusses using the GNU Debugger (GDB) to debug programs. It begins with an introduction to GDB and why it is useful. Examples are then provided of using GDB for interactive debugging, examining core dumps, patching binaries, and advanced tricks. A real-world case study demonstrates using GDB to debug a crash in the GNU C library by examining assembly code and source-level debugging with debug symbols. The document concludes by mentioning another case study involving hijacking file descriptors in GDB.
This document provides an overview of GNU Make, Autotools, and CMake. It begins with a disclaimer and then outlines the topics to be covered, including GNU Make, Autotools, CMake, comparisons between the tools, and a question and answer section. The document recommends using tabs rather than spaces for indentation in code examples.
Debug C/C++ Programs More Comfortably @ 2014.12.14 Trace Code MeetupJian-Hong Pan
?
The document introduces several tools for debugging C/C++ programs within a terminal environment, including vimGdb, clewn, pyclewn, Conque GDB, and cgdb. It recommends cgdb as the best option, describing it as a lightweight curses-based interface that splits the screen to simultaneously display source code and the GNU Debugger terminal.
Este documento proporciona información sobre Date A Live, una serie de novelas ligeras que fueron adaptadas a anime. Resume la trama principal, que gira en torno a Shido Itsuka y su habilidad para sellar los poderes de los espíritus causantes de desastres, los cuales deben enamorarse de él. También presenta a los principales personajes humanos y espíritus que aparecen en la historia.
This document discusses dynamically tracing C code using runtime execution logs. It provides tips for building the environment, using the menuconfig system, makefiles, toolchains and runtime tracing with Qemu logs and Linux ftrace. Ftrace in particular allows tracing kernel functions and call graphs to help understand code flow and task relationships when statically tracing C/C++ sources proves difficult due to macros and configurations.
This document provides an overview of Leman Global Partner, a global transportation and logistics company founded in 1900. It operates 25 offices across 6 countries, with 700 employees and partnerships in over 100 countries. The document summarizes Leman's various transportation services, including air, sea, road, express, and project freight, as well as logistics and warehousing solutions. It highlights capabilities like global networks, flexible solutions, temperature-controlled transport, and expertise in industries like pharmaceuticals.
Re-identification of Anomized CDR datasets using Social networlk DataAlket Cecaj
?
This document discusses re-identifying users across anonymized datasets using social network data. It summarizes related work showing high re-identification rates just from gender, ZIP code and date of birth. The study aims to use social network data to re-identify users in an anonymized call detail records dataset. It describes matching users between the datasets using time and location parameters and probabilistic modeling. The results show the potential and limits of re-identifying users across multiple mobility datasets, and future work is needed to refine the model and address privacy concerns when correlating multiple datasets.
There is a pressing need to distribute accurate timing, i.e., frequency and/or Time of Day (ToD), across Packet Switched Networks (PSNs) for applications such as cellular backhaul. This paper reviews the main issues involved in timing over packet (ToP) demarcation and provides best practices for ToP demarcation and performance monitoring.
13. GNU ld 是啥?問問男人吧
man ld
...
ld combines a number of object and archive files, relocates
their data and ties up symbol references. Usually the last
step in compiling a program is to run ld.
...
ld combines a number of object and archive files, relocates
their data and ties up symbol references. Usually the last
step in compiling a program is to run ld.
...