Visual comparison of Unix-like systems & Virtualisation
10 likes1,808 views
The document provides a comprehensive overview of virtualization in Unix-like systems, detailing various subsystems including process management, memory management, and file systems. It explains the differences between user-level threads, lightweight processes, and kernel threads, as well as virtualization methods and technologies like Xen and KVM. Additionally, it highlights the architecture and management of virtual machines across different platforms, emphasizing the integration and challenges of virtualization technologies.
Visual comparison of Unix-like systems & Virtualisation
- 1. Hello! Visual comparison of Unix-like systems
- 3. Primary Subsystems Process Management Manages all active processes and threads Memory Management Allocates memory for processes at run-time
- 4. Performs translations of physical memory locations to virtual memory locations File Subsystem Virtual File System Layer Abstracts and manages data for different file system types Physical File System Layer Manages access for specific file systems of various types I/O Subsystem Manages system I/O resources, drivers, and data structures
- 5. Drives system configuration process
- 6. Terms Process : A UNIX process includes the user¡¯s address space, stack, and process control block.
- 7. User-level threads : Implemented through a threads library in the address space of a process, these threads are invisible to the OS. A user-level thread (ULT) is a user-created unit of execution within a process.
- 8. Lightweight processes : A lightweight process (LWP) can be viewed as a mapping between user level threads and kernel threads. Each LWP supports ULT and maps to one kernel thread. LWPs are scheduled by the kernel.
- 9. Kernel threads : These are the fundamental entities that can be scheduled and dispatched to run on one of the system processors.
- 10. Kernel Entry Hardware System call interface lightweight syscalls User Level Kernel Level I / O Process Management Subsystem Interpreter User program Libraries Virtual File System Filesystem File Cache LVM device drivers IPC Scheduling Hardware control Memory Management Subsystem
- 11. UNIX v3 304KB
- 13. System Call Interface . .. Virtual File System Framework Kernel Services Clocks & Timers etc Hardware ¨C SPARC / x86 / x64 Hardware Address Translation (HAT) Network ing framework& services . .. Bus & Nexus Drivers sd ssd Processes & Threads t i m e s h a r e Scheduler i n t e r a c t i v e r e a l t i m e f a i r s h a r e f i x e d p r i o r i t y ZFS NFS ProcFS SpecFS TCP/IP Resource managem ent & cont rols Memory Management Virtual Memory Kernel Memory Allocation Resource management & controls Solaris kernel
- 15. HP-UX big picture Kernel Threads VAS Pregion Pregion UAREA user structure kernel stack File Descriptor Table VFS Inode Cache System File Table UAREA user structure kernel stack Pregion Pregion fstore Arena Data Structures VFD DBD bstore File cache Process structures Region vnode vnode FS private data kthread kthread kthread kthread proc_t proc_t proc_t proc_t Memory
- 17. XNU Basis of Mac OS X, iPhone, etc. Made up of three main parts:
- 18. Mach 3.0 kernel code kernel threads, virtual memory management, inter-process communication, interrupt management, real-time support, console I/O FreeBSD 5.x kernel code Process model, Permissions, System calls, Sockets, VFS, System V IPC I/O Kit Device driver model
- 20. XNU kernel User User libraries libc BSD I/O Kit Mach Platform Expert Xnu (kernel)
- 21. Microkernel architecture Hardware Hardware Accounts File Systems Networking Devices Virtual Memory Process Mgmt Virtual Memory Process Mgmt IPC Kernel Microkernel User-Space Servers Applications Applications Traditional (Monolithic) Kernel Microkernel UNIX
- 22. VMS (virtual memory) Solaris DEC OSF/1 -> DIGITAL UNIX HP-UX Tru64 influences System III ULTRIX Mach AIX XENIX OSF/1 Tru64 UNIX SCO UNIX (System V) Tandem NonStop-UX, Integrity XC SunOS UNIX (Bell Labs) SCO XENIX System V BSD
- 23. Mach in Tru64 Process Mgmt Memory Mgmt UNIX Mach Process Mgmt Memory Mgmt IPC Mach System Calls by Select System Services UNIX System Calls by Applications Other Services: File Systems Buffer Caches UNIX IPC and Networking I/O Framework Security and Accounting Etc...
- 25. Tru64 kernel Events and Interrupts I/O Subsystem vm_object Special Files Mount Table nameidata Files Buffer Cache proc utask task thread uthread port Kernel Stack run_queue pcb User Stack wait queue vm_page vm_map vm_map_entry pmap pte Virtual File System Process Virtual Memory System Calls Alpha Architecture Vnode Table File Table Networking Subsystem PALcode
- 26. AdvFS http://advfs.sourceforge.net Code drop of AdvFS
- 28. Linux kernel Process Kernel modules Audit IPC System Call interface Devices SELinux Memory File, I/O Networking
- 29. ?
- 30. Tasks A process, or task, in Linux is represented by a task_struct data structure. The task_struct data structure contains information in a number of categories: Processes in Linux are both containers and schedulable entities; processes can share address space and system resources, making processes effectively usable as threads.
- 31. Most services are implemented in the kernel, with the exception of many networking functions. Thus Linux kernel is relatively big in size.
- 32. Linux provides a unique solution in that it does not recognize a distinction between threads and processes. Using a mechanism similar to lightweight processes, user-level threads are mapped into kernel-level processes. Multiple user-level threads that constitute a single user-level process are mapped into Linux kernel-level processes that share the same group ID. This enables these processes to share resources such as files and memory and to avoid the need for a context switch when the scheduler switches among processes in the same group.
- 33. Context of a Process Process structure Shared memory Heap space Data Text Virtual Address Space
- 34. Context of a task Attributes Instructions Private storage Thread structure Thread stack User structure with register context
- 36. Threading models Hardware Kernel Space Kernel Threads User Space kernel scheduler kernel scheduler kernel scheduler KT KT KT KT UT UT UT UT UT scheduler KT UT UT UT scheduler KT UT UT User Process and User Threads CPU CPU CPU CPU CPU KT file management subsystem I/O management subsystem Process & memory mgmt subsystems network management subsystem
- 37. Scheduling lowest highest longest processor priority waiting scheduler
- 38. Context Switches Save register context of current thread. Restore register context of highest priority thread. UAREA processor kthread structure User structure UAREA kthread structure User structure PCB PCB
- 39. Thread Scheduling Contention Scopes Userspace scheduler in pthread library User Space Kernel Space "unbound" with userspace scheduler. Here, we "bind" to kernel threads. Bound threads
- 40. Scheduler queues CPU0 CPU1 CPU2 CPU3 per-processor run queues
- 41. Books
- 42. A process, or task, in Linux is represented by a task_struct data structure. The task_struct data structure contains information in a number of categories: Processes in Linux are both containers and schedulable entities; processes can share address space and system resources, making processes effectively usable as threads.
- 43. Most services are implemented in the kernel, with the exception of many networking functions. Thus Linux kernel is relatively big in size.
- 44. Linux provides a unique solution in that it does not recognize a distinction between threads and processes. Using a mechanism similar to lightweight processes, user-level threads are mapped into kernel-level processes. Multiple user-level threads that constitute a single user-level process are mapped into Linux kernel-level processes that share the same group ID. This enables these processes to share resources such as files and memory and to avoid the need for a context switch when the scheduler switches among processes in the same group. Virtualisation
- 45. How Server Virtualization Works Strong isolation between VMs
- 46. Dynamic resizing & migration
- 47. Rapid VM instantiation & cloning
- 48. Integrated workload management OEL RHEL MS Physical Server Physical CPUs Hypervisor/VMM Virtual machine monitor OS Virtual Machines Physical Server Hypervisor/VMM Virtual machine monitor OS Physical Server Hypervisor/VMM OS Physical I/O Physical memory File definitions Oracle Database 11g Oracle Database 10g R2 Oracle Database 10g R1
- 50. ?
- 51. KVM Architecture Native Linux kernel is the VMM (kvm module loaded) Xen is different. Xen copy code from linux kernel ; became a VMM itself ; the real linux kernel is going to "sleep" and xen kernel do everything by itself (a lot of maintenance work !! When new linux kernel release, xen need port a lot code ) Guest mode ,each vm looks as a normal linux process
- 52. Intel and AMD Virtualization technologies Intel: VT-x (IA-32)
- 53. VT-i (IA-64)
- 54. VT-d
- 55. VT-C AMD: AMD-V codename "Pacifica" Virtualization Layer Expand or Contract Virtual Processors Virtual Memory Virtual I/O Storage Virtual Machine 1 Expand or Contract Physical System Guest OS 1 Applications
- 56. Intel Virtualization Technologies VT-x (IA-32) and VT-i (IA-64) are Intel virtualization technologies : Implemented at CPU level VT-d is Intel technology for directed I/O Implemented at chipset level
- 57. Support both VT-x and VT-I VT-C (connectivity) Implemented on I/O cards
- 59. virtio
- 60. Before
- 61. After
- 62. Xen overview Fully virtualized Requires no modifications to the operating system
- 63. Requires trapping and emulating all privileged instructions
- 64. Intel and AMD are doing hardware virtualization Support for virtualization of CPU, MMU, and memory
- 65. Allows virtualization of unmodified guests
- 66. I/O performance might still be an issue Xen 3.0 is a hypervisor (virtual machine monitor) Creates and launches virtual machines
- 68. Provides memory management and CPU scheduling
- 70. Event and Memory Channel between domains
- 72. VMware Virtualization leader Efficient proprietary hypervisors
- 73. Mature management tools and eco-system x86 Standard focus HP Proliant USB key Hypervisor VM VM
- 74. Virtual Private Servers (VPS) Managers Single instance of OS Kernel Unique per workload File System
- 75. User Database
- 76. IP addresses
- 77. Process namespace Hardware Operating System Workload 1 Workload 2 Workload 3
- 78. Classification of Virtual Machine Monitors (by implementation) Hosted virtual machines VMM runs in user mode
- 79. Virtual machines run as processes under the host OS
- 80. All device access performed by host OS Example: VMware server (former GSX), HP IVM Hardware Host OS VMM Guest OS 1 Guest OS 2 App App App App App
- 81. Classification of Virtual Machine Monitors (by implementation) Unhosted virtual machines VMM is part of the host kernel Boots on native hardware Device drivers needed for all devices Ex: Xen, Vmware ESX Hardware VMM Guest OS 1 Guest OS 3 Guest OS 2 App App App App App App
- 82. Some Common Virtual Machine Environments User Mode Linux (UML) Unhosted Hosted VMware ESX Hyper-V VMware Server/WS MS Virtual PC HP IVM, KVM Fully virtualized Xen Para-virtualized
- 83. Managing virtual machines Several options on Fedora/RHEL, Ubuntu, etc. to manage virtual machines: virsh
- 84. xend
- 85. xm
- 86. virt-manager
- 87. Thank you
- 88. Hyper-threading Hardware Layer Kernel Layer User Layer H/W Threading Off H/W Threading On LCPUs disabled H/W Threading On LCPUs enabled CPU CPU CPU CPU CPU Core Core Core HT HT HT HT LCPU LCPU LCPU LCPU
Editor's Notes
- #3: There are four primary subsystems covered in this course, which are responsible for the majority of the kernel activity. Process Management The Process Management Subsystem is responsible for managing all of the running processes and threads on the system, maintaining information about process and thread resources, and scheduling threads on the CPU or CPUs. Memory Management Memory is managed by the kernel in a way that allows each process to have its own virtual address space. The Memory Management Subsystem creates an appearance that the sum of process address space is greater than the amount of physical memory in the system. This is the theory behind the virtual address space concept. File Systems The File Subsystem is responsible for managing data on various types of disks. The File Subsystem has provisions to manage different types of file system disk layouts. The ability to do this comes from a level of abstraction known as the vnode layer.
- #10: This slide shows the "Big Picture" of some of the primary kernel data structures. Previous slides have already shown how the process management and memory structures are linked together through the VAS and pregions. In this diagram, we also see how the process management structures are linked to the file system structures through the File Descriptor Table. Each entry in this table is mapped to an entry in the System File Table. We can complete the picture of the process by using the information contained in the System File Table entries. Each of these structures will be discussed in greater detail in later parts of this course.
- #25: The program code The global variables used by the program Contains process- specific information such as the process' state, its threads, signals, size, scheduling priority Address range into which the process context is logically mapped Available range of addresses stored in the virtual address space; can be dynamically allocated by process for private data storage Available range of addresses stored in the virtual address space; can be allocated by process, populated with data, then shared with other processes virtual address space heap space shared memory
- #26: The program code specifically for this thread to execute (e.g., a function) Define thread-specific characteristics Contains thread-specific information such as the scheduling priority, states, signals, and cpu usage. Contains a program counter user structure with register context private storage Area for data; comparable to process heap space, except that unlike the process heap space, thread's private storage is not shared with other process threads Stack space used by thread to call functions