Professional

Linux Kernel Architecture

Welcome to sean’s Guide to Linux Kernel Architecture

리눅스 커널의 구조에 입문하신것을 환영합니다.

본 번역서는 Wolfgang Mauerer 저서의 <Professional Linux Architecture> 를 근간으로 하고 있습니다.

현재 번역중이며… 언제라도 잘못된 부분이 있으면 알려주세요.

If you’d like to help, fork us on GitHub!

contacts: newstein33@gmail.com

Professional Linux Kernel Architecture

본 문서는 리눅스 커널의 구조에 대해서 서술한다.

• Chapter 0: About
  • Thanks to
  • 편저자 의도
  • SEAN’s Paradise
• Chapter 1:Introduction and Overview
  • 1.1 Task of the Kernel
  • 1.2 Implementation Strategies
  • 1.3 Elements of the Kernel
    • 1.3.1 Processes, Task Switching, and Scheduling
    • 1.3.2 Unix Processes
      • Threads
      • Namespaces
    • 1.3.3 Address Spaces and Privilege Levels
      • Privilege Levesls
      • Virtual and Physical Address Spaces
    • 1.3.4 Page Tables
      • Interaction with the CPU
      • Memory Mappings
    • 1.3.5 Allocation of Physical Memory
      • The Buddy System
      • The Slab Cache
      • Swapping and Page Reclaim
    • 1.3.6 Timing
    • 1.3.7 System Calls
    • 1.3.8 Device Drivers, Block and Charater Devices
    • 1.3.9 Networks
    • 1.3.10 Filesystems
    • 1.3.11 Modules and Hotplugging
    • 1.3.12 Caching
    • 1.3.13 List Handling
    • 1.3.14 Object Management and Reference Counting
      • General Kernel Objects
      • Sets of Objects
      • Reference Counting
    • 1.3.15 Data Types
      • Type Definition
      • Byte Order
      • Per-CPU Variables
      • Access to Userspace
    • 1.3.16 … and Beyond the Infinite
    • 1.4 Why the kernel Is Special
    • 1.5 Some Notes on Presentation
    • 1.6 Summary
• Chapter 2 : Process Management and Scheduling
  • 2.1 Process Priorities
  • 2.2 Process Life Cycle
    • 2.2.1 Preemptive Multitasking
  • 2.3 Process Representation
    • 2.3.1 Process Types
    • 2.3.2 Namespaces
      • Concept
      • Implementation
      • The UTS Namespace
      • The User Namespace
  • 2.3.3 Process Identification Numbers
    • 2.3.4 Task Relationships
  • 2.4 ProcessManagement System Calls
    • 2.4.1 Process Duplication
      • Copy on Write
      • Executing System Calls
      • Implementation of do_work
      • Copying Process
      • Special Points When Generating Threads
    • 2.4.2 Kerne Threads
    • 2.4.3 Starting New Programs
      • Implementation of execve
      • Interpreting Binary Formats
    • 2.4.4 Exiting Processes
  • 2.5 Implementation of the Scheduler
    • 2.5.1 Overview
    • 2.5.2 Data Structures
      • Elements in the Task Structure
      • Scheduler Classes
      • Run Queues
      • Scheduling Entities
    • 2.5.3 Dealing with Priorities
      • Kernel Representation of Priorities
      • Computing Priorities
      • Computing Load Weights
    • 2.5.4 Core Scheduler
      • The Periodic Scheduler
  • 2.6 The Completely Fair Scheduling Class
    • Data Structures
    • CFS Operations
    • Queue Manipulation
    • Selecting the Next Task
    • Handling the Periodic Tick
    • 2.6.3 Queue Manipulation
    • 2.6.4 Selecting the Next Task
    • 2.6.5 Handling the Periodic Tick
    • 2.6.6 Wake-up Preemption
    • 2.6.7 Handling New Tasks
  • 2.7 The Real-Time Scheduling Class
    • 2.7.1 Properties
    • 2.7.2 Data Structures
    • 2.7.3 Scheduler Operations
  • 2.8 Scheduler Enhancements
    • 2.8.1 SMP Scheduling
    • Extensions to the Data Structures
    • The Migration Thread
    • Core Scheduler Changes
    • 2.8.2 Scheduling Domains and Control Groups
    • 2.8.3 Kernel Preemption and Low Latency Efforts
    • Kernel Preemption
    • Low Latency
  • Summary
• Chapter 3 : Memory Management
  • 3.2 Organization in the (N)UMA Model
    • 3.2.1 Overview
    • 3.2.2 Data Structures
    • Node Management
    • Memory Zones
    • Calculation of Zone Watermarks
    • Hot-N-Cold Pages
    • Page Frames
    • Definition of page
    • Architecture-Independent Page Flags
    • 3.3 Page Tables
    • 3.3.1 Data Structures
    • Breakdown of Addresses inMemory
    • Format of Page Tables
    • PTE-Specific Entries
    • 3.3.2 Creating and Manipulating Entries
    • 3.4 Initialization of Memory Management
    • 3.4.1 Data Structure Setup
    • Prerequisites
    • System Start
    • Node and Zone Initialization
    • 3.4.2 Architecture-Specific Setup
    • Arrangement of the Kernel in Memory
    • Initialization Steps
    • Initialization of Paging
    • Division of Address Space
    • Alternative Division
    • Splitting the Virtual Address Space
    • Initialization of the Hot-n-Cold Cache
    • Registering Active Memory Regions
    • Registering Regions on IA-32
    • Registering Regions on AMD64
    • Address Space Setup on AMD64
    • 3.4.3 Memory Management during the Boot Process
    • Data Structures
    • Initialization
    • Initialization for IA-32
    • Initialization for AMD64
    • Releasing Memory
    • Disabling the Bootmem Allocator
    • Releasing Initialization Data
    • 3.5 Management of Physical Memory
    • 3.5.1 Structure of the Buddy System
    • 3.5.2 Avoiding Fragmentation
    • Grouping Pages byMobility
    • Data Structures
    • Global Variables and Auxiliary Functions
    • Initializing Mobility-Based Grouping
    • The Virtual Movable Zone
    • Data Structures
    • Implementation
    • 3.5.3 Initializing the Zone and Node Data Structures
    • Managing Data Structure Creation
    • Creating Data Structures for Each Node
  • 3.5.4 Allocator API
    • Allocator API
    • AllocationMasks
    • AllocationMacros
  • 3.5.5 Reserving Page
    • Freeing Pages
    • Allocation of Discontiguous Pages in the Kernel
    • Kernel Mappings
  • 3.6 The Slab Allocator
    • Alternative Allocators
    • Memory Management in the Kernel
    • The Principle of Slab Allocation
    • Implementation
    • General Caches
  • 3.7 Processor Cache and TLB Control
  • Summary
• Chapter 4 : Virtual Process Memory
  • 4.1 Introduction
  • 4.2 Virtual Process Address Space
    • Layout of the Process Address Space
    • Creating the Layout
  • 4.3 Principle ofMemoryMappings
  • 4.4 Data Structures
    • Trees and Lists
    • Representation of Regions
    • The Priority Search Tree
  • 4.5 Operations on Regions
    • Associating Virtual Addresses with a Region
    • Merging Regions
    • Inserting Regions
    • Creating Regions
  • 4.6 Address Spaces
  • 4.7 Memory Mappings
    • Creating Mappings
    • Removing Mappings
    • Nonlinear Mappings
  • 4.8 Reverse Mapping
    • Data Structures
    • Creating a Reverse Mapping
    • Using Reverse Mapping
  • 4.9 Managing the Heap
  • 4.10 Handling of Page Faults
  • 4.11 Correction of Userspace Page Faults
    • Demand Allocation/Paging
    • Anonymous Pages
    • Copy on Write
    • Getting Nonlinear Mappings
  • Kernel Page Faults
  • Copying Data between Kernel and Userspace
  • Summary
• Chapter 5 : Locking and Interprocess Communication
  • 5.1 Control Mechanisms
    • Race Conditions
    • Critical Sections
  • 5.2 Kernel LockingMechanisms
    • Atomic Operations on Integers
    • Spinlocks
    • Semaphores
    • The Read-Copy-Update Mechanism
    • Memory and Optimization Barriers
    • Reader/Writer Locks
    • The Big Kernel Lock
    • Mutexes
    • Approximate Per-CPU Counters
    • Lock Contention and Fine-Grained Locking
  • 5.3 System V Interprocess Communication
    • System V Mechanisms
    • Semaphores
    • Message Queues
    • Shared Memory
  • 5.4 Other IPC Mechanisms
    • Signals
    • Pipes and Sockets
  • Summary
• Chapter 6 : Device Drivers
  • 6.1 I/O Architecture
    • Expansion Hardware
  • 6.2 Access to Devices
    • Device Files
    • Character, Block, and Other Devices
    • Device Addressing Using Ioctls
    • Representation of Major and Minor Numbers
    • Registration
  • 6.3 Association with the Filesystem
    • Device File Elements in Inodes
    • Standard File Operations
    • Standard Operations for Character Devices
    • Standard Operations for Block Devices
  • 6.4 Character Device Operations
    • Representing Character Devices
    • Opening Device Files
    • Reading and Writing
  • 6.5 Block Device Operations
    • Representation of Block Devices
    • Data Structures
    • Adding Disks and Partitions to the System
    • Opening Block Device Files
    • Request Structure
    • BIOs
    • Submitting Requests
    • I/O Scheduling
    • Implementation of Ioctls
  • 6.6 Resource Reservation
    • Resource Management
    • I/O Memory
    • I/O Ports
  • 6.7 Bus Systems
    • The Generic Driver Model
    • The PCI Bus
    • USB
  • Summary
• Chapter 7 : Modules
  • 7.1 Overview
  • 7.2 Using Modules
    • Adding and Removing
    • Dependencies
    • Querying Module Information
    • Automatic Loading
  • 7.3 Inserting and Deleting Modules
    • Module Representation
    • Dependencies and References
    • Binary Structure of Modules
    • Inserting Modules
    • Removing Modules
  • 7.4 Automation and Hotplugging
    • Automatic Loading with kmod
    • Hotplugging
  • 7.5 Version Control
    • Checksum Methods
    • Version Control Functions
  • Summary
• Chapter 8 : The Virtual Filesystem
  • 8.1 Filesystem Types
  • 8.2 The Common File Model
    • Inodes
    • Links
    • Programming Interface
    • Files as a Universal Interface
  • 8.3 Structure of the VFS
    • Structural Overview
    • Inodes
    • Process-Specific Information
    • File Operations
    • Directory Entry Cache
  • 8.4 Working with VFS Objects
    • Filesystem Operations
    • File Operations
  • 8.5 Standard Functions
    • Generic Read Routine
    • The fault Mechanism
    • Permission-Checking
  • Summary
• Chapter 9 : The Extended Filesystem Family
  • 9.1 Introduction
  • 9.2 Second Extended Filesystem
    • Physical Structure
    • Data Structures
    • Creating a Filesystem
    • Filesystem Actions
  • 9.3 Third Extended Filesystem
    • Concepts
    • Data Structures
  • Summary
• Chapter 10 : Filesystems without Persistent Storage
  • 10.1 The proc Filesystem
    • Contents of /proc
    • Data Structures
    • Initialization
    • Mounting the Filesystem
    • Managing /proc Entries
    • Reading and Writing Information
    • Task-Related Information
    • System Control Mechanism
  • 10.2 Simple Filesystems
    • Sequential Files
    • Writing Filesystems with Libfs
    • The Debug Filesystem
    • Pseudo Filesystems
  • 10.3 Sysfs
    • Overview
    • Data Structures
    • Mounting the Filesystem
    • File and Directory Operations
    • Populating Sysfs
  • Summary
• Chapter 11 : Extended Attributes and Access Control Lists
  • 11.1 Extended Attributes
    • Interface to the Virtual Filesystem
    • Implementation in Ext3
    • Implementation in Ext2
  • 11.2 Access Control Lists
    • Generic Implementation
    • Implementation in Ext3
    • Implementation in Ext2
  • Summary
• Chapter 12 : Networks
  • 12.1 Linked Computers
  • 12.2 ISO/OSI and TCP/IP Reference Model
  • 12.3 Communication via Sockets
    • Creating a Socket
    • Using Sockets
    • Datagram Sockets
  • 12.4 The Layer Model of Network Implementation
  • 12.5 Networking Namespaces
  • 12.6 Socket Buffers
    • Data Management Using Socket Buffers
    • Management Data of Socket Buffers
  • 12.7 Network Access Layer
    • Representation of Network Devices
    • Receiving Packets
    • Sending Packets
  • 12.8 Network Layer
    • IPv4
    • Receiving Packets
    • Local Delivery to the Transport Layer
    • Packet Forwarding
    • Sending Packets
    • Netfilter
    • IPv6
  • 12.9 Transport Layer
    • UDP
    • TCP
  • 12.10 Application Layer
    • Socket Data Structures
    • Sockets and Files
    • The socketcall System Call
    • Creating Sockets
    • Receiving Data
    • Sending Data
  • 12.11 Networking from within the Kernel
    • Communication Functions
    • The Netlink Mechanism
  • Summary
• Chapter 13 : System Calls
  • 13.1 Basics of System Programming
    • Tracing System Calls
    • Supported Standards
    • Restarting System Calls
  • 13.2 Available System Calls
  • 13.3 Implementation of System Calls
    • Structure of System Calls
    • Access to Userspace
    • System Call Tracing
  • Summary
• Chapter 14 : Kernel Activities
  • 14.1 Interrupts
    • Interrupt Types
    • Hardware IRQs
    • Processing Interrupts
    • Data Structures
    • Interrupt Flow Handling
    • Initializing and Reserving IRQs
    • Servicing IRQs
  • 14.2 Software Interrupts
    • Starting SoftIRQ Processing
    • The SoftIRQ Daemon
  • 14.3 Tasklets
    • Generating Tasklets
    • Registering Tasklets
    • Executing Tasklets
  • 14.4 Wait Queues and Completions
    • Wait Queues
    • Completions
    • Work Queues
  • Summary
• Chapter 15 : Time Management
  • 15.1 Overview
    • Types of Timers
    • Configuration Options
  • 15.2 Implementation of Low-Resolution Timers
    • Timer Activation and Process Accounting
    • Working with Jiffies
    • Data Structures
    • Dynamic Timers
  • 15.3 Generic Time Subsystem
    • Overview
    • Configuration Options
    • Time Representation
    • Objects for Time Management
  • 15.4 High-Resolution Timers
    • Data Structures
    • Setting Timers
    • Implementation
    • Periodic Tick Emulation
    • Switching to High-Resolution Timers
  • 15.5 Dynamic Ticks
    • Data Structures
    • Dynamic Ticks for Low-Resolution Systems
    • Dynamic Ticks for High-Resolution Systems
    • Stopping and Starting Periodic Ticks
  • 15.6 Broadcast Mode
  • 15.7 Implementing Timer-Related System Calls
    • Time Bases
    • The alarm and setitimer System Calls
    • Getting the Current Time
  • 15.8 Managing Process Times
  • Summary
• Chapter 16 : Page and Buffer Cache
  • 16.1 Structure of the Page Cache
    • Managing and Finding Cached Pages
    • Writing Back Modified Data
  • 16.2 Structure of the Buffer Cache
  • 16.3 Address Spaces
    • Data Structures
    • Page Trees
    • Operations on Address Spaces
  • 16.4 Implementation of the Page Cache
    • Allocating Pages
    • Finding Pages
    • Waiting on Pages
    • Operations with Whole Pages
    • Page Cache Readahead
  • 16.5 Implementation of the Buffer Cache
    • Data Structures
    • Operations
    • Interaction of Page and Buffer Cache
    • Independent Buffers
  • Summary
• Chapter 17 : Data Synchronization
  • 17.1 Overview
  • 17.2 The pdflush Mechanism
  • 17.3 Starting a New Thread
  • 17.4 Thread Initialization
  • 17.5 Performing Actual Work
  • 17.6 Periodic Flushing
  • 17.7 Associated Data Structures
    • Page Status
    • Writeback Control
    • Adjustable Parameters
  • 17.8 Central Control
  • 17.9 Superblock Synchronization
  • 17.10 Inode Synchronization
    • Walking the Superblocks
    • Examining Superblock Inodes
    • Writing Back Single Inodes
  • 17.11 Congestion
    • Data Structures
    • Thresholds
    • Setting and Clearing the Congested State
    • Waiting on Congested Queues
  • 17.12 Forced Writeback
  • 17.13 Laptop Mode
  • 17.14 System Calls for Synchronization Control
  • 17.15 Full Synchronization
    • Synchronization of Inodes
    • Synchronization of Individual Files
    • Synchronization of Memory Mappings
  • Summary
• Chapter 18 : Page Reclaim and Swapping
  • 18.1 Overview
    • Swappable Pages
    • Page Thrashing
    • Page-Swapping Algorithms
  • 18.2 Page Reclaim and Swapping in the Linux Kernel
    • Organization of the Swap Area
    • Checking Memory Utilization
    • Selecting Pages to Be Swapped Out
    • Handling Page Faults
    • Shrinking Kernel Caches
  • 18.3 Managing Swap Areas
    • Data Structures
    • Creating a Swap Area
    • Activating a Swap Area
  • 18.4 The Swap Cache
    • Identifying Swapped-Out Pages
    • Structure of the Cache
    • Adding New Pages
    • Searching for a Page
  • 18.5 Writing Data Back
  • 18.6 Page Reclaim
    • Overview
    • Data Structures
    • Determining Page Activity
    • Shrinking Zones
    • Isolating LRU Pages and Lumpy Reclaim
    • Shrinking the List of Active Pages
    • Reclaiming Inactive Pages
  • 18.7 The Swap Token
  • 18.8 Handling Swap-Page Faults
    • Swapping Pages in
    • Reading the Data
    • Swap Readahead
  • 18.9 Initiating Memory Reclaim
    • Periodic Reclaim with kswapd
    • Swap-out in the Event of Acute Memory Shortage
  • 18.10 Shrinking Other Caches
    • Data Structures
    • Registering and Removing Shrinkers
    • Shrinking Caches
  • Summary
• Chapter 19 : Auditing
  • 19.1 Overview
  • 19.2 Audit Rules
  • 19.3 Implementation
    • Data Structures
    • Initialization
    • Processing Requests
    • Logging Events
    • System Call Auditing
  • Summary
• Appendix A: Architecture Specifics
  • A
    • B
      • C
• Appendix B: Working with the Source Code
  • A
    • B
      • C
• Appendix C: Notes on C
  • A
    • B
      • C
• Appendix D: System Startup
  • A
    • B
      • C
• Appendix E: The ELF Binary Format
  • A
    • B
      • C
• Appendix F: The Kernel Development Process
  • A
    • B
      • C

---

라이센스 정보는 아래와 같다. 또한 스타일 가이드도 첨부한다.

• Donate
• License
• The Guide Style Guide
  • Relevancy
  • Headings
  • Prose
  • Code Examples
  • Externally Linking
  • Linking to Sections in The Guide
  • Notes and Warnings
  • TODOs