Writing an OS Kernel from Scratch | Context Switch — How the CPU Changes the Scene

The scheduler decides to take process A off the CPU and put process B on the CPU — but how does the CPU know which line of code A paused at? Where does B resume from? How are register values saved and restored during a process switch?

This is the core problem that Context Switching solves: save the complete execution state of one process, restore the state of another, so the CPU appears as if it never switched at all.

Today, we’ll thoroughly understand this mechanism.


1. What is Context

The CPU’s execution state at any given moment consists of the following:

Bash
Execution Context Composition:

CPU Registers:
  - General Purpose: rax, rbx, rcx, rdx, rsi, rdi, rbp, rsp, r8-r15
  - Instruction Pointer: rip (address of next instruction to execute)
  - Stack Pointer: rsp
  - Segment Registers: cs, ds, es, fs, gs, ss
  - FLAGS: rflags (condition codes like ZF, SF, OF, etc.)

Control Registers:
  - cr0 (protected mode switch, cache control, etc.)
  - cr2 (Page Fault address)
  - cr3 (page table base address — different per process!)
  - cr4 (PAE, Page Size Extension, etc.)

Floating Point and SIMD State:
  - xmm0-xmm15, ymm0-ymm15, zmm0-zmm31 (AVX512)
  - x87 FPU registers (st0-st7)
  - MXCSR (FPU control/status)

Privilege Level:
  - CPL (Current Privilege Level)
  - RSP (user-mode stack pointer, needs to be saved during switch)

Others:
  - IDTR (Interrupt Descriptor Table location)
  - GDTR (Global Descriptor Table location)

For a process, the most important are: rip, rsp, general-purpose registers. For threads, you also need to save thread-local storage, signal mask, etc.


2. Kernel Stack and Process Switching

Linux uses the kernel stack to save context. Each process has two stacks:

Bash
Each process has two stacks:

User-mode stack (process space, Ring 3):
  → Used when program executes main() / function calls / local variables
  → When switching to kernel, hardware automatically pushes SS/RSP/RFLAGS/CS/RIP onto the kernel stack

Kernel stack (kernel space, Ring 0):
  → Used for system calls, interrupts, exception handling
  → CPU registers are saved here during process switching
  → Fixed size (typically 8KB or 16KB, page-aligned)

┌──────────────────────────────────────┐
│          Kernel Stack (Ring 0)       │
│                                      │
│  ...                                 │
│  SS (User stack selector)           │ ← Auto-pushed at interrupt entry
│  RSP (User stack pointer)           │
│  RFLAGS                              │
│  CS (User code segment selector)    │
│  RIP (User next instruction)        │
│  error code / vector                 │
│  ...                                 │
│  callee-saved registers              │ ← Saved by switch_to
│  ...                                 │
│  task_struct *current                │ ← Used by scheduler
└──────────────────────────────────────┘

3. switch_to: The Core of Process Switching

switch_to is the core assembly function for Linux context switching. It saves the current process’s registers and restores the next process’s registers.

3.1 switch_to Implementation Principle

Asm
# x86_64 switch_to macro (simplified)
# Source: arch/x86/include/asm/switch_to.h

.macro SWITCH_TO next
    # Save current process's callee-saved registers
    pushq   %rbp
    pushq   %rbx
    pushq   %r12
    pushq   %r13
    pushq   %r14
    pushq   %r15

    # Save current stack pointer to task_struct->thread.sp
    movq    %rsp, TASK_thread_sp(%rdi)   # rdi = prev task

    # Load next process's stack pointer
    movq    TASK_thread_sp(%rsi), %rsp   # rsi = next task

    # Load next process's instruction pointer
    # (The return address is on the new stack)

    # Restore next process's callee-saved registers
    popq    %r15
    popq    %r14
    popq    %r13
    popq    %r12
    popq    %rbx
    popq    %rbp
.endm

3.2 What switch_to Actually Does

The key insight of switch_to:

Bash
Before switch_to:
  CPU running process A:
    Kernel stack A (saved by A during a system call / interrupt)
    ┌──────────────────┐
    │ push registers   │ ← current RSP points here
    │ ...              │
    └──────────────────┘
    PCB A (task_struct) stores: .sp = kernel stack A pointer

During switch_to:
  Step 1: Save A's registers → A's kernel stack
  Step 2: Save A's RSP → A's PCB (thread.sp = current RSP)
  Step 3: Load B's RSP from B's PCB (RSP = B's kernel stack pointer)
  Step 4: Pop B's registers from B's kernel stack
  Step 5: RET (return to B's execution point)

After switch_to:
  CPU running process B:
    RSP = B's kernel stack
    All registers restored to B's state
    Fetch instruction from B's RIP → "I was never interrupted!"

4. The Complete Scheduling and Switching Process

Bash
Complete scheduling cycle:

1. Timer interrupt fires (e.g., every 10ms)
   └── CPU enters kernel mode, saves user registers to kernel stack

2. timer interrupt handler:
   └── Calls scheduler_tick()
       └── Current process time slice - 1
       └── If time slice exhausted: set need_resched flag

3. Before returning from interrupt:
   └── Check need_resched flag
       └── If set → call schedule()

4. schedule():
   └── Call scheduler to pick next process (e.g., CFS picks vruntime smallest)
   └── next = pick_next_task()
   └── current->state = TASK_RUNNING (put back in runqueue if preempted)
   └── switch_to(prev, next)
       └── Save prev registers
       └── Restore next registers
       └── Switch CR3 (page table base, if different processes)

5. Return "to user mode" (but actually to the kernel stack of the next process)
   └── Eventually iret / sysret
       └── Pop RIP/RSP/RFLAGS/CS/SS
       └── Back to user-mode execution

5. Key Details of Context Switch

5.1 Why Save Only Callee-Saved Registers?

Bash
Caller-saved (scratch): rax, rcx, rdx, rsi, rdi, r8-r11
  → Automatically saved/restored by the compiler if needed
  → No need for the kernel to save these

Callee-saved: rbx, rbp, r12-r15
  → Functions must save and restore these
  → The kernel (as the "callee") must preserve these for user mode
  → Thus switch_to only saves these

Return address: rip
  → Implicitly saved via call/ret or interrupt entry/exit

5.2 Switching Page Tables (CR3)

When switching between different processes (not threads):

Bash
Before switching to process B:
  CR3 = address of process A's page table
  Process A's virtual address space is active

After switch_to, before returning to user mode:
  CR3 = address of process B's page table
  Process B's virtual address space becomes active

Effect: Both processes think they own the entire 0x0 ~ 0xFFFFFFFF... address space
         But the physical memory behind the pages is completely different!

No CR3 switching needed for threads — threads share the same address space.

5.3 Cache and TLB Impact

Switching CR3 has a significant performance cost:

Bash
TLB (Translation Lookaside Buffer):
  When CR3 changes, TLB is flushed (at least on older CPUs)
  → All virtual-to-physical address translations are invalidated
  → Next instructions will suffer TLB misses
  → Performance penalty: ~hundreds of cycles

Modern improvements:
  PCID (Process Context Identifier):
    → Tags TLB entries with a PCID
    → No TLB flush on CR3 switch if using different PCID
    → Significant performance improvement

6. Summary

  1. Context = all CPU register states of a process
  2. Kernel stack is the foundation for preserving context
  3. switch_to is the core assembly routine for switching — save prev registers, restore next registers
  4. switch_to only saves callee-saved registers; caller-saved registers are handled by the compiler
  5. Switching CR3 switches page tables to implement address space isolation
  6. Cache/TLB are the performance costs of context switching

Next up: Synchronization primitives — how the OS coordinates concurrent access.

OS #KernelDevelopment #ContextSwitch #switch_to #CR3 #TLB #WritingFromScratch

Last modified: 2024年6月10日

Author

Comments

Write a Reply or Comment

Your email address will not be published.