Writing an OS Kernel from Scratch – Part 4: Entering Protected Mode (Detailed Segmented Memory Management)

# Writing an OS Kernel from Scratch – Part 4: Entering Protected Mode (Detailed Segmented Memory Management)

“Protected mode ≠ Paging! Its first line of defense is the segmentation mechanism that modern OSes ‘quietly bypass.'”

In the previous few articles, we used GRUB to easily boot the kernel, output “Hello World,” and understood the Multiboot protocol. But have you ever wondered: why does GRUB help us enter protected mode? What exactly is it protecting?

Today, we manually complete this historic transition — switching from Real Mode to Protected Mode — and deeply understand its core: Segmented Memory Management, including GDT, LDT, and segment selector working principles.

🔥 Even though modern operating systems have almost “abandoned” the segmentation mechanism, understanding it is still essential for mastering the x86 architecture.

—

## 1. What is Real Mode?

Real Mode is the default operating state of the x86 CPU after power-on, originating from the 1978 Intel 8086 processor.

✅ Features:
– 20-bit address lines → maximum addressing of 1MB memory (0x00000 ~ 0xFFFFF)
– Segment:offset addressing: physical address = segment register × 16 + offset
For example: CS:IP = 0x1000:0x0020 → physical address = 0x10000 + 0x20 = 0x10020
– No memory protection: any program can read/write any memory (including BIOS and video RAM)
– No privilege levels: all code runs in “god mode”

💡 Real mode is like an old tractor without a steering wheel or brakes — simple, but extremely dangerous.

❓ Why do modern CPUs still boot in real mode?

For compatibility! PC-compatible machines must be able to run 1980s-era DOS programs, so the CPU “pretends to be an 8086” after power-on.

—

## 2. What is Protected Mode?

Protected Mode was introduced by the Intel 80286 (1982) and became the cornerstone of modern operating systems starting with the 80386.

✅ Core advantages:

| Feature | Description |
| — | — |
| 32-bit addressing | Can access 4GB of memory (later extended via PAE / 64-bit) |
| Memory protection | Prevents programs from accessing out-of-bounds (segment limits, permission checks) |
| Privilege levels (Ring 0~3) | Kernel (Ring 0) vs User programs (Ring 3) |
| Virtual memory foundation | Provides support for Paging |

🌟 Memory access in protected mode is no longer “direct physical address” but is translated and validated through a segment descriptor table.

—

## 3. Key Mechanisms: Segment Descriptors and Segment Registers

In protected mode, segment registers (CS, DS, SS, etc.) no longer directly store addresses, but instead store a Segment Selector.

🔍 Segment Selector structure (16 bits):

Index: index into GDT/LDT table (×8 for byte offset)
TI (Table Indicator): 0 = GDT, 1 = LDT
RPL (Requested Privilege Level): requested privilege level (0~3)

📦 Segment Descriptor (64 bits)

The descriptor pointed to by Index contains:
– Segment base address (32 bits)
– Segment limit
– Type (code/data/read-only/executable, etc.)
– DPL (Descriptor Privilege Level)
– G bit (granularity: 0=byte, 1=4KB)

✅ The CPU looks up descriptors through the GDT (Global Descriptor Table) or LDT (Local Descriptor Table).

—

## 4. GDT vs LDT: Who Manages the “Segments”?

| Comparison | GDT (Global Descriptor Table) | LDT (Local Descriptor Table) |
| — | — | — |
| Scope | Global, shared by all tasks | Local, each task can have its own LDT |
| Count | Only one in the system | Multiple possible (one per process) |
| Usage frequency | Very high (kernel, user code/data segments) | Very low (modern OSes almost never use it) |
| Switching method | LGDT instruction loads | Points to LDT via a special descriptor in GDT |

📌 Modern operating systems (like Linux) have almost completely abandoned LDT, using only GDT with a few necessary segments: kernel code, kernel data, user code, user data, and a TSS (Task State Segment) or a few extra segments.

—

## 5. Practical Implementation: Entering Protected Mode

Below is the core code for switching from real mode to protected mode:

“`asm
; Minimal protected mode switch code (NASM syntax)
; Suitable for boot sector or early boot stage

section .text
[bits 16] ; Still in 16-bit real mode

switch_to_pm:
cli ; 1. Disable interrupts

; 2. Load GDT
lgdt [gdt_descriptor]

; 3. Set Protection Enable bit in CR0
mov eax, cr0
or eax, 0x1 ; Set PE bit (bit 0)
mov cr0, eax

; 4. Far jump to flush pipeline
; This jump loads CS with the new segment selector
jmp CODE_SEG:init_pm

[bits 32] ; Now in 32-bit protected mode

init_pm:
; 5. Update all segment registers
mov ax, DATA_SEG
mov ds, ax
mov ss, ax
mov es, ax
mov fs, ax
mov gs, ax

; 6. Set up kernel stack
mov ebp, 0x90000
mov esp, ebp

; 7. Call kernel main
call kernel_main

; GDT Definition
gdt_start:
; Null descriptor (required by CPU)
dd 0x0
dd 0x0

gdt_code: ; Code segment descriptor
; base=0x0, limit=0xFFFFF, 32-bit, Ring 0, executable
dw 0xFFFF ; Limit [15:0]
dw 0x0 ; Base [15:0]
db 0x0 ; Base [23:16]
db 10011010b ; 1, DPL=00, 1, Code, Non-conforming, Readable
db 11001111b ; G=1, D/B=1, L=0, AVL=0, Limit[19:16] = 0xF
db 0x0 ; Base [31:24]

gdt_data: ; Data segment descriptor
; base=0x0, limit=0xFFFFF, 32-bit, Ring 0, writable
dw 0xFFFF ; Limit [15:0]
dw 0x0 ; Base [15:0]
db 0x0 ; Base [23:16]
db 10010010b ; 1, DPL=00, 1, Data, Expand-up, Writable
db 11001111b ; G=1, D/B=1, L=0, AVL=0, Limit[19:16] = 0xF
db 0x0 ; Base [31:24]

gdt_end:

gdt_descriptor:
dw gdt_end – gdt_start – 1 ; GDT size – 1
dd gdt_start ; GDT base address

CODE_SEG equ gdt_code – gdt_start
DATA_SEG equ gdt_data – gdt_start

kernel_main:
; Kernel starts here!
; Now in protected mode, 32-bit code
; …
“`

—

## 6. Testing and Verification

After successfully entering protected mode:

CR0 PE bit = 1 → protected mode confirmed active
All segment registers loaded with GDT selectors
Memory access uses 32-bit addressing, up to 4GB
Any attempt to access memory beyond segment limits triggers #GP

✅ Successfully switched from real mode to protected mode!

—

## 7. Summary: The Role of Segmentation in Modern OSes

1. **Real mode**: direct physical address access, no protection, addressable up to 1MB
2. **Protected mode**: adds segment descriptor tables (GDT/LDT), privilege levels (Ring 0~3), memory protection
3. **Segmentation: the first line of defense**: segment limit checks prevent memory access violations
4. **Flat model**: Linux sets base=0, limit=4GB for all segments, effectively “bypassing” segmentation
5. **Paging is the true weapon**: memory isolation and virtual memory are implemented through paging

🔥 Even if you bypass segmentation, you must understand it — it’s the foundation of x86’s privilege model and deeply tied to interrupts, TSS, and other mechanisms.

🌟 Learning segmentation is not about using it, but about understanding why modern systems choose to bypass it.

---

Next article: Enabling paging — constructing the foundation of virtual memory.

#OS #KernelDevelopment #ProtectedMode #GDT #Segmentation #RealMode #WritingFromScratch