K08 — Inter-Process Shared Memory: shm and mmap(MAP_SHARED)
Key Insight: The fastest inter-process communication is no communication at all — directly share the same block of memory.
Three Ways to Share Memory Between Processes
Linux supports three mechanisms for inter-process shared memory:
1. System V Shared Memory (shmget/shmat)
- Long history, pre-POSIX API
- Managed through kernel objects (shmid_kernel)
2. POSIX Shared Memory (shm_open/shm_unlink)
- File mapping based on tmpfs
- Operated through file descriptors
3. mmap(MAP_SHARED)
- Anonymous sharing (parent-child processes)
- File-backed sharing (file as backend)
Commonality of all three:
- They all map physical pages into the virtual address space of multiple processes
- Write once, all processes see it immediately (no copying)Performance Comparison:
System V shm POSIX shm(/dev/shm) mmap(MAP_SHARED)
Creation overhead High (kernel obj) Medium (tmpfs file) Low (anon) or Medium (file)
Access latency Same (page table) Same Same
Lifetime Explicit delete Last ref close Last mapping close
Debug convenience ipcs -m ls /dev/shm /proc/PID/maps
POSIX standard No Yes Yes
Cross-node (NUMA) Supported Supported SupportedSystem V Shared Memory Kernel Path
shm kernel data structure path:
Userspace: shmget(key, size, IPC_CREAT|0666)
↓
Kernel: shmid_kernel structure (struct shmid_kernel)
↓
Associated with an anonymous tmpfs file (no real filename)
↓
struct file → struct address_space
↓
Physical pages (from buddy system, added to page cache)
Userspace: shmat(shmid, NULL, 0)
↓
Kernel: Creates a VMA (Virtual Memory Area) for the current process
↓
VMA.vm_ops → shm_vm_ops (file operations)
↓
Page table mapping: virtual address → struct page → physical page
Key Points:
- shm physical pages are part of the page cache
- Can be reclaimed by tmpfs reclaim mechanism (under memory pressure)
- shmctl(IPC_RMID) only marks for deletion; actual page release happens after the last process detaches
Viewing shared memory (ipcs):
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x1234 65536 user 666 1048576 2 dest
nattch = number of processes attached to this segment
status = dest means "marked for deletion, waiting for last process to detach"System V Shared Memory
Using shmget / shmat:
key_t key = ftok("/some/path", 1);
int shmid = shmget(key, 1024*1024, IPC_CREAT | 0666);
if (shmid == -1) { perror("shmget"); exit(1); }
void *ptr = shmat(shmid, NULL, 0);
if (ptr == (void*)-1) { perror("shmat"); exit(1); }
// Now multiple processes can access this memory simultaneously
strcpy(ptr, "Hello from process A");
// ... after use
shmdt(ptr);
// Only the last process to detach needs to delete it
shmctl(shmid, IPC_RMID, NULL);POSIX Shared Memory
POSIX shared memory is file-based:
int fd = shm_open("/myshm", O_CREAT | O_RDWR, 0666);
if (fd == -1) { perror("shm_open"); exit(1); }
ftruncate(fd, 1024*1024);
void *ptr = mmap(NULL, 1024*1024, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (ptr == MAP_FAILED) { perror("mmap"); exit(1); }
close(fd); // The mapping stays, fd can be closed
// Use ptr for shared memory...
// Cleanup
shm_unlink("/myshm");Comparison with System V:
System V shm:
$ ipcs -m # List all shared memory segments
$ ipcrm -m <shmid> # Remove
POSIX shm:
$ ls -la /dev/shm/ # View all POSIX shared memory files
$ rm /dev/shm/myshm # Removemmap(MAP_SHARED) — Anonymous Shared Memory
Anonymous mmap (MAP_ANONYMOUS | MAP_SHARED), typically used between parent and child:
// Parent process
void *ptr = mmap(NULL, 4096, PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_ANONYMOUS, -1, 0);
pid_t pid = fork();
if (pid == 0) {
// Child process: can read/write the same memory as parent
printf("Child reads: %sn", (char*)ptr);
strcpy(ptr, "Written by child");
exit(0);
}
// Parent
wait(NULL);
printf("Parent reads: %sn", (char*)ptr);File-backed shared mmap:
// An existing file as the backend for shared memory
int fd = open("/tmp/shared.dat", O_RDWR | O_CREAT, 0666);
ftruncate(fd, 4096);
void *ptr = mmap(NULL, 4096, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
// Multiple processes can mmap the same file → shared memoryData Structures
shmid_kernel (kernel internal):
struct shmid_kernel {
struct kern_ipc_perm shm_perm; // Permission information
struct file *shm_file; // Corresponding anonymous tmpfs file
unsigned long shm_nattch; // Number of attached processes
unsigned long shm_segsz; // Segment size
time64_t shm_atim; // Last attach time
time64_t shm_dtim; // Last detach time
time64_t shm_ctim; // Last change time
pid_t shm_cprid; // Creator PID
pid_t shm_lprid; // Last operator PID
struct user_struct *mlock_user; // mlock accounting
};
View in /proc:
$ cat /proc/sysvipc/shmPractical Considerations
| Concern | Solution |
|---|---|
| Synchronization | Use mutex/semaphore/futex with shared memory |
| Cross-platform | Prefer POSIX shm or mmap |
| Security | Set permissions correctly, use unique key names |
| Debugging | ipcs, /proc/PID/maps, strace, gdb |
| Performance | Minimize lock contention, align data structures to cache lines |
| Persistence | shm has no persistence (tmpfs is ephemeral) |
Synchronization example (using POSIX semaphore with shared memory):
sem_t *sem = sem_open("/mysem", O_CREAT, 0666, 1);
void *shm = shm_open(...);
// ...
sem_wait(sem);
// Critical section operating on shared memory
sem_post(sem);
Comments