How page fault is handled ?
I was looking at dev_vma_fault in one of the CTF challenge module and I realised that I had no real idea how the CPU ends up calling that function. I know mmap() returns a pointer, I know touching an unmapped page inside it somehow lands in the driver’s .fault callback, but the actual path between “CPU walks the page table to realised to physical memory backing it up” and “ driver’s C function runs” was a abstraction to me.
Since I already traced the SYSCALL path start to end in the syscalls note, I decided to do the same for this as well. So here’s the whole thing, traced through source, ending at all three flavors of fault: anonymous (regular heap/stack memory), a custom driver’s mmap (CTF-style), and a regular file’s mmap (page cache).
the CPU doesn’t call anything, it’s hardware
Unlike SYSCALL, a page fault isn’t something userspace asks for. It’s exception vector 14 (X86_TRAP_PF), raised entirely by hardware the moment the MMU’s page-table walk fails , page not present, wrong permissions, reserved bit set, whatever.
Every exception the CPU can raise gets dispatched through the IDT (Interrupt Descriptor Table) ,a 256-entry array sitting in memory that the CPU indexes directly by vector number.
// arch/x86/kernel/idt.c:71 and :249
INTG(X86_TRAP_PF, asm_exc_page_fault),
So the moment the MMU fails, the CPU:
- Pushes an error code.
- Pushes
ss,rsp,rflags,cs,rip - Latches the actual faulting virtual address into a special register,
CR2becauseriponly tells you which instruction faulted, not which address it was trying to touch - Indexes the IDT at slot 14, jumps to whatever’s there
That’s asm_exc_page_fault. Everything past this point is pure software.
Where asm_exc_page_fault actually comes from
I went looking for SYM_CODE_START(asm_exc_page_fault) expecting to grep straight to it like I did for entry_SYSCALL_64. Couldn’t find it anywhere as literal text. Turns out there’s a nice trick here.
arch/x86/include/asm/idtentry.h:595 has exactly one line:
DECLARE_IDTENTRY_RAW_ERRORCODE(X86_TRAP_PF, exc_page_fault);
- I think this is enough to know the origin because Linux has tons of complex macros to auto expand function names before compilation. It is not worthwhile to go deep into it ,at least for this notes.
- In short, this macro generates ASM to save the state and eventually call the exc_page_fault function.
Step 3 — the asm stub: idtentry_body
Source: arch/x86/entry/entry_64.S:289
.macro idtentry_body cfunc has_error_code:req
ALTERNATIVE "call error_entry; movq %rax, %rsp", \
"call xen_error_entry", X86_FEATURE_XENPV
ENCODE_FRAME_POINTER
movq %rsp, %rdi /* arg0 = &pt_regs */
.if \has_error_code == 1
movq ORIG_RAX(%rsp), %rsi /* arg1 = error code */
movq $-1, ORIG_RAX(%rsp)
.endif
call \cfunc
jmp error_return
.endm
Same shape as the syscall entry: switch onto the proper kernel stack (error_entry), build a struct pt_regs on the stack, put the hardware error code into %rsi, and call the C handler — exc_page_fault(regs, error_code).
Quick reminder on
pt_regssince it shows up again here exactly like it did for syscalls (full layout is in the syscalls note) , it’s just every general-purpose register plusip/cs/flags/sp/ss, laid out on the stack in the order the entry code pushed them. Same struct, same idea, but different entry path filling it in.
exc_page_fault(), the C entry
Source: arch/x86/mm/fault.c:1492
DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault)
{
irqentry_state_t state;
unsigned long address;
address = cpu_feature_enabled(X86_FEATURE_FRED) ? fred_event_data(regs) : read_cr2();
if (kvm_handle_async_pf(regs, (u32)address))
return;
state = irqentry_enter(regs);
instrumentation_begin();
handle_page_fault(regs, error_code, address);
instrumentation_end();
irqentry_exit(regs, state);
}
read_cr2() is where that faulting address from Step 1 finally gets read back out. irqentry_enter/exit just does the accounting kernel entry code always needs (RCU watching, context tracking, etc
The juicy part is here: handle_page_fault(regs, error_code, address).
Handle_page_fault: kernel address or user address?
Source: arch/x86/mm/fault.c
static __always_inline void
handle_page_fault(struct pt_regs *regs, unsigned long error_code, unsigned long address)
{
if (unlikely(fault_in_kernel_space(address)))
do_kern_addr_fault(regs, error_code, address);
else
do_user_addr_fault(regs, error_code, address);
}
One branch on whether the faulting address falls in kernel space or user space. do_kern_addr_fault handles things like vmalloc faults. We care about do_user_addr_fault, since that’s the path both your regular heap/stack memory and a driver’s mmap() region go through.
do_user_addr_fault: finding the VMA
Source: arch/x86/mm/fault.c:1216
void do_user_addr_fault(struct pt_regs *regs, unsigned long error_code, unsigned long address)
{
struct vm_area_struct *vma;
struct mm_struct *mm = current->mm;
...
vma = lock_mm_and_find_vma(mm, address, regs);
if (unlikely(access_error(error_code, vma))) {
bad_area_access_error(regs, error_code, address, mm, vma);
return;
}
fault = handle_mm_fault(vma, address, flags, regs);
...
}
current->mm is a struct mm_struct — every process has exactly one, and it’s the thing that owns the entire address space: the top-level page table (->pgd, the thing that eventually goes into CR3), and the collection of every vm_area_struct that process has mapped.
Which brings us to the struct that matters most for the rest of this note:
// include/linux/mm_types.h:920
struct vm_area_struct {
unsigned long vm_start; // VMA covers [vm_start, vm_end)
unsigned long vm_end;
struct mm_struct *vm_mm; // which process owns this
pgprot_t vm_page_prot; // access permissions
vm_flags_t vm_flags; // VM_READ, VM_WRITE, VM_SHARED, etc.
struct anon_vma *anon_vma; // set if this VMA has anonymous pages
const struct vm_operations_struct *vm_ops; // NULL for anonymous VMAs
unsigned long vm_pgoff; // offset into the file, in PAGE_SIZE units
void *vm_private_data; // driver/filesystem's own pointer — this is exactly
// where your `dev_mmap()` stashed `sbuf`
};
Every mapped region in your address space — a malloc()‘d heap chunk, the stack, a shared library, a mmap()‘d device — is one of these. The important field for where we’re going is vm_ops. If it’s NULL, the VMA is plain anonymous memory (no backing file, no driver). If it’s set, something like a filesystem, a driver installed a .fault handler and wants to be asked for pages itself.
lock_mm_and_find_vma() walks mm’s VMA tree and returns whichever vm_area_struct contains the faulting address. access_error() checks the fault type (read/write/exec) against vma->vm_flags, this is where a write to a read-only mapping gets turned into a SIGSEGV instead of proceeding further.
Then: handle_mm_fault(vma, address, flags, regs). This call is the actual boundary between architecture-specific code (arch/x86/mm/) and the generic memory manager (mm/memory.c) , every architecture Linux supports funnels into this exact same function from here on.
Crossing into generic mm: struct vm_fault
From here the code stops caring whether you’re on x86, ARM, or RISC-V. And the very first thing that happens is the kernel packages up everything about this fault into one struct that gets threaded through every function from now on:
// include/linux/mm.h:730
struct vm_fault {
struct vm_area_struct *vma; // the VMA we found in Step 6
pgoff_t pgoff; // logical page offset *within the VMA*
unsigned long address; // faulting virtual address (masked to page boundary)
enum fault_flag flags; // FAULT_FLAG_WRITE, FAULT_FLAG_USER, etc.
pmd_t *pmd; // page table pointers as the walk finds them
pte_t orig_pte; // the PTE's value at fault time (usually just empty)
struct page *page; // fault handler fills this in awith physical page backeup and hands it back
pte_t *pte; // valid once the actual page table entry is located
spinlock_t *ptl; // lock protecting that pte
};
This is the struct your driver’s fault handler actually receives, the vmf argument in dev_vma_fault(struct vm_fault *vmf). Everything above was building this thing up; everything below is either filling it in or reading it.
handle_mm_fault() → __handle_mm_fault() → handle_pte_fault() (mm/memory.c:6335) walks the page tables (pgd → p4d → pud → pmd → pte) allocating intermediate levels as needed, and once it’s down at the PTE level with nothing there yet:
if (!vmf->pte)
return do_pte_missing(vmf);
The path split: do_pte_missing
Source: mm/memory.c:4561
static vm_fault_t do_pte_missing(struct vm_fault *vmf)
{
if (vma_is_anonymous(vmf->vma))
return do_anonymous_page(vmf);
else
return do_fault(vmf);
}
vma_is_anonymous() is really just checking vma->vm_ops == NULL. That one field, set (or not) way back when the VMA was created, decides which of the two completely different paths this fault takes.
The anonymous path: do_anonymous_page
This is what handles a fault on your regular heap, stack, or a plain mmap(MAP_ANONYMOUS) region, memory with no file or driver behind it.
Source: mm/memory.c:5287
The genuinely interesting bit is that it does two different things depending on whether the fault was a read or a write:
/* Use the zero-page for reads */
if (!(vmf->flags & FAULT_FLAG_WRITE) && !mm_forbids_zeropage(vma->vm_mm)) {
entry = pte_mkspecial(pfn_pte(zero_pfn(vmf->address), vma->vm_page_prot));
...
set_pte_at(vma->vm_mm, addr, vmf->pte, entry);
goto unlock;
}
/* Allocate our own private page. */
folio = alloc_anon_folio(vmf);
...
map_anon_folio_pte_pf(folio, vmf->pte, vma, addr,vmf_orig_pte_uffd_wp(vmf));
- Read fault → map a PTE pointing at
zero_pfn. It is a one single, system-wide, physical page of zeroes, shared read-only across literally every process on the machine that’s ever touched unwritten anonymous memory. No allocation happens at all. - Write fault → now actually allocate a real, private, zeroed page (
alloc_anon_folio) and map it read-write.
This is the actual mechanism behind “anonymous memory is lazily allocated” — malloc() giving you a huge chunk of “memory” instantly isn’t lying, it just hasn’t backed any of it with real physical pages yet. The zero-page trick means even reading newly mmap‘d memory doesn’t cost you a real page , writing is what actually costs you one, on the very first write to each page.
The file/device-backed path: do_fault → vma->vm_ops->fault
vma->vm_ops is non-NULL, so do_pte_missing called do_fault(vmf) instead.
Source: mm/memory.c:5964
static vm_fault_t do_fault(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
...
if (!(vmf->flags & FAULT_FLAG_WRITE))
ret = do_read_fault(vmf);
else if (!(vma->vm_flags & VM_SHARED))
ret = do_cow_fault(vmf);
else
ret = do_shared_fault(vmf);
...
}
All three of those (do_read_fault, do_cow_fault, do_shared_fault) eventually funnel into the same place: __do_fault(vmf).
Source: mm/memory.c:5398
static vm_fault_t __do_fault(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
...
ret = vma->vm_ops->fault(vmf);
...
}
And there it is. vma->vm_ops->fault(vmf) — a plain function pointer call into whatever installed vm_ops. Which brings us all the way back to the driver I started this whole call trace walk from:
static struct vm_operations_struct dev_vm_ops = {
.fault = dev_vma_fault
};
static int dev_mmap(struct file* filp, struct vm_area_struct* vma) {
struct shared_buffer* sbuf = filp->private_data;
...
vma->vm_ops = &dev_vm_ops; // <- the field checked in Step 8
return SUCCESS;
}
static vm_fault_t dev_vma_fault(struct vm_fault *vmf) {
struct vm_area_struct *vma = vmf->vma;
struct shared_buffer *sbuf = vma->vm_private_data;
pgoff_t pgoff = vmf->pgoff;
if (pgoff >= sbuf->pagecount)
return VM_FAULT_SIGBUS;
get_page(sbuf->pages[pgoff]);
vmf->page = sbuf->pages[pgoff]; // <- handing a page back through vm_fault
return SUCCESS;
}
Every field I explained above is being used right here, for exactly the reason it exists. dev_mmap() set vma->vm_ops, which is the one check in Step 8 that routed us down this path instead of the anonymous one. It stashed sbuf in vma->vm_private_data specifically so the fault handler could get it back later. And dev_vma_fault reads vmf->pgoff — the logical page offset the generic mm layer computed for us all the way back at the top — indexes into sbuf->pages[], and hands a real physical page back through vmf->page, exactly like handle_pte_fault expects.
But what about a regular file? open() + mmap() with no driver
Everything above used a custom .fault handler from a CTF driver. But most of the time in the real world there’s no driver at all , just open("some_file", O_RDONLY) then mmap() the fd. Same fork at Step 8, same do_fault → __do_fault → vma->vm_ops->fault(vmf). Only thing that changes is which vm_ops got installed.
open() points file->f_mapping at a struct address_space , the page cache for that inode (deserves its own note, not going deep here). mmap() then goes through file->f_op->mmap, which for a plain file is almost always generic_file_mmap():
// mm/filemap.c:4023
int generic_file_mmap(struct file *file, struct vm_area_struct *vma)
{
...
vma->vm_ops = &generic_file_vm_ops; // kernel's own vm_ops, not a driver's
return 0;
}
generic_file_vm_ops.fault = filemap_fault. So the call a few sections above (vma->vm_ops->fault(vmf)) now lands in filemap_fault(vmf) instead of dev_vma_fault(vmf).
Source: mm/filemap.c:3540
vm_fault_t filemap_fault(struct vm_fault *vmf)
{
struct address_space *mapping = vmf->vma->vm_file->f_mapping;
struct folio *folio = filemap_get_folio(mapping, vmf->pgoff);
if (IS_ERR(folio)) {
// cache miss , VM_FAULT_MAJOR, actually reads the page off disk
// via the filesystem's read_folio, then caches it
}
// cache hit , already in RAM, no I/O at all
...
}
start to end (Thanks to LLM)
CPU: MMU walk fails, raises #PF (vector 14), CR2 = faulting address
│
▼ (hardware: IDT[14] lookup, no software involved)
asm_exc_page_fault [asm, generated from one macro reused for C+asm]
idtentry_body: switch stack, hand-build pt_regs
│
▼
exc_page_fault(regs, error_code) [C]
address = read_cr2()
│
▼
handle_page_fault(regs, error_code, address)
│
├── kernel address ──► do_kern_addr_fault (separate rabbit hole)
│
└── user address ────► do_user_addr_fault
│
▼
lock_mm_and_find_vma() → finds the vm_area_struct
access_error() → permission check
│
▼
handle_mm_fault(vma, address, flags, regs)
│ [ arch-independent boundary ]
▼
__handle_mm_fault → handle_pte_fault → do_pte_missing
│
├── vma->vm_ops == NULL (anonymous)
│ │
│ ▼
│ do_anonymous_page
│ ├── read → map shared zero_pfn page, no alloc
│ └── write → alloc_anon_folio, map real page
│
└── vma->vm_ops != NULL (file/device-backed)
│
▼
do_fault → __do_fault
│
▼
vma->vm_ops->fault(vmf)
│
┌────────────┴────────────┐
▼ ▼
custom driver mmap() regular file mmap()
vm_ops = &dev_vm_ops vm_ops = &generic_file_vm_ops
│ │
▼ ▼
dev_vma_fault(vmf) filemap_fault(vmf)
looks up sbuf->pages[] checks page cache (address_space)
fills vmf->page ├── hit → already in RAM, no I/O
└── miss → read_folio() from disk,
inserted into page cache,
fills vmf->page