Linux SLUB Allocator

#internals#slub

Why SLUB exists

The page allocator (the “buddy allocator”, in mm/page_alloc.c) only ever hands out memory in power-of-two page chunks. But the kernel constantly needs small, fixed-size objects (task_struct, a struct file, a 64-byte buffer), thousands of times a second. Asking the buddy allocator for a fresh page every time would be slow (every call takes a lock, does buddy-merge bookkeeping) and wasteful (a 64-byte object doesn’t need a whole 4KB page).

SLUB’s job: grab pages from the buddy allocator in bulk, carve each page into many same-sized objects, and hand those out / take them back as cheaply as possible

There are three layers stacked on top of each other, and everything in this note is about how a request moves down through them and back up:

Key structs

struct kmem_cache — one per type of object

mm/slab.h:223. One of these exists for each distinct type of object being cached — cred_jar (for struct cred), filp_cachep (for struct file), kmalloc-64, etc. are all separate kmem_cache instances.

struct kmem_cache {
	struct slub_percpu_sheaves __percpu *cpu_sheaves;
	slab_flags_t flags;
	unsigned long min_partial;
	unsigned int size;		/* Object size including metadata */
	unsigned int object_size;	/* Object size without metadata */
	struct reciprocal_value reciprocal_size;
	unsigned int offset;		/* Free pointer offset */
	unsigned int sheaf_capacity;
	struct kmem_cache_order_objects oo;

	struct kmem_cache_order_objects min;
	gfp_t allocflags;
	int refcount;
	void (*ctor)(void *object);
	unsigned int inuse;
	unsigned int align;
	unsigned int red_left_pad;
	const char *name;
	struct list_head list;
#ifdef CONFIG_SLAB_FREELIST_HARDENED
	unsigned long random;		/* per-cache secret, XORed into freelist pointers */
#endif
	...
	struct kmem_cache_per_node_ptrs per_node[MAX_NUMNODES];
};

Important fields to remember for later:

  • size vs object_size — SLUB uses a few bytes inside every object for bookkeeping (the freelist pointer). object_size is what the caller asked for, size is what’s actually reserved.
  • offset — where inside the object the freelist pointer lives.
  • cpu_sheaves — a percpu pointer, i.e. it’s actually one struct per CPU (see below). Every cache gets its own private row of these, allocated once at cache creation via 
  • s->cpu_sheaves = alloc_percpu(struct slub_percpu_sheaves); (mm/slub.c:8657).

Mental model: think of a grid, rows = caches, columns = CPUs. Every cell is an independent per-CPU structure with its own lock and its own state. kmalloc-64 on CPU 2 has nothing to do with cred_jar on CPU 2. Different cells entirely, reached only through cache->cpu_sheaves.

struct slab — one physical run of pages, carved into same-sized objects

mm/slab.h:99. This overlays the same memory as the first struct page of the folio it describes (see the SLAB_MATCH static-asserts at mm/slab.h:119-130) — no separate allocation needed for slab metadata.

struct slab {
	memdesc_flags_t flags;
	struct kmem_cache *slab_cache;   /* which cache owns this slab */
	union {
		struct {
			struct list_head slab_list;
			struct freelist_counters;   /* freelist head, inuse count, frozen bit */
		};
		struct rcu_head rcu_head;
	};
	unsigned int __page_type;
	atomic_t __page_refcount;
#ifdef CONFIG_SLAB_OBJ_EXT
	unsigned long obj_exts;
#endif
};

slab_cache is the important one: it’s how kfree(ptr) figures out which cache owns an arbitrary pointer.

struct kmem_cache_node — per-NUMA-node bookkeeping

struct kmem_cache_node {
	spinlock_t list_lock;
	unsigned long nr_partial;
	struct list_head partial;   /* slabs that are neither full nor empty */
#ifdef CONFIG_SLUB_DEBUG
	atomic_long_t nr_slabs;
	atomic_long_t total_objects;
	struct list_head full;
#endif
};

This is the classic “slow path” territory — a lock shared by every CPU on that node. Everything else in this note exists to avoid touching it as much as possible.

struct slub_percpu_sheaves + struct slab_sheaf — the new per-CPU fast path

mm/slab.h:433. This is a recent (Sept 2025 – Jan 2026) rework that fully replaced the old struct kmem_cache_cpu “one active slab per CPU” design.

struct slub_percpu_sheaves {
	local_trylock_t lock;
	struct slab_sheaf *main;   /* never NULL when unlocked */
	struct slab_sheaf *spare;  /* empty or full, may be NULL */
	struct slab_sheaf *rcu_free;
};

A “sheaf” is just a small array of already-known-good object pointers (void *objects[]) — not a linked list threaded through the objects themselves. Alloc = pop the array; free = push the array.

Old design (still what most kernels in the wild use): struct kmem_cache_cpu held exactly one active slab and its freelist per CPU, and every alloc had to dereference the object about to be handed out to find the next free object and then do an atomic cmpxchg16b to swap the freelist head. This old design affects performance. Therefore, Sheaves replace all of that with a plain array push/pop.

struct node_barn — per-node stash of whole sheaves, shared across CPUs

#define MAX_FULL_SHEAVES	10
#define MAX_EMPTY_SHEAVES	10

struct node_barn {
	spinlock_t lock;
	struct list_head sheaves_full;
	struct list_head sheaves_empty;
	unsigned int nr_full;
	unsigned int nr_empty;
};

Every cache has one barn per NUMA node. This is the layer between “purely local to one CPU” (the sheaf) and “actually touch slab pages” — a shared, but still fairly cheap, stash.

Small supporting structs

/* Bundle of context threaded through the alloc call chain instead of loose args */
struct slab_alloc_context {
	unsigned long caller_addr;
	size_t orig_size;
	unsigned int alloc_flags;
	struct list_lru *lru;
};

/* Page order + object count packed into one word, so it's readable/comparable atomically */
struct kmem_cache_order_objects {
	unsigned int x;
};

kmalloc_caches[][] — the generic-size-class grid

include/linux/slab.h:738-740:

typedef struct kmem_cache * kmem_buckets[KMALLOC_SHIFT_HIGH + 1];
extern kmem_buckets kmalloc_caches[NR_KMALLOC_TYPES];

Another grid: [type][size_index] -> struct kmem_cache *. Rows are allocation type, columns are size class (kmalloc-8, kmalloc-16, … kmalloc-8192).

enum kmalloc_cache_type {
	KMALLOC_NORMAL = 0,
	...
	KMALLOC_RECLAIM,
	KMALLOC_DMA,
	KMALLOC_CGROUP,
	KMALLOC_NO_OBJ_EXT,
	NR_KMALLOC_TYPES
};

struct per_cpu_pages — the buddy allocator’s own per-CPU cache

include/linux/mmzone.h:833-849. this is a completely separate mechanism from SLUB’s sheaves . It lives in the page allocator.

struct per_cpu_pages {
	spinlock_t lock;
	int count;          /* pages currently sitting in this list */
	int high;            /* high watermark — spill to buddy above this */
	int high_min, high_max;
	int batch;            /* chunk size for moving pages to/from real buddy lists */
	short free_count;
	struct list_head lists[NR_PCP_LISTS];
};
  • The SLUB fetched pages from per cpu cache.

  • It calls buddy allocator only if this cache is empty

Both kmalloc()/kmem_cache_alloc() and kfree()/kmem_cache_free() walk the same tiers, just in opposite directions:

kmalloc()

Step 1: the macro entry

#define kmalloc(size, flags)  alloc_hooks(kmalloc_noprof(size, flags))

include/linux/slab.h:1053.

Expands to _kmalloc_noprof(size, flags, token), include/linux/slab.h:979-994:

static __always_inline __alloc_size(1) void *_kmalloc_noprof(size_t size, gfp_t flags, kmalloc_token_t token)
{
	if (__builtin_constant_p(size) && size) {
		unsigned int index;
		if (size > KMALLOC_MAX_CACHE_SIZE)
			return __kmalloc_large_noprof(size, flags);
		index = kmalloc_index(size);
		return __kmalloc_cache_noprof(
				kmalloc_caches[kmalloc_type(flags, token)][index],
				flags, size);
	}
	return __kmalloc_noprof(PASS_TOKEN_PARAMS(size, token), flags);
}

If size is a compile-time constant (the vast majority of real call sites, e.g. kmalloc(sizeof(struct foo), GFP_KERNEL)), the size→cache resolution happens at compile time. If it’s a runtime variable, it falls to __kmalloc_noprof(), which does the same lookup at runtime via kmalloc_slab().

Step 2a: large allocations skip SLUB entirely

KMALLOC_MAX_CACHE_SIZE = 1 << KMALLOC_SHIFT_HIGH = 1 << (PAGE_SHIFT+1) = 8192 bytes on x86-64 (include/linux/slab.h:663,672). Anything bigger goes straight to the page allocator, no kmem_cache involved:

static void *___kmalloc_large_node(size_t size, gfp_t flags, int node)
{
	unsigned int order = get_order(size);
	flags |= __GFP_COMP;
	if (node == NUMA_NO_NODE)
		page = alloc_frozen_pages_noprof(flags, order);   /* <-- straight into the buddy allocator */
	...
	__SetPageLargeKmalloc(page);
	return page_address(page);
}

mm/slub.c:5262-5291. The page is tagged PageLargeKmalloc so kfree() later knows to hand it straight back to the page allocator instead of looking for a struct slab.

Step 2b: normal path — resolving the exact cache

index = kmalloc_index(size);   /* compile-time size -> bucket, e.g. 40 bytes -> index for kmalloc-64 */
return __kmalloc_cache_noprof(kmalloc_caches[kmalloc_type(flags, token)][index], flags, size);

Sizes are rounded up to the nearest bucket (kmalloc(40, ...) gets a 64-byte slot) — the leftover bytes are unused-but-adjacent memory, a classic overflow-into-adjacent-object surface.

Runtime equivalent, used for non-constant sizes, kmalloc_slab() (mm/slab.h:389-404):

static inline struct kmem_cache *
kmalloc_slab(size_t size, kmem_buckets *b, gfp_t flags, kmalloc_token_t token, unsigned int alloc_flags)
{
	enum kmalloc_cache_type type = kmalloc_type(flags, token);
	if (!b)
		b = &kmalloc_caches[type];
	if (size <= 192)
		index = kmalloc_size_index[size_index_elem(size)];
	else
		index = fls(size - 1);
	return (*b)[index];
}

Step 3: __kmalloc_cache_noprof() — rejoins the normal kmem_cache_alloc machinery

mm/slub.c:5476-5491:

void *__kmalloc_cache_noprof(struct kmem_cache *s, gfp_t gfpflags, size_t size)
{
	const struct slab_alloc_context ac = { .orig_size = size, ... };
	ret = slab_alloc_node(s, gfpflags, NUMA_NO_NODE, &ac);
	ret = kasan_kmalloc(s, ret, size, gfpflags);
	return ret;
}

From here it’s the exact same slab_alloc_node() used by any kmem_cache_alloc() call — the tier hierarchy above (sheaf → spare → barn → partial list → new slab) applies identically regardless of whether the caller went through kmalloc() or a direct kmem_cache_alloc(some_specific_cache, ...).

The fast path itself, alloc_from_pcs() (mm/slub.c:4728-4800) — this is the version to remember, no freelist decoding, no cmpxchg, just an array pop:

if (!local_trylock(&s->cpu_sheaves->lock))
	return NULL;
pcs = this_cpu_ptr(s->cpu_sheaves);
if (unlikely(pcs->main->size == 0)) {
	pcs = __pcs_replace_empty_main(s, pcs, gfp, alloc_flags);   /* spare -> barn -> refill from slabs */
	...
}
object = pcs->main->objects[pcs->main->size - 1];
pcs->main->size--;
local_unlock(&s->cpu_sheaves->lock);
return object;

Step 4: the actual bottom — where SLUB touches the buddy allocator

new_slab() → allocate_slab() → alloc_slab_page(), mm/slub.c:3252-3277:

static inline struct slab *alloc_slab_page(gfp_t flags, int node,
                   struct kmem_cache_order_objects oo, bool allow_spin)
{
	unsigned int order = oo_order(oo);
	if (unlikely(!allow_spin))
		page = alloc_frozen_pages_nolock(0, node, order);
	else if (node == NUMA_NO_NODE)
		page = alloc_frozen_pages(flags, order);      /* <-- the buddy allocator call */
	else
		page = __alloc_frozen_pages(flags, order, node, NULL);

	__SetPageSlab(page);
	slab = page_slab(page);      /* reinterpret struct page as struct slab */
	return slab;
}

alloc_frozen_pages() is the exact function that crosses from SLUB into the buddy allocator — same family used by the large-kmalloc bypass in step 2a. oo is s->oo, the packed struct kmem_cache_order_objects computed once at cache creation (“how many pages / how many objects per slab this cache uses”).

Full chain — kmalloc(40, GFP_KERNEL)

kmalloc() macro
  -> _kmalloc_noprof (compile-time size check)
    -> kmalloc_index() + kmalloc_type() resolve s = kmalloc-64
      -> __kmalloc_cache_noprof(s, ...)
        -> slab_alloc_node(s, ...)
          -> alloc_from_pcs()          [usual stopping point: sheaf pop]
             (miss) -> spare -> barn -> refill_sheaf -> new_slab -> allocate_slab
               -> alloc_slab_page -> alloc_frozen_pages()   [buddy allocator]

kfree() — full path, top to bottom

Step 1: kfree() has to figure out the cache — it isn’t told

Unlike kmem_cache_free(cred_jar, ptr), kfree(ptr) only gets a pointer:

void kfree(const void *object)
{
	if (unlikely(ZERO_OR_NULL_PTR(object)))
		return;

	page = virt_to_page(object);   /* which physical page is this address in? */
	slab = page_slab(page);        /* does that page belong to a slab? */
	if (!slab) {
		free_large_kmalloc(page, (void *)object);   /* must've been a large kmalloc */
		return;
	}

	s = slab->slab_cache;          /* struct slab remembers which kmem_cache owns it */
	slab_free(s, slab, x, _RET_IP_);
}

mm/slub.c:6671-6694.

The page itself is the source of truth for “what cache owns this object,” via slab->slab_cache, not anything about the pointer value. This is exactly why type-confusion bugs work at all — if an attacker can get a different cache’s object to occupy a page that used to belong to the target cache (a cross-cache reuse), slab->slab_cache now correctly points at the new owner, and the kernel has no way to know the pointer someone else is holding was meant for the old type.

Step 1b: the large-allocation branch - straight to the buddy allocator

static void free_large_kmalloc(struct page *page, void *object)
{
	unsigned int order = compound_order(page);
	...
	__ClearPageLargeKmalloc(page);
	free_frozen_pages(page, order);     /* <-- straight to the buddy allocator */
}

mm/slub.c:6598-6618. Mirrors the alloc side exactly — free_frozen_pages() is the counterpart to alloc_frozen_pages().

Step 2: normal case — slab_free()

static __fastpath_inline
void slab_free(struct kmem_cache *s, struct slab *slab, void *object, unsigned long addr)
{
	memcg_slab_free_hook(s, slab, &object, 1);
	alloc_tagging_slab_free_hook(s, slab, &object, 1);

	if (unlikely(!slab_free_hook(s, object, slab_want_init_on_free(s), false)))
		return;

	if (likely(can_free_to_pcs(slab)) && likely(free_to_pcs(s, object, true)))
		return;               /* fast path: pushed onto sheaf, done */

	__slab_free(s, slab, object, object, 1, addr);   /* slow path */
	stat(s, FREE_SLOWPATH);
}

mm/slub.c:6370-6385.

slab_free_hook() is where init_on_free zeroing, KASAN poisoning, and freelist-pointer hardening happen. The freelist pointer that gets written into the freed object isn’t stored raw — under CONFIG_SLAB_FREELIST_HARDENED (mm/slub.c:512-541):

encoded = (unsigned long)ptr ^ s->random ^ swab(ptr_addr);

XORed with a per-cache secret (s->random, generated once via get_random_long() at cache creation, mm/slub.c:8619) and the byte-swapped address of the slot storing the pointer . so copying a valid encoded entry to a different memory location decodes to garbage.

Fast path, free_to_pcs() — mirror image of alloc_from_pcs():

bool free_to_pcs(struct kmem_cache *s, void *object, bool allow_spin)
{
	if (!local_trylock(&s->cpu_sheaves->lock))
		return false;
	pcs = this_cpu_ptr(s->cpu_sheaves);
	if (unlikely(pcs->main->size == s->sheaf_capacity)) {
		pcs = __pcs_replace_full_main(s, pcs, allow_spin);   /* spare -> barn -> flush */
		...
	}
	pcs->main->objects[pcs->main->size++] = object;
	local_unlock(&s->cpu_sheaves->lock);
	return true;
}

mm/slub.c:5931-5955. Push onto the array, done — the vast majority of kfree() calls stop right here.

Step 3: the slow path — __slab_free()

This is the function that walks the real freelist and — when the last object in a slab is freed , makes the min_partial decision .

if (unlikely(!new.inuse && n->nr_partial >= s->min_partial))
	goto slab_empty;   /* -> remove_partial() + discard_slab() */

mm/slub.c:5715.

  • new.inuse == 0 → the last object in this slab was just freed.
  • If the node already has >= min_partial other slabs banked as spares → actually discard this one.
  • Otherwise → leave the now-empty slab on the node’s partial list, ready for instant reuse, no buddy call at all.

Step 4: the actual bottom — __free_slab() back to the buddy allocator

static void __free_slab(struct kmem_cache *s, struct slab *slab, bool allow_spin)
{
	struct page *page = slab_page(slab);
	int order = compound_order(page);
	__ClearPageSlab(page);          /* this page is no longer a slab page at all */
	unaccount_slab(slab, order, s, allow_spin);
	if (allow_spin)
		free_frozen_pages(page, order);     /* <-- back to the buddy allocator */
	else
		free_frozen_pages_nolock(page, order);
}

mm/slub.c:3429-3444.

__ClearPageSlab(page) is the critical line — once this runs, page_slab() returns NULL for this page from now on, and it’s genuinely up for grabs by any other cache or non-slab code, once it clears the buddy allocator’s own per-CPU cache (next section).

Full chain summary — kfree(ptr)

kfree(ptr)
  -> virt_to_page + page_slab   (recover s from the page itself, not the pointer)
    -> slab_free(s, ...)
      -> free_to_pcs()            [usual stopping point: sheaf push]
         (sheaf/barn full) -> __slab_free()
           -> (slab now empty AND node already has >= min_partial spares)
             -> discard_slab -> free_slab -> __free_slab
               -> __ClearPageSlab -> free_frozen_pages()   [buddy allocator, PCP]

 alloc_frozen_pages() / free_frozen_pages() and the buddy allocator’s own per-CPU cache

Both directions cross into mm/page_alloc.c through this exact pair of functions. Neither one talks to the real, globally-shared buddy free lists directly on the common path — there’s one more per-CPU caching layer first: the PCP (per_cpu_pages, struct defined above).

Alloc side: rmqueue() checks PCP before the real buddy lists

static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone,
        unsigned int order, gfp_t gfp_flags, unsigned int alloc_flags, int migratetype)
{
	if (likely(pcp_allowed_order(order))) {
		page = rmqueue_pcplist(preferred_zone, zone, order, migratetype, alloc_flags);
		if (likely(page))
			goto out;
	}
	page = rmqueue_buddy(preferred_zone, zone, order, alloc_flags, migratetype);
	...
}

mm/page_alloc.c:3399-3428.

__rmqueue_pcplist() (mm/page_alloc.c:3314-3356) — exact mirror of a sheaf pop:

do {
	if (list_empty(list)) {
		int batch = nr_pcp_alloc(pcp, zone, order);
		alloced = rmqueue_bulk(zone, order, batch, list, migratetype, alloc_flags);  /* refill from real buddy, in bulk */
		pcp->count += alloced << order;
		if (unlikely(list_empty(list)))
			return NULL;
	}
	page = list_first_entry(list, struct page, pcp_list);
	list_del(&page->pcp_list);
	pcp->count -= 1 << order;
} while (check_new_pages(page, order));
return page;

Local list pop, only this CPU’s pcp->lock, never touches zone->lock unless the local list is empty and needs a bulk refill (rmqueue_bulk) or PCP doesn’t apply to this order at all (rmqueue_buddy, real global buddy search under zone->lock).

Free side: free_frozen_page_commit() — the watermark decision

pindex = order_to_pindex(migratetype, order);
list_add(&page->pcp_list, &pcp->lists[pindex]);   /* just push onto this CPU's local list */
pcp->count += 1 << order;
...
high = nr_pcp_high(pcp, zone, batch, free_high);
if (pcp->count < high)
	return true;                                    /* under the watermark — page stays local, nothing else happens */

to_free = nr_pcp_free(pcp, batch, high, free_high);
while (to_free > 0 && pcp->count > 0) {
	to_free_batched = min(to_free, batch);
	free_pcppages_bulk(zone, to_free_batched, pcp, pindex);   /* spill a batch to the REAL buddy lists */
	to_free -= to_free_batched;
}

mm/page_alloc.c:2832-2890.

So a freed slab page just gets appended to this CPU’s local list and the function returns — no global lock, no merging — unless pcp->count has climbed past high.

free_pcppages_bulk() — this is the actual, literal buddy allocator

static void free_pcppages_bulk(struct zone *zone, int count, struct per_cpu_pages *pcp, int pindex)
{
	guard(spinlock_irqsave)(&zone->lock);     /* the real, system-wide, contended lock */
	while (count > 0) {
		...
		page = list_last_entry(list, struct page, pcp_list);
		list_del(&page->pcp_list);
		pcp->count -= nr_pages;
		__free_one_page(page, pfn, zone, order, mt, FPI_NONE);   /* THE buddy-merge algorithm */
	}
}

mm/page_alloc.c:1459-1507. __free_one_page() is the classic buddy-coalescing routine — checks if the page’s buddy is also free, merges into a bigger block, inserts into zone->free_area[order]. This is the first moment the page is genuinely, globally free — visible to any CPU’s rmqueue_buddy() call.