We all know the horror story: a parser thread receives a malformed message, triggers some dormant bug that overflows a buffer, and triggers a memory fault. In a typical Zephyr setup, the whole system goes down. You get a log dump if you’re lucky, a hard fault with no context if you’re not, and then you start debugging blind. With userspace configured properly, the story ends differently: the fault is caught, the offending thread is killed, and the rest of the system keeps running. That’s the pitch for Zephyr’s userspace feature. It’s fundamentally a security feature: its job is to enforce that a thread can’t touch memory it wasn’t granted. And the most practical day-to-day payoff is fault detection and recovery.
Zephyr’s userspace feature runs threads in user mode, backed by the MPU (Memory Protection Unit). A MPU is a hardware circuit baked into most Cortex-M processors. It divides RAM into regions, attaches read/write/execute permissions to each one, and fires a MemManage fault the moment any access violates them.
As we are about to discuss, Zephyr user spacefeature let us ultimately program the MPU’s region registers: telling the hardware exactly what memory each thread is allowed to touch. Cross a boundary, and the fault fires at that exact instruction, before anything is read or written.
With the help of the MPU, the kernel treats user mode threads as untrusted; that is, a flawed or malicious thread cannot leak or modify the private data of the kernel or other threads. When an untrusted thread then tries to read or write outside the allowed boundaries, for example with a stack overflow, a buffer overrun or a bad pointer, the MPU fires a MemManage fault immediately. This prevents trampling of the kernel state or generally any silent corruption outside of the explicitly granted memory for the thread.
The setup involves two concepts: memory partitions and memory domains. A partition is a region of RAM whose variables are tagged at compile time with K_APP_DMEM (for data section) or K_APP_BMEM (for bss section). A memory domain is a collection of partitions; threads are members of a domain and can access all partitions it contains. Both are configured in supervisor mode before the thread enters user mode.
In practice, it could look like:
// parser.c
/* parser's domain and partition, and place the buffer in the partition */
static struct k_mem_domain parser_domain;
K_APPMEM_PARTITION_DEFINE(parser_partition);
K_APP_BMEM(parser_partition) char parse_buffer[32];
/* Queues for exchanging data between kernel and parser */
K_QUEUE_DEFINE(shared_queue_incoming);
K_QUEUE_DEFINE(shared_queue_outgoing);
/* In the thread entry function, set up the domain, grant permissions, then drop into user mode */
void parser_entry(void *p1, void *p2, void *p3) {
/* Initialize memory domain for parser application */
k_mem_domain_init(&parser_domain, 0, NULL);
/* Add memory partition to parser_domain */
k_mem_domain_add_partition(&parser_domain, &parser_partition);
/* Add current thread to parser_domain */
k_mem_domain_add_thread(&parser_domain, k_current_get());
/* Grant access to queues */
k_thread_access_grant(k_current_get(), &shared_queue_incoming,
&shared_queue_outgoing);
/* Run in unprivileged mode from here on */
k_thread_user_mode_enter(parser_thread, NULL, NULL, NULL);
}
After k_thread_user_mode_enter(), the thread can only touch what it was granted. Everything else is MPU-protected.
The attentive reader might have noticed that the example above showed more than just memory partitions and domains. Memory partitions and domains cover access to RAM. The other two pieces to understand for user mode are kernel objects and system calls.
Kernel objects (queues, semaphores, heap handles) are not accessible by user mode threads by default; access must be explicitly granted per object with k_thread_access_grant(), as shown in the setup code above.
Once user mode is enabled, any call into the kernel API automatically goes through a system call gate, where parameters are validated before the kernel acts on them.
Together, these three mechanisms cover different attack surfaces. Memory domains and partitions control which RAM a thread can reach. Access control to kernel objects decides which kernel objects the thread can operate on. And system call validation ensures a thread can’t abuse those objects by passing bad data into the kernel (for example a bad pointer that could corrupt kernel memory indirectly, bypassing the MPU).
By default, k_sys_fatal_error_handler() dumps logs and halts the system. It’s a weak symbol, so you can override it. The key insight from the Zephyr documentation is that if the handler returns, Zephyr aborts the faulting thread and otherwise continues.
For a user mode thread, that’s a safe operation: the MPU guarantee means the thread could not have corrupted memory it wasn’t supposed to touch, and equally could not have exfiltrated kernel data. The isolation that makes it a security boundary is the same isolation that makes returning from the handler safe, although with the missing thread now the system might not be doing what is intended to do. The possible courses of action depends on the system: we could notify the user, send some remote alarm, or just re-schedule the thread.
For anything else (a supervisor mode fault, a usage fault, an unexpected exception) halting remains the honest answer. You cannot trust the system state.
void k_sys_fatal_error_handler(unsigned int reason,
const struct arch_esf *esf) {
if (reason >= K_ERR_ARM_MEM_GENERIC &&
reason <= K_ERR_ARM_MEM_FP_LAZY_STATE_PRESERVATION) {
LOG_ERR("Memory fault. Shutting down this thread only");
/* Returning here causes Zephyr to abort the faulting thread only */
} else {
LOG_ERR("Other fault. Unsafe to continue. Halting the system\n");
LOG_PANIC();
while (true) {
k_cpu_idle();
};
CODE_UNREACHABLE;
}
}
The reason check covers all six Cortex-M MEMFAULT exception codes, from K_ERR_ARM_MEM_GENERIC through K_ERR_ARM_MEM_FP_LAZY_STATE_PRESERVATION. Everything outside that range goes to the halt path. The exact range for the exception codes are taken from include/zephyr/arch/arm/arch.h.
This is the critical distinction from supervisor mode. Zephyr’s own documentation is clear on it: with a user mode memory tampering or stack overflow, there is no risk of data corruption to memory the thread would not otherwise be able to write to.
Once the setup works, it’s easy to get tripped up by subtleties that look completely unrelated to what’s actually wrong.
Use k_queue_alloc_append(), not k_queue_append(). The standard append is intrusive: it writes bookkeeping data directly into your payload buffer. From user mode, that’s an MPU fault even though the pointer and memory are perfectly valid. The alloc variant allocates the container separately from the thread’s resource pool. The above example is kept simple for the sake of this post, but to work with queues you will need to define a struct sys_heap for your allocations and crucially use k_thread_heap_assign() to grant access to it from the user mode thread. Classic trap when porting existing supervisor mode code.
Buffer overflows have a silent zone before the MPU fires. The MPU guard sits at the edge of the allowed memory region, not behind every variable. A small overflow silently corrupts locals and the saved return address before the guard is ever reached; the fault only fires once the overflow crosses into the protected region. If you’re seeing mysterious stack corruption, buffer_address – MMFAR tells you how far from the guard you were and how much silent damage happened first.
Missing sys_heap_init() looks exactly like a permissions bug. Allocating from an uninitialised sys_heap fails in a way that’s indistinguishable from a domain misconfiguration. You’ll waste time rechecking partition definitions and k_thread_access_grant() calls before realising the heap was never initialised. Make sure supervisor mode calls sys_heap_init() before the user mode thread starts.
The reason userspace is worth the setup cost is the guarantee it buys you. A user mode thread cannot corrupt memory it wasn’t given access to. That’s a security property first: it means a compromised parser or a buggy third-party network stack can’t take down the kernel or poison another thread’s data. It also means that when something goes wrong, you can abort the thread and take a recovery action (restart it, flag an error upward, alert the system) instead of losing everything. In supervisor mode, you don’t have that guarantee; the only honest response to a fault is a halt.
Zephyr Samples:
arm documentation about MPU:
Arm Developer portal: Documentation – Arm Developer
Defines for Cortex-M MEMFAULT exceptions: zephyr/include/zephyr/arch/arm/arch.h at v4.4.0 · zephyrproject-rtos/zephyr
Zephyr Documentation:
User Mode: User Mode — Zephyr Project Documentation
Fatal Errors: Fatal Errors — Zephyr Project Documentation
Author’s demo on GitHub:
The demo code referenced in this post is at github.com/lc-EmLogic/nrf-boom. For a cleaner starting point, the official Zephyr samples at samples/userspace/prod_consumer and samples/arch/mpu/mpu_test are a much better starting point.
Lluis is an embedded software engineer with over 10 years of experience developing and verifying software for embedded systems and the Internet of Things. His work spans firmware development, protocol verification, and system integration, covering both low-level embedded programming and higher-level tools and automation.
