Graphics drivers are one of the most complex pieces of software running on your computer, but most people don't really understand how they work. I want to give an overview of what a graphics driver actually does and how the different components fit together.
A graphics driver is software that lets your applications talk to your GPU. Every GPU is different—different capabilities, different instruction sets, different ways of doing things. So they need different drivers, or at least different code paths for different hardware generations.
The main job of a graphics driver is to let applications use your GPU efficiently. This means games can render frames, scientific applications can run compute workloads, and video software can encode/decode streams without hammering your CPU.
Graphics drivers are more complicated than most other hardware drivers because the functionality is incredibly broad and hardware varies a lot between vendors and generations. Here's how a typical graphics driver stack is organized:
You might wonder why we don't just put everything in the kernel. The answer is simple: you don't want your kernel to crash when a game does something stupid. You also don't want a full compiler running in kernel space, taking arbitrary user input and generating code. And you definitely don't want to reboot your system every time there's a driver update.
So the kernel driver (KMD) only handles the bare minimum that absolutely needs kernel privileges:
Everything else—API implementation, shader compilation, command buffer recording—happens in userspace where it's safe to crash without taking down your whole system.
Most modern GPUs have separate processors (not the shader cores) running firmware that handles low-level hardware operations. This firmware parses commands you submit, launches shaders, distributes work across shader cores, manages power states, and coordinates the display engine.
GPU vendors are moving more functionality into firmware over time. This reduces CPU overhead (fewer driver calls needed) but also makes the hardware less transparent—you can't see what the firmware is doing, you just trust it to do the right thing.
If you've ever seen "Compiling shaders..." on a loading screen and wondered why that's necessary: unlike CPUs which have mostly standardized on a few instruction sets (x86-64, ARM), every GPU has its own instruction set architecture (ISA). Even different generations from the same vendor have different ISAs.
This means shaders have to be compiled on your machine, at runtime, specifically for your GPU. The shader compiler takes high-level shader code (or bytecode like SPIR-V) and generates the actual machine instructions your GPU's shader cores will execute.
Shader compilers are complex—they need to do instruction scheduling, register allocation, optimization passes, all while respecting the specific constraints and capabilities of your GPU architecture. This is where a lot of the performance tuning happens.
On Windows, GPU vendors package everything together in one installer: firmware, kernel driver, userspace drivers for all APIs, shader compilers, a control panel application, and various utilities. Users download "the driver" and that's it.
On Linux, the graphics stack is more transparent but also more fragmented. The kernel driver is part of the kernel tree, firmware files are in
linux-firmware, userspace drivers are in Mesa, and shader compilers might be part of Mesa or separate projects like LLVM.
Different distros package these differently, which can lead to confusion.
This doesn't mean Linux drivers are worse—it just means they're organized differently because they're developed by a community rather than a single vendor's internal team.
Working on embedded systems gave me a different perspective on drivers. When you're doing board bring-up on something like the TI AM67A, you realize how many layers of abstraction we take for granted in desktop graphics.
In embedded, you write device tree files to describe hardware connections, configure pinmux for every GPIO, manually set up MMU page tables, and initialize DDR controllers with timing parameters from datasheets. There's no "it just works"—if the UART doesn't print anything, you might be looking at the wrong register offset or forgot to enable a clock tree somewhere.
Graphics drivers operate at a higher level, but the same principles apply: understand the hardware, read the docs (if they exist), and don't assume the abstraction layers are doing what you think they're doing.