redstrate/redstrate.com
1
Fork 0
Code Activity Actions

add first graphics dump

Browse source
This commit is contained in:
Joshua Goins 2022-11-15 16:21:04 -05:00
parent f3a695fa98
commit ca73c50729
3 changed files with 228 additions and 133 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

268
content/blog/linux-graphics-stack.md
View file

@ -1,41 +1,53 @@
--- ---
title: "Deep Dive: Linux Graphics Stack" title: "Graphics Dump: Mesa, Vulkan and DRM"
date: 2022-10-05 date: 2022-11-15
draft: true draft: false
tags: tags:
- Vulkan - Vulkan
- Linux - Linux
- Deep Dive - Graphics Dump
--- ---
The Linux graphics stack is a complex mechanism of many, many projects that function together to The Linux graphics stack is a complex mechanism of many projects that function in unison to deliver images to your screen. How do they work?
deliver images to your screen.
<!--more--> <!--more-->
Since other operating systems hide this away from you, we fortunately have Linux to look
at for how a desktop system might deliver graphics.
You probably noticed this already, but this is a very long article. The idea is to give you a true "vertical slice" of a _Graphics Dump_ is a new articles series I'm starting, geared towards tutorials and documentation for graphics-related things that are obscure but interesting. There are a thousand graphics tutorials covering how to start drawing in OpenGL, how to implement shadow mapping, and so on - but how many cover how stuff like how _Mesa_ works? I hope these fill someone's weird niche, like they do for me - so please enjoy!
typical Vulkan application. Other articles and videos may cover one or two subjects, but I go into excruciating detail
and want to give you the nitty gitty, especially for those who don't know what a "Mesa" is.
## Calling Vulkan {{<toc>}}
Let's begin by making a very simple Vulkan program, say - drawing a triangle. This is good because it covers a lot of ## Introduction
interesting stuff, picking a device to draw with, creating pipelines, render passes and issuing a draw call. I specifically
picked Vulkan for this article _because_ it makes swapchain management explicit which will be covered later.
So the first thing that should be noted is that under most circumstances, the first system in line when you call a Vulkan function Since other operating systems hide these processes away from you, it is fortunate that Linux exists to easily showcase how typical desktop graphics systems function! Of course other systems like Windows, macOS, and other operating systems function differently - but a lot of the same concepts apply (API call dispatch, userspace graphics drivers, and display planes as some good examples.)
is actually not related to Linux graphics at all. It is called the Vulkan Loader.
_Note:_ This is true in most cases, but in some systems (say, a Nintendo Switch) you might link directly to the driver's Vulkan library. However, the scope of this article has ballooned tremendously, so I had to split it into multiple parts in order to get _something_ finished. Right now we'll cover a "vertical slice" or overview of the typical call stack of a Vulkan application, and later articles will deep dive into how we can wrangle GBM to display graphics, how VR headsets fit into this ecosystem, and more interesting topics that require a dedicated article.
The Vulkan Loader has a few jobs, but the one that is relevant to us is that handles ICDs. An ICD is an "Installable Client Driver", This article is geared towards understanding the Linux graphics stack to graphics developers like me, who primarily work with graphics APIs and sort of is confused by Mesa, DRM and other such projects. As such, I kind of hand-waive over stuff like device creation, low-level GPU functions and other terms I expect you should know already or I'm not qualified to cover.
because Vulkan is not specific to one vendor - and one system may have devices from multiple vendors (say, an integrated Intel
chipset and a dedicated AMD or Nvidia GPU), the Vulkan loader must send the right function calls to the right ICDs. The loader ## Vulkan
also has a couple of more important jobs such as handling layers, but that is Vulkan-specific and will not be covered.
I don't intend to talk much about Vulkan itself here, as this is an article dedicated to the graphics stack as a whole, not just one API. However I want to clarify some misconceptions about Vulkan first.
* Vulkan is a graphics and compute API. Without extensions, it has no way to interact with window surfaces, displays, and perform presentation among other things you might expect.
* Vulkan is standardized by Khronos, which are not the ones who typically implement Vulkan (which would be your graphics driver developers.)
* Although Vulkan is spearheaded by Khronos, there are a lot of companies that back, develop for, and contribute back to the ecosystem such as LunarG, Collabora, Intel, AMD, NVIDIA and even Nintendo[^1].
With that out of the way, let's talk about how Vulkan applications typically interact with your system:
### Calling Vulkan
Let's say we have a very simple Vulkan program, all it does is open a window and draw a triangle. The reason why I chose Vulkan as the graphics API for this article is because it is so involved: you need to create a device, create shader pipelines, render passes and submit draw calls and perform explicit presentation sync. Vulkan's explicit swapchain management and synchronization makes it much easier to show what happens for the purpose of these articles. OpenGL and other API hide that information from you.
When we call something like `vkCreatePipeline` (a call that bundles some draw information with one or more shaders) the first library you will encounter is the _"Vulkan Loader"_, not any graphics driver. The Vulkan Loader is an independent project not related to Linux graphics at all.
### Vulkan Loader
The Vulkan Loader does a lot more than wrap Vulkan calls, but the one that is relevant to us is that handling ICDs. "ICD" stands for _"Installable Client Driver"_, this is the actual graphics driver.
The reason ICDs even exist as a concept is hinted in the name: Vulkan is not specific to one vendor and sometimes you may have ICDs from multiple vendors (say, an integrated Intel chipset and a dedicated AMD/Nvidia GPU). You cannot simply link or refer to one specific vendor's implementation of Vulkan, because Vulkan is inherently cross-platform, cross-vendor and a big feature is it's portability[^2].
On Linux, the Vulkan Loader is installed as `libvulkan.so`, but let's dig a little deeper and find out what what ICD we're using. ICDs are typically installed in `/usr/share/vulkan/icd.d`. Here is the `radeon_icd.x86_64.json` found on the Steam Deck[^3]:
The Vulkan loader is usually called `libvulkan.so`, but let's dig a little deeper and find out what the ICD says. You may
find your installed ICDs in `/usr/share/vulkan/icd.d`. For example, here is the `radeon_icd.x86_64.json` found on the Steam Deck:
```json ```json
{ {
"ICD": { "ICD": {
@ -46,51 +58,66 @@ find your installed ICDs in `/usr/share/vulkan/icd.d`. For example, here is the
} }
``` ```
_Note:_ It should be noted that 32-bit Vulkan drivers do exist, and the Vulkan loader handles switching to those as well. For example, ICDs are defined in plain JSON and we can find the driver library in `library_path`. We're focusing on AMD hardware in this article, but you can apply the same processes to Intel hardware and any other Mesa-supported driver.
the Steam Deck also ships with an `radeon_icd.i686.json` file as well.
As you can see, the ICD format is incredibly simple, but the library we're interested in now is `/usr/lib/libvulkan_radeon.so`. ### Inside of an ICD
If we run `objdump` on this, we can get a complete list of functions from this: Let's start by running `objdump` on our driver, so we can get a complete list of functions:
```shell ```shell
$ objdump -T --demangle /usr/lib/libvulkan_radeon.so $ objdump -T --demangle /usr/lib/libvulkan_radeon.so
... ...
``` ```
As you can see, there is a **lot** of functions here - many more than I expected personally. If you notice, there is an interesting If you run this on your system (and of course substitute the library if you're not on an AMD system), you'll see there is a **lot** of functions. If you're keen enough, there is a distinct pattern in function signatures:
pattern with the function signatures, they follow this:
* `wsi` - window screen interface * `wsi` - window screen interface related functions.
* `radv` - radeon, device-specific functions (for Vulkan device extensions) * `radv` - device-specific functions.
* `vk_common` - for common vulkan functions that are not device specific, like `vkFlushMemoryRanges` * `vk_common` - common vulkan functions that are not device specific, like `vkFlushMemoryRanges`. We'll get to why these are included here at all later.
* Some game-specific names, such as `metro_exodus`. We'll come back to those. * Weirdly enough, video game names such as `metro_exodus`. We'll come back to why those are even a thing in a future article.
Seeing this makes you think, "wait a second, what are some instance-level functions doing there?" Let's explain: Seeing this might make you think, _"wait a second, what are some instance-level functions doing there?"_ If you aren't familiar with Vulkan, it's functions have two distinct levels to them:
1. You call a Vulkan function, say - `vkCreateInstance`. * Instance-level functions: `vkCreateInstance`, `vkEnumeratePhysicalDevices` and so on. These do not require a device to be used.
2. The Vulkan loader is actually what gets called, and it sends your call down the chain (through any layers, if needed) and eventually to the ICD. * Device-level functions: `vkCreateGraphicsPipelines`, `vkBeginCommands`. These do need a logical device to operate.
3. The ICD then does what it needs to do (See: https://github.com/Mesa3D/mesa/blob/main/src/vulkan/runtime/vk_instance.c#L44)
4. Whatever that is returned is passed up
But, I chose `vkCreateInstance` for a very good reason. This happens _way_ before you even get to creating devices, so how However, if you haven't noticed already - there is functionally no difference between these two levels: they are technically part of the same driver, **so how does the Vulkan loader know which driver to use**?
does it choose which driver to go with if it could be any of them? Well this is what is called the "loader terminator", and it
specifically deals with this issue. TODO WRITE
Now let's move out of Vulkan and into Mesa, arguably the most important piece of the Linux graphics puzzle. The answer is simple, it just calls all of them! When you call a Vulkan function through the Vulkan Loader system, if it's a device-level call it simply passes it down to the driver that the device belongs to - but if it's an instance-level function, it actually "aggregates" the data from the relevant drivers. This is what is called the "Loader Terminator".
### How the Loader Terminator Works
A good example to showcase this is `vkEnumeratePhysicalDevices`, and as the name implies, gives you a list of the physical devices on the machine. If you have a Intel integrated GPU and say, a NVIDIA GPU, this is what the loader terminator might do:
* Control is eventually handed down to the Vulkan Loader, from your call to `vkEnumeratePhysicalDevices`.
* The Loader realizes this is an instance-level function, and one that needs aggregate data from all devices.
* The Loader then calls your ICDs, which might be `libnvidia_vulkan.so` and `libintel_vulkan.so` (these aren't actual library names, I just made them up!)
* Once control is handed back to the Loader, it then combines the physical device lists from both devices and hands the data back to your application.
This is actually a really smart design, since it would be near-useless if `vkEnumeratePhysicalDevices` only listed devices from _one_ ICD. And then once you create the logical device from that ICD's physical device, the Loader takes care of figuring out which ICD you actually want to talk to without you even realizing it. The fact that I didn't even know what a "loader terminator" was or that it even did this in the first place is a testament to how nicely this system functions.
### Overview
That was a _lot_ of information to parse, so let me give a quick overview of how Vulkan calls eventually each your graphics driver:
1. You call a device-specific function like `vkQueueSubmit`.
2. The Vulkan Loader picks up your call, determines the correct ICD from th device you passed to the function.
3. The graphics driver then does its job. (For example, here's [the implementation of vkCreateInstance](https://gitlab.freedesktop.org/mesa/mesa/-/tree/main/src/vulkan/runtime/vk_instance.c#L44). We'll get into why this is separated from the driver later.)
4. The function result is passed back up, through the call stack, to your application.
This is of course assuming there are no layers activated (which are simply other programs that can intercept Vulkan calls) which may change the route your call makes. Now let's take a look at what your graphics driver actually is, and it's more complex than you might think.
## Mesa ## Mesa
Mesa is the piece of software that is probably doing most of the heavy lifting here, and is an important piece of the Mesa is the next piece of software in this call stack (the ICD). It does most of the heavy lifting here, and is an important piece of the Linux graphics stack. However, I've seen people confused on _what_ Mesa is - so let's take a look at their [website](https://mesa3d.org):
Linux graphics stack. Taken from their website:
``` ```
Open source implementations of OpenGL, OpenGL ES, Vulkan, OpenCL, and more! Open source implementations of OpenGL, OpenGL ES, Vulkan, OpenCL, and more!
``` ```
Oh... alright - well that doesn't really explain much. Let's look at their documentation which explains it better: Oh... alright - well that doesn't really explain much. Let's look at [their documentation](https://docs.mesa3d.org) which explains it better:
```shell ```
The Mesa project began as an open-source implementation of the OpenGL specification - a system for rendering interactive 3D graphics. The Mesa project began as an open-source implementation of the OpenGL specification - a system for rendering interactive 3D graphics.
Over the years the project has grown to implement more graphics APIs, including OpenGL ES, OpenCL, OpenMAX, VDPAU, VA-API, XvMC, Vulkan and EGL. Over the years the project has grown to implement more graphics APIs, including OpenGL ES, OpenCL, OpenMAX, VDPAU, VA-API, XvMC, Vulkan and EGL.
@ -100,130 +127,105 @@ A variety of device drivers allows the Mesa libraries to be used in many differe
Mesa ties into several other open-source projects: the Direct Rendering Infrastructure, X.org, and Wayland to provide OpenGL support on Linux, FreeBSD, and other operating systems. Mesa ties into several other open-source projects: the Direct Rendering Infrastructure, X.org, and Wayland to provide OpenGL support on Linux, FreeBSD, and other operating systems.
``` ```
What's important to note that Mesa runs in _userspace_. As it says in the last line, it interfaces with kernel APIs such as It's important to note that Mesa runs in _userspace_. As it says in that last sentence, it interfaces with kernel APIs such as
DRM (not to be confused with _digital rights management_, and don't worry the acronyms get more confusing). DRM (not to be confused with _digital rights management_, and don't worry the acronyms get more confusing) and a whole lot of other stuff. These other projects will also be covered in this article and in other future ones.
Modern graphics drivers (and this applies to most desktop operating systems such as Windows and macOS) are split into two parts: Linux graphics drivers are split into two parts:
the kernel-space driver that interfaces directly with the hardware, and there is typically one per vendor, and then the userspace driver the kernel-space module that interfaces directly with the hardware, and the userspace driver
that sits on top of the aforementioned layer and interacts with applications. The AMDGPU kernel driver is what we'll be covering today, that sits on top of that kernel layer and interacts with applications. Mesa implements that user space layer for us, for other vendors like NVIDIA and AMD's own proprietary solution have their own userspace layer[^4].
and the Mesa stack is our userspace driver. Specifically, we care about RADV.
Let's take a look at how Mesa is structured, and how they can reuse code between many devices and drivers. This will also explain the curious structure of the `objdump` from earlier, and what the `radv` prefix is referring to.
### Mesa Drivers ### Mesa Drivers
Mesa comes with many drivers, which confusingly can be mixed and matched because they handle supporting different APIs. Mesa comes with many drivers, and each driver has a name (`radv`, `iris`, etc) which is unique even across different APIs:
* RADV is their AMD driver _for_ Vulkan. It has nothing to do with OpenGL support for this hardware (unless you use Zink ;-)) * RADV: the modern AMD driver _for_ Vulkan. It has nothing to do with OpenGL support.
* ANV is their Intel driver _for_ Vulkan. As usual, it has nothing to do with OpenGL support for Intel chipsets. * ANV: the modern Intel driver _for_ Vulkan.
* radeonsi: Modern AMD driver but for OpenGL!
* iris - Modern Intel OpenGL support
* And so on... * And so on...
Mesa not interfaces with the kernel DRI, which in turn calls a bunch of AMD-specific stuff However Mesa has a _lot_ of drivers, and you might think it become unmaintainable - I mean supporting one graphics API is trouble enough, but Mesa somehow supports all of these drivers and all of these APIs? How do they accomplish such a feat?
(in the case of RADV) in order to accomplish its goals. Again, this happens in _userspace_ as the kernel
driver only exists to facilitate I/O, load firmware and other really low level things.
So _what_ is DRI? Well, it's actually comprised of DRM and DRM KMS. So wait, how does this fit into OpenGL again? Well there's
another term, "Gallium".
### Gallium ### Gallium
This is so incredibly fascinating because for the longest time - whenever I think of "Gallium", I think Let's take a look at how their OpenGL support works first, which is easier to explain. This is so incredibly fascinating because for the longest time - whenever I think of "Gallium", and other people
of "Gallium Nine" which is the way to run DX9 almost "natively" on your system via Mesa. But this has nothing to do with probably relate - I think of [Gallium Nine](https://docs.mesa3d.org/gallium-nine.html) which is the way to run DX9 almost natively on your system via Mesa. However, Gallium Nine is simply another frontend to Gallium.
that really, and is just another frontend to this system.
It's just a framework to reduce the amount of driver code needed to write OpenGL-compliant drivers. Unlike the Gallium is a framework to reduce the amount of driver code needed to write OpenGL-compliant drivers. These drivers are located under `src/gallium/drivers` in [the Mesa source code](https://gitlab.freedesktop.org/mesa/mesa/-/tree/main/src/gallium/drivers). OpenGL has a lot of state work and other boring stuff that typically doesn't vary from driver to driver, so this is lifted into the Gallium State Tracker and then the device drivers can focus more on what they're there to do, which is interface with the hardware. So when bringing up a new system that needs OpenGL support, you just need to write a Gallium driver which takes way, way less work than writing a new OpenGL driver from scratch.
Vulkan device drivers, these drivers are located under `src/gallium/drivers`.
* radeonsi - RADV equivalent for OpenGL However you might be wondering how Mesa structures it's installed OpenGL drivers. Mesa provides `libGL.so` which plays a similar role to `libvulkan.so`, and selects the correct OpenGL driver at runtime[^5].
* iris - Intel OpenGL support
* etc
### Overview ### Vulkan
So after going through all of that, I think we should take a step back and see what we have learned here. Now unfortunately Mesa doesn't have a fancy name for its Vulkan driver framework (as far as I know!) because this is mostly a new development. Previously, a lot of Mesa's Vulkan drivers implemented a whole of duplicate Vulkan work (like in instance-level functions, but also a lot of queue-related things) which wasn't a huge deal, but now that Mesa has way more Vulkan drivers (they have RADV, ANV, and soon NVK - and that's not all of them!) the work was piling up and implementations started to drift apart. Apparently a lot of RADV and subsequent Vulkan drivers were based off of ANV, and then common Vulkan code got lifted outside of the drivers later. As Jason Ekstrand says in ["Introducing NVK"](https://www.collabora.com/news-and-blog/news-and-events/introducing-nvk.html):
1. You call `vkCreateInstance` > One of my personal goals for NVK is for it to become the new reference Vulkan driver within Mesa. All of the Vulkan drivers in Mesa can trace their lineage back to the Intel Vulkan driver (ANV) and were started by copying+pasting from it. We won't be there for a while, but my hope is that NVK will eventually become the driver that everyone copies and pastes from. To that end, I'm building NVK with all the best practices we've developed for Vulkan drivers over the last 7.5 years and trying to keep the code-base clean and well-organized.
2. The Vulkan loader terminator eventually calls `vk_common_create_instance`, which exists in the Radeon ICD,
or the `/usr/lib/libvulkan_radeon.so` shared object. This object contains instance-level functions as well as device-specific
functions.
3. If the function is device-specific, the Mesa Vulkan driver knows how to interface with the driver
using the kernel APIs (DRI, which includes DRM KMS for modesetting and DRM for device interfacing).
4. The I/O leaves userspace (Mesa) and heads into the kernel (AMDGPU).
5. ...
6. The I/O returns to Mesa, and goes back up to the Vulkan Loader.
7. If needed (for example, enumerating all physical devices) the Vulkan loader terminator will combine the sources from multiple ICDs, otherwise
just leaves it alone.
Wow, that is a lot! And we learned with our brief OpenGL tangent that the process is very similar with Gallium. Now As we've seen though, the RADV ICD simply contains _both_ the common Vulkan code and then the RADV specific stuff (prefixed with `radv`) and it's all inside of just one object library. The library is then aptly named `libvulkan_radeon.so`.
of course this is nice and all, but what about WSI, swapchains and how those interact with Wayland?
Before we get into that, I would like to revisit the DRM subsystem in the kernel before we head back up to the userspace Now that we covered how Mesa separates and abstracts drivers for Vulkan and OpenGL, how do the device drivers actually function? Well depending on the structure of the hardware and its unique quirks, a lot of care has to go into optimizing the API the driver is handling. But how does it actually _interact_ with the hardware?
portion.
## DRM ## DRI
To remind us, DRM stands for the "Device Rendering Manager" and handles device-specific graphics guff. For us, the part The answer is not so simple. You may have heard terms like DRI, DRM, KMS, DRM KMS and other things but these aren't very well explained. Mesa uses something called DRI.
that's important is that it exposes device-specific APIs to control them from userspace. To find it in the kernel tree, see
`drivers/gpu/drm`.
Since we are just worrying about AMD gpus, you can see the AMD-specific bits in `drivers/gpu/drm/amd/amdgpu`. Inside DRI stands for _"Direct Rendering Infrastructure"_ and is something specific to Linux. The DRI is an umbrella term also covering _DRM_ and _DRM KMS_.
you'll see a metric ton of source and header files related to the amdgpu drm interface. Sweet!
Now I'm curious about how does Mesa then interface with these device-specific stuff? Does it just include the relevant kernel header? Let's find out... ### DRM
If we take a look at the Mesa RADV `meson.build` we get our answer: DRM stands for _"Direct Rendering Manager"_ and refers to the DRM system that exists in the kernel. This is what the AMDGPU kernel module implements, and consequently how Mesa is able to interface with the GPU at all. It basically creates an API for userspace applications to access your GPU. The kernel module handles facilitating I/O, loading firmware and other low level things. To find the in-tree DRM kernel modules, see `drivers/gpu/drm`.
Since we are just covering modern AMD GPUs, the AMDGPU module is located under `drivers/gpu/drm/amd/amdgpu`. Inside you'll see a metric ton of source and header files related to the amdgpu drm interface. Sweet!
Now you might be curious how does Mesa then interface with these device-specific stuff? Does it just include the relevant kernel header? But don't those break often? It would also be weird for Mesa to break between kernel versions too[^6].
If we take a look at [the meson.build for vulkan_radeon](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/vulkan/meson.build#L177) we get our answer:
`dependencies : [ ..., dep_libdrm_amdgpu, ... ]` `dependencies : [ ..., dep_libdrm_amdgpu, ... ]`
Jackpot! This `dep_libdrm_amdgpu` bit is specifically referring to the libdrm project under the Mesa umbrella (https://gitlab.freedesktop.org/mesa/drm) Jackpot! What is this `libdrm` it mentions? It actually is specifically referring to the [libdrm project](https://gitlab.freedesktop.org/mesa/drm).
To get more confusing, this is _not_ DRM (the Linux subsystem) but rather a library sitting on top of it. Yes, really. To get even more
confusing, freedesktop.org also calls this library DRI for some reason. I told you the acronyms were going to get awful.
So, the Mesa RADV driver doesn't interface with the kernel directly, but rather access all the AMD-related So, the Mesa drivers doesn't interface with the kernel directly, but rather access all the device-specific functionality _through_ libdrm. This is because kernel interfaces are not truly stable and the interface isn't really meant for application consumption anyway, libdrm handles all of that for Mesa. This is also might be why Mesa doesn't define a specific kernel requirement, because it technically doesn't depend on their DRM interface.
gubbins _through_ libdrm.
I won't be covering how the AMDGPU kernel driver actually works, partially because I don't know but the main idea is that ### DRM KMS
it basically operates on top of some proprietary firmware that's loaded on startup for the GPU. The DRM kernel driver is basically
responsible for creating a suitable interface, initializing firmware, memory and doing I/O between that and the userspace.
## Windowing The other major part of the DRI project is "DRM KMS", one part we now know is "Direct Rendering Manager" but what is "KMS"? It stands for "Kernel Mode Setting" and is used on modern Linux systems to initialize and configure the framebuffer. You might think that "DRM KMS" stands for a type of "KMS" but really it can't exist _without_ using DRM. You'll also sometimes see it written as "DRM/KMS" which is weird but I guess it's typical in the Linux world (like GNU/Linux.)
This is a huge topic just by itself, but it's incredibly interesting as everything we said before is assuming that you're Modesetting refers to initializing the display, handling EDIDs and all of that fun stuff. "KMS" is then referring to doing this in the kernel instead of in user space. You might be thinking that meant that user space was initializing the GPU's connected screen and you'll be right, which was slow especially when switching to things like the TTY (which is handled by the kernel) since it had no idea what the user space was doing.
either running your Vulkan program without any graphics output (_specifically_ presentation, as you can totally run the graphics
part of Vulkan headless).
For _brevity’s_ sake, and because there's already a lot of X11 information out there - I want to cover what specifically ### Quick Note about DRI versions
happens on **Wayland**. This is especially troubling because there is a lot of misinformation on the web, especially around Wayland.
### On the Vulkan Side You may have noticed this already, but there are things called DRI1, DRI2, DRI3. There are actually three different versions of DRI, which has evolved as the needs of graphics drivers and desktop compositors changed. We're currently on DRI3 and that's what I'll be covering once we get into presentation and synchronization.
Let's momentarily mention what exactly you need to do on Vulkan to get presentation working. We'll keep this short, but ## Conclusion
it's important to get our bearings straight.
This may be a surprise to some, but Vulkan has nothing to do with presentation. This is pretty on par with Khronos APIs I hope this gives you a nice overview of what libraries are involved when dealing with Linux graphics, and an explanation of some of the terms you might have seen before. As stated in the beginning, this has grew and grew in scope so I'm cutting the article here so I can get it published first.
actually, as OpenGL also did not concern itself with presentation (GLX, EGL, and other similar stuff is _not_ related).
In Vulkan, to get presentation you must enable a device extension, specifically `VK_KHR_swapchain`. This is not the only In the next part we'll be covering how windowing, synchronization and presentation works under KWin, a popular but also typical Wayland compositor. We'll also cover how GBM, DRI3 and KMS fit into the picture and why they were created in the first place. This will also include an example of wrangling GBM and Vulkan to display something to the screen and why `VK_KHR_display` isn't being used more.
piece of the puzzle, as you also need a surface to render to. There is a lot of options, but we are only concerning ourselves with two:
* `VK_KHR_surface` - this is the base surface extension I'm not a Mesa nor an AMD developer, so if you see any mistakes feel free to contact me! Otherwise please enjoy and make sure to check out these other fine webpages below.
* `VK_KHR_wayland_surface` - needed to interface with the wayland client
* `VK_KHR_directfb_surface` - we will get into this later, as it provides a way to display vulkan directly to the framebuffer.
### Wayland ## See Also
We'll be exclusively talking about how Vulkan applications under Wayland function, specifically under two scenarios. * [Mesa Documentation](https://docs.mesa3d.org/index.html)
One will be through a typical desktop environment - in this case - KDE Plasma as well as a more barebones example, bare KMS. * [Mesa Source Code](https://gitlab.freedesktop.org/mesa/mesa)
For both cases I will be using SDL2 instead of interacting with the Wayland layer itself, * ["Introducing NVK" on the Collabora blog](https://www.collabora.com/news-and-blog/news-and-events/introducing-nvk.html) - goes over some history of the Mesa Vulkan drivers and how NVK will eventually fit in.
which will be the case for many games. This does not change much though, because unlike OpenGL - SDL2 does not really interact * ["Architecture of the Vulkan Loader Interfaces"](https://github.com/KhronosGroup/Vulkan-Loader/blob/master/docs/LoaderInterfaceArchitecture.md) - Detailed explanation of how the Vulkan Loader functions, a good read if you're still interested in how Vulkan functions outside the scope of this series.
or intercept Vulkan functionality, apart from creating a Vulkan surface. * [DRI on Wikipedia](https://en.wikipedia.org/wiki/Direct_Rendering_Infrastructure) - Really good overview of the DRI system on Wikipedia, actually all of the Linux articles are surprisingly good quality.
* ["The DRM/KMS subsystem from a newbie’s point of view"](https://events.static.linuxfound.org/sites/events/files/slides/brezillon-drm-kms.pdf) - Good overview of how DRM KMS functions and how compares to fbdev, but is geared towards how to use DRM as an API.
* [The Linux Graphics Stack series by Iago Toral](https://blogs.igalia.com/itoral/2014/07/29/a-brief-introduction-to-the-linux-graphics-stack/) - Really good series on the Linux graphics stack by an actual Mesa developer!
I chose these two situations because one is your more typical desktop environment, where the compositor has to juggle many [^1]: See https://www.khronos.org/conformance/adopters/conformant-companies#vulkan for a list of companies that have implemented Vulkan. This is _not_ a list of companies that have pitched some sort of contribution back to Vulkan, but is also a good indicator that they employ individuals or companies that have or they also published Vulkan extensions on their behalf.
windows vying for presentation. The bare KMS example is more relevant to something like a game system, and we can see if
anything is different here in terms how the swapchain and presentation is handled.
### GBM [^2]: While this is true on the desktop (where drivers are installable), Vulkan is equally suited for embedded purposes and game consoles. It's possible to link directly to your driver's Vulkan library if needed. When developing for the Nintendo Switch, the Vulkan Loader is an unnecessary maintenance burden where portability between drivers is a non-issue.
GBM stands for _"generic buffer management"_ and is also part of the Mesa project. It's mostly used to initialize EGL on DRM KMS. [^3]: While we're taking a look at a 64-bit Vulkan driver, 32-bit Vulkan drivers do exist (and needed for those 32-bit games that somehow still get made), and the Vulkan Loader handles switching to those as well. The Steam Deck ships with an `radeon_icd.i686.json`.
[^4]: I've seen a lot of confusion online as to the relation between Mesa, AMDGPU-Pro and NVIDIA. To be clear, Mesa _only_ implements the userspace drivers for OpenGL, Vulkan, OpenCL etc and AMDGPU-Pro, NVIDIA's proprietary solutions completely sidestep that. Curiously, AMDGPU-Pro reuses the existing AMDGPU kernel module used by Mesa's AMD drivers unlike NVIDIA which provides their own kernel modules (which were recently open-sourced). On a typical NVIDIA Linux system, Mesa is typically not touched and everything is routed through NVIDIA's software stack.
[^5]: See Iago Toral's ["Driver loading and querying in Mesa"](https://blogs.igalia.com/itoral/2014/09/04/driver-loading-and-querying-in-mesa/) for a good overview of how Mesa accomplishes this.
[^6]: Some versions of Mesa actually _do_ require specific kernel versions, however as far as I know this is an undocumented requirement.
You may have heard how Nvidia fought against Mesa with it's GBM, with EGLStreams before eventually conceding and
implementing GBM - this is why. Since Wayland compositors need to initialize graphics through DRM KMS, it eventually needs to allocate
memory. It does this via calling the Mesa GBM, which then in turn calls the GPU specific stuff.

83
content/blog/linux-windowing.md Normal file
View file

@ -0,0 +1,83 @@
Windowing
This is a huge topic just by itself, but it's incredibly interesting as everything we said before is assuming that you're
either running your Vulkan program without any graphics output (_specifically_ presentation, as you can totally run the graphics
part of Vulkan headless).
For _brevity’s_ sake, and because there's already a lot of X11 information out there - I want to cover what specifically
happens on **Wayland**. This is especially troubling because there is a lot of misinformation on the web, especially around Wayland.
### On the Vulkan Side
Let's momentarily mention what exactly you need to do on Vulkan to get presentation working. We'll keep this short, but
it's important to get our bearings straight.
This may be a surprise to some, but Vulkan has nothing to do with presentation. This is pretty on par with Khronos APIs
actually, as OpenGL also did not concern itself with presentation (GLX, EGL, and other similar stuff is _not_ related).
In Vulkan, to get presentation you must enable a device extension, specifically `VK_KHR_swapchain`. This is not the only
piece of the puzzle, as you also need a surface to render to. There is a lot of options, but we are only concerning ourselves with two:
* `VK_KHR_surface` - this is the base surface extension
* `VK_KHR_wayland_surface` - needed to interface with the wayland client
* `VK_KHR_directfb_surface` - we will get into this later, as it provides a way to display vulkan directly to the framebuffer.
### Wayland
We'll be exclusively talking about how Vulkan applications under Wayland function, specifically under two scenarios.
One will be through a typical desktop environment - in this case - KDE Plasma as well as a more barebones example, bare KMS.
For both cases I will be using SDL2 instead of interacting with the Wayland layer itself,
which will be the case for many games. This does not change much though, because unlike OpenGL - SDL2 does not really interact
or intercept Vulkan functionality, apart from creating a Vulkan surface.
I chose these two situations because one is your more typical desktop environment, where the compositor has to juggle many
windows vying for presentation. The bare KMS example is more relevant to something like a game system, and we can see if
anything is different here in terms how the swapchain and presentation is handled.
### How swap chains work (quickly)
So how do swap chains work (quickly?) For simplicity, we're talking about a typical MAILBOX swap chain.
Next image is requested by the application.
If the image is still valid, its handle is returned to the application. This is also known as the _backbuffer_.
While the application can hold onto this image, they can draw anything they wish into it.
Once the application has finished rendering, it "presents" (note: the application is usually not directly presenting it to the screen)
### How do swap chains work (in real life)
Okay this sounds all well and good, but how does this work in _real life_? On a typical desktop environment, we have
many different windows, some of which misbehave or malfunction and don't return and present images in time with our
display. Some of them even present _faster_ than the display, so who is managing this?
Now in this case, I'll be covering how Wayland compositors handle this, specifically KWin. While the same might apply to X11 environments,
Wayland is the future and it's way, way simpler to explain.
First we have to remember that KWin itself, _is_ a graphics application and it also follows the same rules as any other. Right now, KWin
is built on OpenGL technologies but uses GBM to present stuff to the screen. Now KWin in this example, is the Wayland _server_ and other
applications are Wayland _clients_. This includes XWayland, which is simply wrapping the Xorg server as a Wayland client.
1. Your Vulkan application wants a Wayland surface, and is running on KWin Wayland.
2. It uses
### What is GBM??
GBM stands for _"generic buffer management"_ and is also part of the Mesa project. It's mostly used to initialize EGL on DRM KMS. This is
Also meaningless to most people, including me. When researching _what_ GBM is, it's infuriating that there's really no simple explanation
to what it enables.
Okay, let's go with an example. You want to implement a desktop compositor, where do you begin?
Here's some simple requirements:
1. You want to make sure you're booted into DRM KMS.
So let's display something to the screen, such as
You may have heard how Nvidia fought against Mesa with it's GBM, with EGLStreams before eventually conceding and
implementing GBM - this is why. Since Wayland compositors need to initialize graphics through DRM KMS, it eventually needs to allocate
memory. It does this via calling the Mesa GBM, which then in turn calls the GPU specific stuff.
## Conclusion
I hope this article clears up any misconceptions about the Linux graphics stack, and I definitely learned a lot while researching
This. While this was a nice overview of the entire stack, it would be really helpful if we could apply in this in a practical fashion,
so that's what I'm going to cover in the next article!

10
themes/red/assets/css/site.css
View file

@ -220,3 +220,13 @@ hr {
height: 1px; height: 1px;
border: 0; border: 0;
} }
code {
white-space: pre-wrap;
}
pre {
background-color: var(--background-primary);
padding: 5px;
}