diff options
Diffstat (limited to 'docs/briefs/tb0006-compile-time-driver-self-registration.rst')
| -rw-r--r-- | docs/briefs/tb0006-compile-time-driver-self-registration.rst | 326 |
1 files changed, 326 insertions, 0 deletions
diff --git a/docs/briefs/tb0006-compile-time-driver-self-registration.rst b/docs/briefs/tb0006-compile-time-driver-self-registration.rst new file mode 100644 index 00000000..9c6293a0 --- /dev/null +++ b/docs/briefs/tb0006-compile-time-driver-self-registration.rst @@ -0,0 +1,326 @@ +Technical Brief 0006: Compile-Time Driver Self-Registration +=========================================================== + +:doc:`/briefs/tb0004-device-tree-ownership-and-registries` covers ``driver_registry`` as one of the three registries built into the device model, and notes in passing that it has no lookup API — only a flat list, tried in full against every device. +This brief covers a question that document deliberately leaves open: **how does anything get into that list in the first place, given that no code anywhere calls a function named "register all drivers"?** + +The answer is a compile-time mechanism shared, in slightly different clothing, by two otherwise-unrelated parts of the kernel: device drivers (``kapi::devices::kernel_driver``/``platform_driver``) and filesystem drivers (``kernel::vfs::driver_module``). +A driver's own ``.cpp`` file defined a small object and the build's linker collects a pointer to it into a dedicated, named section of the final kernel image. +Afterwards, an explicit walk at boot turns that section into real, running driver instances. +No central list exists anywhere in the source tree, and no constructor runs before ``main()`` to build one at runtime. + +The Problem This Replaces +------------------------- + +An obvious implementation for the central device driver list, is a single, hand-maintained function, somewhere in ``kernel/kernel/drivers/init.cpp``. +Such a function might look like the example below: + +.. code-block:: cpp + + auto init() -> void + { + driver_registry::get().add(kstd::make_shared<pit_driver>()); + driver_registry::get().add(kstd::make_shared<null_driver>()); + driver_registry::get().add(kstd::make_shared<zero_driver>()); + // ... one line per driver, forever + } + +This has two real costs, aside of aesthetics. +First, it is a synchronization problem disguised as a list: every new driver requires a matching edit to a file its author may not think to touch. +The failure mode of forgetting such an edit is a driver that compiles and links fine, but never gets bound to any device. +This failure is silent and therefore difficult to debug, since the cause of the issue lies in a physically distinct file from the driver itself. +Second, it is a dependency-direction problem: ``kernel/kernel/drivers/init.cpp`` would need to ``#include`` the header of every driver in the tree. +This means that a change to any single driver's public interface touches a file that has no other reason to know that driver exists. +Both problems get worse, not better, as the driver count grows. +This stands in exact opposition to a mechanism whose whole job is to make adding a driver a local, self-contained change. + +The Core Mechanism +------------------ + +A driver announces itself with a small, private ``driver_descriptor`` type derived from ``kapi::devices::driver_descriptor``. +This descriptor is instantiated directly within the main driver implementation file an automatically placed in a custom linker section. +A complete, real example is the pseudo bus's ``null`` driver (``kernel/kernel/drivers/pseudo/null.cpp``): + +.. code-block:: cpp + + namespace + { + struct descriptor final : kapi::devices::driver_descriptor + { + [[nodiscard]] auto name() const noexcept -> std::string_view override { return "dev_null"; } + [[nodiscard]] auto make_instance() const -> kstd::shared_ptr<kapi::devices::driver> override + { + return kstd::make_shared<null>(); + } + }; + + [[gnu::used]] + constexpr auto registration = kapi::devices::kernel_driver<descriptor>{}; + } // namespace + +``driver_descriptor`` (``kapi/kapi/devices/driver_registry.hpp``) exposes the minimal interface system this needs: a diagnostic name and a factory function that produces the real driver instance on demand. + +.. code-block:: cpp + + struct driver_descriptor + { + virtual ~driver_descriptor() = default; + [[nodiscard]] virtual auto name() const noexcept -> std::string_view = 0; + [[nodiscard]] virtual auto make_instance() const -> kstd::shared_ptr<driver> = 0; + }; + +The registration mechanism itself relies on instances of the type ``kernel_driver<Type>``. +This class template has no member functions at all, and only two ``static`` data members: + +.. code-block:: cpp + + template<typename Type> + struct kernel_driver + { + constexpr auto static instance = Type{}; + [[using gnu: section("kernel_drivers"), used, visibility("hidden")]] constexpr auto static pointer{ + kstd::make_observer<kapi::devices::driver_descriptor const>(&instance), + }; + }; + +An identical class template with a different name (``platform_driver``) exists for architecture specific drivers. + +The ``instance`` member is a instance of the driver's descriptor type that is constructed at compile-time. +They ``pointer`` member points to that instance and is placed into a named section during compilation and linking. +Those named sections (one for ``kernel_drivers`` and one for ``platform_drivers``) contain only a contiguous array of these pointers, one per driver. +Each pointer is contributed independently by every ``.cpp`` file that defines an instance of ``kernel_driver`` or ``platform_driver``. + +The linker script (``arch/x86_64/scripts/kernel.ld``) reserves that region explicitly and exposes its bounds as ordinary symbols: + +.. code-block:: text + + .kernel_rodata ALIGN(4K) : AT (ADDR (.kernel_rodata) - TEACHOS_VMA) + { + /* other read-only data sections go here ... */ + + /* Kernel driver factories */ + PROVIDE(__start_kernel_drivers = .); + KEEP(*(kernel_drivers)); + PROVIDE(__stop_kernel_drivers = .); + + /* Platform driver factories */ + PROVIDE(__start_platform_drivers = .); + KEEP(*(platform_drivers)); + PROVIDE(__stop_platform_drivers = .); + } :kernel_rodata + +Every ``kernel_driver<SomeDescriptor>::pointer`` from every translation unit in the whole kernel image ends up contiguous in memory and is bracketed by ``__start_kernel_drivers`` and ``__stop_kernel_drivers`` — regardless of which ``.cpp`` file defined it, and without that file needing to be named anywhere else. +The ``KEEP()`` linker instruction ensures the linker does not evict these pointers, despite them not being referenced anywhere. + +C++ code recovers those bounds as ordinary ``extern`` declarations and walks the range as a ``std::span`` (``kernel/kernel/drivers/init.cpp``): + +.. code-block:: cpp + + extern "C" + { + extern kstd::observer_ptr<kapi::devices::driver_descriptor> const __start_kernel_drivers; + extern kstd::observer_ptr<kapi::devices::driver_descriptor> const __stop_kernel_drivers; + } + + auto init() -> void + { + kapi::devices::driver_registry::init(); + auto & registry = kapi::devices::driver_registry::get(); + + auto descriptors = std::span{&__start_kernel_drivers, &__stop_kernel_drivers} | + std::views::filter([](auto p) { return p != nullptr; }); + + for (auto driver : descriptors) + { + auto instance = driver->make_instance(); + kstd::println("[OS:DRV] registering driver '{}' ({})", instance->name(), driver->name()); + registry.add(std::move(instance)); + } + } + +``platform_drivers`` is processed the same way, by ``kapi::devices::init_platform_drivers()``. + +Why This Needs No Constructor to Run +------------------------------------ + +The detail that makes this mechanism fit a freestanding, no-exceptions kernel, is that none of the code above executes before ``main()``. +``kernel_driver<Type>::instance`` and ``::pointer`` are both ``constexpr``, requiring the compiler to fully construct their values at compile time. +This includes resolution of the vtable pointer that allows access to ``instance`` via a ``driver_descriptor const &`` through ``pointer``. +A C++ object's vtable pointer is not "set" by running code. +Rather it is baked into the object's storage as a link-time relocation, the same way a global integer's initial value is. +The upshot is that the entire ``kernel_driver<descriptor>{}`` object is present, complete, and correct in the compiled binary before anything has run at all. +Thus, the boot-time "walk" in the shown above merely performs pure read operations. + +This is a genuine alternative to a generally more familiar C++ self-registration idiom: a global object whose *constructor* calls a registration function. +This mechanism relies on the platform's dynamic-initialization machinery to run every one of these constructors at some point in time before user code executes. +That idiom works in many situations, but it depends on the ordering and even existence of dynamic initialization being available and predictable. +This is precisely the kind of platform machinery a freestanding, exceptions-free kernel is written to need as little of as possible, not least because it would invoke the "static initialization order fiasco". +The kernel's implementations sidesteps the initialization order questions, since there is not order to reason about. +All data is constructed during compile time and linked in the final image. + +Two-Phase Registration: Descriptor Now, Instance Later +------------------------------------------------------ + +The split between ``driver_descriptor`` and the actual ``driver`` object ``make_instance()`` produces is deliberate. +It buys control over *when* a driver's constructor actually runs. +A driver's constructor may need to allocate dynamic memory, but the kernel heap is not usable until ``kapi::memory::init()``/``kernel::memory::init_heap()`` have run. +Because ``make_instance()`` is only ever called from inside the explicit ``kernel::drivers::init()``/``init_platform_drivers()`` walks, placed by hand at a specific point in ``kernel::main()`` after memory is ready, a driver author gets a real heap-allocating constructor without ever having to reason about whether it is safe to run yet. + +Three Independent Instances of the Same Idiom +--------------------------------------------- + +This linker-based registration idiom is used in multiple places within the kernel, including: + +* ``kapi::devices::kernel_driver<Type>`` → the ``kernel_drivers`` section, walked by ``kernel::drivers::init()``. +* ``kapi::devices::platform_driver<Type>`` → the ``platform_drivers`` section, walked by ``kapi::devices::init_platform_drivers()``. +* ``kernel::vfs::driver_module<Type>`` → the ``filesystem_drivers`` section, walked by ``kernel::vfs::driver_registry::init()``. + +The filesystem version follows the same general shape, using its own template, in its own namespace, with its own ``driver_descriptor`` type: + +.. code-block:: cpp + + namespace + { + struct descriptor final : kernel::vfs::driver_descriptor + { + [[nodiscard]] auto name() const noexcept -> std::string_view override { return "ext2"; } + [[nodiscard]] auto make_instance() const -> kernel::vfs::filesystem::filesystem_ptr override + { + return kstd::make_shared<kernel::filesystems::ext2::filesystem>(); + } + }; + + [[gnu::used]] + constexpr auto registration = kernel::vfs::driver_module<descriptor>{}; + } // namespace + +The duplication is worth being honest about (see *Drawbacks*, below). +``kernel::vfs::driver_registry::add()`` stores drivers in a ``kstd::flat_map<std::string_view, ...>`` and adds via ``m_drivers.emplace(name, driver)``, then checks the returned ``bool``: + +.. code-block:: cpp + + auto driver_registry::add(std::string_view name, kstd::shared_ptr<filesystem> driver) -> bool + { + auto result = m_drivers.emplace(name, driver); + if (!result.second) + { + kstd::println(kstd::print_sink::stderr, "[OS:VFS] Tried to register duplicate filesystem driver '{}'!", name); + } + return result.second; + } + +A duplicate filesystem name is rejected correctly, relying on ``flat_map::emplace()``'s own semantics rather than being reimplemented by hand. + +Boot Ordering, Precisely +------------------------ + +Both device-driver registration paths run to completion *before* the corresponding device-attachment path runs. +The kernel's ``main()`` function makes the ordering an explicit, readable sequence rather than an accident of link order: + +.. code-block:: cpp + + kapi::devices::init(); // registries themselves come into existence + kernel::drivers::init(); // kernel_drivers section walked → drivers registered + kernel::devices::init(); // generic devices attached → matched against drivers already registered + kapi::devices::init_platform_drivers(); // platform_drivers section walked → drivers registered + kapi::devices::init_platform_devices(); // platform devices attached → matched against drivers already registered + +This ordering is load-bearing, not cosmetic. +``bus::add_child()`` immediately offers a newly-attached device to ``driver_registry::device_attached()`` (:doc:`tb0004-device-tree-ownership-and-registries`), which tries every *currently registered* driver against it. +A driver registered after its matching device was already attached simply never gets a chance to bind, with no error raised anywhere, since a device is only offered once, at attachment time. +Placing both ``*_init()`` calls (drivers) strictly before their corresponding devices call is what makes "define a driver, do nothing else" actually work in practice. +It is the same guarantee the driver guide's pseudo-bus example (:doc:`/guides/device-drivers`) relies on when it notes that ``null`` and ``zero`` are already registered by the time their devices are attached. + +Benefits +-------- + +**Adding a driver is a strictly local change.** + One ``descriptor`` struct and one ``kernel_driver<descriptor>``/``platform_driver<descriptor>``/``driver_module<descriptor>`` line. + Everything is entirely inside the new driver's own ``.cpp`` file. + No other file in the tree needs an edit. + +**No runtime cost paid before a driver is actually used.** + A descriptor is an inert, read-only object until ``make_instance()`` is called on it explicitly. + This call happens at a controlled point during the boot process. + +**No static-initialization-order question to reason about.** + Because nothing here depends on dynamic initialization at all, there is no ordering hazard between two drivers. + Similarly, there is no ordering hazard between a driver and the subsystems it depends on. + The "static initialization order fiasco" is avoided completely. + +Drawbacks and Limitations +------------------------- + +**The mechanism depends on toolchain- and linker-specific behavior.** + ``__attribute__((section(...)))``, ``KEEP()``, and the ``__start_*``/``__stop_*`` symbol convention are GNU-linker (and GNU/Clang-compatible-attribute) features, not standard C++. + Porting the kernel to a fundamentally different toolchain would mean re-deriving this entire mechanism, not recompiling it as-is. + +**Registration order within one section is unspecified.** + Drivers are visited in whatever order the linker happened to place their contributing object files. + There is nothing a driver author can control or can rely on when it comes to the order of descriptor visitation. + Contrast FreeBSD's ``LINKER_SET`` family, which is the same underlying idea but built around named, individually-addressable set members [2]_. + Alternatively, Linux's numbered ``initcallN`` levels give a coarse, explicit priority a driver can opt into [3]_ . + +**A driver silently excluded from the build produces no diagnostic anywhere in this mechanism.** + If a ``.cpp`` file defining a ``kernel_driver<descriptor>`` is accidentally left out of the CMake target's source list, the kernel links, boots, and runs with that driver simply absent. + There is no way of distinguishing this situation from "the driver was not written". + +**The same idiom is implemented three separate times** + ``kernel_driver``, ``platform_driver``, and ``driver_module`` are three distinct templates with three distinct backing sections and three distinct walk functions. + They only differ in the ``driver_descriptor`` type and section name involved. + A generic ``self_registering<Category, DescriptorType>`` template parameterized on both is a plausible unification that has not been attempted. + As of today, a bug fixed in one walk function is not automatically fixed in the other two. + +**There is no compile-time or link-time list of what got registered** + The only way to see the full set of active drivers is to read the boot log's ``[OS:DRV]``/``[ARCH:DRV]``/``[OS:VFS]`` lines at runtime. + Alternatively, the built ELF image's ``kernel_drivers``/``platform_drivers``/``filesystem_drivers`` sections can be inspected directly with a tool like ``nm`` or ``readelf``. + +Parallels in Other Systems +-------------------------- + +**FreeBSD and NetBSD's ``LINKER_SET`` family** + A direct kernel-level ancestor of this idea: a set of macros that place a pointer into a named, linker-collected section, discovered at boot the same way, via ``__start_``/``__stop_``-style bracketing symbols the linker script provides [2]_. + Anyone who has read FreeBSD's ``newbus`` source will recognize this brief's mechanism immediately; TeachOS's version is a small, from-scratch C++ reimplementation of the same underlying trick, not a port of FreeBSD's macros. + +**Linux's ``initcall`` levels** + Linux solves the identical problem, code that must run once, early, without a central list of what to call, with C functions instead of C++ objects, placed into one of several *numbered* sections (``.initcall1.init`` through ``.initcall7.init``). + ``do_initcalls()`` walks in level order during boot [3]_. + The numbered levels are the one capability this brief's mechanism does not have: a Linux subsystem can guarantee it initializes before or after another subsystem by choosing its level. + As of today, in TeachOS, a driver has only "kernel" or "platform" to choose between, and no finer control within either. + +Driver Development Checklist +---------------------------- + +* To add a new driver: write a ``driver_descriptor`` (or ``kernel::vfs::driver_descriptor`` for a filesystem) in an anonymous namespace in your driver's own ``.cpp`` file, and register it with exactly one ``[[gnu::used]] constexpr auto registration = kernel_driver<descriptor>{};`` line (``platform_driver`` or ``driver_module`` as appropriate). + Nothing else in the tree needs to change. +* Choose the section deliberately: ``kernel_driver`` for architecture-independent drivers, ``platform_driver`` for architecture-specific ones. + This choice determines which boot-time walk finds your driver, and therefore which devices will already exist by the time it is registered. +* Never remove ``[[gnu::used]]`` from a registration object "because it isn't referenced anywhere". + That is precisely why it needs the attribute, not a sign that the attribute is unnecessary. +* Do not assume any ordering relative to other drivers in the same section. + If two drivers must initialize in a specific relative order, that guarantee has to come from somewhere else (the device tree's own attach order, or an explicit dependency the drivers express to each other). + Do not rely on any order from this mechanism. +* Confirm a new driver's ``.cpp`` file is actually part of the relevant CMake target's sources — this mechanism gives no diagnostic at all for a driver that was simply never compiled in. + +References +------------- + +.. [1] ISO/IEC TS 19568:2015, *C++ Extensions for Library Fundamentals*, section on ``observer_ptr`` (``std::experimental::observer_ptr``). `Online <https://www.iso.org/standard/65112.html>`_. + +.. [2] FreeBSD Project, ``sys/sys/linker_set.h`` and the ``linker_set(9)`` manual page (``DATA_SET``, ``TEXT_SET``, and the ``SYSINIT``/module registration machinery built on the same section-collection mechanism). `Online <https://man.freebsd.org/cgi/man.cgi?query=linker_set&sektion=9>`_. + +.. [3] Torvalds, L. et al., Linux kernel source, ``include/linux/init.h`` (``module_init()``, ``subsys_initcall()`` and the numbered ``initcall`` levels) and ``init/main.c`` (``do_initcalls()``). See also Corbet, J., Rubini, A., and Kroah-Hartman, G., *Linux Device Drivers*, 3rd ed., O'Reilly, 2005, chapter 2, "Building and Running Modules." + +.. seealso:: + + ``kapi/kapi/devices/driver_registry.hpp`` and ``kernel/kernel/drivers/init.cpp`` / ``arch/x86_64/kapi/devices.cpp`` — the ``kernel_driver``/``platform_driver`` mechanism this brief quotes from directly. + + ``kernel/kernel/vfs/driver_registry.hpp``/``.cpp`` — the independent, structurally parallel ``driver_module`` mechanism for filesystem drivers. + + ``arch/x86_64/scripts/kernel.ld`` — the linker script sections (``kernel_drivers``, ``platform_drivers``, ``filesystem_drivers``) this brief quotes from directly. + + ``kernel/kernel/drivers/pseudo/null.cpp`` and ``kernel/kernel/filesystems/ext2/module.cpp`` — the two complete, real registration examples this brief walks through. + + :doc:`tb0004-device-tree-ownership-and-registries` — ``driver_registry`` itself, and the ``device_registry::add()`` duplicate-name issue this brief contrasts against ``vfs::driver_registry::add()``. + + :doc:`../guides/device-drivers` — self-registration in practical use, and the boot-ordering guarantee this brief derives in full. |