path: root/docs/briefs/tb0006-compile-time-driver-self-registration.rst

Technical Brief 0006: Compile-Time Driver Self-Registration
===========================================================

:doc:`/briefs/tb0004-device-tree-ownership-and-registries` covers ``driver_registry`` as one of the three registries built into the device model, and notes in passing that it has no lookup API — only a flat list, tried in full against every device.
This brief covers a question that document deliberately leaves open: **how does anything get into that list in the first place, given that no code anywhere calls a function named "register all drivers"?**

The answer is a compile-time mechanism shared, in slightly different clothing, by two otherwise-unrelated parts of the kernel: device drivers (``kapi::devices::kernel_driver``/``platform_driver``) and filesystem drivers (``kernel::vfs::driver_module``).
A driver's own ``.cpp`` file defined a small object and the build's linker collects a pointer to it into a dedicated, named section of the final kernel image.
Afterwards, an explicit walk at boot turns that section into real, running driver instances.
No central list exists anywhere in the source tree, and no constructor runs before ``main()`` to build one at runtime.

The Problem This Replaces
-------------------------

An obvious implementation for the central device driver list, is a single, hand-maintained function, somewhere in ``kernel/kernel/drivers/init.cpp``.
Such a function might look like the example below:

.. code-block:: cpp

   auto init() -> void
   {
     driver_registry::get().add(kstd::make_shared<pit_driver>());
     driver_registry::get().add(kstd::make_shared<null_driver>());
     driver_registry::get().add(kstd::make_shared<zero_driver>());
     // ... one line per driver, forever
   }

This has two real costs, aside of aesthetics.
First, it is a synchronization problem disguised as a list: every new driver requires a matching edit to a file its author may not think to touch.
The failure mode of forgetting such an edit is a driver that compiles and links fine, but never gets bound to any device.
This failure is silent and therefore difficult to debug, since the cause of the issue lies in a physically distinct file from the driver itself.
Second, it is a dependency-direction problem: ``kernel/kernel/drivers/init.cpp`` would need to ``#include`` the header of every driver in the tree.
This means that a change to any single driver's public interface touches a file that has no other reason to know that driver exists.
Both problems get worse, not better, as the driver count grows.
This stands in exact opposition to a mechanism whose whole job is to make adding a driver a local, self-contained change.

The Core Mechanism
------------------

A driver announces itself with a small, private ``driver_descriptor`` type derived from ``kapi::devices::driver_descriptor``.
This descriptor is instantiated directly within the main driver implementation file an automatically placed in a custom linker section.
A complete, real example is the pseudo bus's ``null`` driver (``kernel/kernel/drivers/pseudo/null.cpp``):

.. code-block:: cpp

   namespace
   {
     struct descriptor final : kapi::devices::driver_descriptor
     {
       [[nodiscard]] auto name() const noexcept -> std::string_view override { return "dev_null"; }
       [[nodiscard]] auto make_instance() const -> kstd::shared_ptr<kapi::devices::driver> override
       {
         return kstd::make_shared<null>();
       }
     };

     [[gnu::used]]
     constexpr auto registration = kapi::devices::kernel_driver<descriptor>{};
   }  // namespace

``driver_descriptor`` (``kapi/kapi/devices/driver_registry.hpp``) exposes the minimal interface system this needs: a diagnostic name and a factory function that produces the real driver instance on demand.

.. code-block:: cpp

   struct driver_descriptor
   {
     virtual ~driver_descriptor() = default;
     [[nodiscard]] virtual auto name() const noexcept -> std::string_view = 0;
     [[nodiscard]] virtual auto make_instance() const -> kstd::shared_ptr<driver> = 0;
   };

The registration mechanism itself relies on instances of the type ``kernel_driver<Type>``.
This class template has no member functions at all, and only two ``static`` data members:

.. code-block:: cpp

   template<typename Type>
   struct kernel_driver
   {
     constexpr auto static instance = Type{};
     [[using gnu: section("kernel_drivers"), used, visibility("hidden")]] constexpr auto static pointer{
       kstd::make_observer<kapi::devices::driver_descriptor const>(&instance),
     };
   };

An identical class template with a different name (``platform_driver``) exists for architecture specific drivers.

The ``instance`` member is a instance of the driver's descriptor type that is constructed at compile-time.
They ``pointer`` member points to that instance and is placed into a named section during compilation and linking.
Those named sections (one for ``kernel_drivers`` and one for ``platform_drivers``) contain only a contiguous array of these pointers, one per driver.
Each pointer is contributed independently by every ``.cpp`` file that defines an instance of ``kernel_driver`` or ``platform_driver``.

The linker script (``arch/x86_64/scripts/kernel.ld``) reserves that region explicitly and exposes its bounds as ordinary symbols:

.. code-block:: text

   .kernel_rodata ALIGN(4K) : AT (ADDR (.kernel_rodata) - TEACHOS_VMA)
   {
     /* other read-only data sections go here ... */

     /* Kernel driver factories */
     PROVIDE(__start_kernel_drivers = .);
     KEEP(*(kernel_drivers));
     PROVIDE(__stop_kernel_drivers = .);

     /* Platform driver factories */
     PROVIDE(__start_platform_drivers = .);
     KEEP(*(platform_drivers));
     PROVIDE(__stop_platform_drivers = .);
   } :kernel_rodata

Every ``kernel_driver<SomeDescriptor>::pointer`` from every translation unit in the whole kernel image ends up contiguous in memory and is bracketed by ``__start_kernel_drivers`` and ``__stop_kernel_drivers`` — regardless of which ``.cpp`` file defined it, and without that file needing to be named anywhere else.
The ``KEEP()`` linker instruction ensures the linker does not evict these pointers, despite them not being referenced anywhere.

C++ code recovers those bounds as ordinary ``extern`` declarations and walks the range as a ``std::span`` (``kernel/kernel/drivers/init.cpp``):

.. code-block:: cpp

   extern "C"
   {
     extern kstd::observer_ptr<kapi::devices::driver_descriptor> const __start_kernel_drivers;
     extern kstd::observer_ptr<kapi::devices::driver_descriptor> const __stop_kernel_drivers;
   }

   auto init() -> void
   {
     kapi::devices::driver_registry::init();
     auto & registry = kapi::devices::driver_registry::get();

     auto descriptors = std::span{&__start_kernel_drivers, &__stop_kernel_drivers} |
                        std::views::filter([](auto p) { return p != nullptr; });

     for (auto driver : descriptors)
     {
       auto instance = driver->make_instance();
       kstd::println("[OS:DRV] registering driver '{}' ({})", instance->name(), driver->name());
       registry.add(std::move(instance));
     }
   }

``platform_drivers`` is processed the same way, by ``kapi::devices::init_platform_drivers()``.

Why This Needs No Constructor to Run
------------------------------------

The detail that makes this mechanism fit a freestanding, no-exceptions kernel, is that none of the code above executes before ``main()``.
``kernel_driver<Type>::instance`` and ``::pointer`` are both ``constexpr``, requiring the compiler to fully construct their values at compile time.
This includes resolution of the vtable pointer that allows access to ``instance`` via a ``driver_descriptor const &`` through ``pointer``.
A C++ object's vtable pointer is not "set" by running code.
Rather it is baked into the object's storage as a link-time relocation, the same way a global integer's initial value is.
The upshot is that the entire ``kernel_driver<descriptor>{}`` object is present, complete, and correct in the compiled binary before anything has run at all.
Thus, the boot-time "walk" in the shown above merely performs pure read operations.

This is a genuine alternative to a generally more familiar C++ self-registration idiom: a global object whose *constructor* calls a registration function.
This mechanism relies on the platform's dynamic-initialization machinery to run every one of these constructors at some point in time before user code executes.
That idiom works in many situations, but it depends on the ordering and even existence of dynamic initialization being available and predictable.
This is precisely the kind of platform machinery a freestanding, exceptions-free kernel is written to need as little of as possible, not least because it would invoke the "static initialization order fiasco".
The kernel's implementations sidesteps the initialization order questions, since there is not order to reason about.
All data is constructed during compile time and linked in the final image.

Two-Phase Registration: Descriptor Now, Instance Later
------------------------------------------------------

The split between ``driver_descriptor`` and the actual ``driver`` object ``make_instance()`` produces is deliberate.
It buys control over *when* a driver's constructor actually runs.
A driver's constructor may need to allocate dynamic memory, but the kernel heap is not usable until ``kapi::memory::init()``/``kernel::memory::init_heap()`` have run.
Because ``make_instance()`` is only ever called from inside the explicit ``kernel::drivers::init()``/``init_platform_drivers()`` walks, placed by hand at a specific point in ``kernel::main()`` after memory is ready, a driver author gets a real heap-allocating constructor without ever having to reason about whether it is safe to run yet.

Three Independent Instances of the Same Idiom
---------------------------------------------

This linker-based registration idiom is used in multiple places within the kernel, including:

* ``kapi::devices::kernel_driver<Type>`` → the ``kernel_drivers`` section, walked by ``kernel::drivers::init()``.
* ``kapi::devices::platform_driver<Type>`` → the ``platform_drivers`` section, walked by ``kapi::devices::init_platform_drivers()``.
* ``kernel::vfs::driver_module<Type>`` → the ``filesystem_drivers`` section, walked by ``kernel::vfs::driver_registry::init()``.

The filesystem version follows the same general shape, using its own template, in its own namespace, with its own ``driver_descriptor`` type:

.. code-block:: cpp

   namespace
   {
     struct descriptor final : kernel::vfs::driver_descriptor
     {
       [[nodiscard]] auto name() const noexcept -> std::string_view override { return "ext2"; }
       [[nodiscard]] auto make_instance() const -> kernel::vfs::filesystem::filesystem_ptr override
       {
         return kstd::make_shared<kernel::filesystems::ext2::filesystem>();
       }
     };

     [[gnu::used]]
     constexpr auto registration = kernel::vfs::driver_module<descriptor>{};
   }  // namespace

The duplication is worth being honest about (see *Drawbacks*, below).
``kernel::vfs::driver_registry::add()`` stores drivers in a ``kstd::flat_map<std::string_view, ...>`` and adds via ``m_drivers.emplace(name, driver)``, then checks the returned ``bool``:

.. code-block:: cpp

   auto driver_registry::add(std::string_view name, kstd::shared_ptr<filesystem> driver) -> bool
   {
     auto result = m_drivers.emplace(name, driver);
     if (!result.second)
     {
       kstd::println(kstd::print_sink::stderr, "[OS:VFS] Tried to register duplicate filesystem driver '{}'!", name);
     }
     return result.second;
   }

A duplicate filesystem name is rejected correctly, relying on ``flat_map::emplace()``'s own semantics rather than being reimplemented by hand.

Boot Ordering, Precisely
------------------------

Both device-driver registration paths run to completion *before* the corresponding device-attachment path runs.
The kernel's ``main()`` function makes the ordering an explicit, readable sequence rather than an accident of link order:

.. code-block:: cpp

  kapi::devices::init();                    // registries themselves come into existence
  kernel::drivers::init();                  // kernel_drivers section walked → drivers registered
  kernel::devices::init();                  // generic devices attached → matched against drivers already registered
  kapi::devices::init_platform_drivers();   // platform_drivers section walked → drivers registered
  kapi::devices::init_platform_devices();   // platform devices attached → matched against drivers already registered

This ordering is load-bearing, not cosmetic.
``bus::add_child()`` immediately offers a newly-attached device to ``driver_registry::device_attached()`` (:doc:`tb0004-device-tree-ownership-and-registries`), which tries every *currently registered* driver against it.
A driver registered after its matching device was already attached simply never gets a chance to bind, with no error raised anywhere, since a device is only offered once, at attachment time.
Placing both ``*_init()`` calls (drivers) strictly before their corresponding devices call is what makes "define a driver, do nothing else" actually work in practice.
It is the same guarantee the driver guide's pseudo-bus example (:doc:`/guides/device-drivers`) relies on when it notes that ``null`` and ``zero`` are already registered by the time their devices are attached.

Benefits
--------

**Adding a driver is a strictly local change.**
  One ``descriptor`` struct and one ``kernel_driver<descriptor>``/``platform_driver<descriptor>``/``driver_module<descriptor>`` line.
  Everything is entirely inside the new driver's own ``.cpp`` file.
  No other file in the tree needs an edit.

**No runtime cost paid before a driver is actually used.**
  A descriptor is an inert, read-only object until ``make_instance()`` is called on it explicitly.
  This call happens at a controlled point during the boot process.

**No static-initialization-order question to reason about.**
  Because nothing here depends on dynamic initialization at all, there is no ordering hazard between two drivers.
  Similarly, there is no ordering hazard between a driver and the subsystems it depends on.
  The "static initialization order fiasco" is avoided completely.

Drawbacks and Limitations
-------------------------

**The mechanism depends on toolchain- and linker-specific behavior.**
  ``__attribute__((section(...)))``, ``KEEP()``, and the ``__start_*``/``__stop_*`` symbol convention are GNU-linker (and GNU/Clang-compatible-attribute) features, not standard C++.
  Porting the kernel to a fundamentally different toolchain would mean re-deriving this entire mechanism, not recompiling it as-is.

**Registration order within one section is unspecified.**
  Drivers are visited in whatever order the linker happened to place their contributing object files.
  There is nothing a driver author can control or can rely on when it comes to the order of descriptor visitation.
  Contrast FreeBSD's ``LINKER_SET`` family, which is the same underlying idea but built around named, individually-addressable set members [2]_.
  Alternatively, Linux's numbered ``initcallN`` levels give a coarse, explicit priority a driver can opt into [3]_ .

**A driver silently excluded from the build produces no diagnostic anywhere in this mechanism.**
  If a ``.cpp`` file defining a ``kernel_driver<descriptor>`` is accidentally left out of the CMake target's source list, the kernel links, boots, and runs with that driver simply absent.
  There is no way of distinguishing this situation from "the driver was not written".

**The same idiom is implemented three separate times**
  ``kernel_driver``, ``platform_driver``, and ``driver_module`` are three distinct templates with three distinct backing sections and three distinct walk functions.
  They only differ in the ``driver_descriptor`` type and section name involved.
  A generic ``self_registering<Category, DescriptorType>`` template parameterized on both is a plausible unification that has not been attempted.
  As of today, a bug fixed in one walk function is not automatically fixed in the other two.

**There is no compile-time or link-time list of what got registered**
  The only way to see the full set of active drivers is to read the boot log's ``[OS:DRV]``/``[ARCH:DRV]``/``[OS:VFS]`` lines at runtime.
  Alternatively, the built ELF image's ``kernel_drivers``/``platform_drivers``/``filesystem_drivers`` sections can be inspected directly with a tool like ``nm`` or ``readelf``.

Parallels in Other Systems
--------------------------

**FreeBSD and NetBSD's ``LINKER_SET`` family**
  A direct kernel-level ancestor of this idea: a set of macros that place a pointer into a named, linker-collected section, discovered at boot the same way, via ``__start_``/``__stop_``-style bracketing symbols the linker script provides [2]_.
  Anyone who has read FreeBSD's ``newbus`` source will recognize this brief's mechanism immediately; TeachOS's version is a small, from-scratch C++ reimplementation of the same underlying trick, not a port of FreeBSD's macros.

**Linux's ``initcall`` levels**
  Linux solves the identical problem, code that must run once, early, without a central list of what to call, with C functions instead of C++ objects, placed into one of several *numbered* sections (``.initcall1.init`` through ``.initcall7.init``).
  ``do_initcalls()`` walks in level order during boot [3]_.
  The numbered levels are the one capability this brief's mechanism does not have: a Linux subsystem can guarantee it initializes before or after another subsystem by choosing its level.
  As of today, in TeachOS, a driver has only "kernel" or "platform" to choose between, and no finer control within either.

Driver Development Checklist
----------------------------

* To add a new driver: write a ``driver_descriptor`` (or ``kernel::vfs::driver_descriptor`` for a filesystem) in an anonymous namespace in your driver's own ``.cpp`` file, and register it with exactly one ``[[gnu::used]] constexpr auto registration = kernel_driver<descriptor>{};`` line (``platform_driver`` or ``driver_module`` as appropriate).
  Nothing else in the tree needs to change.
* Choose the section deliberately: ``kernel_driver`` for architecture-independent drivers, ``platform_driver`` for architecture-specific ones.
  This choice determines which boot-time walk finds your driver, and therefore which devices will already exist by the time it is registered.
* Never remove ``[[gnu::used]]`` from a registration object "because it isn't referenced anywhere".
  That is precisely why it needs the attribute, not a sign that the attribute is unnecessary.
* Do not assume any ordering relative to other drivers in the same section.
  If two drivers must initialize in a specific relative order, that guarantee has to come from somewhere else (the device tree's own attach order, or an explicit dependency the drivers express to each other).
  Do not rely on any order from this mechanism.
* Confirm a new driver's ``.cpp`` file is actually part of the relevant CMake target's sources — this mechanism gives no diagnostic at all for a driver that was simply never compiled in.

References
-------------

.. [1] ISO/IEC TS 19568:2015, *C++ Extensions for Library Fundamentals*, section on ``observer_ptr`` (``std::experimental::observer_ptr``). `Online <https://www.iso.org/standard/65112.html>`_.

.. [2] FreeBSD Project, ``sys/sys/linker_set.h`` and the ``linker_set(9)`` manual page (``DATA_SET``, ``TEXT_SET``, and the ``SYSINIT``/module registration machinery built on the same section-collection mechanism). `Online <https://man.freebsd.org/cgi/man.cgi?query=linker_set&sektion=9>`_.

.. [3] Torvalds, L. et al., Linux kernel source, ``include/linux/init.h`` (``module_init()``, ``subsys_initcall()`` and the numbered ``initcall`` levels) and ``init/main.c`` (``do_initcalls()``). See also Corbet, J., Rubini, A., and Kroah-Hartman, G., *Linux Device Drivers*, 3rd ed., O'Reilly, 2005, chapter 2, "Building and Running Modules."

.. seealso::

  ``kapi/kapi/devices/driver_registry.hpp`` and ``kernel/kernel/drivers/init.cpp`` / ``arch/x86_64/kapi/devices.cpp`` — the ``kernel_driver``/``platform_driver`` mechanism this brief quotes from directly.

  ``kernel/kernel/vfs/driver_registry.hpp``/``.cpp`` — the independent, structurally parallel ``driver_module`` mechanism for filesystem drivers.

  ``arch/x86_64/scripts/kernel.ld`` — the linker script sections (``kernel_drivers``, ``platform_drivers``, ``filesystem_drivers``) this brief quotes from directly.

  ``kernel/kernel/drivers/pseudo/null.cpp`` and ``kernel/kernel/filesystems/ext2/module.cpp`` — the two complete, real registration examples this brief walks through.

  :doc:`tb0004-device-tree-ownership-and-registries` — ``driver_registry`` itself, and the ``device_registry::add()`` duplicate-name issue this brief contrasts against ``vfs::driver_registry::add()``.

  :doc:`../guides/device-drivers` — self-registration in practical use, and the boot-ordering guarantee this brief derives in full.