TeachOS Kernel ============== TeachOS is a modern kernel with a focus on clean design and simplicity. It is not highly optimized like production grade kernels, but instead aims to be understandable. It is implemented in assembly and C++, where the amount of assembly is kept to a minimum. The primary design goal of the kernel is to be teachable and useful as part of the operating systems lecture track at OST. Development ----------- Development happens primarily on Linux. Other platforms may allow building of the kernel as well, however at the time of writing they are not officially supported. Development on Windows has been tested in the past, see `Notes for Development on Windows`_ Required Tools ~~~~~~~~~~~~~~ A precompiled toolchain, targeting only x86-64 as of the time of writing, is available in the `Devconainers `_ repository. The toolchain is provided both as a downloadable, self-contained archive and an Ubuntu Linux based Docker image. When using the downloadable archive, care must be taken to make the contained executables available in the user's or system path. Also, when using the downloadable archive, further required tools, like ``CMake``, ``QEMU``, ``Ninja``, ``xorisso``, ``grub``, etc. must be installed separately. The primary IDE used for development is `Visual Studio Code `_. A configuration of runnable tasks, debugging settings, etc. as well as suggested extensions is provided. The development container a Additionally, a basic IDE-like configuration for ``neovim`` (version 0.12+) is provided as well. It relies on the presence of the `lazy.nvim `_ to install and load plugins. Other IDEs might work, especially Visual Studio Code related ones, but no support is provided. Repository Structure ~~~~~~~~~~~~~~~~~~~~ The code is grouped into separate "sub-packages", according to the following rules: - ``arch``: The root of all platform-dependent code. - ``kapi``: The Kernel API, implemented by either the kernel itself, or the platform, used by both. - ``kernel``: The root of all platform-independent code - ``libs``: Support libraries, like the TeachOS Standard Library, or Multiboot2 support. When implementing new features, drivers, or infrastructure, care must be taken as to where to place the new code. In general, platform-dependent code, like CPU access, MMU configuration, privilege handling etc., should be placed in the relevant folder under the ``arch`` root. Platform independent-code, like generic memory allocation or scheduling algorithms, or generic drivers for hardware like PCIe devices, should be placed under the ``kernel`` root. Launching ~~~~~~~~~ The default build target generates a bootable image. On x86-64 for example, this image takes the form of a bootable, grub2 based ISO image. These images are designed to be booted in QEMU, and should theoretically also be bootable on real hardware. However, note that not warranty is provided, and the kernel code may irreparably destroy any physical hardware if booted on a real system. The VSCodium IDE configuration provides a launch task using QEMU, available for debugging (via F5) and direct launch as a task. .. _dev-support-windows: Notes for Development on Windows ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ While this repository and the devcontainer can be used from Windows, significant performance issues may occur. To reduce these issues, you can either set up a Linux VM for development, or, if that is not preferred, clone the repository inside WSL and open it from there with Visual Studio Code. | ``git clone `` | ``cd `` | ``code .`` If you use tools such as Git Extensions or GitHub Desktop, access the repository via the WSL network path, for example ``\\wsl.localhost\\``. Kernel Architecture ------------------- The kernel is structured into four primary directories, ensuring a clear division of concerns: - `arch/ `_: Contains all platform-dependent implementations (currently targeting `x86_64`). - `kapi/ `_: The Kernel API, which defines the boundaries between platform-dependent and platform-independent components. - `kernel/ `_: Core platform-independent kernel logic. - `libs/ `_: Standalone support libraries (e.g., custom standard library `kstd`, ACPI, and Multiboot2 parsers). Architecture specific logic (`arch`) and platform-independent (`kernel`) logic must not depend directly on each other. Instead, all interaction must be facilitated via interfaces declared in `kapi/ `_. .. code-block:: mermaid graph TD kernel[kernel: Platform-Independent Core] --> kapi[kapi: Kernel API] arch[arch: Platform-Dependent Code] --> kapi kernel --> libs[libs: Standalone Libraries] arch --> libs kapi --> libs Kernel API (KAPI) Decoupling Design ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The decoupling of interfaces and implementations is structured via two categories of declarations within `kapi/ `_: 1. **Kernel-defined**: Platform-independent implementations located in `kernel/kapi/ `_. For example: - `memory.cpp `_: Wraps physical frame allocation and page mapping. - `filesystem.cpp `_: Wraps Virtual File System (VFS) operations. - `interrupts.cpp `_: Registers and routes interrupts to handlers. 2. **Platform-defined**: Low-level hooks declared in `kapi/ `_ and implemented by the target platform under `arch/x86_64/kapi/ `_. For example: - `cpu.cpp `_: Performs early CPU setup via `init()`. - `memory.cpp `_: Initializes page tables and platform allocator. - `interrupts.cpp `_: Exposes methods to toggle external interrupts. This decoupling is intended to ease the porting of TeachOS to new architectures (e.g. RISC-V or ARM64) by implementing a new subdirectory under `arch` that implements the platform-defined KAPI interfaces. Subsystems ~~~~~~~~~~ Bootstrapping & CPU Initialization .................................. Execution begins in platform-dependent code. Once a basic execution environment has been established, the C++ runtime is initialized via before calling the core `main() `_ function. Memory Management ................. TeachOS separates memory allocation into three specialized systems: * **Physical Memory Manager**: The current implementation manages physical memory using a bitmap. * **Virtual Memory Page Mapper**: The architecture-specific page table management implementation. * **Kernel Heap Allocator**: Memory allocations via `new`/`delete` are currently routed through a `block_list_allocator `_ using a linked list first-fit allocator. * **Memory-Mapped I/O (MMIO)**: The `mmio_allocator `_ reserves and maps memory regions for MMIO. Virtual File System (VFS) ......................... A POSIX-like directory and file abstraction layer is located in `kernel/vfs/ `_: * **Abstractions**: Managed via `directory entries `_, `filesystem nodes `_, `mount hierarchies `_, and the `open file table `_. * **Concrete Filesystems**: Implemented separately, under `kernel/filesystems/ `_: - `RootFS `_: An in-memory filesystem mounted at `/` during early boot. - `Device FS `_: Exposes devices as files under `/dev`. - `Second Extended Filesystem `_: An Ext2 driver. Standalone Support Libraries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Standalone support libraries under `libs/ `_ compile independently of the kernel logic: * `kstd `_: A custom standard library containing containers, support infrastructure, and printing/formatting implementations in lieu of a hosted standard library. * `multiboot2 `_: A Multiboot2 structure parser. * `acpi `_: An ACPI parsing utility. * `elf `_: An ELF file format parser. Semester & Bachelor's Thesis Opportunities ------------------------------------------ Due to its simple and educational design, TeachOS lacks several advanced OS features. This makes the kernel an excellent playground for semester projects and Bachelor's theses. The following list outlines potential areas of development, highlighting the architectural dependencies between key systems: Multitasking, User Space, and System Calls ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ There is a direct dependency between process/thread abstractions and the implementation of user mode execution. To run separate user applications, the kernel must manage task states, perform context switches, and handle system calls. Conversely, user space isolation requires page table management and hardware privilege levels. These projects should be approached sequentially or as a joint effort: 1. **Kernel Multitasking (Cooperative and Preemptive)**: - *Scope*: Design a Thread Control Block (TCB) and Process Control Block (PCB) structure. Implement low-level stack setup and context switching in assembly for the active CPU. - *Extension*: Develop a task scheduler (e.g., Round-Robin, Priority-based, or Multi-Level Feedback Queue) utilizing the local APIC timer for preemptive multitasking. - *Size*: Reducing the scheduling to only support cooperative task switches would likely be suitable to a semester thesis, otherwise this has bachelor's thesis size. 2. **User Mode Isolation and System Call Interface**: - *Prerequisite*: A basic multitasking or thread control subsystem. - *Scope*: Implement user-kernel privilege transitions (Ring 3 to Ring 0) utilizing platform-specific instructions. Configure page tables to separate user space virtual memory from the higher-half kernel mapping. - *Extension*: Establish a system call dispatcher routing file and memory operations from user space to the corresponding `kapi/ `_ implementations. - *Extension*: Load and execute an ELF64 binary as the first user-space process. TeachOS already includes an ELF structure parser (`elf `_), but it only reads headers and sections today — loading segments into a process address space and transferring control to them does not exist yet. - *Size*: At least one, likely two bachelor's theses, especially considering syscalls and elf loading. Interrupt Routing, Multi-Core Bring-Up, and ACPI ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ TeachOS currently boots and runs on a single core, with interrupts routed through the legacy 8259 PICs. Moving beyond that is foundational infrastructure that several other projects on this list — a real scheduler, DMA-capable device drivers, and spec-correct ACPI power management — ultimately build on: 3. **Application Processor (AP) Bring-Up and SMP**: - *Scope*: Implement the INIT-SIPI-SIPI startup sequence to bring additional CPU cores online: a real-mode trampoline, per-core state, and a documented lock-ordering discipline for structures shared across cores. - *Extension*: Use the newly-online cores as the foundation for a genuine multi-core scheduler (see "Kernel Multitasking" above). - *Size*: Bachelor's thesis. 4. **PIC to I/O APIC Transition**: - *Scope*: Implement an I/O APIC device/driver pair, parse the ACPI MADT's Interrupt Source Override entries to correctly resolve legacy ISA IRQs to Global System Interrupts, and switch interrupt delivery away from the legacy 8259 PICs (via the IMCR, or the ACPI-blessed ``_PIC`` control method). - *Extension*: Mask the legacy PICs once the I/O APIC path is confirmed working, and make the kernel's interrupt-acknowledgment path mode-aware. - *Size*: Semester thesis, without the extension. 5. **ACPI AML Interpreter**: - *Scope*: Implement a bytecode interpreter for ACPI Machine Language (AML): object namespace construction and control-method invocation (e.g. ``_STA``, ``_PS0``/``_PS3``, ``_PIC``, ``_S5``). TeachOS currently only parses static ACPI tables (the MADT and similar); AML execution is a substantial, largely self-contained undertaking on top of that and a good fit for a Bachelor's thesis on its own. - *Extension*: Use the interpreter to perform a spec-correct interrupt-mode switch and ACPI S5 shutdown, replacing fixed-register/IMCR shortcuts. - *Size*: For a basic implementation, not processing all AML byte code, a semester thesis. A bachelor's thesis otherwise. Platform Ports (64-bit Architectures) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ TeachOS currently targets only 64-bit x86 (`x86_64`). To validate the platform-independence of the `kapi/ `_ interface, ports to other 64-bit targets are highly encouraged: 6. **ARM64 (AArch64) Port**: - *Scope*: Implement the platform-defined KAPI interfaces for a 64-bit ARM target (e.g., QEMU `virt` board or Raspberry Pi 4). This includes writing the boot startup assembly, configuring the translation table (MMU paging), handling the Generic Interrupt Controller (GIC), and implementing timer ticks. - *Size*: At least one bachelor's thesis. Likely a semester + bachelor's thesis project. 7. **RISC-V 64-bit (RV64G) Port**: - *Scope*: Port TeachOS to the RISC-V 64-bit architecture. This involves implementing boot assembly, configuring page table mappings (Sv39/Sv48), setting up the Core Local Interruptor (CLINT) and Platform-Level Interrupt Controller (PLIC), and managing supervisor/user mode transitions. - *Size*: At least one bachelor's thesis. Likely a semester + bachelor's thesis project. - Memory Management Subsystem Extensions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The current memory management uses a basic bitmap page frame allocator and a first-fit heap allocator. There are significant opportunities to implement standard production-grade memory management schemes: 8. **Buddy Page Allocator**: - *Scope*: Replace the current `bitmap_frame_allocator `_ with a Buddy Allocator system. This manages memory allocations in power-of-two page sizes, significantly reducing external fragmentation and improving allocation speed. - *Size*: Semester thesis. 9. **Slab/Slub/Slob Object Allocator**: - *Scope*: Implement a slab allocator on top of the physical page allocator. This caches kernel objects of identical size (such as inodes, file descriptors, and thread control blocks) to avoid constant heap fragmentation and overhead from the general-purpose `block_list_allocator `_. - *Size*: Semester thesis. 10. **Advanced Virtual Memory (Copy-on-Write, Demand Paging)**: - *Prerequisite*: A basic thread multitasking subsystem. - *Scope*: Implement a page fault handler that dynamically loads executable segments only when touched (demand paging), or implement copy-on-write page table sharing (crucial for implementing Unix-like `fork` semantics). - *Size*: Bachelor's thesis. Filesystem Support and VFS Extensions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The Virtual File System (VFS) is designed to host multiple concurrent filesystem types, but support is currently limited. Project opportunities here include implementing new drivers or core VFS caching features: 11. **Ext2 Write Completion (unlink, rmdir, rename)**: - *Scope*: Ext2 write support — inode and block allocation, directory-entry management, and superblock consistency — is already substantially implemented in the `ext2 `_ driver. The concrete remaining gap is ``unlink()``, ``rmdir()``, and ``rename()``, none of which exist yet anywhere in the VFS-facing filesystem interfaces. - *Extension*: ``rename()`` across a mount boundary is a substantially harder problem than the same-filesystem case, and a good extension once the basic operations land. - *Size*: Part of a FS-centric semester thesis. With the rename extension part of an FS-centric bachelor's thesis. 12. **New Filesystem Drivers (e.g., FAT32, ISO 9660)**: - *Scope*: Implement new filesystem drivers from scratch (such as FAT32 or ISO 9660 for CD-ROMs), allowing TeachOS to interoperate with standard virtual media and flash drives. - *Size*: Depending on the specific filesystem, a semester or bachelor's thesis. 13. **Unified Page Cache and Directory Entry Cache**: - *Scope*: Develop a page caching subsystem that intercepts read/write VFS calls, caching recently accessed filesystem blocks in physical memory frames, and optimize pathname lookup times using a dynamic directory entry (dentry) cache. TeachOS currently has no page cache at all, which is a correctness concern as much as a performance one: two open file descriptors on the same file can see divergent views of the same underlying data. - *Size*: Semester thesis. 14. **Virtual Filesystems (e.g., procfs, sysfs)**: - *Scope*: Create virtual filesystems that dynamically generate contents from current kernel data structures, providing userspace with debugging and configuration access interfaces. - *Size*: Likely not suitable for a standalone thesis at the moment, since no userspace exists. Otherwise a semester thesis. Hardware Buses and Device Drivers .................................. Currently, block storage is simulated via RAM disks loaded as boot modules. Real hardware interaction requires expanding driver support: 15. **PCI/PCIe Bus Discovery**: - *Scope*: Develop a PCI/PCIe bus driver that scans configuration spaces, detects connected devices, and registers them to the virtual root bus using the `kapi::devices `_ interface. - *Size*: At least one bachelor's thesis. Possibility of running as a semester + bachelor's thesis. 16. **PS/2 Keyboard Controller (Intel 8042)**: - *Scope*: Implement a driver for the legacy 8042 keyboard controller: enumerate its keyboard (and, where present, mouse) channels as separate devices, and implement a minimal scancode translation layer. Does not require PCI discovery, which makes it a good smaller-scale companion or precursor project to USB below. - *Size*: Semester thesis, likely plus additional work packages. 17. **Storage Controller Drivers (AHCI/SATA, NVMe, or virtio-blk)**: - *Prerequisite*: PCI bus discovery. - *Scope*: Write a driver for SATA controllers (AHCI), modern NVMe drives, or the virtio-blk paravirtualized interface, routing block read/write operations from the VFS to actual (or emulated) physical disks. AHCI is TeachOS's QEMU machine's native, default-attached storage controller and a natural first target; virtio-blk trades realism for a simpler, more forgiving protocol under emulation. - *Size*: At least one bachelor's thesis, even with reduced scope. 18. **USB Host Controller and Device Support**: - *Prerequisite*: PCI bus discovery (for the xHCI host controller). - *Scope*: Implement an xHCI host controller driver and USB device enumeration (descriptor requests, configuration selection), modeling each USB interface as its own device carrying a class-specific facet (e.g. HID, mass storage). A HID keyboard driver is a natural first target to prove the model end-to-end. A substantial, multi-stage project on its own. - *Size*: At least one bachelor's thesis, even with reduced scope. 19. **Network Stack and Driver Integration**: - *Scope*: Interface with a network adapter (e.g., Intel e1000 or VirtIO-net), and develop a lightweight network stack (ARP, IPv4, UDP) to allow TeachOS to send and receive raw network frames. - *Size*: At least one bachelor's thesis, even with reduced scope.