Crate kernel 

Scarlet Kernel

Scarlet is an operating system kernel written in Rust that implements a transparent ABI conversion layer for executing binaries across different operating systems and architectures. The kernel provides a universal container runtime environment with strong isolation capabilities, comprehensive filesystem support, dynamic linking, and modern graphics capabilities.

Multi-ABI Execution System

The core innovation of Scarlet is its ability to run binaries from different operating systems transparently within the same runtime environment:

ABI Module Architecture

• Modular ABI Implementation: Each ABI module implements its own complete syscall interface using shared kernel APIs, rather than translating between syscalls
• Binary Detection: Automatic identification of binary format and target ABI through ELF header analysis and magic number detection
• Shared Kernel Resources: All ABIs operate on common kernel objects (VFS, memory, devices) ensuring consistent behavior and efficient resource utilization
• Native Implementation: Each ABI provides full syscall implementation using underlying kernel abstractions, enabling complete OS compatibility
• Dynamic Linking: Native dynamic linker support for shared libraries and position-independent executables

Supported ABIs

• Scarlet Native ABI: Direct kernel interface with optimal performance, featuring:

  • Handle-based resource management with capability-based security
  • Modern VFS operations with namespace isolation
  • Advanced IPC mechanisms including pipes and event-driven communication
  • Container-native filesystem operations
  • Dynamic linking support

• Linux Compatibility ABI (in development): Full POSIX syscall implementation

• xv6 Compatibility ABI (in development): Educational OS syscall implementation

Container Runtime Environment

Scarlet provides enterprise-grade containerization features:

Filesystem Isolation

• Mount Namespace Isolation: Per-task filesystem namespaces enabling complete isolation
• Bind Mount Operations: Selective resource sharing between containers
• Overlay Filesystem: Copy-on-write semantics with whiteout support for efficient layering
• Device File Management: Controlled access to hardware through DevFS integration

Resource Management

• Handle-Based Security: Capability-based access control with fine-grained permissions
• Memory Isolation: Per-task memory spaces with controlled sharing mechanisms
• Task Lifecycle Management: Complete process management with environment variable support
• IPC Mechanisms: Pipes, shared memory, and other inter-process communication primitives

Virtual File System v2

Scarlet implements a modern VFS architecture designed for container environments:

Core Architecture

• VfsEntry: Path hierarchy cache providing fast O(1) path resolution with automatic cleanup
• VfsNode: Abstract file entity interface with metadata access and clean downcasting
• FileSystemOperations: Unified driver API consolidating all filesystem operations
• Mount Tree Management: Hierarchical mount point management with O(log n) resolution

Filesystem Drivers

• TmpFS: High-performance memory-based filesystem with configurable size limits

• CpioFS: Read-only CPIO archive filesystem optimized for initramfs and embedded data

• ext2: Full ext2 filesystem implementation with complete read/write support for persistent storage

• FAT32: Complete FAT32 filesystem implementation with directory and file operations

• OverlayFS: Advanced union filesystem with copy-up semantics and whiteout support

• DevFS: Device file system providing controlled hardware access

• Memory Safety: Prevention of use-after-free, double-free, and data races at compile time:

  • The type system ensures resources are not used after being freed
  • Mutable references are exclusive, preventing data races
  • Lifetimes ensure references do not outlive the data they point to

• Trait-based Abstractions: Common interfaces for device drivers and subsystems enabling modularity:

  • The BlockDevice trait defines operations for block-based storage
  • The SerialDevice trait provides a common interface for UART and console devices
  • The FileSystem trait provides unified filesystem operations for VFS v2 integration

Boot Process

Scarlet follows a structured, architecture-agnostic initialization sequence built around the BootInfo structure for unified system startup:

Architecture-Specific Boot Phase

1. Low-level Initialization: CPU feature detection, trap vector setup
2. Hardware Discovery: Parse firmware-provided hardware description (FDT/UEFI/ACPI)
3. Memory Layout: Determine usable memory areas and relocate critical data
4. BootInfo Creation: Consolidate boot parameters into unified structure
5. Kernel Handoff: Call start_kernel() with complete BootInfo

Unified Kernel Initialization

6. Early Memory Setup: Heap allocator initialization using BootInfo memory areas
7. Early Subsystems: Critical kernel subsystem initialization via early initcalls
8. Driver Framework: Device driver registration and basic driver initcalls
9. Virtual Memory: Kernel virtual memory management and address space setup
10. Device Discovery: Hardware enumeration from BootInfo device source
11. Graphics Subsystem: Framebuffer and graphics device initialization
12. Interrupt Infrastructure: Interrupt controller setup and handler registration
13. Timer Subsystem: Kernel timer initialization for scheduling and timekeeping
14. Virtual File System: VFS initialization and root filesystem mounting
15. Initial Filesystem: Initramfs processing if provided in BootInfo
16. Initial Process: Create and load first userspace task (/system/scarlet/bin/init)
17. Scheduler Activation: Begin task scheduling and enter normal operation

BootInfo Integration Benefits

• Architecture Abstraction: Unified interface across RISC-V, ARM, x86 platforms
• Modular Design: Clean separation between arch-specific and generic initialization
• Memory Safety: Structured memory area management prevents overlaps and corruption
• Extensibility: Easy addition of new boot parameters without breaking existing code
• Debugging: Centralized boot information for diagnostics and troubleshooting

Each stage validates successful completion before proceeding, with comprehensive logging available through the early console interface. The BootInfo structure ensures all necessary information is available throughout the initialization process.

System Integration

Core Subsystems

• Task Management: Complete process lifecycle with environment variables and IPC
• Memory Management: Virtual memory with per-task address spaces and shared regions
• Device Framework: Unified device interface supporting block, character, and platform devices
• Interrupt Handling: Event-driven architecture with proper context switching
• Handle System: Capability-based resource access with fine-grained permissions

ABI Module Integration

Each ABI module integrates with the kernel through standardized interfaces:

• Binary Loading: ELF loader with format detection and validation
• Syscall Dispatch: Per-ABI syscall tables with transparent routing
• Resource Management: Shared kernel object access through common APIs
• Environment Setup: ABI-specific process initialization and cleanup
• Mount Operations: mount(), umount(), pivot_root() for dynamic filesystem management
• Process Management: execve(), fork(), wait(), exit() with proper cleanup
• IPC Operations: Pipe creation, communication, and resource sharing

Architecture Support

Currently implemented for RISC-V 64-bit architecture with comprehensive hardware support:

• Interrupt Handling: Complete trap frame management with timer and external interrupts
• Memory Management: Virtual memory with page tables and memory protection
• SBI Interface: Supervisor Binary Interface for firmware communication
• Instruction Abstractions: RISC-V specific optimizations with compressed instruction support

Rust Language Features

Scarlet leverages Rust’s advanced features for safe and efficient kernel development:

Memory Safety

• Zero-cost Abstractions: High-level constructs compile to efficient machine code
• Ownership System: Automatic memory management without garbage collection overhead
• Lifetime Validation: Compile-time prevention of use-after-free and dangling pointer errors
• Borrowing Rules: Exclusive mutable access prevents data races at compile time
• No Buffer Overflows: Array bounds checking and safe pointer arithmetic

Type System Features

• Trait-based Design: Generic programming with zero-cost abstractions for device drivers
• Pattern Matching: Exhaustive matching prevents unhandled error cases
• Option/Result Types: Explicit error handling without exceptions or null pointer errors
• Custom Test Framework: #[test_case] attribute for no-std kernel testing
• Const Generics: Compile-time array sizing and type-level programming

No-std Environment

• Embedded-first Design: No standard library dependency for minimal kernel footprint
• Custom Allocators: Direct control over memory allocation strategies
• Inline Assembly: Direct hardware access when needed with type safety
• Custom Panic Handler: Controlled kernel panic behavior for debugging
• Boot-time Initialization: Static initialization and controlled startup sequence

Development Framework

Testing Infrastructure

Scarlet provides a comprehensive testing framework designed for kernel development:

#[test_case]
fn test_vfs_operations() {
    // Kernel unit tests run in privileged mode
    let vfs = VfsManager::new();
    // ... test implementation
}

• Custom Test Runner: #[test_case] attribute for kernel-specific testing
• No-std Testing: Tests run directly in kernel mode without standard library
• Integration Tests: Full subsystem testing including multi-ABI scenarios
• Hardware-in-the-Loop: Testing on real hardware and QEMU emulation
• Performance Benchmarks: Kernel performance measurement and regression testing

Debugging Support

• Early Console: Serial output available from early boot stages
• Panic Handler: Detailed panic information with stack traces
• GDB Integration: Full debugging support through QEMU’s GDB stub
• Memory Debugging: Allocation tracking and leak detection
• Tracing: Event tracing for performance analysis and debugging

Build System Integration

The kernel integrates with cargo-make for streamlined development:

• cargo make build-debug-riscv64 / cargo make build-debug-aarch64: Full build with user programs
• cargo make test-riscv64 / cargo make test-aarch64: Run kernel tests
• cargo make debug-riscv64 / cargo make debug-aarch64: Launch kernel with GDB support
• cargo make run-riscv64 / cargo make run-aarch64: Quick development cycle execution

Entry Points

The kernel provides multiple entry points for different scenarios:

• start_kernel(): Main bootstrap processor initialization
• start_ap(): Application processor startup for multicore systems
• test_main(): Test framework entry point when built with testing enabled

Module Organization

Core kernel modules provide focused functionality:

• abi/: Multi-ABI implementation modules (Scarlet Native, Linux, xv6)
• arch/: Architecture-specific code (currently RISC-V 64-bit)
• drivers/: Hardware device drivers (UART, block devices, VirtIO)
• fs/: Filesystem implementations and VFS v2 core
• task/: Task management, scheduling, and process lifecycle
• mem/: Memory management, allocators, and virtual memory
• syscall/: System call dispatch and implementation
• object/: Kernel object system with handle management
• interrupt/: Interrupt handling and controller support

Note: Currently, Scarlet Native ABI is fully implemented. Linux and xv6 ABI support are under development and will be available in future releases.

Modules

abi

ABI module.

arch

Architecture-specific code for Scarlet kernel

device

Device module.

drivers

Device drivers module.

earlycon

Early console for generic architecture.

environment

executor

Transparent Executor Module

fs

Virtual File System (VFS) Module - Version 2 Architecture

initcall

Initcall System

interrupt

Interrupt management system

ipc

Inter-Process Communication (IPC) module

library

Library module for the kernel.

mem

Memory management module.

network

Network functionality for Scarlet

object

Kernel object management system

profiler

Simple profiler for measuring function execution time.

random

Kernel Random Number Generator

sched

Scheduler module.

sync

Synchronization primitives module

syscall

System call interface module.

task

Task module.

time

Time utilities for the kernel

timer

Kernel timer module.

traits

vm

Virtual memory module.

Macros

defer

Macro to defer execution of a block of code. This macro allows you to specify a block of code that will be executed when the current scope is exited. It is similar to the defer function but provides a more concise syntax.

driver_initcall

A macro used to register driver initialization functions to be called during the system boot process.

early_initcall

early_print

early_println

late_initcall

print

println

profile_scope

register_abi

Structs

BootInfo

Boot information structure containing essential system parameters

Enums

DeviceSource

Represents the source of device information during boot

Functions

panic 🔒

A panic handler is required in Rust, this is probably the most basic one possible

start_ap

start_kernel

Main kernel entry point for the boot processor