System functionality

system
User space interfaces, System calls
Driver Model
modules
buses, PCI
hardware interfaces, [re]booting

This article describes infrastructures used to support and manage other kernel functionalities. This functionality is named after system calls and sysfs.

User space communication

🔧 TODO

⚲ APIs:

kernel space API for user space

uapi _inc

arch/x86/include/uapi _src

man 2 ioctl

System calls

Device files

user space API for kernel space

linux/uaccess.h _inc:

copy_to_user _id

copy_from_user _id

📚 References

User-space API guides _doc

User space

Linux kernel interfaces

ULK3 Chapter 11. Signals

System calls

Table of syscalls

⚙️ Internals:

linux/syscalls.h _inc

syscall_init _id

entry_SYSCALL_64 _id

do_syscall_64 _id

man 2 syscall

man 2 syscalls

📚 References

System call

Directory of system calls, man section 2

Anatomy of a system call, part 1 and part 2

syscalls _ltp

💾 Historical

ULK3 Chapter 10. System Calls

Device files

Classic UNIX devices are Char devices used as byte streams with man 2 ioctl.

⚲ API:

ls /dev
cat /proc/devices 
cat /proc/misc

Examples: misc_fops _id usb_fops _id memory_fops _id

Allocated devices _doc

drivers/char _src - actually byte stream devices

Chapter 13. I/O Architecture and Device Drivers

hiddev

⚠️ Warning: confusion. hiddev isn't real human interface device! It reuses USBHID infrastructure. hiddev is used for example for monitor controls and Uninterruptible Power Supplies. This module supports these devices separately using a separate event interface on /dev/usb/hiddevX (char 180:96 to 180:111) (⚙️ HIDDEV_MINOR_BASE _id)

⚲ API:

uapi/linux/hiddev.h _inc

HID_CONNECT_HIDDEV _id

⚙️ Internals:

CONFIG_USB_HIDDEV

linux/hiddev.h _inc

hiddev_event _id

drivers/hid/usbhid/hiddev.c _src, hiddev_fops _id

📚 References:

HIDDEV - Care and feeding of your Human Interface Devices _doc

📚 References:

Device file

Administration

🔧 TODO

📚 References

man 7 netlink

The Linux kernel user’s and administrator’s guide _doc

procfs

The proc filesystem (procfs) is a special filesystem that presents information about processes and other system information in a hierarchical file-like structure, providing a more convenient and standardized method for dynamically accessing process data held in the kernel than traditional tracing methods or direct access to kernel memory. Typically, it is mapped to a mount point named /proc at boot time. The proc file system acts as an interface to internal data structures in the kernel. It can be used to obtain information about the system and to change certain kernel parameters at runtime.

/proc includes a directory for each running process —including kernel threads— in directories named /proc/PID, where PID is the process number. Each directory contains information about one process, including the command that originally started the process (/proc/PID/cmdline), the names and values of its environment variables (/proc/PID/environ), a symlink to its working directory (/proc/PID/cwd), another symlink to the original executable file —if it still exists— (/proc/PID/exe), a couple of directories with symlinks to each open file descriptor (/proc/PID/fd) and the status —position, flags, ...— of each of them (/proc/PID/fdinfo), information about mapped files and blocks like heap and stack (/proc/PID/maps), a binary image representing the process's virtual memory (/proc/PID/mem), a symlink to the root path as seen by the process (/proc/PID/root), a directory containing hard links to any child process or thread (/proc/PID/task), basic information about a process including its run state and memory usage (/proc/PID/status) and much more.

📚 References

procfs

man 5 procfs

man 7 namespaces

man 7 capabilities

linux/proc_fs.h _inc

fs/proc _src

sysfs

sysfs is a pseudo-file system that exports information about various kernel subsystems, hardware devices, and associated device drivers from the kernel's device model to user space through virtual files. In addition to providing information about various devices and kernel subsystems, exported virtual files are also used for their configuring. Sysfs is designed to export the information present in the device tree, which would then no longer clutter up procfs.

Sysfs is mounted under the /sys mount point.

⚲ API:

linux/sysfs.h _inc

📚 References

sysfs

man 5 sysfs

sysfs - The filesystem for exporting kernel objects _doc

fs/sysfs _src

devtmpfs

devtmpfs is a hybrid kernel/userspace approach of a device filesystem to provide nodes before udev runs for the first time.

📚 References

Device file

drivers/base/devtmpfs.c _src

Driver Model

or Device Model, or just DM. DM core structure consists of DM classes, DM buses, DM drivers and DM devices.

🔧 TODO

⚲ Infrastructure API:

linux/kobject.h _inc

🔧 TODO

Classes

A class is a higher-level view of a device that abstracts out low-level implementation details. Drivers may see a NVME storage or a SATA storage, but, at the class level, they are all simply block_class _id devices. Classes allow user space to work with devices based on what they do, rather than how they are connected or how they work. General DM classes structure match composite pattern.

⚲ API:

ls /sys/class/

class_register _id registers class _id

linux/device/class.h _inc

👁 Examples: input_class _id, block_class _id net_class _id

Buses

A peripheral bus is a channel between the processor and one or more peripheral devices. A DM bus is proxy for a peripheral bus. General DM buses structure match composite pattern. For the purposes of the device model, all devices are connected via a bus, even if it is an internal, virtual, platform_bus_type _id. Buses can plug into each other. A USB controller is usually a PCI device, for example. The device model represents the actual connections between buses and the devices they control. A bus is represented by the bus_type _id structure. It contains the name, the default attributes, the bus' methods, PM operations, and the driver core's private data.

⚲ API:

ls /sys/bus/

bus_register _id registers bus_type _id

linux/device/bus.h _inc

👁 Examples: usb_bus_type _id, hid_bus_type _id, pci_bus_type _id, scsi_bus_type _id, platform_bus_type _id

Peripheral buses

Drivers

⚲ API:

ls /sys/bus/:/drivers/

module_driver _id - simple common driver initializer, 👁 for example used in module_pci_driver _id

driver_register _id registers device_driver _id - basic device driver structure, one per all device instances.

linux/device/driver.h _inc

👁 Examples: hid_generic _id usb_register_device_driver _id

Platform drivers

module_platform_driver _id registers platform_driver _id (platform wrapper of device_driver _id) with platform_bus_type _id

linux/platform_device.h _inc

👁 Examples: gpio_mouse_device_driver _id

Devices

⚲ API:

ls /sys/devices/

device_register _id registers device _id - the basic device structure, per each device instance

linux/device.h _inc

linux/dev_printk.h _inc

👁 Examples: platform_bus _id mousedev_create

Platform devices

platform_device _id - platform wrapper of struct device - the basic device structure _doc, contains resources associated with the devie

it is can be created dynamically automatically by platform_device_register_simple _id or platform_device_alloc _id. Or registered with platform_device_register _id.

platform_device_unregister _id - releases device and associated resources

👁 Examples: add_pcspkr _id

⚲ API: 🔧 TODO

platform_device_info platform_device_id platform_device_register_full platform_device_add

platform_device_add_data platform_device_register_data platform_device_add_resources

attribute_group dev_pm_ops

---

⚙️ Internals:

linux/dev_printk.h _inc

lib/kobject.c _src

drivers/base/platform.c _src

drivers/base/core.c _src

📚 References

Device drivers infrastructure _doc

Everything you never wanted to know about kobjects, ksets, and ktypes _doc

Driver Model _doc

The Linux Kernel Device Model _doc

Platform Devices and Drivers _doc

Linux Device Model, by linux-kernel-labs

Modules

Article about modules

lsmod
cat /proc/modules

kernel/kmod.c _src

📚 References

LDD3: Building and Running Modules

http://www.xml.com/ldd/chapter/book/ch02.html

http://www.tldp.org/LDP/tlk/modules/modules.html

http://www.tldp.org/LDP/lkmpg/2.6/html/ The Linux Kernel Module Programming Guide

Peripheral buses

⚲ API:

Shell interface: ls /proc/bus/ /sys/bus/

See also Buses of Driver Model

See Input: keyboard, mouse etc

PCI

⚲ Shell API:

lspci -vv

column -t /proc/bus/pci/devices

Main article: PCI

USB

⚲ Shell API:

lsusb -v

ls /sys/bus/usb/

cat /proc/bus/usb/devices

⚙️ Internals:

drivers/usb _src

📚 References:

USB _doc

LDD3:USB Drivers

Other buses

drivers/bus _src

Buses for 🤖 embedded devices:

linux/gpio/driver.h _inc linux/gpio.h _inc drivers/gpio _src tools/gpio _src

drivers/i2c _src https://i2c.wiki.kernel.org

SPI

⚲ API:

linux/spi/spi.h _inc

tools/spi _src

⚙️ Internals:

drivers/spi _src

spi_register_controller _id

spi_controller_list _id🚧

📚 References:

SPI _doc

Hardware interfaces

Interrupts

I/O ports and registers

⚲ API:

linux/regmap.h _inc — register map access API

asm-generic/io.h _inc — generic I/O port emulation.

ioport_map _id

ioread32 _id / iowrite32 _id ...

readl _id/ writel _id ...

The {in,out}[bwl] macros are for emulating x86-style PCI/ISA IO space:

inl _id/ outl _id ...

linux/ioport.h _inc — definitions of routines for detecting, reserving and allocating system resources.

request_mem_region _id

arch/x86/include/asm/io.h _src

Functions for memory mapped registers:

ioremap _id ...

Hardware Device Drivers

Keywords: firmware, hotplug, clock, mux, pin

⚙️ Internals:

drivers/acpi _src

drivers/base _src

drivers/sdio _src - Secure Digital Input Output

drivers/virtio _src

drivers/hwmon _src

drivers/thermal _src

drivers/pinctrl _src

drivers/clk _src

📚 References

Pin control subsystem _doc

Linux Hardware Monitoring _doc

Firmware guide _doc

Devicetree _doc

https://hwmon.wiki.kernel.org/

LDD3:The Linux Device Model

http://www.tldp.org/LDP/tlk/dd/drivers.html

http://www.xml.com/ldd/chapter/book/

http://examples.oreilly.com/linuxdrive2/

Booting and halting

Kernel booting

This is loaded in two stages - in the first stage the kernel (as a compressed image file) is loaded into memory and decompressed, and a few fundamental functions such as essential hardware and basic memory management (memory paging) are set up. Control is then switched one final time to the main kernel start process calling start_kernel _id, which then performs the majority of system setup (interrupts, the rest of memory management, device and driver initialization, etc.) before spawning separately, the idle process and scheduler, and the init process (which is executed in user space).

Kernel loading stage

The kernel as loaded is typically an image file, compressed into either zImage or bzImage formats with zlib. A routine at the head of it does a minimal amount of hardware setup, decompresses the image fully into high memory, and takes note of any RAM disk if configured. It then executes kernel startup via startup_64 (for x86_64 architecture).

arch/x86/boot/compressed/vmlinux.lds.S _src - linker script defines entry startup_64 _id in

arch/x86/boot/compressed/head_64.S _src - assembly of extractor

extract_kernel _id - extractor in language C

prints

Decompressing Linux... done.
Booting the kernel.

Kernel startup stage

The startup function for the kernel (also called the swapper or process 0) establishes memory management (paging tables and memory paging), detects the type of CPU and any additional functionality such as floating point capabilities, and then switches to non-architecture specific Linux kernel functionality via a call to start_kernel _id.

↯ Startup call hierarchy:

arch/x86/kernel/vmlinux.lds.S _src – linker script

arch/x86/kernel/head_64.S _src – assembly of uncompressed startup code

arch/x86/kernel/head64.c _src – platform depended startup:

x86_64_start_kernel _id

x86_64_start_reservations _id

init/main.c _src – main initialization code

start_kernel _id 200 SLOC

mm_init _id

mem_init _id

vmalloc_init _id

sched_init _id

rcu_init _id – Read-copy-update

rest_init _id

kernel_init _id - deferred kernel thread #1

kernel_init_freeable _id This and following functions are defied with attribute __init _id

prepare_namespace _id

initrd_load _id

mount_root _id

run_init_process _id obviously runs the first process man 1 init

kthreadd _id – deferred kernel thread #2

cpu_startup_entry _id

do_idle _id

start_kernel _id executes a wide range of initialization functions. It sets up interrupt handling (IRQs), further configures memory, starts the man 1 init process (the first user-space process), and then starts the idle task via cpu_startup_entry _id. Notably, the kernel startup process also mounts the initial ramdisk (initrd) that was loaded previously as the temporary root file system during the boot phase. The initrd allows driver modules to be loaded directly from memory, without reliance upon other devices (e.g. a hard disk) and the drivers that are needed to access them (e.g. a SATA driver). This split of some drivers statically compiled into the kernel and other drivers loaded from initrd allows for a smaller kernel. The root file system is later switched via a call to man 8 pivot_root / man 2 pivot_root which unmounts the temporary root file system and replaces it with the use of the real one, once the latter is accessible. The memory used by the temporary root file system is then reclaimed.

⚙️ Internals:

arch/x86/Kconfig.debug _src

📚 References:

Article about booting of the kernel

Initial RAM disk _doc

Linux startup process

init

Linux (U)EFI boot process

The kernel’s command-line parameters _doc

Boot time memory management _doc

Kernel booting process

Kernel initialization process

💾 Historical

http://tldp.org/HOWTO/Linux-i386-Boot-Code-HOWTO/

http://www.tldp.org/LDP/lki/lki-1.html

http://www.tldp.org/HOWTO/KernelAnalysis-HOWTO-4.html

Halting or rebooting

🔧 TODO

⚲ API:

linux/reboot.h _inc

reboot_mode _id

sys_reboot _id calls

machine_restart _id or

machine_halt _id or

machine_power_off _id

linux/reboot-mode.h _inc

reboot_mode_driver _id

devm_reboot_mode_register _id

⚙️ Internals:

kernel/reboot.c _src

arch/x86/kernel/reboot.c _src

Softdog Driver

Power management

Keyword: suspend, alarm, hibernation.

⚲ API:

/sys/power/

/sys/kernel/debug/wakeup_sources

⌨ hands-on:

sudo awk '{gsub("^ ","?")} NR>1 {if ($6) {print $1}}' /sys/kernel/debug/wakeup_sources

linux/pm.h _inc

linux _inc

linux/pm_qos.h _inc

linux/pm_clock.h _inc

linux/pm_domain.h _inc

linux/pm_wakeirq.h _inc

linux/pm_wakeup.h _inc

wakeup_source _id

wakeup_source_register _id

linux/suspend.h _inc

pm_suspend _id suspends the system

Suspend and wakeup depend on

man 2 timer_create and man 2 timerfd_create with clock ids CLOCK_REALTIME_ALARM _id or CLOCK_BOOTTIME_ALARM _id will wake the system if it is suspended.

man 2 epoll_ctl with flag EPOLLWAKEUP _id blocks suspend

See also man 7 capabilities CAP_WAKE_ALARM _id, CAP_BLOCK_SUSPEND _id

⚙️ Internals:

CONFIG_PM _id

CONFIG_SUSPEND _id

kernel/power _src

alarm_init _id

kernel/time/alarmtimer.c _src

drivers/base/power _src: wakeup_sources _id

📚 References:

PM administration _doc

CPU and Device PM _doc

Power Management _doc

https://lwn.net/Kernel/Index/#Power_management

PowerTOP

cpupower

tlp – apply laptop power management settings

ACPI – Advanced Configuration and Power Interface

Runtime PM

Keywords: runtime power management, devices power management opportunistic suspend, autosuspend, autosleep.

⚲ API:

/sys/devices/.../power/:

async autosuspend_delay_ms control runtime_active_kids runtime_active_time runtime_enabled runtime_status runtime_suspended_time runtime_usage

linux/pm_runtime.h _inc

pm_runtime_mark_last_busy _id

pm_runtime_enable _id

pm_runtime_disable _id

pm_runtime_get _id – asynchronous get

pm_runtime_get_sync _id

pm_runtime_resume_and_get _id – preferable synchronous get

pm_runtime_put _id

pm_runtime_put_noidle _id – just decrement usage counter

pm_runtime_put_sync _id

pm_runtime_put_autosuspend _id

SET_RUNTIME_PM_OPS _id

👁 Example: ac97_pm _id

⚙️ Internals:

CONFIG_PM_AUTOSLEEP _id

📚 References:

Runtime Power Management Framework for I/O Devices _doc

Sysfs devices PM API

Power Management for USB

Opportunistic suspend

Building and Updating

Updating