human interfaces
text interfaces
security
debugging
multimedia subsystems
human interface devices, input devices
HI drivers

Welcome to the first article of the book. The article is named after USB class and Linux facility Human Interface Devices (HID). HID facility in Linux supports keyboard, mouse and other input devices. Console, multimedia (or just media), sound (audio), video, graphics also are in the focus of this article. Additionally, this chapter covers security and debugging topics as they are closely linked to user and human interactions.



Text interfaces edit

In the world of Linux, a text terminal emulators and consoles are essential components of the operating system that allow users to interact with applications trough kernel. A text terminal is a device that provides a text-based interface for communicating with the kernel, while the console is the physical device that houses the terminal and displays the output of the kernel.

The Linux kernel includes a built-in console driver that provides a basic interface for communicating with the console and controlling the terminal. The console driver also supports various input and output devices, such as keyboards and displays, to enable users to interact with the system through a terminal.

The use of text terminals and system consoles in Linux can be traced back to the early days of computing, when graphical user interfaces were not yet widely available. Despite the widespread adoption of graphical user interfaces, text-based interfaces and consoles remain popular among Linux users and developers for their simplicity, efficiency, and flexibility. Overall, the text terminal and console play a crucial role in the Linux kernel, providing users with a powerful interface for managing and interacting with the operating system.


Char devices edit

cdev id – "character device" is a type of device driver that provides an implementation for character device file in the /dev directory. The word "device" here means abstract interface, proxy to a usually peripheral physical device. A character device is a type of device that can be accessed as a stream of bytes, rather than as a block of data like a block device. Cdev drivers are commonly used for devices that provide a stream of data, such as keyboards, mouses, terminals, serial ports, and printers. They are also used for devices that provide access to memory-mapped I/O regions, such as frame buffers and network devices. A cdev driver typically consists of a set of functions that implement the low-level I/O operations for the device, such as open, read and write. These functions are called by the kernel when a user space program accesses the character device file. To create a cdev driver, a kernel developer must first initialize a cdev structure using cdev_init id or cdev_alloc id. The cdev structure contains information about the device, such as its major and minor numbers and the set of I/O functions that the driver implements. Once the cdev structure has been initialized, it can be registered with the kernel using the cdev_add id function. This function creates the character device file in the /dev directory and associates it with the cdev driver.

You can find a list of registered char devices on the beginning the listing of /proc/devices. Input devices keyboard and mouse are examples of char devices.

Tip: Browse the cross-referencing site to explore nearby API and use cases


πŸ’Ύ Historical: It is one of the most simple, fundamental and oldest concepts derived from UNIX.


⚲ API:

linux/cdev.h inc:
dev_t id - device id consists of MAJOR id and MINOR id numbers
cdev id - core char device struct
cdev_init id or cdev_alloc id
cdev_device_add id - helper function, uses:
cdev_add id - common key function to add a char device to the system.
register_chrdev id - obviously registers char device by major number, name and file operations
unregister_chrdev id
alloc_chrdev_region id / register_chrdev_region id,
unregister_chrdev_region id
uapi/linux/major.h inc - static definitions of many major numbers, including obsolete.


βš™οΈ Internals:

fs/char_dev.c src
chrdevs id


πŸ“– References

Char devices doc
Character device drivers, linux-kernel-labs
Character device files, on opensourceforu


πŸ’Ύ Historical

LDD3:Char Drivers
LDD3:Advanced Char Driver Operations
LDD1:#3
LDD1:#5

Text terminals and console edit

πŸ—οΈ Acronyms:

tty - πŸ’Ύ historically TeleTYpewriter, means just terminal
pty - pseudoterminal
pts - pseudoterminal slave
ptmx - pseudoterminal master


⚲ API:

To find out current terminal:
readlink /proc/self/fd/0
man 1 tty
man 1 who -m
linux/tty.h inc
register_console id obviously registers console id
πŸ‘ example virtio_console id
linux/console.h inc
man 2 ioctl_console


βš™οΈ Internals:

drivers/tty src
fs/devpts src
fs/proc/proc_tty.c src
drivers/tty/vt/vt.c src


πŸ“– References

man 4 tty – controlling terminal
man 4 ptmx and pts – pseudoterminal master and slave
man 7 pty – pseudoterminal interfaces
console doc


πŸ’Ύ Historical:

LDD3:TTY Drivers

Security edit

The goal of security is to restrict access through interfaces. From access control and authentication mechanisms to secure boot and memory protection, the Linux kernel employs a variety of techniques to safeguard the system and its users. Basic Linux security is quite simple. It consists of tree ownership classes and tree access modes. One of the most frequently executed functions is may_open id. It rejects access of unauthorized users to open a file.


See article Security for new features.


Authorization edit

Authorization is the function of specifying access rights/privileges to system resources. The main goal of authorization is prevention of privilege escalation under any circumstances.

πŸ”§ TODO. Keywords: permission, capabilities, ownership, mitigation.


⚲ API:

linux/stat.h inc
uapi/linux/stat.h inc


Basic classic UNIX authorization is based on ownership and tree access modes: reading, writing and execution.

Ownership is encoded by owning user id uid_t id and owning group id gid_t id.

umode_t id - just typedef used for encoding access mode. S_IRUSR id - minimal "read only by user/owner" access mode. S_IALLUGO id - full access mode. Please read the source for details for other modes.


Binary Access Control Matrix of access modes:

modes bits Read Write Execute
bit offset 2 1 0
Others 0-2 or ow ox
Group 3-5 gr gw gx
User 6-8 ur uw ux


man 2 chown β†ͺ do_fchownat id changes ownership for file or directory
man 2 chmod β†ͺ do_fchmodat id changes access mode for file or directory
man 2 access, man 2 faccessat β†ͺ do_faccessat id checks access rights


Common authorization errors:

EPERM id – "Operation not permitted"
EACCES id – "Permission denied"


πŸš€ Advanced features:

man 5 acl posix_acl id
uapi/linux/capability.h inc
man 2 capset and capget – set/get capabilities of thread(s)
man 3 libcap
man 1 setpriv – run a program with different privilege settings

βš™οΈ Internals:

may_open id rejects unauthorized file opening
inode_permission id checks for access rights to a given inode
kernel/capability.c src


πŸ“– References

File-system permissions
man 7 capabilities

Credentials edit

πŸ”§ TODO. Keywords: authentication, user IDs, group IDs, Process group ID, session ID.


⚲ API:

uapi/asm-generic/stat.h inc
arch/x86/include/uapi/asm/stat.h src
linux/cred.h inc
struct cred id - the security context of a task
man 1 id, man 1 test - shell utilities
man 2 getuid β†ͺ current_uid id
man 2 getgid
man 2 geteuid is used by utility man 1 whoami
Real, effective, and saved user/group IDs:
man 2 getresuid, getresgid
man 2 setreuid, setregid
man 2 setfsuid - set user identity used for filesystem checks
man 2 umask - sets file mode creation mask
man 1 stat, man 2 stat β†ͺ vfs_fstat id, vfs_fstatat id
man 2 statx β†ͺ do_statx id


βš™οΈ Internals:

kstat id
make_kuid id etc
from_kuid_munged id etc


πŸ“– References

Credentials in Linux doc
man 7 credentials
https://www.geeksforgeeks.org/real-effective-and-saved-userid-in-linux/

Cryptography edit

πŸ”§ TODO


πŸ—οΈ Acronyms:

AES - Advanced Encryption Standard


⚲ API:

AF_ALG id - User Space Interface doc
linux/crypto.h inc - Scatterlist Cryptographic API.
crypto inc


βš™οΈ Internals:

crypto src
drivers/crypto src
lib/crypto src
arch/x86/crypto src
fs/crypto src - per-file encryption
fs/ecryptfs src eCrypt FS - Encrypted filesystem that operates on the VFS layer.
dm-crypt, drivers/md/dm-crypt.c src


πŸ“– References

Linux Kernel Crypto API doc
Crypto API (Linux)
devicetree/bindings/crypto
crypto ltp


Audit edit

kernel/audit.h src
kernel/audit.c src
kernel/auditsc.c src
kernel/audit_tree.c src
kernel/audit_watch.c src
kernel/audit_fsnotify.c src
kernel/auditfilter.c src


πŸ“– References

https://capsule8.com/blog/auditd-what-is-the-linux-auditing-system/
https://wiki.archlinux.org/title/Audit_framework
man 8 auditctl


See also eBPF and BPF


Appendix for Security:


πŸ”§ TODO:

man 2 fcntl β†ͺ do_fcntl id
man 2 seccomp β†ͺ do_seccomp id
man 2 add_key β†ͺ security/keys/keyctl.c src
chroot, man 2 chroot
Address space layout randomization
man 8 setarch / man 2 personality


πŸ“– References

Security doc
LSM - Linux Security Modules doc Perf events and tool security
Hardware vulnerabilities doc
Linux Security Modules
linux/security.h inc β‡Ύ security src
keys inc
linux/verification.h inc
certs src
security ltp
cve ltp
http://kernsec.org/wiki/index.php/Main_Page
SELinux http://selinuxproject.org/

Debugging edit

See Debugging Linux kernel

Multimedia subsystems edit

Graphics edit

Old graphics (not to be confused with v4l):

⚲ API:

video inc

βš™οΈ Internals:

drivers/video src

Direct Rendering Manager (DRM) edit

DRM is responsible for interfacing with GPUs of modern video cards. DRM exposes an API that user-space programs can use to send commands and data to the GPU and perform operations such as configuring the mode setting of the display. User-space programs can use the DRM API to command the GPU to do hardware-accelerated 3D rendering and video decoding, as well as GPGPU computing.


⚲ API:

/sys/class/drm/
uapi/drm inc
uapi/drm/drm.h inc
DRM_IOCTL_BASE id
drm_version id

βš™οΈ Internals:

drm inc
drm_gem.h inc – Graphics Execution Manager Driver Interfaces
drm_dev_register id registers drm_device id

Advanced Linux Sound Architecture (ALSA) edit

ALSA is a software framework and part of the Linux kernel that provides an API for sound card device drivers. Some of the goals of the ALSA project at its inception were automatic configuration of sound-card hardware and graceful handling of multiple sound devices in a system.

The sound servers PulseAudio, JACK (low-latency professional-grade audio editing and mixing) and PipeWire, the higher-level abstraction APIs OpenAL, SDL audio, etc. work on top of ALSA and implemented sound card device drivers. On Linux systems, ALSA succeeded the older Open Sound System (OSS).


⚲ API:

/proc/asound/cards, /sys/class/sound/
snd_card id - central struct
snd_card_new id
snd_card_register id
snd_device_ops id
snd_device_new id creates an ALSA device component
uapi/sound/asound.h inc
sound/core.h inc


βš™οΈ Internals:

sound src
sound/core/device.c src
See ASoC


πŸ“– References

ALSA (sound) doc
Writing an ALSA Driver doc
sound ltp

Video4Linux (V4L2) edit

V4L is a collection of device drivers and an API for supporting realtime video capture on Linux systems. It supports many USB webcams, TV tuners, and related devices, standardizing their output, so programmers can easily add video support to their applications. MythTV, tvtime and Tvheadend are typical applications that use the V4L framework.


⚲ API:

v4l2_device_register id registers v4l2_device id
video_register_device id registers video_device id
πŸ‘ examples drivers/media/test-drivers src


πŸ“– References

Video4Linux
media doc
V4L doc
Media subsystem kernel internal API

HID edit

Generic human interface devices. Don't confuse with hiddev.

Input devices edit

Input device files are kind of char devices with id INPUT_MAJOR id. Classic input devices are keyboard and mouse.


⚲ API:

In shell: cat /proc/bus/input/devices
linux/input.h inc
devm_input_allocate_device id, input_register_device id, input_register_handler id, input_dev id
input_report_key id input_sync id


πŸ‘ Examples:

drivers/input/mousedev.c src
drivers/input/keyboard/atkbd.c src
drivers/input/evbug.c src


⌨️ Hands on:

sudo hexdump /dev/input/mice # dump your mouse movements events from your kernel


βš™οΈ Internals:

drivers/input/input.c src
input_event id


πŸ“– References

Input doc
input ltp

HID devices edit

πŸ”§ TODO


⚲ API:

hid_device id - device report descriptor. Operations: hid_allocate_device id, hid_add_device id . πŸ‘ Example usbhid_probe id
uapi/linux/hid.h inc
linux/hid.h inc


Camera edit

πŸ”§ TODO


⚲ API:

uapi/linux/uvcvideo.h inc

πŸ“– References

UVC doc
drivers/media/usb/uvc src

HI device drivers edit

This section is about low level drivers to human interface peripheral devices.


⚲ HID API:

linux/hidraw.h inc
module_hid_driver id registers hid_driver id
hid_hw_start id


βš™οΈ Internals

hid_bus_type id
drivers/hid src
drivers/hid/hid-core.c src
drivers/accessibility src
drivers/leds src
samples/uhid/uhid-example.c src - πŸ‘ example of user mode HID driver
drivers/input srcΒ : keyboard & mouse, misc, serio, tablet, touchscreen, gameport, joystick
⌨️ Hands on:
echo "module atkbd +pfl" | sudo tee /sys/kernel/debug/dynamic_debug/control


USB HID

⚲ HID API:

USB_INTERFACE_CLASS_HID id == USB_CLASS_HID id


βš™οΈ Internals:

drivers/hid/usbhid src
drivers/hid/usbhid/usbkbd.c src: usb_kbd_driver id
drivers/hid/usbhid/usbmouse.c src: usb_mouse_driver id


πŸ“– References

USB HID class doc

Graphics edit

πŸ”§ TODO


πŸ—οΈ Acronyms:

FB - Framebuffer
GPU - Graphics processing unit
TFT (LCD) - Thin-film-transistor liquid-crystal display used for πŸ€– embedded devices
MIPI - πŸ“± Mobile Industry Processor Interface
DBI - Display Bus Interface
DSI - Display Serial Interface
DCS - The Display Command Set


⚲ API:

cat /proc/fb
ls -l /sys/class/graphics
video/mipi_display.h inc
linux/fb.h inc
register_framebuffer id
FBTFT_REGISTER_DRIVER id
fbtft_display id


βš™οΈ Internals

drivers/video src
drivers/gpu src

πŸ‘ Examples:

vivid_fb_init id
fbtft_register_framebuffer id


πŸ“– References

GPU Driver Developer’s Guide doc
The Frame Buffer Device doc
Frame Buffer Library doc
LWN: Graphics

Sound SoC - ASoC edit

ALSA System on Chip (ASoC) layer for or πŸ€– embedded systems. ASoC is designed to handle complex audio processing and routing on low-power and resource-constrained systems, making it an ideal solution for embedded devices such as smartphones, tablets, and other IoT devices.

ASoC provides a comprehensive framework for audio drivers, enabling the creation of modular audio drivers that can be easily integrated with the rest of the kernel. It also supports a wide range of audio interfaces, including I2S, PCM, AC97, and SPDIF, making it highly versatile and capable of handling a variety of audio formats.

One of the key features of ASoC is its ability to handle audio routing and processing using Digital Signal Processing (DSP) techniques.

This enables ASoC to support advanced audio features such as noise reduction, echo cancellation, and dynamic range compression, among others.

Overall, ASoC is a powerful and flexible subsystem that enables Linux to support a wide range of audio hardware in embedded devices. It has become an essential component of many embedded Linux distributions and is widely used in the development of modern audio-enabled devices.

⚲ API:

sound/soc.h inc
snd_soc_card id
is registered by devm_snd_soc_register_card id β‡Ύ snd_soc_register_card id
sound/soc-component.h inc
snd_soc_component id
snd_soc_component_driver id
snd_soc_card id
snd_card id
snd_soc_register_component id snd_soc_component_get_drvdata id snd_soc_component_read id snd_soc_component_update_bits id snd_soc_component_write id
sound/soc-dai.h inc - DAI - Digital Audio Interface doc: AC97, I2S, PCM
snd_soc_dai id snd_soc_dai_driver id snd_soc_dai_get_drvdata id
sound/soc-dpcm.h inc - DPCM - Dynamic PCM doc
sound/soc-dapm.h inc - DAPM - Dynamic Audio Power Management doc
snd_soc_dapm_route id, snd_soc_dapm_to_component id, snd_soc_dapm_widget id


πŸ‘ Examples:

sound/soc/generic/simple-card.c src
sound/soc/generic/audio-graph-card.c src uses sound/graph_card.h inc


βš™οΈ Internals:

sound/soc src
snd_soc_card id
snd_soc_dai_link id


πŸ“– References

ASoC - ALSA SoC Layer doc
ASoC Core API doc
https://www.alsa-project.org/wiki/ASoC
https://www.alsa-project.org/wiki/DAPM

πŸ—οΈ Acronyms SAI could be

STM Serial Audio Interface: sound/soc/stm/stm32_sai.h src
Freescale (FSL) Synchronous Audio Interface: sound/soc/fsl/fsl_sai.h src



βš™οΈ Internals:

drivers/media src