Human interfaces

human interfaces
text interfaces
security
debugging
multimedia subsystems
human interface devices, input devices
HI drivers

Welcome to the first article of the book. The article is named after USB class and Linux facility Human Interface Devices (HID). HID facility in Linux supports keyboard, mouse and other input devices. Console, multimedia (or just media), sound (audio), video, graphics also are in the focus of this article. Additionally, this chapter covers security and debugging topics as they are closely linked to user and human interactions.

Text interfaces

In the world of Linux, a text terminal emulators and consoles are essential components of the operating system that allow users to interact with applications trough kernel. A text terminal is a device that provides a text-based interface for communicating with the kernel, while the console is the physical device that houses the terminal and displays the output of the kernel.

The Linux kernel includes a built-in console driver that provides a basic interface for communicating with the console and controlling the terminal. The console driver also supports various input and output devices, such as keyboards and displays, to enable users to interact with the system through a terminal.

The use of text terminals and system consoles in Linux can be traced back to the early days of computing, when graphical user interfaces were not yet widely available. Despite the widespread adoption of graphical user interfaces, text-based interfaces and consoles remain popular among Linux users and developers for their simplicity, efficiency, and flexibility. Overall, the text terminal and console play a crucial role in the Linux kernel, providing users with a powerful interface for managing and interacting with the operating system.

Char devices

cdev_id – "character device" is a type of device driver that provides an implementation for character device file in the /dev directory. The word "device" here means abstract interface, proxy to a usually peripheral physical device. A character device is a type of device that can be accessed as a stream of bytes, rather than as a block of data like a block device. Cdev drivers are commonly used for devices that provide a stream of data, such as keyboards, mouses, terminals, serial ports, and printers. They are also used for devices that provide access to memory-mapped I/O regions, such as frame buffers and network devices. A cdev driver typically consists of a set of functions that implement the low-level I/O operations for the device, such as open, read and write. These functions are called by the kernel when a user space program accesses the character device file. To create a cdev driver, a kernel developer must first initialize a cdev structure using cdev_init_id or cdev_alloc_id. The cdev structure contains information about the device, such as its major and minor numbers and the set of I/O functions that the driver implements. Once the cdev structure has been initialized, it can be registered with the kernel using the cdev_add_id function. This function creates the character device file in the /dev directory and associates it with the cdev driver.

You can find a list of registered char devices on the beginning the listing of /proc/devices. Input devices keyboard and mouse are examples of char devices.

Tip: Browse the cross-referencing site to explore nearby API and use cases

💾 Historical: It is one of the most simple, fundamental and oldest concepts derived from UNIX.

⚲ API

linux/cdev.h_inc

dev_t_id - device id consists of MAJOR_id and MINOR_id numbers

cdev_id - core char device struct

cdev_init_id or cdev_alloc_id

cdev_device_add_id - helper function, uses

cdev_add_id - common key function to add a char device to the system.

register_chrdev_id - obviously registers char device by major number, name and file operations

unregister_chrdev_id

alloc_chrdev_region_id / register_chrdev_region_id,

unregister_chrdev_region_id

uapi/linux/major.h_inc - static definitions of many major numbers, including obsolete.

⚙️ Internals

fs/char_dev.c_src

chrdevs_id

📖 References

Char devices_doc

Character device drivers, linux-kernel-labs

Character device files, on opensourceforu

💾 Historical

LDD3:Char Drivers

LDD3:Advanced Char Driver Operations

LDD1:#3

LDD1:#5

Text terminals and console

🗝️ Acronyms

tty - 💾 historically TeleTYpewriter, means just terminal

pty - pseudoterminal

pts - pseudoterminal slave

ptmx - pseudoterminal master

⚲ API

To find out current terminal:

readlink /proc/self/fd/0

man 1 tty

man 1 who -m

linux/tty.h_inc

register_console_id obviously registers console_id

👁 example virtio_console_id

⚙️ Internals

fs/proc/proc_tty.c_src

drivers/tty/vt/vt.c_src

📖 References

man 4 tty – controlling terminal

man 4 ptmx and pts – pseudoterminal master and slave

man 7 pty – pseudoterminal interfaces

console_doc

💾 Historical

LDD3:TTY Drivers

Security

The goal of security is to restrict access through interfaces. From access control and authentication mechanisms to secure boot and memory protection, the Linux kernel employs a variety of techniques to safeguard the system and its users. Basic Linux security is quite simple. It consists of tree ownership classes and tree access modes. One of the most frequently executed functions is may_open_id. It rejects access of unauthorized users to open a file.

See article Security for new features.

Authorization

Authorization is the function of specifying access rights/privileges to system resources. The main goal of authorization is prevention of privilege escalation under any circumstances.

🔧 TODO. Keywords: permission, capabilities, ownership, mitigation.

⚲ API

linux/stat.h_inc

uapi/linux/stat.h_inc

Basic classic UNIX authorization is based on ownership and tree access modes: reading, writing and execution.

Ownership is encoded by owning user id uid_t_id and owning group id gid_t_id.

umode_t_id - just typedef used for encoding access mode. S_IRUSR_id - minimal "read only by user/owner" access mode. S_IALLUGO_id - full access mode. Please read the source for details for other modes.

Binary Access Control Matrix of access modes:


modes	bits	Read	Write	Execute
bit offset		2	1	0
Others	0-2	or	ow	ox
Group	3-5	gr	gw	gx
User	6-8	ur	uw	ux

man 2 chown ↪ do_fchownat_id changes ownership for file or directory

man 2 chmod ↪ do_fchmodat_id changes access mode for file or directory

man 2 access, man 2 faccessat ↪ do_faccessat_id checks access rights

Common authorization errors

EPERM_id – "Operation not permitted"

EACCES_id – "Permission denied"

🚀 Advanced features

man 5 acl posix_acl_id

uapi/linux/capability.h_inc

man 2 capset and capget – set/get capabilities of thread(s)

man 3 libcap

man 1 setpriv – run a program with different privilege settings

⚙️ Internals

may_open_id rejects unauthorized file opening

inode_permission_id checks for access rights to a given inode

kernel/capability.c_src

📖 References

File-system permissions

man 7 capabilities

Credentials

🔧 TODO. Keywords: authentication, user IDs, group IDs, Process group ID, session ID.

⚲ API

uapi/asm-generic/stat.h_inc

arch/x86/include/uapi/asm/stat.h_src

linux/cred.h_inc

struct cred_id - the security context of a task

man 1 id, man 1 test - shell utilities

man 2 getuid ↪ current_uid_id

man 2 getgid

man 2 geteuid is used by utility man 1 whoami

Real, effective, and saved user/group IDs:

man 2 getresuid, getresgid

man 2 setreuid, setregid

man 2 setfsuid - set user identity used for filesystem checks

man 2 umask - sets file mode creation mask

man 1 stat, man 2 stat ↪ vfs_fstat_id, vfs_fstatat_id

man 2 statx ↪ do_statx_id

⚙️ Internals

kstat_id

make_kuid_id etc

from_kuid_munged_id etc

📖 References

Credentials in Linux_doc

man 7 credentials

https://www.geeksforgeeks.org/real-effective-and-saved-userid-in-linux/

Cryptography

🔧 TODO

🗝️ Acronyms

AES - Advanced Encryption Standard

⚲ API

AF_ALG_id - User Space Interface_doc

linux/crypto.h_inc - Scatterlist Cryptographic API.

⚙️ Internals

fs/crypto_src - per-file encryption

fs/ecryptfs_src eCrypt FS - Encrypted filesystem that operates on the VFS layer.

dm-crypt, drivers/md/dm-crypt.c_src

📖 References

Linux Kernel Crypto API_doc

Crypto API (Linux)

devicetree/bindings/crypto

crypto_ltp

Audit

kernel/audit.h_src

kernel/audit.c_src

kernel/auditsc.c_src

kernel/audit_tree.c_src

kernel/audit_watch.c_src

kernel/audit_fsnotify.c_src

kernel/auditfilter.c_src

📖 References

https://capsule8.com/blog/auditd-what-is-the-linux-auditing-system/

https://wiki.archlinux.org/title/Audit_framework

man 8 auditctl

Debugging

See Debugging Linux kernel

Multimedia subsystems

⚙️ Internals

drivers/media_src

Graphics

Old graphics (not to be confused with v4l):

⚲ API

video_inc

⚙️ Internals

drivers/video_src

Direct Rendering Manager (DRM)

DRM is responsible for interfacing with GPUs of modern video cards. DRM exposes an API that user-space programs can use to send commands and data to the GPU and perform operations such as configuring the mode setting of the display. User-space programs can use the DRM API to command the GPU to do hardware-accelerated 3D rendering and video decoding, as well as GPGPU computing.

⚲ API

/sys/class/drm/

⚙️ Internals

drm_gem.h_inc – Graphics Execution Manager Driver Interfaces

drm_dev_register_id registers drm_device_id

Advanced Linux Sound Architecture (ALSA)

ALSA is a software framework and part of the Linux kernel that provides an API for sound card device drivers. Some of the goals of the ALSA project at its inception were automatic configuration of sound-card hardware and graceful handling of multiple sound devices in a system.

The sound servers PulseAudio, JACK (low-latency professional-grade audio editing and mixing) and PipeWire, the higher-level abstraction APIs OpenAL, SDL audio, etc. work on top of ALSA and implemented sound card device drivers. On Linux systems, ALSA succeeded the older Open Sound System (OSS).

⚲ API

/proc/asound/cards, /sys/class/sound/

snd_card_id - central struct

snd_card_new_id

snd_card_register_id

snd_device_ops_id

snd_device_new_id creates an ALSA device component

uapi/sound/asound.h_inc

sound/core.h_inc

⚙️ Internals

sound_src

sound/core/device.c_src

See ASoC

📖 References

ALSA (sound)_doc

Writing an ALSA Driver_doc

sound_ltp

Video4Linux (V4L2)

V4L is a collection of device drivers and an API for supporting realtime video capture on Linux systems. It supports many USB webcams, TV tuners, and related devices, standardizing their output, so programmers can easily add video support to their applications. MythTV, tvtime and Tvheadend are typical applications that use the V4L framework.

⚲ API

v4l2_device_register_id registers v4l2_device_id

video_register_device_id registers video_device_id

👁 examples drivers/media/test-drivers_src

📖 References

Video4Linux

media_doc

V4L_doc

Media subsystem kernel internal API

HID

Generic human interface devices. Don't confuse with hiddev.

Input devices

Input device files are kind of char devices with id INPUT_MAJOR_id. Classic input devices are keyboard and mouse.

⚲ API

In shell: cat /proc/bus/input/devices

linux/input.h_inc

devm_input_allocate_device_id, input_register_device_id, input_register_handler_id, input_dev_id

input_report_key_id input_sync_id

👁 Examples

drivers/input/mousedev.c_src

drivers/input/keyboard/atkbd.c_src

drivers/input/evbug.c_src

⌨️ Hands onInternals

sudo hexdump /dev/input/mice # dump your mouse movements events from your kernel

⚙️ Internals

drivers/input/input.c_src

input_event_id

📖 References

Input_doc

input_ltp

HID devices

🔧 TODO

⚲ API

hid_device_id - device report descriptor. Operations: hid_allocate_device_id, hid_add_device_id . 👁 Example usbhid_probe_id

uapi/linux/hid.h_inc

linux/hid.h_inc

Camera

🔧 TODO

⚲ API

uapi/linux/uvcvideo.h_inc

📖 References

UVC_doc

drivers/media/usb/uvc_src

HI device drivers

This section is about low level drivers to human interface peripheral devices.

⚲ HID API

linux/hidraw.h_inc

module_hid_driver_id registers hid_driver_id

hid_hw_start_id

⚙️ Internals

hid_bus_type_id

drivers/hid_src

drivers/hid/hid-core.c_src

drivers/accessibility_src

drivers/leds_src

samples/uhid/uhid-example.c_src - 👁 example of user mode HID driver

drivers/input_src : keyboard & mouse, misc, serio, tablet, touchscreen, gameport, joystick

⌨️ Hands on

echo "module atkbd +pfl" | sudo tee /sys/kernel/debug/dynamic_debug/control

USB HID

⚲ HID API

USB_INTERFACE_CLASS_HID_id == USB_CLASS_HID_id

⚙️ Internals

drivers/hid/usbhid_src

drivers/hid/usbhid/usbkbd.c_src: usb_kbd_driver_id

drivers/hid/usbhid/usbmouse.c_src: usb_mouse_driver_id

📖 References

USB HID class_doc

Graphics

🔧 TODO

🗝️ Acronyms

FB - Framebuffer

GPU - Graphics processing unit

TFT (LCD) - Thin-film-transistor liquid-crystal display used for 🤖 embedded devices

MIPI - 📱 Mobile Industry Processor Interface

DBI - Display Bus Interface

DSI - Display Serial Interface

DCS - The Display Command Set

⚲ API

cat /proc/fb

ls -l /sys/class/graphics

video/mipi_display.h_inc

linux/fb.h_inc

register_framebuffer_id

FBTFT_REGISTER_DRIVER_id

⚙️ Internals

👁 Examples

fbtft_register_framebuffer_id

📖 References

GPU Driver Developer’s Guide_doc

The Frame Buffer Device_doc

Frame Buffer Library_doc

LWN: Graphics

Sound SoC - ASoC

ALSA System on Chip (ASoC) layer for or 🤖 embedded systems. ASoC is designed to handle complex audio processing and routing on low-power and resource-constrained systems, making it an ideal solution for embedded devices such as smartphones, tablets, and other IoT devices.

ASoC provides a comprehensive framework for audio drivers, enabling the creation of modular audio drivers that can be easily integrated with the rest of the kernel. It also supports a wide range of audio interfaces, including I2S, PCM, AC97, and SPDIF, making it highly versatile and capable of handling a variety of audio formats.

One of the key features of ASoC is its ability to handle audio routing and processing using Digital Signal Processing (DSP) techniques.

This enables ASoC to support advanced audio features such as noise reduction, echo cancellation, and dynamic range compression, among others.

Overall, ASoC is a powerful and flexible subsystem that enables Linux to support a wide range of audio hardware in embedded devices. It has become an essential component of many embedded Linux distributions and is widely used in the development of modern audio-enabled devices.

⚲ API

sound/soc.h_inc

snd_soc_card_id

is registered by devm_snd_soc_register_card_id ⇾ snd_soc_register_card_id

sound/soc-component.h_inc

snd_soc_component_id

snd_soc_component_driver_id

snd_soc_card_id

snd_card_id