Embedded Linux

Embedded Linux systems can range from consumer electronics like smartphones and smart TVs to industrial machinery, medical devices, automotive systems, communication equipment and more. Linux brings the power, flexibility, and stability to these resource-constrained devices.

Core components of an Linux on embedded systems are boot loader, custom Linux kernel and root file system. Source code is compiled by GCC cross compiler. Common boot loader is Das U-Boot – the Universal Boot Loader. Root FS usually is based on BusyBox – software suite that provides many Unix utilities in a single executable file. Local debugging often is limited, so gdbserver can be used.

Basic embedded Linux system can be compiled manually from modules above. There are tools for complete system generation from old and simple Buildroot to huge Yocto Project, which creates customized embedded Linux distributions. PC distributions Debian, Ubuntu ant others have build for ARM processors.

Raspberry Pi, BeagleBoard and Nvidia Jetson are examples of popular single-board computers supporting embedded Linux.

Unlike regular desktop or server distributions of Linux these versions of Linux usually have much fewer libraries and application, firmware is smaller and typically boot from flash memory or SD card.

Projects

Build frameworks and environments

Wear OS – Android for smartwatches and other wearables.

AOSP – Android Open Source Project

Buildroot – old simple patch and make based build environment

Yocto Project and OpenEmbedded – a BitBake based build framework and environment

BitBake – make-like build tool

OpenWrt – open wireless router

Embeddable Linux Kernel Subset – for ancient 16-bit x86 machines and emulators

Components

The Linux Kernel

"init" implementations: OpenRC, runit

GUIs: LXQt, LXDE, Xfce, FLTK

coreboot – lightweight BIOS

C standard library variants

General purpose GNU C Library glibc, 7.9 MB

uClibc, 560 KB

musl, 527 KB

dietlibc, 185 KB

Newlib

klibc – primarily for booting Linux systems

Bionic – originally developed by Google for the Android embedded system operating system

Distributions

Distributions for ARM processors:

Arch Linux ARM – port of Arch Linux for ARM processors

Armbian – Debian and Ubuntu based

RedSleeve – port of Red Hat Enterprise Linux

emteria.OS – proprietary Android based

Lightweight cloud oriented systems:

Alpine Linux – uses musl, BusyBox and OpenRC.

Fedora CoreOS

RHCOS – Red Hat Enterprise Linux CoreOS for OpenShift Container Platform

https://MicroShift.io/ – lightweight Kubernetes distribution designed for edge and IoT computing environments

Lightweight multi-platform distributions:

Common multi-platform distributions:

Features of Linux kernel for embedded systems

I/O and buses:

GPIO

drivers/gpio_src, tools/gpio_src, General Purpose Input/Output_doc

SPI

drivers/spi_src, tools/spi_src, Serial Peripheral Interface_doc

I²C

⚲ UAPI: Instantiate I2C devices from user-space_doc

drivers/i2c_src, https://i2c.wiki.kernel.org

I2C/SMBus Subsystem_doc

drivers/net/can_src – Controller Area Network bus

drivers/mfd_src – Multi-Function Devices

drivers/usb/gadget_src – peripheral USB device implementation

drivers/iio_src – Industrial I/O

drivers/pwm_src – Pulse-width modulation

drivers/regulator_src – Generic Voltage and Current Regulator support

drivers/fpga_src

drivers/slimbus_src

SLOB – Simple List Of Blocks memory allocator

Devicetree – describes hardware components for the kernel

Multimedia

Audio

ASoC – ALSA System on Chip

snd_soc_card_id

ASoC – ALSA SoC Layer_doc

https://www.alsa-project.org/wiki/ASoC

TFT (LCD) – Thin-film-transistor liquid-crystal display

fbtft_display_id

File systems and storage

UBIFS – Unsorted Block Image FS

SquashFS – a compressed read-only FS with low overhead

drivers/mtd_src – Memory Technology Device block device

More: YAFFS, JFFS2

MMU less

Linux was originally designed on a processor with a memory management unit (MMU). Most embedded systems do not have a MMU, as we discussed earlier (Embedded Systems/Memory) and are called microcontrollers. Microcontrollers without a MMU are cheaper consumes less power.

Benefits of using a processor with a MMU:

can isolate running "untrusted" machine code from running "critical" code, so the "untrusted" code is guaranteed (in the absence of hardware failures) not to interfere with the "critical" code

makes it easier for the OS to present the illusion of virtual memory

can run "normal" Linux (could also run "μClinux", but what's the point?)

μClinux – "MicroController Linux" is a version of the Linux kernel that supports Altera NIOS, ARM, Freescale M68K (including DragonBall, ColdFire, PowerQUICC and others), Hitachi H8, MIPS, and Xilinx MicroBlaze processors.

Disabling parameter CONFIG_MMU enables nommu mode.

Real-time

People use a variety of methods to combine real-time tasks with Linux:

Run the real-time tasks on a dedicated microcontroller; communicate with a (non-real-time) PC that handles non-real-time tasks. This is pretty much your only choice if you need real-time response times below 1 microsecond.
Run the real-time tasks in a "underlying" dedicated real-time operating system; run Linux as a "nested operating system" inside one low-priority task on top of the real-time operating system. Some of these systems claim real-time response times below 500 microseconds.
use a Linux kernel designed to emphasize real-time tasks, and run the real-time tasks with a high priority (perhaps even as a kernel thread).

Examples of RT distributions:

Red Hat Enterprise Linux for Real Time

Real-time Ubuntu

Several RT features provide real time capabilities:

RT preemption

RT scheduling

CPU partitioning and isolation

Dynticks (Tickless, nohz)

RT preemption

The Linux Foundation's Real-Time Linux (RTL) collaborative project is focused on improving the real-time capabilities of Linux and advancing the adoption of real-time Linux in various industries, including aerospace, automotive, robotics, and telecommunications.

Parameter CONFIG_PREEMPT_RT_id enables real-time preemption.

RT scheduling policies

Scheduling policies for RT:

SCHED_FIFO_id, SCHED_RR_id

implemented in kernel/sched/rt.c_src

SCHED_DEADLINE

implemented in kernel/sched/deadline.c_src

API:

man 1 chrt – manipulate the real-time attributes of a process

man 2 sched_rr_get_interval – get the SCHED_RR interval for the named process

man 2 sched_setscheduler, sched_getscheduler – set and get scheduling policy/parameters

man 2 sched_get_priority_min, sched_get_priority_max – get static priority range

RT synchronization

⚲ APIs

migrate_disable_id + spin_lock_id

local_lock_id calls migrate_disable(); rt_spin_lock_id(this_cpu_ptr_id((__lock)));

📖 References

Usage of migrate_disable

PREEMPT_RT caveats: spinlock_t, rwlock_t, migrate_disable and local_lock_doc

⚙️ Internals

Spinlock

linux/spinlock_rt.h_inc used via linux/spinlock_types.h_inc

spinlock_t_id

rt_mutex_base_id

rt_spin_lock_id ...

__rt_spin_lock_id ...

rtlock_lock_id ...

kernel/locking/spinlock_rt.c_src

RT Mutex

linux/rtmutex.h_inc used via linux/mutex_types.h_inc

rt_mutex_base_id

rt_mutex_id

rt_mutex_lock_id ...

kernel/locking/rtmutex_api.c_src

kernel/locking/rtmutex_common.h_src

kernel/locking/rtmutex.c_src

rwbase_rt

linux/rwbase_rt.h_inc used via linux/rwlock_types.h_inc

rwlock_t_id

rwbase_rt_id – used to implement real-time read/write locks

rt_read_trylock_id ...

kernel/locking/rwbase_rt.c_src

kernel/locking/spinlock_rt.c_src

Testing RT capabilities

The testing process for Real-Time Linux typically involves several key aspects. First and foremost, it is crucial to verify the accuracy and stability of the system's timekeeping mechanisms. Precise time management is fundamental to real-time applications, and any inaccuracies can lead to timing errors and compromise the system's real-time capabilities.

Another essential aspect of testing is evaluating the system's scheduling algorithms. Real-Time Linux employs advanced scheduling policies to prioritize critical tasks and ensure their timely execution. Testing the scheduler involves assessing its ability to allocate resources efficiently, handle task prioritization correctly, and prevent resource contention or priority inversion scenarios.

Furthermore, latency measurement is a critical part of Real-Time Linux testing. Latency refers to the time delay between the occurrence of an event and the system's response to it. In real-time applications, minimizing latency is crucial to achieving timely and predictable behavior. Testing latency involves measuring the time it takes for the system to respond to various stimuli and identifying any sources of delay or unpredictability.

Additionally, stress testing plays a significant role in assessing the system's robustness under heavy workloads. It involves subjecting the Real-Time Linux system to high levels of concurrent activities, intense computational loads, and input/output operations to evaluate its performance, responsiveness, and stability. Stress testing helps identify potential bottlenecks, resource limitations, or issues that might degrade the real-time behavior of the system.