Network functionality

networking
sockets access
address families: inet, unix, ...
network storage
protocols
network interfaces
network drivers

Linux kernel network functionality spans from sockets interface through protocols to network cards.

⚲ Shell interfaces:

man 8 netstat prints network connections, routing tables, interface statistics and other details

man 8 ip shows and configures routing, network devices, interfaces and tunnels

man 8 ss - socket statistics utility

Sockets

⚲ API:

sys/socket.h – main user mode sockets header

Basic common and client side interface:

man 2 socket ↪ __sys_socket _id creates an endpoint for communication

struct sockaddr _id - abstract socket address

man 2 connect ↪ __sys_connect _id;

man 2 shutdown shuts down part of a full-duplex connection

man 2 send ↪ __sys_sendto _id sends a message on a socket

man 2 recv ↪ __sys_recvfrom _id, __sys_recvmsg _id receives a message from a socket

Additional server side interface:

man 2 bind ↪ __sys_bind _id - binds a sockaddr to a socket

man 2 listen ↪ __sys_listen _id - listens for connections on a socket

man 2 accept ↪ __sys_accept4 _id - accepts a connection on a socket

⚙️ Internals

struct socket _id @ linux/net.h _inc contains

struct proto_ops _id - abstract protocols interface

struct sock _id - network layer representation of sockets net/sock.h _inc

__sys_socket _id ↯ call hierarchy:

sock_create _id

__sock_create _id

security_socket_create _id

sock_alloc _id

net_proto_family _id->create.

for example inet_create _id. See Address families for another options.

__sys_connect _id ↯ call hierarchy:

move_addr_to_kernel _id

audit_sockaddr _id

__sys_connect_file _id

sock_from_file _id

security_socket_connect _id

proto_ops _id->connect.

for example inet_stream_connect _id. See Protocols for another options.

net/socket.c _src

📚 References

man 7 socket

linux/socket.h _inc

Berkeley sockets

Network storage

🚀 advanced topic

🔧 TODO

⚲ API:

man 2 sendfile ↪ do_sendfile _id.

See also Zero-copy between file descriptors

Application layer: Network File System

NFS _doc

init_nfs_fs _id, nfs4_fs_type _id, nfs_fs_type _id,

init_nfsd _id, nfsd_fs_type _id

CIFS _doc

init_cifs _id

cifs_fs_type _id, smb3_fs_type _id cifs_smb3_do_mount _id

target and iSCSI Interfaces Guide _doc

Logistics

Names

⚲ API: man 2 uname, man 2 sethostname, man 2 gethostname, man 2 setdomainname man 2 getdomainname

↪ utsname _id

⚙️ Details

utsname _id returns writable pointer to new_utsname _id from uts_namespace _id from nsproxy _id from current _id task_struct _id.

CLONE_NEWUTS _id, setns _id

kernel/utsname.c _src

📚 References:

man 7 namespaces

man 7 network_namespaces

man 7 uts_namespaces

Address families

⚲ API

man 2 getsockname

man 2 getpeername

Address Family (AF) domain defines address format and address length socklen_t.

man 3 inet_ntop, man 3 inet_pton (derive socklen_t from AF)

Common AF: AF_UNIX _id, AF_INET _id, AF_NETLINK _id.

PF - Protocol Family index (PF_MAX _id) actually is the same as Address Family index (AF).

⚙️ Internals of some AF

man 7 unix ↪ unix_family_ops _id - sockets for local IPC

unix_create _id

man 7 ip ↪ inet_family_ops _id - IPv4

inet_create _id

man 7 netlink ↪ netlink_family_ops _id - communication between kernel and user space

netlink_create _id

man 7 vsock ↪ vsock_family_ops _id - communication between VM and hypervisor

vsock_create _id

man 7 packet ↪ packet_family_ops _id - device level interface

packet_create _id

bt_sock_family_ops _id - Bluetooth

bt_sock_create _id

Totally there are more than 40 AFs (see AF_MAX _id)

⚙️ Internals

sock_register _id - registers net_proto_family _id. See references to this identifiers to find more than 30 protocol families.

__sock_create _id

📚 Further reading

man 8 ip-address – protocol address management

Internet layer

man 7 address_families

Protocols

Each Protocol Family (PF, same index as Address Family AF) consists of several protocol implementations.

Directory /proc/net contains various files and subdirectories containing information about the networking layer. File /proc/net/protocols lists available and used protocols.

In each PF protocols are classified to different types sock_type _id, for example stream, datagram and raw socket. TCP is type of stream, UDP is type of datagram, raw and ping are type of raw.

proto_register _id - registers struct proto _id - protocol implementations:

In inet_init _id initcall, inetsw_array _id, proto_ops _id and proto _id :

inet_stream_ops _id & tcp_prot _id tcp_sendmsg _id ...

inet_dgram_ops _id & udp_prot _id udp_sendmsg _id ...

inet_sockraw_ops _id

raw_prot _id raw_sendmsg _id ...

ping_prot _id ping_v4_sendmsg _id ...

In af_unix_init _id initcall:

unix_family_ops _id

unix_create _id

unix_stream_ops _id unix_stream_sendmsg _id ...

unix_dgram_ops _id unix_dgram_sendmsg _id ...

unix_seqpacket_ops _id unix_seqpacket_sendmsg _id ...

📚 References:

man 7 tcp

man 7 udp

man 7 raw

Transport layer and TCP

RDMA

🚀 advanced topic

🗝️ Acronyms:

IB — InfiniBand, an interconnect standard, competes with Ethernet, Fibre Channel

IPoIB — IP network emulation layer over InfiniBand networks

SRP — SCSI RDMA Protocol

ULP — Upper-layer protocols

iSER — iSCSI Extensions for RDMA

⚲ Interfaces:

https://github.com/linux-rdma/rdma-core

man 8 rdma

man 7 rdma_cm — RDMA communication manager

include/uapi/rdma _src

include/rdma _src

⚙️ Internals:

drivers/infiniband _src

drivers/infiniband/ulp _src — Upper-layer protocols

drivers/infiniband/sw _src — software drivers

drivers/infiniband/hw _src — hardware device drivers

📚 References:

InfiniBand _doc

InfiniBand and RDMA Interfaces _doc

Netfilter

🚀 advanced topic

⚲ Interface:

man 8 ebtables-nft

man 8 arptables-nft

man 8 xtables-nft

man 8 iptables

man 8 ip6tables

man 8 ebtables

man 8 arptables

ipset

linux/netfilter.h _inc

uapi/linux/netfilter _inc

net/netfilter _inc

net/netns/netfilter.h _inc

linux/netfilter _inc

⚙️ Internals:

net/netfilter _src

📚 References:

Netfilter Sysfs variables _doc

Netfilter Conntrack Sysfs variables _doc

Netfilter’s flowtable infrastructure _doc

nftables

https://wiki.nftables.org/

https://lwn.net/Kernel/Index/#Networking-Packet_filtering

Network device interfaces

⚲ Interfaces

ip -brief link show

ls -l /sys/class/net

devm_register_netdev _id registers net_device _id, net_device_ops

sk_buff _id socket buffer (skb)

dev_queue_xmit _id queues socket buffers into transmit queue

linux/netdevice.h _inc

linux/skbuff.h _inc

👁 Example: drivers/net/loopback.c _src - the most famous and simple interface lo

⚙️ Internals

net/core/dev.c _src

function loopback_xmit _id receives skb and passes it back with netif_rx _id

📚 Further reading

man 8 ip-link – network device configuration

man 8 ip-stats – manage and show interface statistics

man 7 netdevice – low-level access to Linux network devices

man 7 packet – packet interface on device level

Queueing in the Linux Network Stack

💾 Historical

LDP TLK Chapter 10 Networks

Network drivers

linux/etherdevice.h _inc

netif_rx _id - before NAPI

input_pkt_queue _id

NAPI

NAPI Driver design

⚲ API:

netif_napi_add _id adds napi_struct _id

napi_schedule _id - called by an IRQ handler to schedule a poll

netif_receive_skb _id - instead netif_rx, finally calls ip_rcv _id

napi_complete_done _id - called from custom napi->poll()

⚙️ Internals:

net_dev_init _id

net_rx_action _id

napi_poll _id calls custom napi->poll()

👁 example

e1000_intr _id calls __napi_schedule _id

custom napi->poll() e1000e_poll _id calls napi_complete_done _id

ether_setup _id setups Ethernet network device

👁 An example of Ethernet driver: e1000_probe _id

⚙️ Internals:

drivers/net _src

drivers/net/wireless _src

drivers/net/ethernet _src

📚 References:

man 8 ethtool – query or control network driver and hardware settings

Data link layer: Ethernet

GRO - Generic Receive Offload

Segmentation Offloads _doc

https://wireless.wiki.kernel.org

---

💾 Historical:

LDD2:Network Drivers

LDD3:Network Drivers

Kernel Analysis: Networking, 2003

network_overview

📖 Further reading about networking

Networking interfaces _doc

Networking _doc

https://lwn.net/Kernel/Index/#Networking

https://lartc.org/ – Linux Advanced Routing & Traffic Control

man 8 ip – show / manipulate routing, network devices, interfaces and tunnels

man 8 tc – show / manipulate traffic control settings

bcc/ebpf networking tools

eBPF-based Networking, Security, and Observability

Retis – tracing packets in the Linux networking stack & friends