OpenSSH/SSH Protocols

OpenSSH uses the SSH protocol which connects over TCP. Normally, one SSH session per TCP connection is made, but multiple sessions can be multiplexed over a single TCP connection if planned that way. The current set of Secure Shell protocols is SSH2. It is a rewrite of the old, deprecated SSH1 protocol. It contains significant improvements in security, performance, and portability. The default is now SSH2 and SSH1 support has been removed from both the client and server.

Sequence Diagram for SSH Password Authentication

The Secure Shell protocol is an open standard. As such, it is vendor-neutral and maintained by the Internet Engineering Task Force (IETF). The current protocol is described in RFC 4250 through RFC 4256 and standardized by the IETF secsh working group. The overall structure of SSH2 is described in RFC 4251, The Secure Shell (SSH) Protocol Architecture.

The SSH protocol is composed of three layers: the transport layer, the authentication layer, and the connection layer.

SSH-CONNECT – The connection layer runs over the user authentication protocol. It multiplexes many different concurrent encrypted channels into logical channels over the authenticated connection. It allows for tunneling of login sessions and TCP-forwarding. It provides a flow control service for these channels. Additionally, various channel-specific options can be negotiated. This layer manages the SSH session, session multiplexing, X11 forwarding, TCP forwarding, shell, remote program execution, invoking SFTP subsystem.

SSH-USERAUTH – The user authentication layer authenticates the client-side to the server. It uses the established connection and runs on top of the transport layer. It provides several mechanisms for user authentication. These include password authentication, public-key or host-based authentication mechanisms, challenge-response, pluggable authentication modules (PAM), Generic Security Services API (GSSAPI) and even dongles.

SSH-TRANS – The transport layer provides server authentication, confidentiality and data integrity over TCP. It does this through algorithm negotiation and a key exchange. The key exchange includes server authentication and results in a cryptographically secured connection: it provides integrity, confidentiality and optional compression.

Among the differences between the current protocol, SSH2, and the deprecated SSH1 protocol, is that SSH2 uses host keys for authentication. Whereas SSH1 used both server and host keys to authenticate. There's not much which can be added about the protocols which is not already covered with more detail and authority in RFC 4251 .

SSH File Transfer Protocol (SFTP)

The SSH File Transfer Protocol (SFTP) is a binary protocol to provide secure file transfer, access and management.

SFTP was added by Markus Friedl on the server side in time for the 2.3.0 release of OpenSSH in November 2000. Damien Miller added support for SFTP to the client side in time for 2.5.0. Since then, many have added to both the client and the server.

SFTP is not FTPS

For basic file transfer, nothing more is needed than an account on the machine with the OpenSSH server. SFTP support is built into the OpenSSH server package. The SFTP protocol, in contrast to old FTP, has been designed from the ground up to be as secure as possible for both login and data transfer.

Samba is an interoperability suite providing fast file and print services for any client which can use the SMB/CIFS protocol. It is most often used for file sharing at the level of the local area network.

AFS, or the Andrew File System, is a distributed file system providing file sharing at an institutional level across a geographically diverse institution or set of collaborating institutions. Notably it provides a set of trusted servers and a homogeneous, location-transparent file name space to all the client workstations.

Unless the use-case calls for publicly available, read-only, downloads, don't worry about trying to fiddle with FTP. It is the protocol FTP itself that is inherently insecure. It's great for read-only, public data transfer. The programs vsftpd and proftpd, for example, are secure insofar as the server software itself goes, although the protocol itself is still insecure. In other words the program itself is more or less fine and if you need to provide read-only, publicly available downloads then FTP maybe the right tool. Otherwise forget about FTP. Nearly always when users ask for "FTP" they don't mean specifically the old file transfer protocol from 1971 as described in RFC 114, but a generic means of file transfer and there are many ways to solve that problem. This is especially true since the next part of their request is usually how to make it secure. The name "FTP" is frequently mis-used generically to mean any file transfer utility, much the same way as the term "Coke" is used in some of Southern United States to mean any carbonated soft drink, not just Coca-Cola. Consider SFTP or, for larger groups, even SSHFS, Samba, or AFS. While old FTP succeeded very well in achieving its main goal to promote use of networked computers by allowing users at any host on the network to use the file system of any cooperating host, it cannot be made secure. There's nothing to be done about that, so it is past time to get over it.

Again, it is the protocol itself, FTP, which is the problem. With FTP, the data, passwords and user name are all sent back and forth unencrypted. Anyone on the client's subnet, the server's subnet or any subnet in between can 'sniff' the passwords and data when FTP is used. With extra effort it is possible to wrap FTP inside SSL or TLS, thus creating FTPS. However, tunneling FTP over SSL/TLS is complex to do and far from an optimum solution.

Unfortunately because of name confusion combined with the large number of posts and discussions created by complex, nit-picky tasks like wrapping FTP in SSL to provide FTPS, the wrong way still turns up commonly in web searches regarding file transfer. In contrast, easy, relatively painless solutions vanish because it is rarely necessary to post how to do those. Also, an easy solution can be summed up in very few lines and maybe a single answer. Thus, there is still a lot of talk online about 'securing' FTP and very little mention of using SFTP. It's a vicious cycle that this book hopes to help break: Difficult tasks mean lots of discussion and noise, lots of discussion and noise means strong web presence, strong web presence means high Google ranking.

SFTP tools are very common, but might be taken for granted and thus overlooked. SFTP tools are easy to use and more functional than old FTP clients. In fact a lot of improvements have been made in usability. There is no shortage of common, GUI-based SFTP clients to transfer files: Filezilla, Konqueror, Dolphin, Nautilus, Cyberduck, Fugu, and Fetch top the list but there are many more. Most are Free Software. Again, these SFTP clients are very easy to use. For example, in Konqueror, just type in the URL to the sftp server, where the server name or address is xx.yy.zz.aa.

sftp://xx.yy.zz.aa

If it is desirable to start with a particular directory, then that too can be specified.

sftp://xx.yy.zz.aa/var/www/pictures/

One special client worth knowing about is sshfs. With sshfs as an SFTP client the other machine is accessible as an open folder on your machine's local file system. In that way any program you normally have to work with files, such as LibreOffice, Inkscape or Gimp can access the remote machine via that folder.

Background of FTP

FTP is from the 1970s. It's a well proven workhorse, but from an era when if you were on the net you were supposed to be there and if there was trouble it could usually be cleared up with a short phone call or an e-mail or two. It sends the login name, password and all of the data unencrypted for anyone to intercept. FTP clients can connect to the FTP server in either passive or active modes. Both active and passive modes for FTP use two ports, one for control and one for data. In FTP Active mode, after the client makes a connection to the FTP server it then allows an incoming connection to be initiated from the server to for data transfer. In FTP Passive mode, after the client makes a connection to the FTP server, the server then responds with information about a second port for data transfer and the client initiates the second connection. FTP is most relevant now as Anonymous FTP, which is still excellent for read-only downloads without login. FTP is still one way to go for transferring read-only data, as would be using the web (HTTP or HTTPS), or a P2P protocol like Bittorrent. So there are other options than FTP for offering read-only downloads. Preference is given lately to HTTPS for small files and Bittorrent for large files or large groups of files.

Using tcpdump to show FTP activity

An illustration of how the old protocol, FTP, is insecure can be had from the utility tcpdump. It can show what is going over the network during an Anonymous FTP session, or for that matter any FTP session. Look at the manual page for tcpdump for an explanation of the individual arguments, the usage example below displays the first FTP or FTP-Data packets going from the client to the server and vice versa.

The output below shows an excerpt from the output of tcpdump which captured packets between an FTP client and the FTP server, one line per packet.

$ sudo tcpdump -q -s 0 -c 10 -A -i eth0 \
"tcp and (port ftp or port ftp-data)"
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
… 
:18:36.010820 IP desk.55227 > server.ftp: tcp 16 E..D..@.@.....1.[.X.....G.r. ........l..... ."......USER anonymous 
:18:36.073192 IP server.ftp > desk.55227: tcp 0 E..4jX@.7.3.[.X...1..... ...G.r#........... .....".. 
:18:36.074019 IP server.ftp > desk.55227: tcp 34 E..VjY@.7.3.[.X...1..... ...G.r#....Y...... ....."..331 Please specify the password. 
:18:36.074042 IP desk.55227 > server.ftp: tcp 0 E..4..@.@..+..1.[.X.....G.r# ..)........... ."...... 
:18:42.098941 IP desk.55227 > server.ftp: tcp 23 E..K..@.@.....1.[.X.....G.r# ..)....gv..... .".w....PASS user@example.net 
:18:42.162692 IP server.ftp > desk.55227: tcp 23 E..KjZ@.7.3.[.X...1..... ..)G.r:........... .....".w230 Login successful.
… 
:18:43.431827 IP server.ftp > desk.55227: tcp 14 E..Bj\@.7.3.[.X...1..... ..SG.rF.....j..... ....."..221 Goodbye.
…

As can be seen in lines 3 and 7, data such as text from the server is visible. In lines 1 and 5, text entered by the user is visible and in this case it includes the user name and password used to log in. Fortunately the session is Anonymous FTP, which is read-only and used for downloading. Anonymous FTP is a rather efficient way to publish material for download. For Anonymous FTP, the user name is always "anonymous" and the password is the user’s e-mail address and the server's data always read-only.

If you have the package for the OpenSSH server already installed, no further configuration of the server is needed to start using SFTP for file transfers. Though comparatively speaking, FTPS is significantly more secure than FTP. If you want remote remote login access, then both FTP and FTPS should be avoided. A very large reason to avoid both is to save work.

On FTPS

FTPS is FTP tunneled over SSL or TLS. A goal of FTP was to encourage the use of remote computers which, along with the web, has succeeded. A goal of FTPS was to secure logins and transfers, and it was a necessary step in securing file transfers with the legacy protocol. However, since SFTP is so much easier to deploy and most systems now include both graphical and text-based SFTP clients, FTPS can really be considered deprecated for most occasions.

Some good background material can be found in the Request for Comments (RFCs) for FTP and FTPS. There, SFTP and even HTTPS are better matches and largely supersede FTPS. See the section on Client Applications for an idea of the SFTP clients available.

Privilege Separation

Privilege separation is when a process is divided into sub-processes, each of which have just enough access to just the right parts of the system to do their part of the job. The goal of privilege separation is to compartmentalize any corruption and prevent a corrupt process from accessing other parts of the system. An underlying principle is that of least privilege, which is where each process has exactly enough privileges to accomplish a task, neither more nor less. Currently it's like this :

sshd [listener]
   |
 exec
   |
sshd-session [privsep monitor]
  |      |
  |    exec
  |      |
  |    sshd-auth: [net unpriv]
  |
 fork (after auth completes)
  |
sshd-session [postauth unpriv]

First, a privileged binary listens for incoming connections.

$ ps -a -x -w -o user,pid,ppid,args | grep '[s]shd'
root      2460     1 sshd: /usr/sbin/sshd [listener] 0 of 10-100 startups (sshd)

Once a connection starts, it is passed off to a separate privileged monitor binary (seen as PID 46495 below) which makes an unprivileged exec (seen as PID 96923 below) under the account 'sshd' to address authentication.

$ ps -a -x -w -o user,pid,ppid,args | grep '[s]shd'
root      2460     1 sshd: /usr/sbin/sshd [listener] 1 of 10-100 startups (sshd)
root     46495  2460 sshd-session: fred [priv] (sshd-session)
sshd     96923 46495 sshd-auth: fred [net] (sshd-auth)

The fork owned by 'sshd' is terminated, and if the authentication succeeds a new process owned by the authenticated account is forked (seen as PID 94635 below) from the monitor to deal with the session itself.

$ ps -a -x -w -o user,pid,ppid,args | grep '[s]shd'
root      2460     1 sshd: /usr/sbin/sshd [listener] 0 of 10-100 startups (sshd)
root     46495  2460 sshd-session: fred [priv] (sshd-session)
fred     94635 46495 sshd-session: fred@ttyp1 (sshd-session)

This is a work in progress and future developments may move the post authentication, unprivileged process into a separate binary. The portable edition might acquire a PAM helper binary which would be called by the privilege separation monitor.

Privilege separation has been the default in OpenSSH since version 3.3 Starting with version 5.9, privilege separation further applies mandatory restrictions on which system calls the privilege separated child can perform. The intent is to prevent a compromised privilege separated child from being used to attack other hosts either by opening sockets and proxying or by probing the local kernel attack surface. Since version 6.1, this sandboxing has been the default.

Sequence Diagram for OpenSSH Privilege Separation

Old Privilege Separation Prior to 9.8

Privilege separation is applied in OpenSSH by using several levels of access, some higher some lower, to run sshd(8) and its subsystems and components. The SSH server ➊ starts out with a privileged process ➋ which then creates an unprivileged process ➌ to work with the network traffic. Once the user has authenticated, another unprivileged process is created ➍ with the privileges of that authenticated user. See the "Sequence Diagram for OpenSSH Privilege Separation". As seen in the diagram, a total of four processes get run to create an SSH session. One process, the server, remains and listens for new connections and spawn new child processes.

$ ps -ax -o user,pid,ppid,state,start,command | awk '/sshd/ || NR==1' 
USER       PID  PPID STAT   STARTED  COMMAND
root      1473     1 I     05:44:01  sshd: /usr/sbin/sshd [listener] 0 of 10-10

It is this privileged process that listens for the initial connection from clients. Here it is seen waiting and listening on port 22.

$ netstat -ntlp | awk '/sshd/ || NR<=2'
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1473/sshd       
tcp6       0      0 :::22                   :::*                    LISTEN      1473/sshd

After the initial connection while waiting for password authentication from user 'fred', a privileged monitor process supervises an unprivileged process by user 'sshd' which handles the contact with the remote user's client.

$ ps -ax -o user,pid,ppid,state,start,command | awk '/sshd/ || NR==1' 
USER       PID  PPID S  STARTED COMMAND
root      1473     1 S 05:44:12 sshd: /usr/sbin/sshd [listener] 1 of 10-10
root      9481  1473 S 14:40:37 sshd: fred [priv]   
sshd      9482  9481 S 14:40:37 sshd: fred [net]

Then after authentication is completed and a session established for user 'fred', a new privileged monitor process is created to supervise the process running as user 'fred'. At that point the other process running as user 'sshd' has gone away.

$ ps -ax -o user,pid,ppid,state,start,command | awk '/sshd/ || NR==1' 
USER       PID  PPID S  STARTED COMMAND
root      1473     1 S 05:44:12 sshd: /usr/sbin/sshd [listener] 0 of 10-10
root      9481  1473 S 14:40:37 sshd: fred [priv]   
fred      9579  9481 S 14:42:02 sshd: fred@pts/30

Again, that is for the older method of privilege separation.