OpenSSH/Cookbook/File Transfer with SFTP

 

Basic SFTP service requires no additional setup, it is a built-in part of the OpenSSH server and it is the subsystem sftp-server(8) which then implements an SFTP file transfer. See the manual page for sftp-server(8). Alternately, the subsystem internal-sftp can implement an in-process SFTP server which may simplify configurations using ChrootDirectory to force a different filesystem root on clients.

On the client, the same options and tricks are available for SFTP as for the regular SSH clients. However, some client options may have to be specified with the full option name using the -o argument. For many dedicated graphical SFTP clients, it is possible to use a regular URL to point to the target. Many file managers nowadays have built-in support for SFTP. See the section "GUI Clients" above.

Basic SFTP

SFTP provides a very easy to use and very easy to configure option for accessing a remote system. Just to say it again, regular SFTP access requires no additional changes from the default configuration. The usual clients can be used or special ones like sshfs(1).

Automated SFTP

SFTP uploads or downloads can be automated. The prerequisite is key-based authentication. Once key-based authentication is working, a batch file can be used to carry out activities via SFTP. See the batchfile option -b in sftp(1) for details.

$ sftp -b /home/fred/cmds.batch -i /home/fred/.ssh/foo_key_rsa server.example.org:/home/fred/logs/

If a dash (-) is used as the batch file name, SFTP commands will be read from stdin.

$ echo "put /var/log/foobar.log" | sftp -b - -i /home/fred/.ssh/foo_key_rsa server.example.org:/home/fred/logs/

More than one SFTP command can be sent, but then it is better to use the batch file mode.

$ echo -e "put /var/log/foobar.log\nput /var/log/munged.log" | sftp -b - -i /home/fred/.ssh/foo_key_rsa server.example.org:/home/fred/logs/

The batch file mode can be very useful in cron jobs and in scripting.

SFTP-only Accounts

Using the Match directive in sshd_config(5), it is possible to limit members of a specific group to using only SFTP for interaction with the server.

Subsystem sftp internal-sftp

Match Group sftp-only
	AllowTCPForwarding no
	X11Forwarding no
	ForceCommand internal-sftp

Note that disabling TCP forwarding does not improve security unless users are also denied shell access, as they can in principle install their own forwarders.

See PATTERNS in ssh_config(5) for more information on patterns available to Match.

It is common for a group of accounts to need to read and write files to their home directories on the server while having little or no reason to access the rest of the file system. SFTP provides a very easy to use and very easy to configure chroot. In some cases, it is enough to chroot users to their home directories. This may not be as straight forward because in most cases home directories aren't owned by root and allow writing by at least one user. However, since SFTP chroot requires that the chroot target directory and all parent directories are owned by root and not writable by any others, this causes some difficulty but is necessary. Without ownership restrictions, it is quite feasible to escape the chroot.

One way around the difficulties imposed by this restriction is to have the home directory owned by root and have it populated with a number of other directories and files that are owned by the regular account to which the user can actually write to.

Match Group sftp-only
	ChrootDirectory %h
	AllowTCPForwarding no
	X11Forwarding no
	ForceCommand internal-sftp

In that case the root user will have to populate the target directory with the needed files and subdirectories and then change their ownership to that of the unprivileged account.

Three Ways of Setting Permissions for Chrooted SFTP-Only Accounts

There are at least three other ways set permissions for home directories for chrooted SFTP-only accounts. They each have strong advantages and a few drawbacks and thus will suit some particular situations but not others.

Option 1: Automatically Setting the Start Directory

If it is not practical to have the various home directories owned by root, a compromise can be made. ChrootDirectory can point to /home, which must be owned by root anyway, and then ForceCommand can then designate the user's home directory as the starting directory using the -d option.

Match Group sftp-only
        ChrootDirectory /home/
        ForceCommand internal-sftp -d %u

Nesting another directory under the chroot is another way to do this. The subdirectory would be writable by the unprivileged account, the chroot target would not.

Match Group sftp-only
        ChrootDirectory /home/%u
        AllowTCPForwarding no
        X11Forwarding no
        ForceCommand internal-sftp -d %u

If it is necessary to hide the contents of the home directories from other users, chmod(1) can be used. Permissions could be 0111 for /home and 0750, 0700, 0770, or 2770, and so on for the home directories, be sure to check the group memberships as well.

Option 2: Nested Home Directories

Alternately, for a similar effect but with more isolation, home directories can be nested one level deeper for the chrooted accounts. Note the ownership and permissions for the following directories:

$ ls -lhd /home/ /home/*/ /home/*/*/
drwxr-xr-x  4 root  root  4.0K Aug  4 20:47 /home/
drwxr-x---  3 root  user1 4.0K Aug  4 20:47 /home/user1/
drwxr-x---  3 root  user2 4.0K Aug  4 20:47 /home/user2/
drwxr-x--- 14 user1 user1 4.0K Aug  4 20:47 /home/user1/user1/
drwxr-x--- 14 user2 user2 4.0K Aug  4 20:47 /home/user2/user2/

Then the ChrootDirectory directive can lock the user to the directory above their home and the ForceCommand directive can put the user in their own home directory using the -d option. Once logged in they can only ever see their own files. This arrangement also makes it easier to add chrooted shell access later as system directories can be added to the chroot without being available to other accounts.

Another common case is to chroot access to a web server's document root or server root. If each site has its own hierarchy under /var/www/ such as /var/www/site1/ then chroot can be put to use as follows:

Match Group team1
	ChrootDirectory /var/www/
	ForceCommand internal-sftp -d site1

Match Group team2
	ChrootDirectory /var/www/
	ForceCommand internal-sftp -d site2

The site directories can then be group writable while the parent /var/www/ remains read-only for non-root users.

Option 3: Split Ownership of the Home Directory and Its Contents

As seen above there are several ways to deal with a chrooted SFTP service. A third way would be to set the directory to be owned by root, in another group, but not writable by that group. So with the following settings, the account 'fred' can log in and use any of the pre-made subdirectories or files in the usual way, but cannot add anything to the chroot target itself.

$ ls -lhd /home/ /home/fred/ /home/fred/*
drwxr-xr-x  68 root root 4.0K Sep 4 15:40 /home/
drwxr-xr-x  21 root fred 4.0K Sep 4 15:41 /home/fred/
drwxr-xr-x   8 fred fred 4.0K Sep 4 15:44 /home/fred/Documents
drwxr-xr-x   9 fred fred 4.0K Sep 4 15:41 /home/fred/Music
drwxr-xr-x 145 fred fred 4.0K Sep 4 15:41 /home/fred/Pictures
drwxr-xr-x   5 fred fred 4.0K Sep 4 15:41 /home/fred/Videos
drwxr-xr-x  98 fred fred 4.0K Sep 4 15:41 /home/fred/WWW

The corresponding lines for ssh_config(5) would be the following, with the account 'fred' being a member of the 'team1' group:

Match Group team1
	ChrootDirectory /home/%u
	ForceCommand internal-sftp

This method is quick to set up but the drawback is that it requires system administrator intervention any time a new directory or file is to be added to the home directory. That requires root permission to do even if the new file or directory is changed to being owned by the unprivileged account.

Umask

Starting with OpenSSH 5.4, sftp-server(8) can set a umask to override the default one set by the user’s account. The in-process SFTP server, internal-sftp, accepts the same options as the external SFTP subsystem.

Subsystem sftp internal-sftp -u 0022

However it is important to remember that umasks only restrict permissions, never loosen them.

Earlier versions can do the same thing through the use of a helper script, but this complicates chrooted directories very much. The helper script can be a regular script or it can be embedded inline in the configuration file though neither works easily in a chroot jail. It’s often easier to get a newer version of sshd(8) which supports umask as part of the server’s configuration. Here is an inline helper script for umask in OpenSSH 5.3 and earlier, based on one by gilles@

Subsystem sftp /bin/sh -c 'umask 0022; /usr/libexec/openssh/sftp-server'

Either way, this umask is server-side only. The original file permissions on the client side will usually, but not always, be used when calculating the final file permissions on the server. This depends on the client itself. Most clients pass the file permissions on to the server, FileZilla being a notable exception. As such, permissions can generally be tightened but not loosened. For example, a file that is mode 600 on the client will not be automatically become 664 or anything else less than the original 600 regardless of the server-side umask. That is unless the client does not forward the permissions, in which case only the server's umask will be used. So for most clients, if you want looser permissions on the uploaded file, change them on the client side before uploading.

Further Restricting Chrooted SFTP

There are several additional common scenarios where chrooted access is restricted further.

Chrooted SFTP to Shared Directories

Another common case is to chroot a group of users to different levels of the web server they are responsible for. For obvious reasons, symbolic links going from inside the jail to parts of the filesystem outside the chroot jail are not accessible to the chrooted users. So directory hierarchies must be planned more carefully if there are special combinations of access. See the earlier section on chrooted SFTP-only accounts.

In these kinds of directories, it may be useful to give different levels of access to more than just one group. In that case, ACLs might be needed.

Chrooted SFTP Accounts Accessible Only from Particular Addresses

More complex matching can be done. It is possible to allow a group of users to use SFTP, but not a shell login, only if they log in from a specific address or range of addresses. If they log in from the right addresses, then get SFTP and only SFTP, but if they try to log in from other addresses they will be denied access completely. Both conditions, the affirmative and negative matches, need to be accounted for.

Subsystem sftp internal-sftp

Match Group sftp-only, Address 192.0.2.10
        AllowTCPForwarding no
        X11Forwarding no
        ForceCommand internal-sftp
        ChrootDirectory  /home/servers/

Match Group sftp-only, Address *,!192.0.2.10
        DenyGroups sftp-only

Note that for negation a wildcard must be specified first and then the address or range to be excluded following it. Mind the spaces or lack thereof. Similar matching can be done for a range of addresses by specifying the addresses in CIDR address/mask format, such as 192.0.2.0/24. Any number of criteria can be specified and only if all of them are met then the directive in the subsequent lines take effect.

The first Match block that fits is the one that takes effect, so care must be taken when constructing conditional blocks to make them fit the precise situation desired. Also, any situations that don't fit a Match conditional block will fall through the cracks. Those will get the general configuration settings whatever they may be. Specific user and source address combinations can be tested with the configurations using the -T and -C options with the server for more options. See the section Debugging a Server Configuration for more.

Chrooted SFTP with Logging

If the internal-sftp in-process SFTP server is not used then the logging daemon must establish a socket in the chroot directory for the sftp-server(8) subsystem to access as /dev/log See the section on Logging.

Chrooted Login Shells

Making a chroot jail for interactive shells is difficult. The chroot and all its components must be root-owned directories that are not writable by any other user or group. The ChrootDirectory must contain the necessary files and directories to support the user's session. For an interactive session this requires at least a shell, typically bash(1), ksh(1), or sh(1), and basic device nodes inside /dev such as null(4), zero(4), stdin(4), stdout(4), stderr(4), arandom(4), and tty(4) devices. Paths may contain the following tokens that are expanded at runtime once the connecting user has been authenticated: %% is replaced by a literal '%', %h is replaced by the home directory of the user being authenticated, and %u is replaced by the username of that user.

sshfs(1) - SFTP File Transfer Via Local Folders

Another way to transfer files back and forth, or even use them remotely, is to use sshfs(1) It is a user-space file system client based on SFTP and utilizes the server's SFTP-subsystem. It can make a directory on the remote server accessible as if it were a directory on the local file system which can be accessed by any program. The user must have read-write privileges for mount point to use sshfs(1).

The following creates the mount point, mountpoint, in the home directory if none exists. Then sshfs(1) mounts the remote server.

$ test -d ~/mountpoint || mkdir --mode 700 ~/mountpoint
$ sshfs fred@server.example.org:. ~/mountpoint

Reading or writing files to the mount point is actually transferring data to or from the remote system. The amount of bandwidth consumed by the transfers can be reduced using compression. That can be important if the network connection has bandwidth caps or per-unit fees. However, if speed is the only issue, compression can make the transfer slower if the processors on either end are busy or not powerful enough. About the only way to be sure is to test and see which method is faster. Below, compression is specified with -C.

$ sshfs -C fred@server.example.org:. ~/mountpoint

Or try with debugging output:

$ sshfs -o sshfs_debug fred@server.example.org:. /home/fred/mountpoint

Named pipes will not work over sshfs(1). Use fusermount -u to umount these remote directories and close the SFTP session.

Using sshfs(1) With A Key

The ssh_command option is used to pass parameters on to ssh(1). In this example it is used to have ssh(1) point to a key used for authentication to mount a remote directory, /usr/src, locally as /home/fred/src.

$ sshfs -o ssh_command="ssh -i /home/fred/.ssh/id_rsa" fred@server.example.org:/usr/src /home/fred/src/

If a usable key is already loaded into the agent, then ssh(1) should find it and use it on behalf of sshfs(1) without needing intervention.