In this seventh part of a nineteen-part article series on SSH, we'll delve more deeply into the passive mode, and tell you the sequence of steps you must take to set up data-connection forwarding. This article is excerpted from chapter 11 of the book SSH, The Secure Shell: The Definitive Guide, Second Edition, written by Daniel J. Barrett, Richard E. Silverman and Robert G. Byrnes (O'Reilly; ISBN-10: 0596008953).
126.96.36.199 Passive mode in depth
Recall that in a passive-mode transfer, the client initiates a connection to the server. Specifically, instead of listening on a local socket and issuing a PORT command to the server, the client issues a PASV command. In response, the server selects a socket on its side to listen on and reveals it to the client in the response to thePASVcommand. The client then connects to that socket to form the data connection, and issues the file-transfer command over the control connection. With command line–based clients, the usual way to do passive-mode transfers is to use the passive command. Again, an example:
$ ftp -d aaor.lionaka.net Connected to aaor.lionaka.net. 220 aaor.lionaka.net FTP server (SunOS 5.7) ready. ---> USER res 331 Password required for res. Password: ---> PASS XXXX 230 User res logged in. ---> SYST 215 UNIX Type: L8 Version: SUNOS Remote system type is UNIX. Using binary mode to transfer files. ftp> passive Passive mode on. ftp> ls ---> PASV 227 Entering Passive Mode (219,243,169,52,128,73) ---> LIST 150 ASCII data connection for /bin/ls (188.8.131.52,2538) (0 bytes). total 360075 drwxr-xr-x98 res 500 7168 May 5 17:13 . dr-xr-xr-x 2root root 2 May 5 01:47 .. -rw-rw-r-- 1 res 500 596 Apr 25 1999 .FVWM2-errors -rw------- 1 res 500 332 Mar 24 01:36 .ICEauthority -rw------- 1 res 500 50 May 5 01:45 .Xauthority -rw-r--r-- 1 res 500 1511 Apr 11 00:08 .Xdefaults 226 ASCII Transfer complete. ftp> quit ---> QUIT 221 Goodbye.
Note that after the user gives the ls command, the client sendsPASVinstead ofPORT. The server responds with the socket on which it will listen. The client issues theLISTcommand to list the contents of the current remote directory, and connects to the remote data socket; the server accepts and confirms the connection, then transfers the directory listing over the new connection.
An interesting historical note, which we alluded to earlier, is that the PASV command wasn’t originally intended for this use; it was designed to let an FTP client direct a file transfer between two remote servers. The client makes control connections to two remote servers, issues a PASV command to one causing it to listen on a socket, issues aPORTcommand to the other telling it to connect to the other server on that socket, then issues the data-transfer command (STOR,RETR, etc.). These days, most people don’t even know this is possible, and will pull a file from one server to the local machine, and transfer it again to get it to the second remote machine. It’s so uncommon that many FTP clients don’t support this mode, and some servers prevent its use for security reasons. [184.108.40.206]
220.127.116.11 FTP with the default data ports
The third file-transfer mode occurs if the client issues neither a PORT nor a PASV command. In this case, the server initiates the data connection from the well-known ftp-data port (20) to the source socket of the control connection, on which the client must be listening (these sockets are the “default data ports” for the FTP session). The usual way to use this mode is with the FTP client command sendport, which switches on and off the client’s feature of using aPORT command for each data transfer. For this mode, we want it turned off, and it is generally on by default. So, the sequence of steps is this:
The client initiates the control connection from local socket C to server:21.
The user gives the sendport command, and then a data-transfer command, such as put or ls. The FTP client begins listening on socket C for an incoming TCP connection.
The server determines the socket C at the other end of the control connection. It doesn’t need the client to send this explicitly via the FTP protocol, since it can just ask TCP for it (e.g., with thegetpeername()sockets API routine). It then opens a connection from its ftp-data port to C, and sends or receives the requested data over that connection.
Now, this is certainly a simpler way of doing things than using a different socket for each data transfer, and so it begs the question of whyPORTcommands are the norm. If you try this out, you will discover why. First off, it might fail on the client side with the message “bind: Address already in use”. And even if it does work, it does so only once. A second ls elicits another address-related error, this time from the server:
aaor% ftp syrinx.lionaka.net Connected to syrinx.lionaka.net. 220 syrinx.lionaka.net FTP server (Version wu-2.5.0(1) Tue Sep 21 16:48:12 EDT 331 Password required for res. Password: 230 User res logged in. ftp> sendport Use of PORT cmds off. ftp> ls 150 Opening ASCII mode data connection for file list. keep fichier.txt 226 Transfer complete. 19 bytes received in 0.017 seconds (1.07 Kbytes/s) ftp> ls 425 Can't build data connection: Cannot assign requested address. ftp> quit
These problems are due to a technicality of the TCP protocol. In this scenario, every data connection is between the same two sockets, server:ftp-data and C. Since a TCP connection is fully specified by the pair of source and destination sockets, these connections are indistinguishable as far as TCP is concerned; they are different incarnations of the same connection and can’t exist at the same time. In fact, to guarantee that packets belonging to two different incarnations of a connection aren’t confused, there’s a waiting period after one incarnation is closed, during which a new incarnation is forbidden. In the jargon of TCP, on the side that performed an “active close” of the connection, the connection remains in a state called TIME_WAIT. This state lasts for a period that is supposed to be twice the maximum possible lifetime of a packet in the network (or “2MSL,” for two times the Maximum Segment Lifetime). After that, the connection becomes fully closed, and another incarnation can occur. The actual value of this timeout varies from system to system, but is generally in the range of 30 seconds to 4 minutes.*
As it happens, some TCP implementations enforce even stronger restrictions. Often, a port that is part of a socket in the TIME_WAIT state is unavailable for use, even as part of a connection to a different remote socket. We have also run into systems that disallow listening on a socket that is currently an endpoint of some connection, regardless of the connection state. These restrictions aren’t required by the TCP protocol, but they are common. Such systems usually provide a way to avoid the restrictions, such as the SO_REUSEADDR option of the Berkeley sockets API. An FTP client generally uses this feature, of course, but it doesn’t always work!
This address-reuse problem comes up in two places in a default-port FTP transfer. The first one is when the client must start listening on its default data port, which by definition is currently the local endpoint of its control connection. Some systems simply don’t allow this, even if the program requests address reuse; that’s why the attempt might fail immediately with the message, “address already in use.”
The other place is on a second data transfer. When the first transfer is finished, the server closes the data connection, and that connection on the server side moves into the TIME_WAIT state. If you try another data transfer before the 2MSL period has elapsed, the server tries to set up another incarnation of the same connection, and it will fail saying “cannot assign requested address.” This happens regardless of the address reuse setting, since the rules of the TCP require it. You can transfer a file again within a few minutes, of course, but most computer users aren’t good at waiting a few seconds, let alone minutes. It is this problem that prompts the use of a PORT command for every transfer; since one end of the connection is different every time, the TIME_WAIT collisions don’t occur.
Because of these problems, the default-port transfer mode isn’t generally used. It has, however, an important property for us: it is the only mode in which the data connection destination port is fixed and knowable before the data-transfer command is given. With this knowledge, some patience, and a fair amount of luck, it is possible to forward your FTP data connections through SSH.