FTP in batch programming

                            FTP in batch programming

On this page I will show some examples of unattended FTP download (or upload, the difference in script commands is small) scripts.

Command Line Syntax

	FTP [-v] [-d] [-i] [-n] [-g] [-s:filename] [-a] [-w:windowsize] [host]
	where:
	-v	Suppresses display of remote server responses.
	-n	Suppresses auto-login upon initial connection.
	-i	Turns off interactive prompting during multiple file transfers.
	-d	Enables debugging.
	-g	Disables filename globbing (see GLOB command).
	-s:filename	Specifies a text file containing FTP commands; the commands will automatically run after FTP starts.
	-a	Use any local interface when binding data connection.
	-A	Login as anonymous.
	-w:buffersize	Overrides the default transfer buffer size of 4096.
	host	Specifies the host name or IP address of the remote host to connect to.

Notes:	(1)	mget and mput commands take y/n/q for yes/no/quit.
	(2)	Use Control-C to abort commands.

The -s switch is the most valuable switch for batch files that take care of unattended downloads and uploads:
FTP -s:ftpscript.txt
On some operating systems redirection may do the same:
FTP < ftpscript.txt
However, unlike the -s switch its proper functioning cannot be guaranteed.

FTP's Interactive Commands

The following table shows the FTP commands available in Windows NT 4. The difference with other operating systems is marginal.
The actual commands available can be found by starting an FTP session and then typing a question mark at the FTP> prompt.
To get a short description af a particular command, type a question mark followed by that command: (user input shown in bold italics):

C:\>ftp ftp> ? get get             receive file ftp> ? mget mget            get multiple files ftp> bye C:\>

FTP commands
Command	Description
!	escape to the shell
?	print local help information
append	append to a file
ascii	set ascii transfer type
bell	beep when command completed
binary	set binary transfer type
bye	terminate ftp session and exit
cd	change remote working directory
close	terminate ftp session
debug	toggle debugging mode
delete	delete remote file
dir	list contents of remote directory
disconnect	terminate ftp session
get	receive file
glob	toggle metacharacter expansion of local file names
hash	toggle printing `#' for each buffer transferred
help	print local help information
lcd	change local working directory
literal	send arbitrary ftp command
ls	nlist contents of remote directory
mdelete	delete multiple files
mdir	list contents of multiple remote directories
mget	get multiple files
mkdir	make directory on the remote machine
mls	nlist contents of multiple remote directories
mput	send multiple files
open	connect to remote tftp
prompt	force interactive prompting on multiple commands
put	send one file
pwd	print working directory on remote machine
quit	terminate ftp session and exit
quote	send arbitrary ftp command
recv	receive file
remotehelp	get help from remote server
rename	rename file
rmdir	remove directory on the remote machine
send	send one file
status	show current status
trace	toggle packet tracing
type	set file transfer type
user	send new user information
verbose	toggle verbose mode

Creating Unattended FTP Scripts

Suppose an interactive FTP session looks like this (user input shown in bold italics):

C:\>ftp ftp.myhost.net Connected to ftp.myhost.net. 220 *** FTP SERVER IS READY *** User (ftp.myhost.net:(none)): MyUserId 331 Password required for MyUserId. Password: **** 230- Welcome to the FTP site 230- Available space: 8 MB 230 User MyUserId logged in. ftp> cd files/pictures 250 CWD command successful. "files/pictures" is current directory. ftp> binary 200 Type set to B. ftp> prompt n Interactive mode Off. ftp> mget *.* 200 Type set to B. 200 Port command successful. 150 Opening data connection for firstfile.jpg. 226 File sent ok 649 bytes received in 0.00 seconds (649000.00 Kbytes/sec) 200 Port command successful. 150 Opening data connection for secondfile.gif. 226 File sent ok 467 bytes received in 0.00 seconds (467000.00 Kbytes/sec) ftp> bye 221 Goodbye. C:\>

An FTP script for unattended file transfer would then look like this:

USER MyUserId
MyPassword
cd files/pictures
binary
prompt n
mget *.*

Note that I left out the BYE (or QUIT) command, it isn't necessary to specify this command in unattended FTP scripts (though it doesn't do any harm either).
As you can see, using a script like this is a potential security risk: the password is stored in the script in a readable form.
As Tom Lavedas once pointed out in the alt.msdos.batch newsgroup, it is safer to create the script "on the fly" and delete it afterwards:

@ECHO OFF
:: Check if the password was given
IF "%1"=="" GOTO Syntax
:: Create the temporary script file
> script.ftp ECHO USER MyUserId
>>script.ftp ECHO %1
>>script.ftp ECHO cd files/pictures
>>script.ftp ECHO binary
>>script.ftp ECHO prompt n
>>script.ftp ECHO mget *.*
:: Use the temporary script for unattended FTP
:: Note: depending on your OS version you may have to add a '-n' switch
FTP -v -s:script.ftp ftp.myhost.net
:: For the paranoid: overwrite the temporary file before deleting it
TYPE NUL >script.ftp
DEL script.ftp
GOTO End

:Syntax
ECHO Usage: %0 password

:End

Sometimes it may be necessary to make the script completely unattended, without the user having to know the password, or even the user ID, but with the possibility to check for errors during transfer.
There are several ways to do this.
One is to redirect FTP's output to a log file and either display it to the user or use FIND to search the log file for any error messages.
Another way to do this, on the fly, is by displaying FTP's output on screen, in the mean time using FIND /V to hide the output you do not want the user to see (like the password and maybe even the USER command):

FTP -s:script.ftp ftp.myhost.net | FIND /V "USER" | FIND /V "%1"

It is important not to use FTP's -v switch in either case.

Summary

To create a semi interactive FTP script, you may need to split it into several smaller parts, like an unattended FTP script to read a list of remote files, the output of which is redirected to a temporary file, which in turn is used by a batch file to create a new unattended FTP script on the fly to download and/or delete some of these files.
Create these files by writing down every command and all screen output in an interactive FTP session, analyze this "log" thoroughly, and test, test, and test again!
And don't forget to log the results by redirecting the script's output to a log file. You may need it later for debugging purposes...

Alternatives

Instead of Windows' own native FTP command, you can choose from a multitude of "third party" alternatives.
I'll discuss three of those alternatives here: a command-line tool, a GUI-tool and VBScript with a third party ActiveX component.

Note:

The alternatives discussed here, GNU WGET, Fileaze, and VBScript (with WinHTTP), handle HTTP downloads just as easily.

WGET

WGET is a port of the UNIX wget command.
WGET is perfect for anonymous FTP or HTTP downloads (sorry, no uploads), but it can be used for downloads requiring authentication too.
GNU WGET comes with help both in the (text mode) console and in Windows Help format.
The basic syntax for an FTP download doesn't get any simpler than this:
WGET ftp://ftp.mydomain.com/path/file.ext
for anonymous downloads, or:
WGET ftp://user:password@ftp.mydomain.com/path/file.ext
when authentication is required.

Note:

This is not secure, as you would need to store your user ID and password in unencrypted format in the batch file. Besides that, the user ID and password will be logged together with the rest of the URL on all servers associated with the file transfer. Read the GNU WGET help file for more information on securing user IDs and passwords.

WinSCP

WinSCP is a free open-source SFTP and FTP client with a command line/scripting interface as well as a GUI.
WinSCP can be used for uploads and downloads.

Fileaze

Prefer a GUI to create interactive as well as unattended jobs?
To be honest, I hardly ever use the text mode native FTP command myself.
For these web pages, my photographs, or any other FTP job, I have always used Coffeecup FreeFTP (no longer available, unfortunately).
Recently, I automated my own web page uploads using Fileaze.
Fileaze is a great GUI program to automate file related tasks like rename, copy, upload and download, send by e-mail, etcetera, in batch (no, not in batch files but in batch mode, i.e. in groups of files, and unattended).
Have a look at the tutorial I wrote, based on my own "efforts" to generate an automated FTP upload job.
And did I mention the uploads are fast?
You can download the free Fileaze trial version or buy it here.
You can also download the free Fileaze LITE version from the Fileaze website. The LITE version may be used indefinitely, without limitations, but it has limited functionality (i.e. no FTP and no e-mail).
Make sure you read the tutorials included in the help file.
The program may not be that hard to learn without help, but speaking from experience: reading them will save you a lot of time.
And once you created one or more jobs, make sure you make a backup.
Fileaze stores the data (jobs, encrypted account settings, etcetera) in the registry key "HKEY_CURRENT_USER\Software\Resolware".
The registry as a whole is included in Windows's SystemState backups, but it doesn't support selectively restoring registry keys.
So (of course) I made myself a batch file which I scheduled, to make unattended backups. View this batch file's source, or download the ZIPped batch file (for Windows NT 4 and later).

VBScript

In my VBScript Scripting Techniques section, sample scripts are available for FTP and HTTP file transfers.
The FTP scripts require one of these free ActiveX components:

• Chilkat FTP ActiveX
• Sapien FTP Automation Component