Deleting files older than N days from FTP server

Many hosting companies provide customers with FTP space for storing their backups (e.g. Hetzner, HostEurope).

Problem is that unless you want the backup space to fill up after a while you have two possibilities:

  • use a naming scheme for your backups which overwrites old files like hostname-DAYOFWEEK.tar.gz, hostname-DAYOFMONTH.tar.gz
  • automatically delete old files from your FTP space

I like to name my backups with the full date, e.g. hostname-YYYY-MM-DD.tar.gz, so I went for the latter.

Unfortunately deleting old files from an FTP server is not as simple as running find . -mtime +30 -delete because usually you don’t get shell access to your FTP space. Everything must be done via FTP.

Here comes a simple perl script I wrote that does the trick. It requires the Net::FTP module.

Hope it will be useful to you too!
#!/usr/bin/perl

use strict;
use Net::FTP;

my $host        = "ftp.hostname.com";
my $dir         = "/";
my $user        = "foo";
my $passwd      = "bar";
my $interval    = 3600*24*62; # 62 day
my $passive     = 1; # my FTP server only works in PASV mode
my $DEBUG       = 0;

my $ftp = Net::FTP->new($host,
 Debug => $DEBUG,
 Passive => $passive)
 or die "Can't open $host\n";
$ftp->login($user, $passwd) or die "Can't log in as $user\n";
$ftp->cwd($dir) or die "Can't chdir to $dir\n";
my @files = $ftp->ls();

foreach my $file (@files) {
my $file_mdtm = $ftp->mdtm($file) or die "Can't find $file in $dir\n";
if (time - $file_mdtm >= $interval) {
print "File $file is older than $interval secs: deleted\n";
$ftp->delete($file) unless $DEBUG;
}
}

The password is stored in the script itself so make sure permissions are set properly! You don’t want it to be world readable :)

chmod 700 ./ftp-purge-backups.pl

Share

9 thoughts on “Deleting files older than N days from FTP server

  1. This was just what I was looking for. Awesome of you to share. I just setup a new dedicated server with a hosting company. It’s a Windows 2008 iis7 and I just installed the latest 64bit version of perl from Active State successfully.

    Though — How can I check to see if the Net::FTP module is installed? And if it’s not, do you know anywhere there is info on downloading and installing it? I have googled with no luck. (I know it’s hard to believe)

  2. I’m a big newb when it comes to perl. That being said, how would you incorporate only deleting a certain file type into this script? Thanks!

  3. If by “file type” you mean “file extension”, then it could be as simple as modifying the script like this:

    if ( (time - $file_mdtm >= $interval) and ($file =~ /\.txt/i) ) {

    This would delete all files with extension .txt, case insensitive.

  4. I spoke a bit too soon. I didn’t notice it wasn’t working right away, due to the fact that I needed to wait a week to find out. When I added the addition of the file extension, I now simply get an error stating this:

    “Can’t find in outgoing/prod”

    So investigating a little further, I added a line to the top of the script “use diagnostics;”. Now I get this error:

    Use of uninitialized value $_ in pattern match (m//) at C:/Perl/lib/Net/Cmd.pm
    line 222 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined. It was interpreted as a “” or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you the
    name of the variable (if any) that was undefined. In some cases it cannot
    do this, so it also tells you what operation you used the undefined value
    in. Note, however, that perl optimizes your program and the operation
    displayed in the warning may not necessarily appear literally in your
    program. For example, “that $foo” is usually optimized into “that ”
    . $foo, and the warning will refer to the concatenation (.) operator,
    even though there is no . in your program.

    Use of uninitialized value in join or string at C:/Perl/lib/Net/Cmd.pm line 221
    (#1)
    Use of uninitialized value $file in concatenation (.) or string at
    D:\UHCFTP\script.pl line 23 (#1)
    Uncaught exception from user code:
    Can’t find in outgoing/prod
    at D:\UHCFTP\script.pl line 23

    I used the package manager to make sure net::Ftp was installed
    I even tried that command you listed up above for someone else, and it seems to be installed, however when I run that command I get:

    D:\UHCFTP>perl -MCPAN -we install "Net::FTP"

    It looks like you don’t have a C compiler and make utility installed. Trying
    to install dmake and the MinGW gcc compiler using the Perl Package Manager.
    This may take a a few minutes…

    Downloading MinGW-5.1.4.1…done
    Downloading dmake-4.11.20080107…done
    Unpacking MinGW-5.1.4.1…done
    Unpacking dmake-4.11.20080107…done
    Generating HTML for MinGW-5.1.4.1…done
    Generating HTML for dmake-4.11.20080107…done
    Updating files in site area…done
    1070 files installed

    Please use the dmake program to run commands from a Makefile!

    Set up gcc environment – 3.4.5 (mingw-vista special r3)
    Unrecognized character \x91 in column 1 at -e line 1.

    D:\UHCFTP>perl -MCPAN -we install "Net::FTP"
    Set up gcc environment – 3.4.5 (mingw-vista special r3)
    Unrecognized character \x91 in column 1 at -e line 1.

    Any help would be appreciated… I can’t seem to find any other script or article that attempts to do what this script can. Thank you.

  5. sdjustin005: first of all, you do have Net::FTP installed, otherwise the script would die immediately, and not at line 25.

    In my previous post I suggested to modify line 26 from:

    if (time – $file_mdtm >= $interval) {

    to:

    if ( (time – $file_mdtm >= $interval) and ($file =~ /\.txt/i) ) {

    Did you touch anything else?

    Is it possible that another process is deleting the same files while the script is running? Your error message seems to indicate exactly that.
    Did you do that?

  6. Thanks for the response…

    I am sure that nothing else is deleting the files. All the files are still there and I can login and manually delete them from an ftp clients.

    Secondly, here is my script that I am running (removed the IP, username & pass):

    #!/usr/bin/perl

    use diagnostics;
    use strict;
    use Net::FTP;

    my $host = “1.2.3.4”;
    my $dir = “outgoing/prod”;
    my $user = “USERNAME”;
    my $passwd = “PASSWORD”;
    my $interval = 3600*24*8; # 8 days
    my $passive = 1; # my FTP server only works in PASV mode
    my $DEBUG = 0;

    my $ftp = Net::FTP->new($host,
    Debug => $DEBUG,
    Passive => $passive)
    or die “Can’t open $host\n”;
    $ftp->login($user, $passwd) or die “Can’t log in as $user\n”;
    $ftp->cwd($dir) or die “Can’t chdir to $dir\n”;
    my @files = $ftp->get(“*.zip”);

    foreach my $file (@files) {
    my $file_mdtm = $ftp->mdtm($file) or die “Can’t find $file in $dir\n”;
    if ( (time – $file_mdtm >= $interval) and ($file =~ /\.zip/i) ) {
    print “File $file is older than $interval secs: deleted\n”;
    $ftp->delete($file) unless $DEBUG;
    }
    }

    Let me know what you think. Thank you!

  7. Ok, so I figured out what it was….

    When I was messing with the script trying to get it to also only delete the zip files, I changed the line:
    my @files = $ftp->ls();

    to

    my @files = $ftp->get(“*.zip”);

    Then when you gave me the correct modification I never switched it back. My apologies and thank you again!

Leave a Reply