D&C GLug - Home Page

[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]

Re: [LUG] Large Backups - Advice?

 

On 11/10/10 16:33, Gibbs wrote:
  Hi all,

I've been assigned a task (in a couple of weeks) to automate the way our
company backs up its data.

Currently we have an inhouse server (Ubuntu Server 10.04), 2 computers
(one Windows Vista, one Ubuntu 10.10) and a laptop (Windows Vista again
sigh)... I have no expertise in the field of backups whatsoever but I
manage our nifty little server. We need offsite backups.

What I was thinking... Sync all computers to the server and then have
the server periodically archive (password protect?) then upload to some
remote location. I have no idea what the remote location will be but I
assume it would be relatively cheap. The computers syncing would
probably make the backup size in the region of 100-200GB? I can't be
sure yet but that's probably a reasonably accurate estimate.

So. Any advice, solutions or ideas?

Cheers,

Gibbs



I've done this before for a client who was a radio station (so 600GB+ of uncompressed WAV audio) over an ADSL connection.

At first they were simply running a copy between the two sites which were linked via VPN. Both sites had reasonably slow ADSL connections (2 Meg and 1 Meg) with low upload speeds (IIRC they were 256K upload speeds). A simple copy didn't work, they were lucky if 2 or 3 files got uploaded every night.

In the end I implemented rsync over SSH for them. I basically did a first copy of everything they had on the server to a USB drive (which ran over a weekend) and then this drive went to the remote site.

From there I got rsync to backup only the files which had changed (probably about 10 or so files). I also enabled gzip compression to try and help speed things up a bit.

I've also done this with virtual machine images too, rather than going on time stamps, I've rsynced the files using checksums. I understand that only the changes in the files are copied over again saving on bandwidth.

Either way I'm guessing that any changes to files that you'd have wouldn't run into the hundreds of gigs every night.

Using SSH with rsync means that the data when it is being transferred is encrypted. Mainly this is because the remote host is a trusted machine (usually at another site or at a directors home).

If however you are looking at changes of hundreds of gigs per night then maybe you might want to look into backing up on to tape/USB or SATA hard drive every night and having a few drives to swap around so you have an up to date off-site backup. Personally I'd suggest LTO tape as it's much quicker than a hard drive (that's working on Veritas Backup Exec backing up 5 servers, approx 100GB of data on to a USB 2.0 1TB hard drive and the same data on to a LTO-2 tape).

Rob

--
The Mailing List for the Devon & Cornwall LUG
http://mailman.dclug.org.uk/listinfo/list
FAQ: http://www.dcglug.org.uk/listfaq