Backup strategy for Unix

[Bob Daniel]

In article <6226 at spdcc.SPDCC.COM> rbraun at spdcc.COM (Rich Braun) writes:
>
>The backup software which comes with SCO Unix is obsolete, compared with
>software I've seen for other systems.  I'd like to have something which
>has enough flexibility and reliability to handle backups totally automatically
>without requiring a lot of expertise on the part of a sys admin.  (If I
>don't have software to do this, guess who gets stuck with the task...)
>
>I can probably talk my company into paying decent money for the right
>software.  If you've had experience setting this type of thing up and can
>offer your opinions, I'd love to get your advice.  And if I've missed
>something about 'backup' which redeems itself as a useful utility, I'd
>also be interested in gaining insight.


I'm system administrator for about 70 systems and have written my own auto
backup program because I have not seen one that I liked and I never know if
I can trust it.

First of all, a backup that doesn't verify is not a backup.  Make sure your
backup program does a verify.  It should also be documented about how it goes
about doing the verify.  A simple index of the tape doesn't hack it in my book.

Make sure you know how the incremental backup works.  There are two kinds. One
kind will backup all files changed since the last COMPLETE backup.  The other 
will backup changed files since the last INCREMENTAL backup.  I would only use a
backup program that does its incremental backups from the last COMPLETE backup.
Example:  If you did a COMPLETE backup on Friday, Mondays incremental backup
will backup changed files since Friday.  When doing Tuesdays incremental backup,
it should backup changed files since Friday's complete backup, NOT the changed 
files since Monday's incremental. This is a real mess if you don't understand
how the incremental works.

How I do backups...

I recommend doing complete backups atleast once a month.  Incremental backup
is then automatically done using the cron.  I have 5 tapes for Monday-Friday
incremental backups.  The tape for that day is put in before going home.  The
cron then starts up an incremental in the middle of the night.  It does a
double verify, and then mails the system administrator the results of the
backup.  


Suprisingly, there are many backup programs that don't do a verify.  Then
there are some that will only rescan the tape to show a table of contents of
the tape which only checks for a cpio error.  The way I perform a verify is
by first creating an index of the files that will be stored.  Backup the files
in the index, and then rescan the tape and compare it against the index. This
verify takes a long time but because I use the cron to perform it in the
middle of the night, it doesn't waste anyones time.

If a tape is not in the drive or the verify fails, the system administrator
will get mail about the error.  If the backup is successful, the sys adm still
gets mail that it was successful and states how many blocks were backed up.
Also, if the incremental is too large to fit on the tape, the sysadm is 
notified that it is time to do a complete backup which will then allow
incrementals to fit on one tape.

The best method I found to do incrementals is by backing up files based on
the time marked on a special file created (/etc/.lastbackup) when the complete
backup was made.  Then 'find' all files newer than /etc/.lastbackup.

I've been using this method for a couple years and have had to restore crashed
systems with backups using this method.  It has proven to be a reliable method.


If your tape drive is large enough to hold a complete backup on a single tape,
then incrementals aren't necessary.  I'd do a complete backup daily and assign
it to the cron with the same verify method mentioned above. Then, when a
complete backup get's to large to fit on one tape, goto incrementals.


Some sys adm's will do a complete backup once a month and then just do a
SELECTIVE backup each day.  This is when you only specify 'important' files
to be backed up.  This may be OK but it doesn't backup ALL changed files
throughout the rest of the system.  I'd recommend using incrementals over
selectives.


Now for tapes... 
Never only have one tape for incremental and one tape for complete backups.
You should rotate several tapes for incremental (I use 5), and rotate atleast
more than one for complete (I use 3).  For one thing, the more tapes you have,
the less frantic you will be when you get a crashed hard disk <murphy's law>.
But the other advantage of several tapes is that if a file got screwed up
a few days ago and you don't realize till now, you can go back to the tape
5 days ago and get the file back. The more tapes the better.


Whatever backup program you choose to use, make sure you understand how it
does it's backups.  Make sure it verifies and that the docs describe how the
verify is performed (unfortunately not too many do).  I have not seen a backup
program I was satisfied with so wrote my own.  

Another thing.  I personally would use a backup program that is written through
a shell script than one that is compiled.  That way, I can see how it is done.