=item *
+The http/cgi user interface has internationalization (i18n) support,
+currently providing English, French, German, Spanish and Italian.
+
+=item *
+
No client-side software is needed. On WinXX the standard smb
protocol is used to extract backup data. On linux or unix clients,
-tar over ssh/rsh/nfs is used to extract backup data (or alternatively
-Samba can be installed on the linux or unix client to provide smb shares).
+rsync or tar (over ssh/rsh/nfs) is used to extract backup data.
+Alternatively, rsync can also be used on WinXX (using cygwin),
+and Samba could be installed on the linux or unix client to
+provide smb shares).
=item *
any backup directly from the CGI interface. Zip or Tar archives
for selected files or directories from any backup can also be
downloaded from the CGI interface. Finally, direct restore to
-the client machine (using SMB or tar) for selected files or
+the client machine (using smb or tar) for selected files or
directories is also supported from the CGI interface.
=item *
=item Full Backup
-A full backup is a complete backup of a share. BackupPC can be configured to
-do a full backup at a regular interval (often weekly). BackupPC can also
-be configured to keep a certain number of full backups, and to keep
-a smaller number of very old full backups.
+A full backup is a complete backup of a share. BackupPC can be
+configured to do a full backup at a regular interval (typically
+weekly). BackupPC can be configured to keep a certain number
+of full backups. Exponential expiry is also supported, allowing
+full backups with various vintages to be kept (for example, a
+settable number of most recent weekly fulls, plus a settable
+number of older fulls that are 2, 4, 8, or 16 weeks apart).
=item Incremental Backup
An incremental backup is a backup of files that have changed (based on their
-modification time) since the last successful full backup. To be safe,
-BackupPC backups all files that have changed since one hour prior to the
-start of the last successful full backup. BackupPC can also be configured
-to keep a certain number of incremental backups, and to keep a smaller
-number of very old incremental backups. (BackupPC does not support
-multi-level incremental backups, although it would be easy to do so.)
+modification time) since the last successful full backup. For SMB and
+tar, BackupPC backups all files that have changed since one hour prior to the
+start of the last successful full backup. Rsync is more clever: any files
+who attributes have changed (ie: uid, gid, mtime, modes, size) since the
+last full are backed up. Deleted and new files are also detected by
+Rsync incrementals (SMB and tar are not able to detect deleted files or
+new files whose modification time is prior to the last full dump.
+
+BackupPC can also be configured to keep a certain number of incremental
+backups, and to keep a smaller number of very old incremental backups.
+(BackupPC does not support multi-level incremental backups, although it
+would be easy to do so.)
BackupPC's CGI interface "fills-in" incremental backups based on the
last full backup, giving every backup a "full" appearance. This makes
browsing and restoring backups easier.
+=item Partial Backup
+
+When a full backup fails or is canceled, and some files have already
+been backed up, BackupPC keeps a partial backup containing just the
+files that were backed up successfully. The partial backup is removed
+when the next successful backup completes, or if another full backup
+fails resulting in a newer partial backup. A failed full backup
+that has not backed up any files, or any failed incremental backup,
+is removed; no partial backup is saved in these cases.
+
+The partial backup may be browsed or used to restore files just like
+a successful full or incremental backup.
+
+With the rsync transfer method the partial backup is used to resume
+the next full backup, avoiding the need to retransfer the file data
+already in the partial backup.
+
=item Identical Files
BackupPC pools identical files using hardlinks. By "identical
likely means an exact image of a linux/unix file system can be made.
BackupPC saves backups onto disk. Because of pooling you can relatively
-economically keep several weeks of old backups. But BackupPC does not
-provide permanent storage to tape. Other Open Source applications can do
-this by backing up BackupPC's pool directories to tape.
+economically keep several weeks of old backups.
At some sites the disk-based backup will be adequate, without a
secondary tape backup. This system is robust to any single failure: if a
of the server disk failing can be made very small by spending more money
on increasingly better RAID systems.
-At other sites a secondary tape backup will be required. This tape
-backup can be done perhaps weekly from the BackupPC pool file system.
-
-One comment: in the US in particular, permanent backups of things like
-email are becoming strongly discouraged by lawyers because of discovery
-prior to possible litigation. Using BackupPC without tape backup allows
-recent file changes or losses to be restored, but without keeping a
-history more than a month or two old (although this doesn't avoid the
-problem of old emails languishing in user's email folders forever).
+At other sites a secondary tape backup or cd/dvd will be required. This
+backup can be done perhaps weekly using the archive function of BackupPC.
=back
=item Mail lists
-Two BackupPC mailing lists exist for announcements (backuppc-announce)
-and reporting information, asking questions, discussing development or
-any other topic relevant to BackupPC (backuppc-users).
+Three BackupPC mailing lists exist for announcements (backuppc-announce),
+developers (backuppc-devel), and a general user list for support, asking
+questions or any other topic relevant to BackupPC (backuppc-users).
-You are encouraged to subscribe to either the backuppc-announce
-or backuppc-users mail list on sourceforge.net at either:
+You can subscribe to these lists by visiting:
http://lists.sourceforge.net/lists/listinfo/backuppc-announce
http://lists.sourceforge.net/lists/listinfo/backuppc-users
+ http://lists.sourceforge.net/lists/listinfo/backuppc-devel
The backuppc-announce list is moderated and is used only for
important announcements (eg: new versions). It is low traffic.
-You only need to subscribe to one list: backuppc-users also
-receives any messages on backuppc-announce.
+You only need to subscribe to one of backuppc-announce and
+backuppc-users: backuppc-users also receives any messages on
+backuppc-announce.
+
+The backuppc-devel list is only for developers who are working on BackupPC.
+Do not post questions or support requests there. But detailed technical
+discussions should happen on this list.
To post a message to the backuppc-users list, send an email to
=item Other Programs of Interest
If you want to mirror linux or unix files or directories to a remote server
-you should look at rsync, L<http://rsync.samba.org>.
-
-Two popular open source packages that do tape backup are
-Amanda (L<http://www.amanda.org>) and
-afbackup (L<http://sourceforge.net/projects/afbackup>).
+you should consider rsync, L<http://rsync.samba.org>. BackupPC now uses
+rsync as a transport mechanism; if you are already an rsync user you
+can think of BackupPC as adding efficient storage (compression and
+pooling) and a convenient user interface to rsync.
+
+Unison is a utility that can do two-way, interactive, synchronization.
+See L<http://www.cis.upenn.edu/~bcpierce/unison>.
+
+Three popular open source packages that do tape backup are
+Amanda (L<http://www.amanda.org>),
+afbackup (L<http://sourceforge.net/projects/afbackup>), and
+Bacula (L<http://www.bacula.org>).
Amanda can also backup WinXX machines to tape using samba.
These packages can be used as back ends to BackupPC to backup the
BackupPC server data to tape.
+Various programs and scripts use rsync to provide hardlinked backups.
+See, for example, Mike Rubel's site (L<http://www.mikerubel.org>),
+J. W. Schultz's dirvish (L<http://www.pegasys.ws/dirvish>),
+Ben Escoto's rdiff-backup (L<http://rdiff-backup.stanford.edu>),
+and John Bowman's rlbackup (L<http://www.math.ualberta.ca/imaging/rlbackup>).
+
+BackupPC provides many additional features, such as compressed storage,
+hardlinking any matching files (rather than just files with the same name),
+and storing special files without root privileges. But these other scripts
+provide simple and effective solutions and are worthy of consideration.
+
=back
=head2 Road map
=item *
-Adding support for rsync as a transport method, in addition to
-smb and tar. This will give big savings in network traffic for
-linux/unix clients. I haven't decided whether to save the pool file
-rsync checksums (that would double the number of files in the pool, but
-eliminate most server disk reads), or recompute them every time. I expect
-to use native rsync on the client side. On the server, rsync would
-need to understand the compressed file format, the file name mangling
-and the attribute files, so I will either have to add features to rsync
-or emulate rsync on the server side in perl.
+Adding hardlink support to rsync.
=item *
=item *
-Resuming incomplete completed full backups. Useful if a machine
-(eg: laptop) is disconnected from the network during a backup,
-or if the user manually stops a backup. This would work by
-excluding directories that were already complete.
+Allow editing of config parameters via the CGI interface. Users should
+have permission to edit a subset of the parameters for their clients.
+Additionally, allow an optional self-service capability so that users
+can sign up and setup their own clients with no need for IT support.
+
+=item *
+
+Add backend SQL support for various BackupPC metadata, including
+configuration parameters, client lists, and backup and restore
+information. At installation time the backend data engine will
+be specified (eg: MySQL, ascii text etc).
+
+=item *
+
+Replacing smbclient with the perl module FileSys::SmbClient. This
+gives much more direct control of the smb transfer, allowing
+incrementals to depend on any attribute change (eg: exist, mtime,
+file size, uid, gid), and better support for include and exclude.
+Currently smbclient incrementals only depend upon mtime, so
+deleted files or renamed files are not detected. FileSys::SmbClient
+would also allow resuming of incomplete full backups in the
+same manner as rsync will.
+
+=item *
+
+Support --listed-incremental or --incremental for tar,
+so that incrementals will depend upon any attribute change (eg: exist,
+mtime, file size, uid, gid), rather than just mtime. This will allow
+tar to be to as capable as FileSys::SmbClient and rsync.
+
+=item *
+
+For rysnc (and smb when FileSys::SmbClient is supported, and tar when
+--listed-incremental is supported) support multi-level incrementals.
+In fact, since incrementals will now be more "accurate", you could
+choose to never to full dumps (except the first time), or at a
+minimum do them infrequently: each incremental would depend upon
+the last, giving a continuous chain of differential dumps.
=item *
around 15-20%, which isn't spectacular, and likely not worth the
implementation effort. The program xdelta (v1) on SourceForge (see
L<http://sourceforge.net/projects/xdelta>) uses an rsync algorithm for
-doing efficient binary file deltas.
+doing efficient binary file deltas. Rather than using an external
+program, File::RsyncP will eventually get the necessary delta
+generation code from rsync.
=back
evaluated BackupPC but didn't use it because it doesn't ...".
Beyond being a satisfied user and telling other people about it, everyone
-is encouraged to add links to backuppc.sourceforge.net (I'll see then
-via Google) or otherwise publicize BackupPC. Unlike the commercial
-products in this space, I have a zero budget (in both time and money)
-for marketing, PR and advertising, so it's up to all of you!
+is encouraged to add links to L<http://backuppc.sourceforge.net> (I'll
+see then via Google) or otherwise publicize BackupPC. Unlike the
+commercial products in this space, I have a zero budget (in both
+time and money) for marketing, PR and advertising, so it's up to
+all of you!
Also, everyone is encouraged to contribute patches, bug reports, feature
and design suggestions, code, and documentation corrections or
=item *
Perl version 5.6.0 or later. BackupPC has been tested with
-version 5.6.0 and 5.6.1. If you don't have perl, please see
-L<http://www.cpan.org>.
+version 5.6.0, 5.6.1 and 5.8.0. If you don't have perl, please
+see L<http://www.cpan.org>.
=item *
-Perl modules Compress::Zlib and Archive::Zip. Try "perldoc Compress::Zlib"
-and "perldoc Archive::Zip" to see if you have these modules. If not,
-fetch them from L<http://www.cpan.org> and see the instructions below
-for how to build and install them.
+Perl modules Compress::Zlib, Archive::Zip and File::RsyncP. Try "perldoc
+Compress::Zlib" and "perldoc Archive::Zip" to see if you have these
+modules. If not, fetch them from L<http://www.cpan.org> and see the
+instructions below for how to build and install them.
+
+The File::RsyncP module is available from L<http://perlrsync.sourceforge.net>
+or CPAN. You'll need to install the File::RsyncP module if you want to use
+Rsync as a transport method.
=item *
If you are using smb to backup WinXX machines you need smbclient and
nmblookup from the samba package. You will also need nmblookup if
you are backing up linux/unix DHCP machines. See L<http://www.samba.org>.
-Version >= 2.2.0 of Samba is required (smbclient's tar feature in
+Version 2.2.0 or later of Samba is required (smbclient's tar feature in
2.0.8 and prior has bugs for file path lengths around 100 characters
and generates bad output when file lengths change during the backup).
1.13.7 at a minimum, with version 1.13.20 or higher recommended. Use
"tar --version" to check your version. Various GNU mirrors have the newest
versions of tar, see for example L<http://www.funet.fi/pub/gnu/alpha/gnu/tar>.
-As of July 2002 the latest versons is 1.13.25.
+As of June 2003 the latest version is 1.13.25.
+
+=item *
+
+If you are using rsync to backup linux/unix machines you should have
+version 2.5.5 or higher on each client machine. See
+L<http://rsync.samba.org>. Use "rsync --version" to check your
+version.
+
+For BackupPC to use Rsync you will also need to install the perl
+File::RsyncP module, which is available from
+L<http://perlrsync.sourceforge.net>. Version 0.44 or later is required.
=item *
cannot be read by smbclient whenever Outlook is running. See the
L<Limitations|limitations> section for more discussion of this problem.
+In addition to total disk space, you shold make sure you have
+plenty of inodes on your BackupPC data partition. Some users have
+reported running out of inodes on their BackupPC data partition.
+So even if you have plenty of disk space, BackupPC will report
+failures when the inodes are exhausted. This is a particular
+problem with ext2/ext3 file systems that have a fixed number of
+inodes when the file system is built. Use "df -i" to see your
+inode usage.
+
=head2 Step 1: Getting BackupPC
Download the latest version from L<http://backuppc.sourceforge.net>.
=head2 Step 2: Installing the distribution
-First off, to enable compression, you will need to install Compress::Zlib
-from L<http://www.cpan.org>. It is optional, but strongly recommended.
-Also, to support restore via Zip archives you will need to install
-Archive::Zip, also from L<http://www.cpan.org>. You can run
-"perldoc Compress::Zlib" to see if this module is installed.
-To build and install these packages you should run these commands:
+First off, there are three perl modules you should install.
+These are all optional, but highly recommended:
+
+=over 4
+
+=item Compress::Zlib
+
+To enable compression, you will need to install Compress::Zlib
+from L<http://www.cpan.org>.
+You can run "perldoc Compress::Zlib" to see if this module is installed.
+
+=item Archive::Zip
+
+To support restore via Zip archives you will need to install
+Archive::Zip, also from L<http://www.cpan.org>.
+You can run "perldoc Archive::Zip" to see if this module is installed.
+
+=item File::RsyncP
+
+To use rsync and rsyncd with BackupPC you will need to install File::RsyncP.
+You can run "perldoc File::RsyncP" to see if this module is installed.
+File::RsyncP is available from L<http://perlrsync.sourceforge.net>.
+Version 0.44 or later is required.
+
+=back
+
+To build and install these packages, fetch the tar.gz file and
+then run these commands:
tar zxvf Archive-Zip-1.01.tar.gz
cd Archive-Zip-1.01
make test
make install
+The same sequence of commands can be used for each module.
+
Now let's move onto BackupPC itself. After fetching
BackupPC-__VERSION__.tar.gz, run these commands as root:
=item *
-As an environment variable PASSWD set before BackupPC starts. If you
-start BackupPC manually this PASSWD must be set manually first.
+As an environment variable BPC_SMB_PASSWD set before BackupPC starts.
+If you start BackupPC manually the BPC_SMB_PASSWD variable must be set
+manually first. For backward compatibility for v1.5.0 and prior, the
+environment variable PASSWD can be used if BPC_SMB_PASSWD is not set.
+Warning: on some systems it is possible to see environment variables of
+running processes.
=item *
-Alternatively the PASSWD setting can be included in /etc/init.d/backuppc,
-in which case you must make sure this file is not world (other) readable.
+Alternatively the BPC_SMB_PASSWD setting can be included in
+/etc/init.d/backuppc, in which case you must make sure this file
+is not world (other) readable.
=item *
=item Host name
-If this host is a static IP address this must the machine's IP host name
-(ie: something that can be looked up using nslookup or DNS). If this is
-a host with a dynamic IP address (ie: DHCP flag is 1) then the host
-name must be the netbios name of the machine. The host name should
-be in lower case.
+This is typically the host name or NetBios name of the client machine
+and should be in lower case. The host name can contain spaces (escape
+with a backslash), but it is not recommended.
+
+Please read the section L<How BackupPC Finds Hosts|how backuppc finds hosts>.
+
+In certain cases you might want several distinct clients to refer
+to the same physical machine. For example, you might have a database
+you want to backup, and you want to bracket the backup of the database
+with shutdown/restart using $Conf{DumpPreUserCmd} and $Conf{DumpPostUserCmd}.
+But you also want to backup the rest of the machine while the database
+is still running. In the case you can specify two different clients in
+the host file, using any mnemonic name (eg: myhost_mysql and myhost), and
+use $Conf{ClientNameAlias} in myhost_mysql's config.pl to specify the
+real host name of the machine.
=item DHCP flag
-Set to 0 if this host has a static IP address (meaning it can be looked
-up by name in the DNS). If the host's IP address is dynamic (eg, it is
-assigned by DHCP) then set this flag to 1.
+Starting with v2.0.0 the way hosts are discovered has changed and now
+in most cases you should specify 0 for the DHCP flag, even if the host
+has a dynamically assigned IP address.
+Please read the section L<How BackupPC Finds Hosts|how backuppc finds hosts>
+to understand whether you need to set the DHCP flag.
-The hosts with dhcp = 1 are backed up as follows. If you have
-configured a DHCP address pool ($Conf{DHCPAddressRanges}) then
-BackupPC will check the NetBIOS name of each machine in the
-range. Any hosts that have a valid NetBIOS name (ie: matching
-an entry in the hosts file) will be backed up.
+You only need to set DHCP to 1 if your client machine doesn't
+respond to the NetBios multicast request:
+
+ nmblookup myHost
+
+but does respond to a request directed to its IP address:
+
+ nmblookup -A W.X.Y.Z
+
+If you do set DHCP to 1 on any client you will need to specify the range of
+DHCP addresses to search is specified in $Conf{DHCPAddressRanges}.
+
+Note also that the $Conf{ClientNameAlias} feature does not work for
+clients with DHCP set to 1.
=item User name
receive email or be allowed to stop/start/browse/restore backups
for this host. Administrators will still have full permissions.
+=item More users
+
+Additional user names, separate by commas and with no white space,
+can be specified. These users will also have full permission in
+the CGI interface to stop/start/browse/restore backups for this host.
+These users will not be sent email about this host.
+
=back
The first non-comment line of the hosts file is special: it contains
Here's a simple example of a hosts file:
- host dhcp user
- farside 0 craig
- larson 1 gary
-
-The range of DHCP addresses to search is specified in
-$Conf{DHCPAddressRanges}.
+ host dhcp user moreUsers
+ farside 0 craig jim,dave
+ larson 1 gary andy
=head2 Step 5: Client Setup
-Two methods for getting backup data from a client are
-supported: smb and tar. Smb is the preferred method for WinXX clients
-and tar is preferred method for linux/unix clients.
+Two methods for getting backup data from a client are supported: smb and
+tar. Smb is the preferred method for WinXX clients and tar is preferred
+method for linux/unix clients.
The transfer method is set using the $Conf{XferMethod} configuration
setting. If you have a mixed environment (ie: you will use smb for some
=item WinXX
The preferred setup for WinXX clients is to set $Conf{XferMethod} to "smb".
-
-You need to create shares for the data you want to backup.
-Open "My Computer", right click on the drive (eg: C), and
-select "Sharing..." (or select "Properties" and select the
-"Sharing" tab). In this dialog box you can enable sharing,
-select the share name and permissions.
+(Actually, for v2.0.0, rsyncd is the better method for WinXX if you are
+prepared to run rsync/cygwin on your WinXX client. More information
+about this will be provided via the FAQ.)
+
+If you want to use rsyncd for WinXX clients you can find a pre-packaged
+zip file on L<http://backuppc.sourceforge.net>. The package is called
+cygwin-rsync. It contains rsync.exe, template setup files and the
+minimal set of cygwin libraries for everything to run. The README file
+contains instructions for running rsync as a service, so it starts
+automatically everytime you boot your machine.
+
+Otherwise, to use SMB, you need to create shares for the data you want
+to backup. Open "My Computer", right click on the drive (eg: C), and
+select "Sharing..." (or select "Properties" and select the "Sharing"
+tab). In this dialog box you can enable sharing, select the share name
+and permissions. Many machines will be configured by default to share
+the entire C drive as C$ using the administrator password.
If this machine uses DHCP you will also need to make sure the
NetBios name is set. Go to Control Panel|System|Network Identification
mounted file system).
Also, to make sure that file names with 8-bit characters are correctly
-transferred by smbclient you should add this to samba's smb.conf file:
+transferred by smbclient you should add this to samba's smb.conf file
+for samba 2.x:
[global]
# Accept the windows charset
client code page = 850
character set = ISO8859-1
+For samba 3.x this should instead be:
+
+ [global]
+ unix charset = ISO8859-1
+
This setting should work for western europe.
See L<http://www.oreilly.com/catalog/samba/chapter/book/ch08_03.html>
for more information about settings for other languages.
=item Linux/Unix
The preferred setup for linux/unix clients is to set $Conf{XferMethod}
-to "tar".
+to "rsync", "rsyncd" or "tar".
-You can use either smb or tar for linux/unix machines. Smb requires that
-the Samba server (smbd) be run to provide the shares. Since the smb
+You can use either rsync, smb, or tar for linux/unix machines. Smb requires
+that the Samba server (smbd) be run to provide the shares. Since the smb
protocol can't represent special files like symbolic links and fifos,
-tar is the recommended transport method for linux/unix machines.
+tar and rsync are the better transport methods for linux/unix machines.
(In fact, by default samba makes symbolic links look like the file or
directory that they point to, so you could get an infinite loop if a
symbolic link points to the current or parent directory. If you really
need to use Samba shares for linux/unix backups you should turn off the
"follow symlinks" samba config setting. See the smb.conf manual page.)
-The rest of this section describes the tar setup.
+The requirements for each Xfer Method are:
+
+=over 4
+
+=item tar
You must have GNU tar on the client machine. Use "tar --version"
or "gtar --version" to verify. The version should be at least
-1.13.7, and 1.13.20 or greater is recommended.
+1.13.7, and 1.13.20 or greater is recommended. Tar is run on
+the client machine via rsh or ssh.
+
+The relevant configuration settings are $Conf{TarClientPath},
+$Conf{TarShareName}, $Conf{TarClientCmd}, $Conf{TarFullArgs},
+$Conf{TarIncrArgs}, and $Conf{TarClientRestoreCmd}.
+
+=item rsync
+
+You should have at least rsync 2.5.5, and the latest version 2.5.6
+is recommended. Rsync is run on the remote client via rsh or ssh.
+
+The relevant configuration settings are $Conf{RsyncClientPath},
+$Conf{RsyncClientCmd}, $Conf{RsyncClientRestoreCmd}, $Conf{RsyncShareName},
+$Conf{RsyncArgs}, and $Conf{RsyncRestoreArgs}.
+
+=item rsyncd
+
+You should have at least rsync 2.5.5, and the latest version 2.5.6
+is recommended. In this case the rsync daemon should be running on
+the client machine and BackupPC connects directly to it.
+
+The relevant configuration settings are $Conf{RsyncdClientPort},
+$Conf{RsyncdUserName}, $Conf{RsyncdPasswd}, $Conf{RsyncdAuthRequired},
+$Conf{RsyncShareName}, $Conf{RsyncArgs}, and $Conf{RsyncRestoreArgs}.
+$Conf{RsyncShareName} is the name of an rsync module (ie: the thing
+in square brackets in rsyncd's conf file -- see rsyncd.conf), not a
+file system path.
+
+Be aware that rsyncd will remove the leading '/' from path names in
+symbolic links if you specify "use chroot = no" in the rsynd.conf file.
+See the rsyncd.conf manual page for more information.
+
+=back
-For linux/unix machines you should no backup "/proc". This directory
+For linux/unix machines you should not backup "/proc". This directory
contains a variety of files that look like regular files but they are
special files that don't need to be backed up (eg: /proc/kcore is a
regular file that contains physical memory). See $Conf{BackupFilesExclude}.
(eg: backing up /dev/hda5 just saves the block-special file information,
not the contents of the disk).
+Alternatively, rather than backup all the file systems as a single
+share ("/"), it is easier to restore a single file system if you backup
+each file system separately. To do this you should list each file system
+mount point in $Conf{TarShareName} or $Conf{RsyncShareName}, and add the
+--one-file-system option to $Conf{TarClientCmd} or add --one-file-system
+(note the different punctuation) to $Conf{RsyncArgs}. In this case there
+is no need to exclude /proc explicitly since it looks like a different
+file system.
+
Next you should decide whether to run tar over ssh, rsh or nfs. Ssh is
the preferred method. Rsh is not secure and therefore not recommended.
Nfs will work, but you need to make sure that the BackupPC user (running
files. Ssh is setup so that BackupPC on the server (an otherwise low
privileged user) can ssh as root on the client, without being prompted
for a password. There are two common versions of ssh: v1 and v2. Here
-are some instructions for one way to setup ssh v2:
+are some instructions for one way to setup ssh. (Check which version
+of SSH you have by typing "ssh" or "man ssh".)
+
+=item Mac OS X
+
+In general this should be similar to Linux/Unix machines.
+
+=over 4
+
+=item OpenSSH Instructions
+
+Depending upon your OpenSSH installation, many of these steps can be
+replaced by running the scripts ssh-user-config and ssh-host-config
+included with OpenSSH. You still need to manually exchange the keys.
+
+=over 4
+
+=item Key generation
+
+As root on the client machine, use ssh-keygen to generate a
+public/private key pair, without a pass-phrase:
+
+ ssh-keygen -t rsa -N ''
+
+This will save the public key in ~/.ssh/id_rsa.pub and the private
+key in ~/.ssh/id_rsa.
+
+=item BackupPC setup
+
+Repeat the above steps for the BackupPC user (__BACKUPPCUSER__) on the server.
+Make a copy of the public key to make it recognizable, eg:
+
+ ssh-keygen -t rsa -N ''
+ cp ~/.ssh/id_rsa.pub ~/.ssh/BackupPC_id_rsa.pub
+
+See the ssh and sshd manual pages for extra configuration information.
+
+=item Key exchange
+
+To allow BackupPC to ssh to the client as root, you need to place
+BackupPC's public key into root's authorized list on the client.
+Append BackupPC's public key (BackupPC_id_rsa.pub) to root's
+~/.ssh/authorized_keys2 file on the client:
+
+ touch ~/.ssh/authorized_keys2
+ cat BackupPC_id_rsa.pub >> ~/.ssh/authorized_keys2
+
+You should edit ~/.ssh/authorized_keys2 and add further specifiers,
+eg: from, to limit which hosts can login using this key. For example,
+if your BackupPC host is called backuppc.my.com, there should be
+one line in ~/.ssh/authorized_keys2 that looks like:
+
+ from="backuppc.my.com" ssh-rsa [base64 key, eg: ABwBCEAIIALyoqa8....]
+
+=item Fix permissions
+
+You will probably need to make sure that all the files
+in ~/.ssh have no group or other read/write permission:
+
+ chmod -R go-rwx ~/.ssh
+
+You should do the same thing for the BackupPC user on the server.
+
+=item Testing
+
+As the BackupPC user on the server, verify that this command:
+
+ ssh -l root clientHostName whoami
+
+prints
+
+ root
+
+You might be prompted the first time to accept the client's host key and
+you might be prompted for root's password on the client. Make sure that
+this command runs cleanly with no prompts after the first time. You
+might need to check /etc/hosts.equiv on the client. Look at the
+man pages for more information. The "-v" option to ssh is a good way
+to get detailed information about what fails.
+
+=back
+
+=item SSH2 Instructions
=over 4
=item Key generation
-As root on the client machine, use ssh2-keygen to generate a
+As root on the client machine, use ssh-keygen2 to generate a
public/private key pair, without a pass-phrase:
ssh-keygen2 -t rsa -P
+or:
+
+ ssh-keygen -t rsa -N ''
+
+(This command might just be called ssh-keygen on your machine.)
+
This will save the public key in /.ssh2/id_rsa_1024_a.pub and the private
key in /.ssh2/id_rsa_1024_a.
=item Fix permissions
You will probably need to make sure that all the files
-in /.ssh2 have no group or other read/write permission:
+in ~/.ssh2 have no group or other read/write permission:
- chmod -R go-rwx /.ssh2
+ chmod -R go-rwx ~/.ssh2
You should do the same thing for the BackupPC user on the server.
man pages for more information. The "-v" option to ssh2 is a good way
to get detailed information about what fails.
-=item ssh version 1 instructions
+=back
+
+=item SSH version 1 Instructions
The concept is identical and the steps are similar, but the specific
commands and file names are slightly different.
to BackupPC's ~/.ssh/config on the server.
-Next, run "chmod -R go-rwx ~/.ssh" on the server and "chmod -R go-rwx /.ssh"
+Next, run "chmod -R go-rwx ~/.ssh" on the server and "chmod -R go-rwx ~/.ssh"
on the client.
Finally, test using:
chmod u-s __CGIDIR__/BackupPC_Admin
To tell Apache to use mod_perl to execute BackupPC_Admin, add this
-to Apache's httpd.conf file:
+to Apache's 1.x httpd.conf file:
<IfModule mod_perl.c>
PerlModule Apache::Registry
SetHandler perl-script
PerlHandler Apache::Registry
Options ExecCGI
+ PerlSendHeader On
</Location>
</IfModule>
+Apache 2.0.44 with Perl 5.8.0 on RedHat 7.1, Don Silvia reports that
+this works (with tweaks from Michael Tuzi):
+
+ LoadModule perl_module modules/mod_perl.so
+ PerlModule Apache2
+
+ <Directory /path/to/cgi/>
+ SetHandler perl-script
+ PerlResponseHandler ModPerl::Registry
+ PerlOptions +ParseHeaders
+ Options +ExecCGI
+
+ Order deny,allow
+ Deny from all
+ Allow from 192.168.0
+ AuthName "Backup Admin"
+ AuthType Basic
+ AuthUserFile /path/to/user_file
+ Require valid-user
+ </Directory>
+
There are other optimizations and options with mod_perl. For
example, you can tell mod_perl to preload various perl modules,
which saves memory compared to loading separate copies in every
-Apache process after they are forked. See Stas's definitive
+Apache process after they are forked. See Stas's definitive
mod_perl guide at L<http://perl.apache.org/guide>.
=back
require valid-user
</Location>
-If you want to defeat the user authentication you can force a
-particular user name by getting Apache to set REMOTE_USER, eg,
-to hardcode the user to www you could add this to httpd.conf:
+If you want to disable the user authentication you can set
+$Conf{CgiAdminUsers} to '*', which allows any user to have
+full access to all hosts and backups. In this case the REMOTE_USER
+environment variable does not have to be set by Apache.
+
+Alternatively, you can force a particular user name by getting Apache
+to set REMOTE_USER, eg, to hardcode the user to www you could add
+this to Apache's httpd.conf:
<Location /cgi-bin/BackupPC/BackupPC_Admin> # <--- change path as needed
Setenv REMOTE_USER www
Finally, you should also edit the config.pl file and adjust, as necessary,
the CGI-specific settings. They're near the end of the config file. In
particular, you should specify which users or groups have administrator
-(privileged) access. Also, the configure.pl script placed various
+(privileged) access: see the config settings $Conf{CgiAdminUserGroup}
+and $Conf{CgiAdminUsers}. Also, the configure.pl script placed various
images into $Conf{CgiImageDir} that BackupPC_Admin needs to serve
up. You should make sure that $Conf{CgiImageDirURL} is the correct
URL for the image directory.
+See the section L<Debugging installation problems|debugging installation problems> for suggestions on debugging the Apache authentication setup.
+
+=head2 How BackupPC Finds Hosts
+
+Starting with v2.0.0 the way hosts are discovered has changed. In most
+cases you should specify 0 for the DHCP flag in the conf/hosts file,
+even if the host has a dynamically assigned IP address.
+
+BackupPC (starting with v2.0.0) looks up hosts with DHCP = 0 in this manner:
+
+=over 4
+
+=item *
+
+First DNS is used to lookup the IP address given the client's name
+using perl's gethostbyname() function. This should succeed for machines
+that have fixed IP addresses that are known via DNS. You can manually
+see whether a given host have a DNS entry according to perls'
+gethostbyname function with this command:
+
+ perl -e 'print(gethostbyname("myhost") ? "ok\n" : "not found\n");'
+
+=item *
+
+If gethostbyname() fails, BackupPC then attempts a NetBios multicast to
+find the host. Provided your client machine is configured properly,
+it should respond to this NetBios multicast request. Specifically,
+BackupPC runs a command of this form:
+
+ nmblookup myhost
+
+If this fails you will see output like:
+
+ querying myhost on 10.10.255.255
+ name_query failed to find name myhost
+
+If this success you will see output like:
+
+ querying myhost on 10.10.255.255
+ 10.10.1.73 myhost<00>
+
+Depending on your netmask you might need to specify the -B option to
+nmblookup. For example:
+
+ nmblookup -B 10.10.1.255 myhost
+
+If necessary, experiment on the nmblookup command that will return the
+IP address of the client given its name. Then update
+$Conf{NmbLookupFindHostCmd} with any necessary options to nmblookup.
+
+=back
+
+For hosts that have the DHCP flag set to 1, these machines are
+discovered as follows:
+
+=over 4
+
+=item *
+
+A DHCP address pool ($Conf{DHCPAddressRanges}) needs to be specified.
+BackupPC will check the NetBIOS name of each machine in the range using
+a command of the form:
+
+ nmblookup -A W.X.Y.Z
+
+where W.X.Y.Z is each candidate address from $Conf{DHCPAddressRanges}.
+Any host that has a valid NetBIOS name returned by this command (ie:
+matching an entry in the hosts file) will be backed up. You can
+modify the specific nmblookup command if necessary via $Conf{NmbLookupCmd}.
+
+=item *
+
+You only need to use this DHCP feature if your client machine doesn't
+respond to the NetBios multicast request:
+
+ nmblookup myHost
+
+but does respond to a request directed to its IP address:
+
+ nmblookup -A W.X.Y.Z
+
+=back
+
=head2 Other installation topics
=over 4
+=item Removing a client
+
+If there is a machine that no longer needs to be backed up (eg: a retired
+machine) you have two choices. First, you can keep the backups accessible
+and browsable, but disable all new backups. Alternatively, you can
+completely remove the client and all its backups.
+
+To disable backups for a client there are two special values for
+$Conf{FullPeriod} in that client's per-PC config.pl file:
+
+=over 4
+
+=item -1
+
+Don't do any regular backups on this machine. Manually
+requested backups (via the CGI interface) will still occur.
+
+=item -2
+
+Don't do any backups on this machine. Manually requested
+backups (via the CGI interface) will be ignored.
+
+=back
+
+This will still allow that client's old backups to be browsable
+and restorable.
+
+To completely remove a client and all its backups, you should remove its
+entry in the conf/hosts file, and then delete the __TOPDIR__/pc/$host
+directory. Whenever you change the hosts file, you should send
+BackupPC a HUP (-1) signal so that it re-reads the hosts file.
+If you don't do this, BackupPC will automatically re-read the
+hosts file at the next regular wakeup.
+
+Note that when you remove a client's backups you won't initially recover
+a lot of disk space. That's because the client's files are still in
+the pool. Overnight, when BackupPC_nightly next runs, all the unused
+pool files will be deleted and this will recover the disk space used
+by the client's backups.
+
=item Copying the pool
If the pool disk requirements grow you might need to copy the entire
=head2 Debugging installation problems
This section will probably grow based on the types of questions on
-the BackupPC mail list.
+the BackupPC mail list. Eventually the FAQ at
+L<http://backuppc.sourceforge.net/faq/> will include more details
+than this section.
+
+=over 4
+
+=item Check log files
Assuming BackupPC can start correctly you should inspect __TOPDIR__/log/LOG
for any errors. Assuming backups for a particular host start, you
should be able to look in __TOPDIR__/pc/$host/LOG for error messages
-specific to that host.
+specific to that host. Always check both log files.
-The most likely problems will relate to connecting to the smb shares on
-each host. On each failed backup, a file __TOPDIR__/pc/$host/XferERR will
-be created. This is the stderr output from smbclient. The first line
-will show the full smbclient command that was run. Based on the error
-messages you should figure out what is wrong. Possible errors on the
-server side are invalid host, invalid share name, bad username or password.
-Possible errors on the client side are misconfiguration of the share,
-username or password.
+=item CGI script doesn't run
-You should run smbclient manually and verify that you can connect to
-the host in interactive mode, eg:
+Perhaps the most common program with the installation is getting the
+CGI script to run. Often the setuid isn't configured correctly, or
+doesn't work on your system.
- smbclient '\\hostName\shareName' -U userName
+First, try running BackupPC_Admin manually as the BackupPC user, eg:
-shareName should match the $Conf{SmbShareName} setting and userName
-should match the the $Conf{SmbShareUserName} setting.
+ su __BACKUPPCUSER__
+ __CGIDIR__/BackupPC_Admin
-You will be prompted for the password. You should then see this prompt:
+Now try running it as the httpd user (which ever user apache runs as);
- smb: \>
+ su httpd
+ __CGIDIR__/BackupPC_Admin
-Verify that "ls" works and then type "quit" to exit.
+In both cases do you get normal html output?
+
+If the first case works but the second case fails with an error that
+the wrong user is running the script then you have a setuid problem.
+(This assumes you are running BackupPC_Admin without mod_perl, and
+you therefore need seduid to work. If you are using mod_perl then
+apache should run as user __BACKUPPCUSER__.)
+
+First you should make sure the cgi-bin directory is on a file system
+that doesn't have the "nosuid" mount option.
+
+Next, experiment by creating this script:
+
+ #!/bin/perl
+
+ printf("My userid is $> (%s)\n", (getpwuid($>))[0]);
+
+then chown it to backuppc and chmod u+s:
+
+ root# chown backuppc testsetuid
+ root# chmod u+s testsetuid
+ root# chmod a+x testsetuid
+ root# ls -l testsetuid
+ -rwsr-xr-x 1 backuppc wheel 76 Aug 26 09:46 testsetuid*
+
+Now run this program as a normal user. What uid does it print?
+Try changing the first line of the script to directly call sperl:
+
+ #!/usr/bin/sperl5.8.0
-Secondly, you should also verify that nmblookup correctly returns
-the netbios name. This is essential for DHCP hosts, and depending
-upon the setting of $Conf{FixedIPNetBiosNameCheck} might also be
-required for fixed IP address hosts too. Run this command:
+(modify according to your version and path). Does this work
+instead?
+
+Finally, you should invoke the CGI script from a browser, using
+a URL like:
+
+ http://myHost/cgi-bin/BackupPC/BackupPC_Admin
+
+You should make sure REMOTE_USER is being set by apache (see the
+earlier section) so that user authentication works. Make sure
+the config settings $Conf{CgiAdminUserGroup} and $Conf{CgiAdminUsers}
+correctly specify the privileged administrator users.
+
+=item You cannot access per-host information in the CGI interface
+
+If you get the error
+
+ Only privileged users can view information about host xyz
+
+it means that BackupPC_Admin is unable to match the user's login
+name (supplied by Apache via the REMOTE_USER environment variable)
+with either that host's user name (in the conf/hosts file) or
+with the administrators specified in the $Conf{CgiAdminUsers}
+or $Conf{CgiAdminUserGroup} settings.
+
+The most common problem is that REMOTE_USER is not set because the
+Apache authentication is not correctly configured. In this case
+BackupPC_Admin will report this additional error:
+
+ Note: $ENV{REMOTE_USER} is not set, which could mean there is an
+ installation problem. BackupPC_Admin expects Apache to authenticate
+ the user and pass their user name into this script as the REMOTE_USER
+ environment variable. See the documentation.
+
+You should review the configuration instructions to setup Apache
+authentication correctly. To test if REMOTE_USER is being set
+correctly, there is a simple script called printenv that is
+included with Apache. This is a simple CGI script that prints
+out all the environment variables. Place this script in the
+same directory as BackupPC_Admin and run it with a URL like:
+
+ http://myHost/cgi-bin/BackupPC/printenv
+
+Check the value of the REMOTE_USER environment variable.
+Here's a copy of the printenv script:
+
+ #!/usr/bin/perl
+ ##
+ ## printenv -- demo CGI program which just prints its environment
+ ##
+
+ print "Content-type: text/plain\n\n";
+ foreach $var (sort(keys(%ENV))) {
+ $val = $ENV{$var};
+ $val =~ s|\n|\\n|g;
+ $val =~ s|"|\\"|g;
+ print "${var}=\"${val}\"\n";
+ }
+
+=item Can't ping or find host
+
+Please read the section L<How BackupPC Finds Hosts|how backuppc finds hosts>.
+
+The BackupPC_dump command now has a -v option, so the easiest way to
+debug backup problems on a specific host is to run BackupPC_dump
+manually as the BackupPC user:
+
+ su __BACKUPPCUSER__
+ __INSTALLDIR__/bin/BackupPC_dump -v -f hostName
+
+This will run a full dump on hostName (replace with your host name).
+It will show each command (eg: ping, nmblookup and the full dump
+commands) and the output from each command. Reading the output
+carefully should show you what the problem is.
+
+You can also verify that nmblookup correctly returns the netbios name.
+This is essential for DHCP hosts, and depending upon the setting of
+$Conf{FixedIPNetBiosNameCheck} might also be required for fixed IP
+address hosts too. Run this command:
nmblookup -A hostName
The first name, converted to lower case, is used for the host name.
+=item Transport method doesn't work
+
+The BackupPC_dump command has a -v option, so the easiest way to
+debug backup problems on a specific host is to run BackupPC_dump
+manually as the BackupPC user:
+
+ su __BACKUPPCUSER__
+ __INSTALLDIR__/bin/BackupPC_dump -v -f hostName
+
+This will run a full dump on hostName (replace with your host name)
+and will print all the output from each command, including the log
+output.
+
+The most likely problems will relate to connecting to the smb shares on
+each host. On each failed backup, a file __TOPDIR__/pc/$host/XferLOG.bad.z
+will be created. This is the stderr output from the transport program.
+You can view this file via the CGI interface, or manually uncompress it
+with;
+
+ __INSTALLDIR__/bin/BackupPC_zcat __TOPDIR__/pc/$host/XferLOG.bad.z | more
+
+The first line will show the full command that was run (eg: rsync, tar
+or smbclient). Based on the error messages you should figure out what
+is wrong. Possible errors on the server side are invalid host, invalid
+share name, bad username or password. Possible errors on the client
+side are misconfiguration of the share, username or password.
+
+You should try running the command manually to see what happens.
+For example, for smbclient you should it manually and verify that
+you can connect to the host in interactive mode, eg:
+
+ smbclient '\\hostName\shareName' -U userName
+
+shareName should match the $Conf{SmbShareName} setting and userName
+should match the the $Conf{SmbShareUserName} setting.
+
+You will be prompted for the password. You should then see this prompt:
+
+ smb: \>
+
+Verify that "ls" works and then type "quit" to exit.
+
+=back
+
=head1 Restore functions
BackupPC supports several different methods for restoring files. The
Each of these programs reside in __INSTALLDIR__/bin.
+=head1 Archive functions
+
+BackupPC supports archiving to removable media. For users that require
+offsite backups, BackupPC can create archives that stream to tape
+devices, or create files of specified sizes to fit onto cd or dvd media.
+
+Each archive type is specified by a BackupPC host with its XferMethod
+set to 'archive'. This allows for multiple configurations at sites where
+there might be a combination of tape and cd/dvd backups being made.
+
+BackupPC provides a menu that allows one or more hosts to be archived.
+The most recent backup of each host is archived using BackupPC_tarCreate,
+and the output is optionally compressed and split into fixed-sized
+files (eg: 650MB).
+
+The archive for each host is done by default using
+__INSTALLDIR__/BackupPC_archiveHost. This script can be copied
+and customized as needed.
+
+=head2 Configuring an Archive Host
+
+To create an Archive Host, add it to the hosts file just as any other host
+and call it a name that best describes the type of archive, e.g. ArchiveDLT
+
+To tell BackupPC that the Host is for Archives, create a config.pl file in
+the Archive Hosts's pc directory, adding the following line:
+
+$Conf{XferMethod} = 'archive';
+
+To further customise the archive's parameters you can adding the changed
+parameters in the host's config.pl file. The parameters are explained in
+the config.pl file. Parameters may be fixed or the user can be allowed
+to change them (eg: output device).
+
+The per-host archive command is $Conf{ArchiveClientCmd}. By default
+this invokes __INSTALLDIR__/BackupPC_archiveHost, which you can
+copy and customize as necessary.
+
+=head2 Starting an Archive
+
+In the web interface, click on the Archive Host you wish to use. You will see a
+list of previous archives and a summary on each. By clicking the "Start Archive"
+button you are presented with the list of hosts and the approximate backup size
+(note this is raw size, not projected compressed size) Select the hosts you wish
+to archive and press the "Archive Selected Hosts" button.
+
+The next screen allows you to adjust the parameters for this archive run.
+Press the "Start the Archive" to start archiving the selected hosts with the
+parameters displayed.
+
=head1 BackupPC Design
=head2 Some design issues
Identical files on multiples backups are represented by hard links.
Hardlinks are used so that identical files all refer to the same
physical file on the server's disk. Also, hard links maintain
-reference counts so that BackupPC knows when to deleted unused files
+reference counts so that BackupPC knows when to delete unused files
from the pool.
For the computer-science majors among you, you can think of the pooling
true for backups in v1.4.0 and above. False for all backups prior
to v1.4.0.
+=item xferMethod
+
+Set to the value of $Conf{XferMethod} when this dump was done.
+
+=item level
+
+The level of this dump. A full dump is level 0. Currently incrementals
+are 1. But when multi-level incrementals are supported this will reflect
+each dump's incremental level.
+
=back
=item restores
space if all the files in a directory have the same attributes across
multiple backups, which is common.
+=head2 Optimizations
+
+BackupPC doesn't care about the access time of files in the pool
+since it saves attribute meta-data separate from the files. Since
+BackupPC mostly does reads from disk, maintaining the access time of
+files generates a lot of unnecessary disk writes. So, provided
+BackupPC has a dedicated data disk, you should consider mounting
+BackupPC's data directory with the noatime attribute (see mount(1)).
+
=head2 Limitations
BackupPC isn't perfect (but it is getting better). Here are some
in a single large file and keeps it locked whenever it is running.
Since many users keep Outlook running all the time their machine
is up their Outlook file will not be backed up. Sadly, this file
-is the most important file to backup. How do commercial backup
-systems solve this problem?
+is the most important file to backup. As one workaround, Microsoft has
+a user-level application that periodically asks the user if they want to
+make a copy of their outlook.pst file. This copy can then be backed up
+by BackupPC. See L<http://office.microsoft.com/downloads/2002/pfbackup.aspx>.
Similarly, all of the data for WinXX services like SQL databases,
-Exchange etc won't be backed up.
+Exchange etc won't be backed up. If these applications support
+some kind of export or utility to save their data to disk then this
+can =used to create files that BackupPC can backup.
So far, the best that BackupPC can do is send warning emails to
the user saying that their outlook files haven't been backed up in
knows about this bug and can recover the correct file size. So smbclient
transport works up to 4GB file sizes.
+Rsync running on Cygwin is limited to either 2GB or 4GB file sizes.
+More testing needs to be done to verify the file size limit for
+rsync on various platforms.
+
=back
=item Some tape backup systems aren't smart about hard links
ln -s ../../conf/ConfigWinDesktop.pl config.pl
or, better yet, create a config.pl file in __TOPDIR__/pc/$host
-that contains this line:
+that includes the default config.pl file using perl's "do"
+command:
- do "../../conf/ConfigWinDesktop.pl";
+ do "__TOPDIR__/conf/ConfigWinDesktop.pl";
This alternative allows you to set other configuration options
-specific to each host (perhaps even overriding the settings in
-the included file).
+specific to each host after the "do" command (perhaps even
+overriding the settings in the included file).
Note that you could also include snippets of configuration settings
from the main configuration file. However, be aware that the
=head1 Copyright
-Copyright (C) 2001-2002 Craig Barratt
+Copyright (C) 2001-2004 Craig Barratt
=head1 Credits
+Xavier Nicollet, with additions from Guillaume Filion, added the
+internationalization (i18n) support to the CGI interface for v2.0.0.
+Xavier provided the French translation fr.pm, with additions from
+Guillaume.
+
Ryan Kucera contributed the directory navigation code and images
-for v1.5.0. He also contributed the first skeleton of BackupPC_restore.
+for v1.5.0. He contributed the first skeleton of BackupPC_restore.
+He also added a significant revision to the CGI interface, including
+CSS tags, in v2.1.0.
Guillaume Filion wrote BackupPC_zipCreate and added the CGI support
for zip download, in addition to some CGI cleanup, for v1.5.0.
+Guillaume continues to support fr.pm updates for each
+new version.
+
+Josh Marshall implemented the Archive feature in v2.1.0.
+
+Javier Gonzalez provided the Spanish translation, es.pm for v2.0.0.
+
+Manfred Herrmann provided the German translation, de.pm for v2.0.0.
+Manfred continues to support de.pm updates for each new version.
+
+Lorenzo Cappelletti provided the Italian translation, it.pm for v2.1.0.
-Several people have reported bugs or made useful suggestions; see the
-ChangeLog.
+Many people have reported bugs, made useful suggestions and helped
+with testing; see the ChangeLog and the mail lists.
Your name could appear here in the next version!
projects / BackupPC.git / blobdiff