More on Incremental Backups

The example below shows the data included in a series of backups between January 1 and January 4. The January 1 backup is a full backup and includes all files and directories in the policy backup selections list. The subsequent backups are differential incremental backups and include only the data that changed since the last full or differential incremental backup. If the disk fails sometime on January 4 (after the backup), the full and all three of the incremental backups are required for the recovery.

A cumulative incremental backs up the data that has changed since the last full backup. The example below shows the data included in a series of backups between January 1 and January 4. The January 1 full backup includes all files and directories in the policy backup selections list. Each of the cumulative incremental backups include the data changed since the last full backup. If the disk fails sometime on January 4 (after the backup), the full backup and the last cumulative incremental backup are required for the recovery.

The following table compares the retention requirements for differential and cumulative incremental backups.

Backup and Restore Times

The following table compares the relative backup and restore times for differential and cumulative incremental backups.

It is possible to use a combination of cumulative and differential incremental backups in order to obtain some of the advantages of both methods. For example, assume a set of schedules with the following backup frequencies and retention periods (notice that the differential incremental backups occur more often.)

This set of schedules results in the series of backups shown below:

• Every other day a differential incremental backup occurs, which usually has a minimum backup time.
• On alternate days, a cumulative incremental backup occurs, which requires more time than the differential backup, but not as much time as a full backup. The differential backup can now be expired.
• To recover all files requires, at most, two incremental backups in addition to the most recent full backup. This typically means less restore time than if all differential incremental backups were used. The full backups can be done less often if the amount of data being backed up by the incremental backups is small.

Determining Files Due for Backup on Windows Clients

On Windows clients, NetBackup performs incremental backups of files based on the Perform Incrementals Based on Archive Bit setting. This setting is found in the Backup, Archive and Restore client interface, under File > NetBackup Client Properties, on the General tab.

If the Perform Incrementals Based on Archive Bit check box is checked, incremental backups for this client are based on the state of the archive bit of each file. The operating system sets the bit whenever a file is changed and it remains set until cleared by NetBackup. The conditions under which NetBackup clears the bit depend on the type of backup being performed.

• For a full backup, NetBackup backs up files regardless of the state of their archive bit. After a full backup, the archive bit is always cleared.
• For a differential incremental backup, NetBackup backs up files that have the archive bit set and have therefore been changed. When the client receives a response from the server indicating that the backup was successful (or partially successful) the archive bits are cleared. This allows the next differential incremental to back up only files that have changed since the previous full or differential incremental backup.
• For a cumulative incremental backup, NetBackup backs up files that have the archive bit set, but does not clear the archive bits after the backup. This allows the next cumulative incremental to back up not only changed files, but also files that were in this cumulative incremental.

If the Perform Incrementals Based on Archive Bit check box is clear, NetBackup includes a file in an incremental backup only if the datetime stamp of the file has been changed since the last backup. The datetime stamp indicates when the file was last backed up.

• For a full backup, NetBackup backs up files regardless of the datetime stamp.
• For a differential incremental backup, NetBackup compares the datetime stamp of the file against the last full or incremental backup.
• For a cumulative incremental backup, NetBackup compares the datetime stamp of the file against the last full backup.

If you install or copy files from another computer, the new files retain the datetime stamp of the originals. If the original date is before the last backup date on this computer, then the new files are not be backed up until the next full backup.

Determining Files Due for Backup on UNIX Clients

When performing incremental backups on NetBackup UNIX clients, all relevant files and directories are looked at to determine if they are due for backup based on a reference date (that is, back up all files changed since date X).

UNIX files and directories have three times associated with them:

• mtime: The file modification time. The mtime for a file or directory is updated by the file system each time the file is modified. Prior to modifying a file, an application can save the mtime of the file, then reset it after the modification using the utime(2) system call.
• atime: The file access time. The atime for a file or directory is updated by the file system each time the file is accessed (read or write). Prior to accessing a file, an application can save the atime of the file, and then reset it after the file access using the utime(2) system call.
• ctime: The inode change time. The ctime for a file or directory is updated each time the file or directory's inode is changed; examples of this are changing permissions, ownership, link-counts, and so on. The ctime for a file or directory cannot be saved before and reset after a change. Another significant fact is that the ctime of a file or directory is changed when resetting the mtime and atime (using the utime(2) system call) for the file.

UNIX man pages contain a definition of these attributes.

When NetBackup reads the data for a file that is included in a backup, it does not affect the file modification time, but does affect the access time of the file. For this reason, NetBackup saves the atime and mtime of the file prior to reading the file, and (by default) resets the atime and mtime using the utime(2) system call. By doing it this way, NetBackup does not cause problems for storage migration products or administrator scripts that are utilizing file access times (atime) as criteria for their operations. While this benefit is obvious, a side effect is that it does update the ctime of the file.

As an option to a NetBackup configuration, customers can choose to have NetBackup not reset the access time of the file after it reads a file. Additionally, customers can choose to have NetBackup use the ctime of the file, in addition to the mtime, when determining what files to back up in an incremental. Normally, these two options are used together, but there may be sites which want to use one without the other. By default, NetBackup uses only the mtime of the file to determine what files and directories to back up.

When a file is moved from one location to another, the ctime of the file changes, but the mtime remains unchanged. If NetBackup is only using the file modification time (mtime) to determine files due to be backed up during an incremental backup, it will not detect these moved files. For sites where this is an issue, the ctime should also be used (if possible) to determine files due to be included in an incremental backup, using the bp.conf attributes USE_CTIME_FOR_INCREMENTALS and DO_NOT_RESET_FILE_ACCESS_TIME.

When a directory is moved from one location to another, the ctime of the directory changes, but the mtime remains unchanged. Neither the mtime nor the ctime are changed for the files or directories within the moved directory. Using file timestamps, there is no reliable method for determining that files within a moved directory need to be included in an incremental backup.

In either case, these moved files and directories are included in subsequent full backups.