Author Topic: nokuBT Backup Tools Manual excerpt - Chapter 7. Backing Up: General Concepts  (Read 928 times)

Talus

  • Administrator
  • Jr. Member
  • *****
  • Posts: 60



(The full nokuBT Backup Tools Manual is available to nokuBT Backup Tools Forum subscribers.)




Backing Up: General Concepts

nokuBT Backup Tools is designed to assist with management of chained multi-drive backup projects that are safe, secure, redundant and internet independent.

The general travel path for backup files in a four-drive computer backup system is:


           WS desktop drive ==> Transport Drive ==> Off-Site desktop drive ==> Safe-Site drive


Any networked storage location that can be mapped with a drive letter may be substituted for a physical drive.

Ideally, the file transfer process should be fast and efficient. Special user attention should only be necessary when available space is depleted, for routine inspection of backup logs, and occasional testing of backup files.


Drive Terminology

In the context of nokuBT operations, a drive may be any type of re-writable storage device that can be recognized with a drive letter, including hard drives, flash drives, SSD drives, NAS (Network Attached Storage) devices and mapped network paths. A storage space must be represented by a drive letter and path pointing to a root backup directory or subdirectory, with lower subdirectories holding saved backup files.

A "Work Site backup drive" may be an internal drive, an external drive, a NAS device or mapped networked storage.

A "Transport Drive" would typically be a high-capacity portable drive that is specifically engineered to withstand movement and minor disturbances when detached and without attached electrical power.

"Off-Site" and "Safe-Site" backup drives may be internal drives or external desktop drives. A decision to use drives hardened for transport will depend on the system design in consideration of users' needs.

A "Consolidation Drive" is a generic term describing a backup drive dedicated to receiving backup files from multiple backup projects. A Consolidation Drive may be situated at any location, be it a local Work Site drive, or a local or remote networked location, and the drive may be used in complex backup systems that include backup storage for multiple Work Sites and/or multiple backup strategies. nokuBT has no pertinent need for specifying if a backup drive is a Consolidation Drive. This is a matter for backup system design. A Consolidation Drive would most likely be incorporated into nokuBT settings for a backup project as a Work Site backup drive, as the top-level source drive in a redundant backup drive chain.



Synchronization of Backup Drives

Several recommendations for the synchronization of backup drives follow below. Every user and every system will have different needs and circumstances. Consider these suggestions as a starting point for configuring a backup system that meets your needs.


1. The Golden Rule of Backup Drive Management

Avoid manual selection and deletion of sets of backup files in an effort to increase available space on backup drives.

As the number of computers included in a backup project increases, the more golden this rule becomes.

The time, effort, costs and risks of replicated manual file deletions across backup drives will quickly exceed the costs and benefits of procuring and installing new replacement drives. Plan ahead. nokuBT available space notifications will provide ample advance notice of pending space issues.


2. Configure a true synchronization for Work Site to Transport drive file transfers.

In a true synchronization, changes on either drive are made to the other paired drive.

Configuring a true synchronization for Work Site to Transport drive transfers allows the backups administrator to apply backup file retention policies in the backup software to control file bloat, and hence available drive space, on the two drives, not just on the source Work Site backup drive. Efficiency is improved because the administrator is then relieved of persistent available space concerns for these two drives.

Traditionally, true synchronization of backup files across drives is considered to be ill-advised because of the risk that files inadvertently or maliciously overwritten or deleted on one drive will then be overwritten or deleted on the paired drive.

Adherence to the Golden Rule of Backup Drive Management, in combination with nokuBT file validation routines, reduce the risks otherwise inherent to true synchronization. The true synchronization becomes an applied contribute operation, with exceptions for systematic deletions implemented by backup file retention policies.


3. Configure a contribute synchronization for Transport drive to Off-Site drive file transfers, and for Off-Site to Safe-Site drive file transfers

For a contribute synchronization task, all new files on the source drive are transferred to the target drive. There should be no deletions and no file re-names on the target drive. File changes on the target drive cannot travel backwards, or up the redundant backup drive chain.

This is where the user must use caution by fully understanding the details of synchronization methods offered by a chosen Drive Synchronization Application. Sometimes, re-names may be duplicated from the source drive to the target drive within a contribute task. A duplicate file changed in any way is not a backup. nokuBT will recognize file name changes for the most recent pre-existing backup files and block synchronization.

By contributing new files from the source drive to the target drive, backup files are allowed to accrue on the target Off-Site and Safe-Site drives. These drives may then serve as archive drives.


4. Consider replacing the Safe-Site and Off-Site Backup drives once they are full

Contribute synchronization will cause the Safe-Site and Off-Site Backup drives to serve as archive drives, holding all backups created for all computers. Once full, if not already dictated as a matter of company or personal policy, the user must decide whether to replace the old drives with new units, or if the old drives should be emptied and re-used.

The relatively small cost of replacing and configuring backup drives will often be greatly exceeded by the benefits of keeping those older archive drives. For example, one five-year-old good copy of a corrupt file, now sorely needed, is far more valuable when it turns out the file became corrupt four years ago.

In addition, older drives are just that - old. New replacement drives will be more reliable than those already subject to time and wear. The most recent backups are far more valuable than older backups. Keep those new backups going to newer and more reliable backup drives by replacing full Safe-Site and Off-Site backup drives with new units.


Directory Structure for Backup Drives

nokuBT, in and of itself, exhibits no prejudice for directory layout on backup drives. In nokuBT settings, the location and name of the root backups directory can differ across all drives.

There are good practices, however, that are recommended. These may be applied to a new backup project, or an existing backup project may be modified to simplify and ease backup drive management.


Use a consistent upper-level directory structure to accommodate software operations and methods

First, create a generic directory under the root of the backup drive. Then in the generic directory, create a directory named "bkups", or similar, that will serve as the target directory for drive synchronization.  Under this directory, create a unique "SITENAME" directory that serves as the unique backup project directory referenced by nokuBT WS and nokuBT OS. Under the "SITENAME" directory, create unique computer or station directories, followed by any desired directories that will hold backup files built under dual backup strategies.

Apply this directory structure identically across all backup drives. If building a new backup project, manual creation of all directories on the Work Site backup drive, and the first two higher-level subdirectories on the other backup drives, will be adequate. The Drive Synchronization Application will propagate the remaining directory tree structure across the remaining drives down the redundant backup drive chain as backup drives are synchronized.

Let's assume that the higher-level directories on the backup drive are

    "[drive letter]:\data\bkups"

and that the next two subdirectory levels, including multiple stations, are

    "[drive letter]:\data\bkups\SITENAME\Station1"
    "[drive letter]:\data\bkups\SITENAME\Station2"
    "[drive letter]:\data\bkups\SITENAME\Station3"
    "[drive letter]:\data\bkups\SITENAME\Station4"
    "[drive letter]:\data\bkups\SITENAME\Station5"
              .
              .
              .
    "[drive letter]:\data\bkups\SITENAME\StationXX"

where "SITENAME" represents the unique backup project site name supplied by the user when installing nokuBT WS and nokuBT OS.

The example directory tree appears as:


Fig. 7.1. Example directory tree.


In nokuBT WS and nokuBT OS, specify the root backups directory in nokuBT settings as

    "[drive letter]:\data\bkups\SITENAME"

for each drive, again where "SITENAME" represents the unique backup project site name.

Designate the "bkups" directory as the target synchronized directory in the Drive Synchronization Application on the Work Site and Off-Site backups computers. Ideally, any overhead files created and used by the Drive Synchronization Application are then written to the "bkups" directory on each drive. These potentially dynamic files will be safely above nokuBT operations and will not interfere with nokuBT file validation routines.



Notes for Advanced Users

"SITENAME" and %sitename%

In the example above, it is recommended, but not required, that the directory named "SITENAME" should be the same as the user-input value for %sitename% provided during new paired installs of nokuBT WS and nokuBT OS. See Chapter 11 for a detailed explanation.



Special storage needs on backup drives

The "data" directory is then available for creation of additional non-backups directories for special needs. The user must be aware that any such directory will never be synchronized across drives and will never be accessed by nokuBT. Create and use any such directories under "data" sparingly, if at all, and only for special uses.


Retaining long-term base backups

It may be desirable to retain long-term base backups for special use at a later time. For example, a disk image of a new computer may be retained should there be a need to re-purpose a computer. Alternatively, a disk image created after a computer has been fully configured and customized may be useful for restoring a computer to an early and fully-functional operating system state.

Storing these long-term base backups under computers' backups directories becomes problematic when backup drives are replaced. The disparate locations of each computer's holding directories and included long-term base backup files will require manual searches and time-consuming transfers from the old to the new replacement drives.

It is highly recommended that a special directory (and systematically-named subdirectories for individual computers) be created for storage of all long-term base backups. This allows for the rapid transfer of long-term base backups from an old backup drive to a new drive.

The recommended location of the special long-term base backups directory would include it within the scope of daily synchronization tasks and within the scope of nokuBT operations. The reasoning is straight-forward. At the Work Site, new base backups from existing computers, or from new or replacement computers in a network, would coincide with placement of those long-term base backup files under the special directory on the Work Site backups drive. Transfers of the new long-term backups would proceed routinely down the backup drive chain without further attention.

An example location of the special directory would be

    "[drive letter]:\data\bkups\%sitename%\BaseStorage"

with the following possible subdirectories where the base backups would actually be stored:

    "[drive letter]:\data\bkups\%sitename%\BaseStorage\Station1"
    "[drive letter]:\data\bkups\%sitename%\BaseStorage\Station2"
    "[drive letter]:\data\bkups\%sitename%\BaseStorage\Station3"
    "[drive letter]:\data\bkups\%sitename%\BaseStorage\Station4"
    "[drive letter]:\data\bkups\%sitename%\BaseStorage\Station5"
              .
              .
              .
    "[drive letter]:\data\bkups\%sitename%\BaseStorage\StationXX"


The one maintenance issue introduced will be that as computers are decommissioned, the backups administrator may wish to manually and simultaneously remove the newly irrelevant base backups from the "BaseStorage" directories on the Work Site and Transport drives.


Admin Notes files

Should the backups administrator wish to create and utilize one or more backup project notes files that are passed down the backup drive chain, these must be created and used in a new directory on the Work Site backup drive in a specific location that will be synchronized across backup drives without encumbering nokuBT file validation routines. Any notes files will be dynamic files. If incorrectly located in or under the "%sitename%" directory, they will be detected as suspect changed files by nokuBT, which will then block the startup of the Drive Synchronization Application in nokuBT WS and nokuBT OS.

Instead, create an appropriately named admin notes directory under the "bkups" directory:

    "[drive letter]:\data\bkups\%sitename%_AdminNotes"

where %sitename% represents the unique backup project name. The resulting location will be outside the scope of nokuBT operations, but within the scope of daily synchronization tasks.

The directory "%instdir%_AdminNotes" and any contained files will then be synchronized down the backup drive chain, but will not encumber nokuBT file validation routines.

Considerations:

(a) Strongly encrypt any admin notes files, just as backup files are encrypted. If a backup drive is lost or stolen, admin notes will then be inaccessible to unauthorized parties.

(b) Admin notes will be truly synchronized across the Work Site and Transport drives (warning!), but will be contributed down the chain from the Transport drive to the Safe-Site and Off-Site backup drives.

Changes to admin notes files on the Work Site backup drive will work their way down the backup drive chain.

Manual changes to admin notes files on the Safe-Site drive will not survive, and will ultimately be overwritten by new file versions from the Transport drive.

Similarly, manual changes to admin notes files on the Off-Site drive will not survive, and will ultimately be overwritten by new versions from the Safe-Site drive.


Example Full Directory Structure of a Backup Drive

A final and full directory structure for a backup drive then accommodates base disk images, scheduled backups and admin notes.


Fig. 7.2. Example full directory tree.


In review, the "bkups" directory (Fig. 7.2) serves as the target directory for drive synchronization. All files in and below the "bkups" directory will be synchronized across all chained backup drives. Dynamic database and journal files created and managed by the Drive Synchronization Application should be stored in this directory.

The "SITENAME" directory identifies the unique backup project, and serves as the root backups directory. The directory structure below the root backups directory is laid out to organize stored backup files for the backup project.

The "BaseStorage" directory holds base disk images that can be used to restore a drive to an early critical operating system point, such as rebuilding the computer for a different purpose or use.

The "SITENAME_AdminNotes" directory may hold any files or notes documents needed for administrator reference.

The example of Fig. 7.2 may serve as a starting point for a customized backup drive directory structure. For nokuBT, the important directory structure components are the targeting of a "bkups" directory for drive synchronization, and a "SITENAME" directory, unique to a backup project, that is located below the "bkups" directory.


Naming Conventions for Backup Files

Do not rely solely on directory names for identification of backup files. Including identifying information in the name of the backup file will save time when a restore is necessary because the file will not have to be opened to reveal contents. Importantly, an inadvertently incorrect target directory for a backup task may then be easily resolved by referring to the backup file name alone.

When configuring backup tasks in Disk Image and File Backup Applications, always insert as much identifying information into the file name as feasible. Select a format, and apply it systematically. Ensure that the name of the target computer, the date and time of the backup, and the purpose of the backup are included in the file name. Consider the format and order of the information included. Depending on backup system configuration and personal preferences, a site location and computer name prefix immediately followed by a datetime stamp may be invaluable for sorting in a file manager. For datetime stamps, consider using the format YYYY-MM-DD-hh_mm_ss because it is sort friendly.

Double-check the file name mask to ensure it is correct. Check again after creating an initial or test backup file.


Static Versus Dynamic Files

The storage of static files in and under the "%sitename%" directory is preferred and highly recommended. Static files, once written, are never changed.

For some Drive Image, File Backup and Drive Synchronization Applications, it may be the case that certain types of saved files are dynamic, meaning the contents change while maintaining the same file name. These dynamic files may be numerous and sometimes large. Show preference for applications, application options and configurations that do not save dynamic files in your synchronized backups directory structure.

For example, some Drive Synchronization Applications may manage synchronization by storing dynamic overhead files in every directory, and then apply hidden attributes to the files to hide them from the user. These dynamic files may be catastrophic to the integrity of the backup system, and further, will encumber nokuBT File Validation routines.

If the creation and use of dynamic overhead files cannot be avoided, attempt to build and utilize a backups directory structure and software settings that elevate dynamic files to a storage location above the "%sitename%" directory. This approach is only sensible for synchronization overhead. Backup files must remain in and below the "%sitename%" directory.

Dynamic files will cause nokuBT File Validation to become a nuisance and of no value for identifying inadvertent or malicious file changes. If dynamic files are deemed necessary, consider disabling nokuBT File Validation in nokuBT settings.


Versioning

Versioning is the persistent backup of targeted files each time those files are changed. Each version of a targeted file is saved or backed up. Sometimes just changed portions of files are saved in an incremental chain, in which case restore opportunities may be diminished should any corruption be incurred along that chain. Versioning has its benefits and reasons for being, but may cause significant impacts for your chained redundant backup drives when the versioned files are large. Disk available space may be quickly diminished. Be selective with versioning source files and target storage. Consider directing versioning activities to a backup project separate from, and independent of, your primary backup project.

Versioning may be applied solely to specified user files or directories by a software application, or may be used by backup software in certain configurations. Consider the impacts of versioning in both circumstances.


Full, Differential and Incremental Backups, and a Dual Backup Strategy

A full backup of a hard drive or user files is a true backup. Anything else is an attempt to manage storage space in a resource-limited world. A restore from a full backup is a one-step process. It may be argued, and correctly so, that the sheer size of full backups increases the risk of incurring file corruption, potentially destroying a restore opportunity when all that is needed are the most recent, say, 20 files created, modified or accidentally deleted.

A differential backup task refers to the last full backup as a base, and builds a new backup file at each scheduled time that holds all new and changed files written since the last full backup was created. Each subsequent differential backup file grows larger, and larger, until a new full backup file is created. Restores are a simple two-step process. First restore the last full backup, then restore the most recent differential backup. The last full backup file had best be intact, because if corrupted, later differential backups may either not restore successfully, or may not hold the files needed.

An incremental backup task starts with the last full backup, and builds a new backup file at each scheduled time that holds only new and changed files since the last incremental backup was created. A restore from incremental backups will be laborious and frustrating, but much less disk storage space is needed. Each and every backup file had best be intact, because any corruption along the chain of backup files may ruin the day or an entire business.

Hard drive storage space has become inexpensive. It is now feasible to implement a dual backup strategy where two or more different methods are used to back up files. If disaster occurs and a restore is needed, the discovered failure of one backup method is then countered by a second backup method. Essentially, one may backup a backup method by applying a dual backup strategy.

nokuBT is configured to work with a Disk Image Application and a File Backup Application. Truly, this is just nomenclature, because nokuBT simply accesses the log files of the backup applications to improve management efficiency. The important point is that nokuBT facilitates administration of a dual backup strategy that best suits user circumstances.


About the Transport Drive

The portable Transport drive is a key piece of infrastructure. It is a storage location. Make use of it.

When synchronization with the Transport drive is complete, safely remove it and immediately return it to its normal location - someplace safe, outside of the building, is best. A special pocket under your car seat is a good, cool and safe location. Regardless, make it an objective to immediately remove it from the building, to be safe from localized hazards.


Summary

Using true synchronization and backup file retention policies for the backup drives at the Work Site, in combination with contribute-driven accrued backups on the Safe-Site and Off-Site backup drives, allows for increased available disk space at the Work Site and reduced monitoring demands for the administrator. More available backup drive space at the Work Site backups computer means more flexibility for the selected backup types and strategies, allowing for faster restores with a greater likelihood of success. Meanwhile, the Off-Site and Safe-Site backup drives are always covering your back.

Consider the purpose and featuers of nokuBT Backup Tools. It is not only consolidating tools for backup management and troubleshooting, and it is doing more than validating files to help when malicious actors get into your system and target your backup files. nokuBT is assisting with the formal organization of your backup projects and streamlining the daily synchronization process.







6. Installation    7. Backing Up: General Concepts    8. nokuBT WS Backup Tools






nokuBT Backup Tools Manual  |  nokuBT version 1.05.00  |  Copyright 2019 nokuBT

A good backup is the one that you need, that you have, and that restores successfully without errors.