keywords: backup, free, automated, incremental, encrypted, truecrypt, xxcopy, batch, script, scheduled task, system, project, files, network drive, byte-level verification, hidden files
**UPDATE! - (02/27/08)
Hopefully I won’t need to keep updating this.
Improvements:
Also, an observation:
You can actually use the freeware version of XXCOPY with my backup scripts! I tested it with XXCOPY Freeware Version 2.95.3. Since this script copies to the mounted TC container and not a net path, XXCOPY thinks it’s a local drive and doesn’t error out with an advertisement for XXCOPY PRO. Normally, you must purchase the PRO version to mount network paths/drives (I already purchased it a while back for this very purpose). In any case, please abide by the XXCOPYs licensing terms for the freeware version.
**
UPDATE! - Fixed a minor problem with the script (2/26/08)
Here is the latest version of my daily backup scripts for backing up important files on my development computer that others might be interested in using themselves. Terms of service, and more detailed comments are in the batch file script code.
In a nutshell, my method is to copy local files to a Truecrypt container hosted remotely on a network file server. I use XXCOPY to do the actual copying. XXCOPY is nice because it is a command-line tool similar to the old DOS XCOPY command and allows some useful features including copying hidden files, doing incremental copies (only copy to destination if destination doesn’t already have the exact same file), doing byte-wise verification of copies, and allowing user prompting to be turned off. Admittedly I started using XXCOPY for simple backups years ago because it was free. Then the author started charging money for the network capable version. So unfortunately this is not a completely free solution I am providing. This IS a free method of doing backups (with caveats). See update above. To automate the backups I have a Windows “Scheduled Task” that runs my script each night (computer stays on all the time).
I realize there are much better commercial and open source solutions out there, but this batch file method has been working for me for years and I have been slowly improving it as needed. Hopefully sometime soon I’ll be able to get an Apple Mac Pro and then I’ll use Time Machine. I am also interested in getting an Infrant ReadyNAS NV+ (dead link removed, see below) in which case the included ECM Retrospect backup software could be a better backup option than my script for MSFT Windows machines.
The batch scripts follow…or download the zip file:
backup_scripts_2008_02_27.zip
backup.bat
@ECHO OFF
:: Jeff Wilson's xxcopy batch file for doing "snapshot" backups of
:: select directories and files. I use it for backing up files
:: to a location on my network.
::
:: See xxbkup.bat for details of the actual backing up.
::
:: This file preps network and Truecrypt mounts so that backing up
:: can actually take place. There's a good bit of error checking going on
:: to reduce the possibility of getting the dreaded "delayed write error."
:: This is caused by losing network connectivity and is common if you leave
:: a remote TC container mounted all the time and your file server reboots
:: to apply security patches (or network temporarily goes down). The
:: delayed write errors will engulf you in pop-up notifications.
::
:: Basically, this script attempts to mount/map a net path to a virtual local
:: drive. After this, an attempt is made to mount a Truecrypt volume on this
:: net drive. If all of this is successful, the xxbkup.bat script is run
:: to back up files to this TC container. Once complete, the TC container is
:: unmounted.
::
:: This program assumes that TC already has the password for the container
:: cached in memory. Otherwise, a prompt (should) pop up asking for it.
::
:: I have this script run daily as a Windows "Scheduled Task." My computer
:: is always on (for Remote Desktop availability and some other services) so
:: the scheduled backup task runs as my regular user account. When I boot up
:: Truecrypt starts and auto-mounts my backup container (the remote network
:: drive where the TC container resides also auto-connects). Note that
:: Truecrypt must be configured through its GUI to auto-connect. This auto-mount
:: causes me to be prompted for the password. I enter the password, which
:: caches it, and then I unmount the container (passwd still cached). Now,
:: when the scheduled task runs a password does not need to be interactively
:: entered and I don't need to store the password in plain text in this script.
:: Alternatively, a local keyfile could be used (with no password). Note
:: that password caching must be enabled in the Truecrypt GUI to use my method.
::
:: Disclaimer:
:: This script is provided "AS IS". Author makes no claim that this will
:: work correctly. Author accepts NO responsibility for any damages
:: caused by this script. It is up to the user to determine if the
:: script will work in the desired fashion.
::
:: Copyright:
:: Jeff Wilson holds the copyright for this script, except for Eric
:: Phelps counter script code.
:: Anyone may use this script so long as they don't sell it (unless they
:: get written permission).
:: Those planning to redistribute this script must maintain documentation
:: that I'm the original author (as well as my contact info).
:: Anyone may modify this script, and redistribute so long as the above
:: requirements are followed.
::
:: Jeff's contact info:
:: user: jeffrey.brian.wilson
:: domain: acm.org
:: put the two together above (with an @) for my email
:: YOU WILL NEED TO CONFIGURE SEVERAL OF THE PATHS BELOW IN ORDER TO USE THIS SCRIPT
:: BEGIN USER CONFIG SECTION
:: Init paths
:: do a local drive mount of a net path (0), or directly use net path (1)?
set USE_NET_PATH=1
:: name of the Truecrypt container file (with no path)
set CNTNR_NAME=user.tc
:: drive letter where the samba net share should be mounted (if USE_NET_PATH==0)
set NET_DRIVE_LTR=J
:: net path used for doing the local drive mount
set NET_PATH=\\fileserver\backup\user
:: full net path to TC container for alternate direct mounting
set TC_NET_CNTNR_PATH=%NET_PATH%\%CNTNR_NAME%
:: path to the TC container using the locally mounted net drive
IF %USE_NET_PATH%==0 set TC_CONTAINER_PATH=%NET_DRIVE_LTR%:\%CNTNR_NAME%
IF %USE_NET_PATH%==1 set TC_CONTAINER_PATH=%TC_NET_CNTNR_PATH%
:: path to Truecrypt executable
set TC_PATH=c:\progra~1\truecrypt\truecrypt.exe
:: desired drive letter for TC container to be mounted to
set TC_DRIVE_LTR=Z
:: a path to a valid directory inside the TC container (used for existence test purposes)
set TC_TEST_PATH=%TC_DRIVE_LTR%:\random\stuff
:: TC dismount command
set TC_DISMOUNT_CMD=%TC_PATH% /dismount %TC_DRIVE_LTR% /force /quit /silent
:: TC mount command
set TC_MOUNT_CMD=%TC_PATH% /volume %TC_CONTAINER_PATH% /letter %TC_DRIVE_LTR% /quit /silent
:: # of sec to wait to allow time for TC mount to complete (in sec) (we can't block until completion unfortunately)
set TC_MOUNT_WAIT=5
:: # of sec to wait before attempting to connect after a failure. (in sec) failure probably because network or server is down.
set TC_RETRY_WAIT=300
:: # of retries before giving up completely.
:: multiply 10 times TC_RETRY_COUNT times TC_MOUNT_WAIT for estimate of longest possible run time
:: currently this digit is compared to ten's place in the hack counter below
:: modification of the script will be required if you don't want a multiple of 10 less than 100
set TC_RETRY_COUNT=3
set XXCOPY_SCRIPT=xxbkup.bat
:: END USER CONFIG SECTION
:: INIT hack counter because batch files can't do math (directly)
set E0=0
set E1=0
set E2=0
GOTO FORCE_RECONNECT
:BEGIN
:: Unfortunately, in some cases the network can be down and this will eval to true
:: that explains the GOTO FORCE_RECONNECT above
IF EXIST %TC_TEST_PATH% GOTO CONTINUE
echo. %TC_TEST_PATH% not found!
:: Dismount from truecrypt just in case we are in some locked up state
:: of partial mount
:FORCE_RECONNECT
echo. Dismounting truecrypt mount (if mounted)
%TC_DISMOUNT_CMD%
:NETWORK
IF %USE_NET_PATH%==1 echo. Checking %NET_PATH% to see if TC container (%CNTNR_NAME%) is available.
IF %USE_NET_PATH%==0 echo. Checking net drive (%NET_DRIVE_LTR%:) to see if TC container (%CNTNR_NAME%) is available.
::check net drive for connectivity
IF EXIST %TC_CONTAINER_PATH% GOTO TRUECRYPT
::if we are using a net path directly, then go to delay and wait
IF %USE_NET_PATH%==1 GOTO DELAY
echo. Net drive not connected. Attempting reconnect!
net use %NET_DRIVE_LTR%: %NET_PATH%
echo. Error level from 'NET USE %NET_DRIVE_LTR%: %NET_PATH%': %ERRORLEVEL%
:: assume errorlevel 0 is no error, otherwise wait a bit and try again
IF ERRORLEVEL 1 GOTO DELAY
:TRUECRYPT
echo. %TC_CONTAINER_PATH% is available!
:: stupid truecrypt blocks on mounting network shares, tf I can't wait for err
:: start used for non-block start
echo. Truecrypt trying to open %CNTNR_NAME%!
start %TC_MOUNT_CMD%
echo. Waiting a bit for truecrypt to finish....
::sleep to allow connection to hopefully complete
ping -n %TC_MOUNT_WAIT% 127.0.0.1 >NUL
:: Test for exact ERRORLEVEL 0
IF ERRORLEVEL 0 IF NOT ERRORLEVEL 1 GOTO BEGIN
echo. Inturrupted with CTRL-C
GOTO EXIT_FAIL
:DELAY
echo. Waiting %TC_RETRY_WAIT% seconds before reconnect attempt!
echo. Attempt Number: %E2%%E1%%E0%
:: wait %TC_RETRY_WAIT% seconds
ping -n %TC_RETRY_WAIT% 127.0.0.1 >NUL
GOTO INCR_COUNTER
:IC_RETURN
if %E1%==%TC_RETRY_COUNT% echo. Error: too many tries to reconnect!!! (# %E2%%E1%%E0%)
if %E1%==%TC_RETRY_COUNT% GOTO EXIT_FAIL
:: Test for exact ERRORLEVEL 0
IF ERRORLEVEL 0 IF NOT ERRORLEVEL 1 GOTO NETWORK
echo. Inturrupted with CTRL-C
GOTO EXIT_FAIL
:INCR_COUNTER
:: This script code for the counter came from Eric Phelps
:: [http://www.ericphelps.com/batch/samples/addition.txt](http://www.ericphelps.com/batch/samples/addition.txt)
:: Increments a three digit number
:: Works by comparing each digit
:: E2=hundreds, E1=tens, E0=ones
if [%E2%]==[] set E2=0
if [%E1%]==[] set E1=0
if [%E0%]==[] set E0=0
:E0
if %E0%==9 goto E1
if %E0%==8 set E0=9
if %E0%==7 set E0=8
if %E0%==6 set E0=7
if %E0%==5 set E0=6
if %E0%==4 set E0=5
if %E0%==3 set E0=4
if %E0%==2 set E0=3
if %E0%==1 set E0=2
if %E0%==0 set E0=1
goto IC_DONE
:E1
set E0=0
if %E1%==9 goto E2
if %E1%==8 set E1=9
if %E1%==7 set E1=8
if %E1%==6 set E1=7
if %E1%==5 set E1=6
if %E1%==4 set E1=5
if %E1%==3 set E1=4
if %E1%==2 set E1=3
if %E1%==1 set E1=2
if %E1%==0 set E1=1
goto IC_DONE
:E2
set E1=0
if %E2%==9 set E2=0
if %E2%==8 set E2=9
if %E2%==7 set E2=8
if %E2%==6 set E2=7
if %E2%==5 set E2=6
if %E2%==4 set E2=5
if %E2%==3 set E2=4
if %E2%==2 set E2=3
if %E2%==1 set E2=2
if %E2%==0 set E2=1
goto IC_DONE
:IC_DONE
GOTO IC_RETURN
:EXIT_FAIL
echo. FAILED! Exiting.
GOTO END
:CONTINUE
echo. We are ready to backup now! Here goes...
start /WAIT %XXCOPY_SCRIPT%
echo. Backup complete! Check the log for success.
:END
:: leave the drive dismounted
:: The author wants this because his network (at work) tends to go
:: down and cause delay write errors
echo. Dismount TC container (if mounted).
%TC_DISMOUNT_CMD%
xxbkup.bat
@ECHO OFF
:: Jeff Wilson's xxcopy batch file for doing "snapshot" backups of
:: select directories and files. I use it for backing up files
:: to a location on my network.
::
:: This program uses the shareware program, xxcopy (http://www.xxcopy.com).
:: I have tested with version 2.92.6 Pro Edition of xxcopy.
:: I think the registered version of xxcopy is needed to copy across
:: network shares. If you use this script with a different xxcopy version,
:: please double-check all the command line switches to see if they
:: still do the right thing!
::
:: This only does one backup (no support for multiple snapshots)
:: It will fail on open and exclusively locked files unfortunately.
:: In practice, I find that is not a big deal as I run this script
:: daily and eventually locked files get backed up.
:: In my case, I only have problems with MSFT Outlook PST files of
:: archived email.
::
:: Things this script won't help with:
:: If you delete one or more files accidentally and then run this script
:: successfully afterwards, the file will be lost (same goes for any unintended
:: modification of a file such as edits or virus infection).
::
:: Basically, this script will only help recover data after a catastrophic
:: hard drive failure or user error that is immediately realized.
::
:: Disclaimer:
:: This script is provided "AS IS". Author makes no claim that this will
:: work correctly. Author accepts NO responsibility for any damages
:: caused by this script. It is up to the user to determine if the
:: script will work in the desired fashion.
::
:: Copyright:
:: Jeff Wilson holds the copyright for this script.
:: Anyone may use this script so long as they don't sell it (unless they
:: get written permission).
:: Those planning to redistribute this script must maintain documentation
:: that I'm the original author (as well as my contact info).
:: Anyone may modify this script, and redistribute so long as the above
:: requirements are followed.
::
:: Jeff's contact info:
:: user: jeffrey.brian.wilson
:: domain: acm.org
:: put the two together above (with an @) for my email
set XXCOPY_PATH=xxcopy
:: for xxcopy args, mainly the clone setting is used,
:: but with some extras for forcing execution and byte-wise validation
set XXCOPY_ARGS=/CLONE/C/YY/V2/WD0
set XXCOPY_LOG_CREATE_ARG=/oN
set XXCOPY_LOG_APPEND_ARG=/oA
:: used to display log file upon completion
set TEXT_VIEWER=notepad.exe
:: These env vars are all user/situation specific. You may need to add more
:: depending on what you need backed up (or delete some here and below)
:: PLEASE PAY ATTENTION TO WHERE YOU ARE COPYING FROM AND WHERE YOU ARE COPYING TO.
:: THIS SCRIPT COULD ERASE ALL YOU DATA IF YOU GET THINGS BACKWARDS!!!!!
:: location for a log file
set LOG_PATH=C:\user\BACKUP\bkuplog.txt
:: various directories to backup and where to backup to
set ORIG_MY_DOCS=C:\DOCUME~1\user\MYDOCU~1
set BKUP_MY_DOCS=z:\MYDOCU~1
set ORIG_DESKTOP=C:\DOCUME~1\user\Desktop
set BKUP_DESKTOP=z:\Desktop
set ORIG_DEV=c:\dev
set BKUP_DEV=z:\dev
set ORIG_DB=c:\database
set BKUP_DB=z:\database
::Some XXCOPY notes
::/H copy hidden and/or system files
::/R overwrite read-only files
::/KS keep the file attributes of the source files (including read-only)
::/Q supresses display for skipped file
::NOT USED: /BN back up newer files only
::NOT USED: /DA copies newer files and brand new files
::/E copies directories and subdir (including empty ones)
::/C continues copying even on errors
::/Y overwrite exsisting files without prompting
::/V2 verify copied data byte by byte
::/oN<fname> Output a new logfile named fname
::/oA<fname> append to logfile fname
::/WD0 supresses warning for copying a non-directory source
::/BI back up incrementally (different by size or date)
::/ZY remove extra files or subdirectories in destination
::/ZE disables the use of all env vars for xxcopy (not sure why this is needed?)
::/YY Super yes! Answer yes to everything!!
::NOT USED: /CLONE is /KS/H/E/R/Q/Y/BI/ZY/ZE/oD0
:: /KS /Q /ZE/oD0
echo. Backing up: %ORIG_MY_DOCS%
@ECHO ON
%XXCOPY_PATH% "%ORIG_MY_DOCS%" "%BKUP_MY_DOCS%" %XXCOPY_ARGS%%XXCOPY_LOG_CREATE_ARG%%LOG_PATH%
@ECHO OFF
echo. Backing up: %ORIG_DESKTOP%
@ECHO ON
%XXCOPY_PATH% "%ORIG_DESKTOP%" "%BKUP_DESKTOP%" %XXCOPY_ARGS%%XXCOPY_LOG_APPEND_ARG%%LOG_PATH%
@ECHO OFF
echo. Backing up: %ORIG_DEV%
@ECHO ON
%XXCOPY_PATH% "%ORIG_DEV%" "%BKUP_DEV%" %XXCOPY_ARGS%%XXCOPY_LOG_APPEND_ARG%%LOG_PATH%
@ECHO OFF
echo. Backing up: %ORIG_DB%
@ECHO ON
%XXCOPY_PATH% "%ORIG_DB%" "%BKUP_DB%" %XXCOPY_ARGS%%XXCOPY_LOG_APPEND_ARG%%LOG_PATH%
@ECHO OFF
:: show the log file to the user
start %TEXT_VIEWER% %LOG_PATH%
:: This exit is for another batch file to call this one and wait for it to finish
:: calling this batch file directly will cause you shell to exit when it is done (if you're in one)
exit
Editor’s note: Original link no longer resolves; updated to an archived copy from the Wayback Machine.
Dead links