For new Linux users, the hardest thing can be trying to get an answer to one simple question: "Which Linux distribution should I use?" Back in the world of that other OS, the choice is pretty simple since you had no choice, or as Henry Ford might have put it, you can have "any color you want as long as it's black". In the Linux world, you can get black, yellow, red, blue, green, and every color in between. I personally think it is a wonderful thing that so many Linux distributions exist. Aside from creating a rich OS landscape, it furthers creativity and fosters innovation in software design. This can only be a good thing. While this makes for a colorful world, it can be very confusing for the new user. The DistroWatch Web site alone lists over 350 Linux distributions.

So which distribution should you get? Well, every Linux vendor does things a little differently. If you think of this in terms of cars, it starts to make sense. Every single car out there is basically an engine on wheels with seats and some kind of steering mechanism so that drivers can get to where they want to go. What kind of car you buy depends on what else you expect from a car, whether that is comfort, style, the vendor's reputation, or any great number of other choices.

If you've come this far and you are reading what I'm writing here, then you probably want me to do what no sane person is willing to do, go out on a limb, take sides, and recommend something. All right, here goes . . . but I'm going start by dividing all of you into two groups.

Those of you in the first group haven't yet decided whether they are ready to give up on Windows. You know it's a good idea and you're going to switch at some point soon, but you would like to try Linux without having to actually commit to anything. For you, I'm going to recommend Knoppix, a wonderful Linux distribution that runs entirely from the CD without the need to install. There are other so-called live CDs out there (and I invite you to try others) but Knoppix is a great place to start. Download and burn a copy or pick up the first issue of TUX which will include a special version of Knoppix, remastered to include all of the software we are covering in the first issue.

Incidentally, because Knoppix can boot and run entirely from the CD and it can read the files on your hard disk, it has saved many a system and recovered many important files that would otherwise have been gone forever.

It's also a great item to carry with you when you go PC shopping. When you're looking at that spiffy new PC, ask the salesperson to let you boot it up with Knoppix to verify that your system is well supported.

Those of you in the second group have decided to just go for it and install Linux. It's a done deal and now you just need to know which Linux distribution to pick. If you are just getting started with desktop Linux, I would probably suggest that you look at either Mandrake or SUSE (now owned by Novell). Both are excellent, well-engineered, and beginner-friendly Linux distributions. Mandrake tends to be more geared to home users with excellent multimedia and game support (it's the one I usually install for friends and family). SUSE, meanwhile, tends to feel more polished and is certainly more business friendly. In both cases, you can buy boxed sets or download images which you can then burn to a CD.

This is by no means an exhaustive list and it certainly doesn't imply that other distributions aren't great and worthy of all the accolades that come their way. RedHat, Fedora, Xandros, Libranet, Linspire, and MEPIS are all great beginner friendly Linux distributions. Rest assured that under the skin, all of these remain Linux distributions.

Which brings me to what might be the single best suggestions for selecting a distribution. Sometimes, the ideal choice is to install what your friends running Linux are currently using. After all, one of the most powerful attractions to Linux is the enthusiastic and knowledgeable community support that is available. There's a lot to be said for taking advantage of an existing community of support, in this case, your friends and family.

Getting Linux

Getting Linux is the easy part. One way to get Linux is to buy a copy. Head down to your local computer software store and ask for your favorite distribution. If you live in the Bay Area, just send us an email requesting your distribution of choice and we'll arrange to get it to you. Alternatively, visit one of these vendors' Web site, whether it be Mandrake, SUSE, Red Hat, or any of the many different distributions listed on the DistroWatch DistroWatch, TLM-Project.org or LinuxISO.org Web sites and order one online.

You might be asking the question, If I can get a free copy of Linux, why would I want to pay for one? As it turns out, there is more than one answer to that question. The first is that buying a boxed set usually gets you some amount of technical support from the vendor. If you are feeling nervous about your first Linux installation, this might be a good reason. Furthermore, working with a vendor may offer other advantages such as certification of certain enterprise applications (eg: Oracle) to run with your specific platform. You'll often see messages stating that this database or this CRM system is certified to work on a particular version of Red Hat or SUSE.

Second, the boxed set usually contains some kind of manual or manuals specific to that version of Linux. That will inevitably lead to another question as to what makes this Linux different from that one. Finally, in purchasing a boxed set, you are supporting the company that put leather on the seats or tinted the windows. It's a way of saying, "Thanks for all the hard work."

Because it is possible to get a free copy of Linux, you don't have to shell out the dollars if you don't want to. At most, you'll need a fast Internet connection, a CD burner, and some blank CDs—or a helpful friend who has these.

The final option is to buy a PC with Linux preinstalled. A number of companies now offer systems with Linux preinstalled and that may be the way you want to go. Best of all, Linux PCs are often much less expensive than those preloaded with that other OS.

Here at Harmony Service, we assume you GNU/Linux users out there are already highly experienced technically savvy power users who know all the ins and outs of Linux. So why write a tutorial on something as simple as a backup? Well, let's just say that we all need a reminder to backup, and, heck, maybe that new Linux user out there has become so absorbed with all things Linux that he's forgotten to backup his great new works. Besides, we'll show you an easy, secure, safe and complete backup solution.

15 years in the technical profession has shown us just about every sort of data loss. Working in a college town has provided even more immediate experience. We've lost count of how many dissertations, term-papers, multi-media projects, and overdue homework assignments we've been asked to recover from dead media. Although we can recover just about anything, it's usually expensive and not 100% sure. We love to give the "computers are worthless without a good backup" speech to grieving customers.

Here are some facts about computers:

1. Hard drives are likely to fail.

2. Writable CD's and DVD's get scratched and wear out.

3. Zip disks are just about the least reliable hardware ever invented.

4. Tape drives are expensive, slow and difficult to recover.

5. Floppy disks are too small.

6. Flash drives lose their formatting and get lost.

So, if everything in our computing world eventually dies, whatever shall we do to keep our valuable data among the living? The simple answer: backup to a remote host using rsync and ssh. This answers all local dilemmas: no need to buy hardware, someone else is responsible for hardware integrity, data is off-site and is quickly available for recovery to any system anywhere. The possible drawbacks to remote backups are speed and space. The speed depends on your internet connection and the space depends on your remote host. If you have 200 gigs of music to back up, you are better off using a large external firewire hard drive. But for 95% of your personal data backup needs, remote backups will work perfectly. Of course, Harmony Service always recommends redundant backups of important data on as many types of media as possible.

Requirements:

To get started, we first assume you have an internet connection and a remote host. A high-speed internet connection is best for all upload and download tasks but certainly not necessary. Because our backup strategy can run in the background, as long as your computer is on and connected to the internet, the backup will happen. A remote host is any other computer providing log-in and storage services. In this tutorial, we use our web host as the remote host. The advantages to using your web host as your backup destination are speed (they'll have the fastest connection), reliability (they must honor their "always on" promises), redundancy (they keep their own backups of your domain), availability (you can recover your data anywhere there is an internet connection), and security (if you have a competent web host, they'll have cutting edge security measures installed by default). If you follow the Harmony How-To titled: Get Personalized Email and a Website, you'll have everything you need already setup.

Now, on to the how-to:

Rsync is a command line utility traditionally used in synchronizing files between two computers, but rsync can also be used as an effective backup tool. This free and powerful tool is simple enough for anyone to use on their Linux desktop.

First, make sure you have rsync by entering rsync --version at the command line. If you see rsync version 2.X.X protocol version X, you have it. If you see "command not found" or a similar message, you need to download and install rsync. Use your distribution's package management system to do this, or else download and install the source from the rsync Web site. Make sure your version is greater than 2.6.0.

Now it's time to consider what to back up, to where, and when?

What should be backed up? Do you want to run a full system or a partial system backup? A full system backup creates a second copy of everything on your hard drive. This has the advantage of providing a means to quickly restore your system to the exact state it was in when you made the backup. Full system backups take a long time to complete, take up a lot of disk space, and are often unnecessary. When you run full system backups, make sure to use rsync's --exclude parameter. Certain directories, such as /proc, should not be backed up. See the backup script below as an example.

Partial system backups are faster and more space-efficient, because you copy only important hand-selected data. For instance, you may want to backup only the /home directory, which contains users' documents, music, and program settings. The operating system files, such as those under /usr (programs) and /var (log files, email, etc.) can be easily reinstalled and don't need to be backed up.

Where should it be backed up? Your imagination is the limit when it comes to rsync's backup destination options. The scope of this tutorial, however, is limited to a remote web host.

When should it be backed up? Automated daily backups are a good choice for most Linux desktop scenarios. You can use Linux's built in scheduler, the cron daemon, with shell scripts to automate your backups.

Using rsync

The basic implementation of rsync is: rsync -a source/ target/. This command copies the source directory to the target as if you were executing cp -a source/. target/. Unlike cp, rsync uses the rsync algorithm to check for differences between source and destination files. Since it copies only new changes, a technique known as incremental backup, rsync provides a very fast method for updating your backups.

Make exact copies using the --delete flag. You can apply the --delete flag when making system backups, which causes rsync to delete any files found in the target that are not present in the source. This ensures that the target is an exact copy of the source, so that if you delete an unwanted document, it is also removed from your backup. Rsync preserves files found in the target and not in the source by default, allowing for multiple sources to be added to a single target destination. To get around this behavior, use a command like the following: rsync -a --delete source/ target/

Keeping multiple backups. It is a good idea to keep a few days' worth of backups so that you can return to a particular day if necessary. You can do this by rotating the oldest backup to the current one and updating it using rsync. The following script executes a single day to day backup. You'll find the code for a 3 day rotating backup system at the end of the tutorial.

#!/bin/sh
# Modified code from Brice Burgess -
# backup.sh -- backup to a local drive using rsync

# Directories to backup. Separate with a space. Exclude trailing slash!
SOURCES="/home/wendy /home/daisy/.thunderbird /var/mail"

# Directory to backup to. This is where your backup(s) will be stored before compressing and sending it to the remote host. Make this a directory inside your backup directory for easy cataloging.
# Exclude trailing slash!
TARGET="/backups/current"

# Your EXCLUDE_FILE tells rsync what NOT to backup. Leave it unchanged if you want
# to backup all files in your SOURCES. If performing a FULL SYSTEM BACKUP, ie.
# Your SOURCES is set to "/", you will need to make use of EXCLUDE_FILE.
# The file should contain directories and filenames, one per line.
# An example of a EXCLUDE_FILE would be:
# /proc/
# /tmp/
# /mnt/
# *.SOME_KIND_OF_FILE

# Comment out the following line to backup everything in the SOURCES variable.
EXCLUDE_FILE="/path/to/your/exclude_file.txt"

# Comment out the following line to disable verbose output
VERBOSE="-v"
###########################

if [ ! -x $TARGET ]; then
echo "Backup target does not exist or you don't have permission!"
echo "Exiting..."
exit 2
fi

echo "Verifying Sources..."
for source in $SOURCES; do
echo "Checking $source..."
if [ ! -x $source ]; then
echo "Error with $source!"
echo "Directory either does not exist, or you do not have proper permissions."
exit 2
fi
done

if [ -f $EXCLUDE_FILE ]; then
EXCLUDE="--exclude-from=$EXCLUDE_FILE"
fi

echo "Sources verified. Running rsync..."
for source in $SOURCES; do

# Create directories in $TARGET to mimic source directory hierarchy
if [ ! -d $TARGET/$source ]; then
mkdir -p $TARGET/$source
fi

rsync $VERBOSE --exclude=$TARGET/ $EXCLUDE -a --delete $source/ $TARGET/$source/

done

tar -zcpf /backups/currentbackup.tar.gz /backups/current

# Make sure you have a directory on your web host named "backup."
cat /backups/currentbackup.tar.gz |ssh -l sshusername sshhostname "cd backup; cat > currentbackup.tar.gz"

exit 0


Copy and paste the script into your favorite text editor (we've used vi), save it somewhere in your $PATH (/bin or /usr/local/bin) as backup.sh, and make it executable with the command chmod +x backup.sh. Change the SOURCES variable to the paths you'd like to backup. Be considerate on how large your backup will be compared to how much space you have available on your host. You should have at least 1 gig on your host (good for most backups). Change the TARGET variable to the path where you'd like your backups to be saved on the local computer before compressing and sending to the remote host. Make this a directory inside your backup directory for easy cataloging.

You'll notice that we are using the tar command at the end of the script to compress the backup and then the cat and ssh commands to send the compressed backup to the remote host. We do this for the best compression and security. Before you make your first backup, it's important to make sure your ssh connection to the web host is working. Make sure you have your ssh (or FTP) username, password, and hostname URL (usually your domain name). If you are hosting at 1and1.com, go to your web control panel and check the Web Space/Access SSH area to get this info. Then execute the command ssh -l sshusername hostnameURL. Enter your password when asked. You should then see a welcome screen with a new command prompt. You are now inside your web host at their command line! Neat! You can automate the ssh login process if you want to execute automatic backups. This is best if you want to have your backup run at a specific time every day; otherwise you'll have to enter your ssh password every time you backup.

To be able to logon to another server without being prompted for your password, you need to generate a key that will be trusted by the remote host, where your backups will be sent to. To accomplish this, follow the following steps as the user you will use (harmony here).

ssh-keygen -b 2048 -t dsa

You will then be prompted for a file name. Leave it as the default by simply pressing "Enter".

Generating public/private dsa key pair.
Enter file in which to save the key (/home/harmony/.ssh/id_dsa):

The last step of the key creation is the passphrase. Since the purpose of this is to not enter a password, hence being able to automate the backup, just hit "Enter" twice, leaving them blank.

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/harmony/.ssh/id_dsa.
Your public key has been saved in /home/harmony/.ssh/id_dsa.pub.
The key fingerprint is:
a6:84:5a:a8:cf:ff:31:38:21:85:ca:46:93:88:7a:50 harmony@localmachine

This just created 2 files in the user's home directory: ~/.ssh/id_dsa (the private key) and ~/.ssh/id_dsa.pub. The id_dsa.pub is your public key, which you share with the remote host. The id_dsa is your private key, and this is only for you. Do not lose it or share it with anyone, as this is your passkey! Make sure the file is not readable by anyone: chmod 600 ~/.ssh/id_dsa. Anyone having a copy of this key could steal your identity and login to this host as you. It is not any more dangerous to use this method as to use a traditional password, but I will not enter into a debate here.

Now upload the public key to your host with ssh,:

cat ~/.ssh/id_dsa.pub |ssh -l sshusername sshhostname "cat > id_dsa.pub"

log into your web host with ssh and create the .ssh directory if it isn't already there,:

mkdir .ssh

create the authorization key,

cat id_dsa.pub >> .ssh/authorized_keys

delete the public dsa file,

rm id_dsa.pub

and protect the new authorization file.

chmod 600 .ssh/authorized_keys

Log out by typing exit and then test the connection using:

ssh -l sshusername sshhostname

You should be logged in automatically without being prompted for a password! Log out again and move on.

You're now ready to make your first backup. Type backup.sh to start the process. The script can take some time to complete the first time you run it, because rsync must make a copy of each file rather than update just changed files. Later runs will complete much faster. So if your cursor blinks under the command for a long time, just be patient. If you notice something is wrong, press Ctrl-C to stop the process. Upon completion of the script, you should have a replica of your SOURCES in your TARGET and a gzip file of it on your host. Have a look.

Automating the process. Assuming backup.sh ran successfully, and that you now have a copy of your important files on the host, it is time to automate the process. We'll use Linux's built-in scheduler, the cron daemon, to do this. The cron daemon uses "crontab" files to schedule tasks. The system's main crontab file can be accessed by becoming the superuser (either by logging in as root or typing su at the command line) and executing crontab -e.

You'll want to schedule a time for your backup.sh to execute. Crontab syntax is:

[minute] [hour] [day] [month] [dayofweek] [command]

Thus, adding the line:

0 4 * * * /path/to/backup.sh

will execute backup.sh at 4:00am every day. When you're finished adding the line, save the file and exit.

That's all there is to it. Rsync is a very powerful tool, and you should pat yourself on the back for applying some of its potential. In the future we'll cover how to backup to a remote machine, show examples on how to keep multiple backups in rotation, and even run rsync within Microsoft Windows. In the meantime, check out Mike Rubel's excellent resource on rsync to learn how to perform daily and even hourly backups. If you have any problems with this script, please send us an email or log in and open a help desk ticket.

If you have enough space on your remote host, you don't have to include the compression part of the script. If you backup everything "as is" to the server (the first run of the script will take a long time), rsync will then only backup new data from your backup source (the next runs of the scripts will be much quicker). To do this, copy and paste this script into your favorite text editor, name it "backup.sh (or anything)," and chmod +x it. Change the source paths to match your backup directories. Change the destination path to match a backup directory on your remote host. Use a forward slash at the end of the source path if you want to only backup the contents of that directory. Omit the trailing slash to backup the contents and the directory to the remote host.

#!/bin/sh

rsync -av --delete -e ssh /path/to/source /another/path/to/source sshusername@remotehostURL:~/destination

exit 0

Here is the script for multiple backup rotations. The modifications will keep a designated number of backups in the target directory named after the date they were executed (YYYY-MM-DD_Hour-Minute).

#!/bin/sh
# Modified code from Brice Burgess -
# multi_backup.sh -- backup to a remote host using rsync.
# Uses hard-link rotation to keep multiple backups.

# Directories to backup. Separate with a space. Exclude trailing slash!
SOURCES="/home/wendy /home/daisy/.thunderbird /var/mail"

# Directory to backup to. This is where your backup(s) will be stored. No spaces in names!
# :: NOTICE :: -> Make sure this directory is empty or contains ONLY backups created by #this script and NOTHING else. Exclude trailing slash!
TARGET="/backup/current"

# Set the number of backups to keep (greater than 1). Ensure you have adequate space.
ROTATIONS=3

# Your EXCLUDE_FILE tells rsync what NOT to backup. Leave it unchanged if you want
# to backup all files in your SOURCES. If performing a FULL SYSTEM BACKUP, ie.
# Your SOURCES is set to "/", you will need to make use of EXCLUDE_FILE.
# The file should contain directories and filenames, one per line.
# A good example would be:
# /proc
# /tmp
# *.SOME_KIND_OF_FILE. Comment out the following line to backup everything in the SOURCES variable.
EXCLUDE_FILE="/path/to/your/exclude_file.txt"

# Comment out the following line to disable verbose output
VERBOSE="-v"

#######################################
########DO_NOT_EDIT_BELOW_THIS_POINT#########
#######################################

# Set name (date) of backup.
BACKUP_DATE="`date +%F_%H-%M`"

if [ ! -x $TARGET ]; then
echo "Backup target does not exist or you don't have permission!"
echo "Exiting..."
exit 2
fi

if [ ! $ROTATIONS -gt 1 ]; then
echo "You must set ROTATIONS to a number greater than 1!"
echo "Exiting..."
exit 2
fi

#### BEGIN ROTATION SECTION ####

BACKUP_NUMBER=1
# incrementor used to determine current number of backups

# list all backups in reverse (newest first) order, set name of oldest backup to $backup
# if the retention number has been reached.
for backup in `ls -dXr $TARGET/*/`; do
if [ $BACKUP_NUMBER -eq 1 ]; then
NEWEST_BACKUP="$backup"
fi

if [ $BACKUP_NUMBER -eq $ROTATIONS ]; then
OLDEST_BACKUP="$backup"
break
fi

let "BACKUP_NUMBER=$BACKUP_NUMBER+1"
done

# Check if $OLDEST_BACKUP has been found. If so, rotate. If not, create new directory for this backup.
if [ $OLDEST_BACKUP ]; then
# Set oldest backup to current one
mv $OLDEST_BACKUP $TARGET/$BACKUP_DATE
else
mkdir $TARGET/$BACKUP_DATE
fi

# Update current backup using hard links from the most recent backup
if [ $NEWEST_BACKUP ]; then
cp -al $NEWEST_BACKUP. $TARGET/$BACKUP_DATE
fi
#### END ROTATION SECTION ####

# Check to see if rotation section created backup destination directory
if [ ! -d $TARGET/$BACKUP_DATE ]; then
echo "Backup destination not available. Make sure you have write permission in TARGET!"
echo "Exiting..."
exit 2
fi

echo "Verifying Sources..."
for source in $SOURCES; do
echo "Checking $source..."
if [ ! -x $source ]; then
echo "Error with $source!"
echo "Directory either does not exist, or you do not have proper permissions."
exit 2
fi
done

if [ -f $EXCLUDE_FILE ]; then
EXCLUDE="--exclude-from=$EXCLUDE_FILE"
fi

echo "Sources verified. Running rsync..."
for source in $SOURCES; do

# Create directories in $TARGET to mimic source directory hierarchy
if [ ! -d $TARGET/$BACKUP_DATE/$source ]; then
mkdir -p $TARGET/$BACKUP_DATE/$source
fi

rsync $VERBOSE --exclude=$TARGET/ $EXCLUDE -a --delete $source/ $TARGET/$BACKUP_DATE/$source/

done

tar -zcpf /backups/currentbackup.tar.gz /backups/current

# Make sure you have a directory on your web host named "backup"
cat /backups/currentbackup.tar.gz |ssh -l sshusername sshhostname "cd backup; cat > currentbackup.tar.gz"

exit 0

Here at HarmonyService, we assume you Mac users out there know what you know and stick with it. We hope that backing up your data is one of those things you know. Just in case, we'll show you an easy, secure, safe and complete backup solution for your OSX systems.

15 years in the technical profession has shown us just about every sort of data loss. Working in a college town has provided even more immediate experience. We've lost count of how many dissertations, term-papers, multi-media projects, and overdue homework assignments we've been asked to recover from dead media. Although we can recover just about anything, it's usually expensive and not 100% sure. We love to give the "computers are worthless without a good backup" speech to grieving customers.

Here are some facts about computers:

1. Hard drives are likely to fail.

2. Writable CD's and DVD's get scratched, wear out and are very slow.

3. Zip disks are just about the least reliable hardware ever invented.

4. Tape drives are expensive, slow and difficult to recover.

5. Floppy disks are too small.

6. Flash drives lose their formatting and get lost.

So, if everything in our computing world eventually dies, whatever shall we do to keep our valuable data among the living? The simple answer: backup to a remote host using rsync and ssh. This answers all local dilemmas: no need to buy hardware, someone else is responsible for hardware integrity, data is off-site and is quickly available for recovery to any system anywhere. The possible drawbacks to remote backups are speed and space. The speed depends on your internet connection and the space depends on your remote host. If you have a dialup internet connection, this is not for you. If you have 200 gigs of music to back up, you are better off using a large external firewire hard drive. But for 95% of your personal data backup needs, remote backups will work perfectly. Of course, HarmonyService always recommends redundant backups of important data on as many types of media as possible.

Requirements:

To get started, we first assume you have an internet connection and a remote host. A high-speed internet connection is best for all upload and download tasks but certainly not necessary. Because our backup strategy can run in the background, as long as your computer is on and connected to the internet, the backup will happen. A remote host is any other computer providing log-in and storage services. You can use another Mac on the same network running OS X if you like. Simply open System Preferences, click sharing and note your hostname (something.local). The ssh username and password will be the admin username and password of your other Mac. Then, check the box next to "Remote Login." However, for this tutorial, we use our web host as the remote host.

The advantages to using your web host as your backup destination are speed (they'll have the fastest connection), reliability (they must honor their "always on" promises), redundancy (they keep their own backups of your domain), availability (you can recover your data anywhere there is an internet connection), and security (if you have a competent web host, they'll have cutting edge security measures installed by default).

If you follow the Harmony How-To titled: Get Personalized Email and a Website, you'll have everything you need already setup. Be sure that you select a hosting plan that provides ssh access and plenty of space. The 1and1.com Linux Business Plan now offers ssh and 200 gigabytes! Before continuing, make sure you have your ssh username, password and domain name.

Now, on to the how-to:

Rsync is a command line utility traditionally used in synchronizing files between two computers, but rsync can also be used as an effective backup tool. This free and powerful tool is simple enough for anyone to use on their OS X desktop.

If you want to learn more about rsync, checkout the rsync Web site.

We are quite aware that Mac folks are by default GUI users (Graphical User Interface - all those windows, pretty buttons, folders, etc). This means that you never dabble in the command line. And you should never have to. Although for this task, the command line is recommended (and most powerful), checkout RsyncX or Psync for free GUI based backup programs. Apple also provides the .Mac backup application called Backup that will sync your files with your .Mac account. We recommend against this for several reasons, the best being that your .Mac account provides precious little space at a very expensive price, the second best: it's extremely slow. The new Leopard (10.5) operating system now has Time Machine! Time Machine will probably work fine for all local backups even if it does access the hard drive too often. However, for offsite or network backups, the techniques below are superior.

On the other hand, OS X is built on a form of UNIX called FreeBSD, which is the king of the command line. So we think you should do this tutorial to prove that you are really a Mac enthusiast. Impress your friends!! Besides, this way is cool and that's the most important thing, to be cool.... If you aren't turned off by the thought of the command line (once you work with it a bit, you might actually be seduced by it), please read on to execute the best backup imaginable. Moving on....

Now it's time to consider what to back up, to where, and when?

What should be backed up? Do you want to run a full system or a partial system backup? A full system backup creates a second copy of everything on your hard drive. This has the advantage of providing a means to quickly restore your system to the exact state it was in when you made the backup. Full system backups take a long time to complete, take up a lot of disk space, and are often unnecessary. If you don't have your OSX installer CD and all your program CD's, you might just have to do a full hard drive backup.

Otherwise, it's best to simply backup your HOME folder. OS X has the handy feature of storing all of your data, including system settings, email, bookmarks, desktop items, dock settings, etc in your little house folder located in the Users folder in your hard drive. Of course, this means that you have to be vigilant about not saving stuff outside the HOME folder. Don't worry, the Desktop stuff is part of the HOME folder. Either way, you must have enough room on your web server to hold whatever you wish to backup.

Where should it be backed up? Your imagination is the limit when it comes to rsync's backup destination options. The scope of this tutorial, however, is limited to a remote web host.

When should it be backed up? Automated daily backups are a good choice for most OS X desktop scenarios. You can use OS X's iCal scheduler to automate your backups. Read on....

Using rsync

The basic command line implementation of rsync is: rsync -a source/ target/. This command copies the source folder to the target as if you were copying by dragging one folder to a hard drive. Unlike copying, rsync uses the rsync algorithm to check for differences between source and destination files. Since it copies only new changes, a technique known as incremental backup, rsync provides a very fast method for updating your backups.

Make exact copies using the --delete flag. You can apply the --delete flag when making system backups, which causes rsync to delete any files found in the target that are not present in the source. This ensures that the target is an exact copy of the source, so that if you delete an unwanted document, it is also removed from your backup. Rsync preserves files found in the target and not in the source by default, allowing for multiple sources to be added to a single target destination.

Let's get down now into the dirt of it....

The Terminal

Step 1

Open the Terminal.  This should be an icon on your dock - hee hee.  If not, open your Macintosh HD/Applications/Utilities/Terminal. Now don't freak out, just flow with it, m'kay?!

When you open Terminal, you should see something that looks like this:

Welcome to Darwin!
The-Universe:~ harmony$

Except that instead of "The-Universe:~ harmony$" you will see something like:  "SomethingSomething:~ yourusername$"

The "$" symbol is the command prompt.  You will type commands there and hit the return key after every command.

For example, try typing the word "date," without the quotes or comma and press the "return" key. Now try typing the word "cal," without the quotes or comma and press the "return" key. Pretty cool eh? No? Whatever!

Step 2

To be able to logon to your web host without being prompted for your password, you need to generate a key that will be trusted by the host, where your backups will be sent to. To accomplish this, type the following command at the $ prompt - then press the "return" key (you might want to copy and paste the command exactly as it's written here to avoid errors):

ssh-keygen -b 2048 -t dsa

If you are using the Tiger operating system or above as the server (10.4 or above), you should use this command from the client:

ssh-keygen -b 1024 -t dsa

After a long pause, you will then be prompted for a file name. Leave it as the default by simply pressing "Enter".

Generating public/private dsa key pair.
Enter file in which to save the key (/Users/harmony/.ssh/id_dsa):

The last step of the key creation is the passphrase. Since the purpose of this is to not enter a password, hence being able to automate the backup, just hit "Enter" twice, leaving them blank.

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/harmony/.ssh/id_dsa.
Your public key has been saved in /Users/harmony/.ssh/id_dsa.pub.
The key fingerprint is:
a6:84:5a:a8:cf:ff:31:38:21:85:ca:46:93:88:7a:50 harmony@localmachine

Don't worry about all this text, just let it go. This command just created 2 files in your home folder; one named "id_dsa" and one named "id_dsa.pub." So you open your home folder and don't see the files?? Bwahahhahaha. I shouldn't tell you, but they are located in a hidden folder named ".ssh." Don't worry about this yet (except that if you want to make a folder or file invisible, type a period at the beginning of its name - careful - you won't be able to see it except in the Terminal).

The id_dsa.pub is your public key, which you share with the remote host. The id_dsa is your private key, and this is only for you. Do not lose it or share it with anyone, as this is your passkey! Let's make sure the file is not readable by anyone by cutting and pasting this command into the $ prompt and pressing the "return" key:

chmod 600 ~/.ssh/id_dsa

Anyone having a copy of this key could steal your identity and login to this host as you. It is not any more dangerous to use this method then to use a traditional password, but I will not enter into a debate here. This procedure is safer than shopping online, and you do want your crap backed up right?

Step 3

Now you'll upload the public key to your host with this command using ssh. Remember "sshusername" and "domainname" is the ssh username and hostname of your website (or other computer). You will be asked for your ssh password after you press "return":

cat ~/.ssh/id_dsa.pub |ssh -l sshusername domainname "cat > id_dsa.pub"

Log into your web host with ssh and create the ".ssh" directory if it isn't already there:

ssh -l sshusername domainname

Example: ssh -l ednacrabapple pricipleskinner.com

or

ssh -l usernameOfOtherMacComputer hostnameOfOtherMac.local

mkdir .ssh

Believe it or not, you are now logged into your webhost (or other computer) through the command line. That's right, you are working on their computer right now!

To complete the automated authorization process, create the authorization key:

cat id_dsa.pub >> .ssh/authorized_keys

delete the public dsa file,

rm id_dsa.pub

and protect the new authorization file.

chmod 600 .ssh/authorized_keys

Log out of the web host by typing:

exit

Press "return" and then we'll test the connection. Now that you and your host have the secret authorization key, you should be able to "ssh" into your web host without entering a password - sneaky....Give it a try by entering the following command and pressing "return":

ssh -l sshusername domainname

You should be logged in automatically without being prompted for a password! Remember, we went through that complex (not!) process so that we could do automated backups. Create your backup folder (it's best if this folder lives outside of your web root):

mkdir myBackup

(you can change the name "myBackup" to be anything you want, as long as it's under 13 characters, has no punctuation, and has no spaces.)

Log out again by typing exit and move on.

Build a Program

Groovy, now we are back at our own command prompt, that trusty $ sign. As Bob Ross used to say: "Let's get crazy."

In order to actually backup your crap, we now have the privilege of building a computer program. Never thought you'd do that, eh? So gather yourself and stare at that $ prompt. Or better yet, do the following:

Step 1

We need more power to create programs, so we'll activate the Super User (root). Zoinks. The Super User has absolute power over your system, so be sure to use him wisely (you could conceivably delete your entire system with him). If you are not an "administrator" on your computer, you'll have to use the 'sudo' command. But we like living on the edge, so we'll tell you how to activate the Super User. You've been warned. We'll deactivate the Super User when we are finished, just in case.

Open your hard drive/Applications/Utilities/NetInfo Manager. Click on the Security menu and click Authenticate. Enter your OS X system password. Then go back to the Security menu and click "Enable Root User." You will be asked to enter a new password for the Super User. It would be best not to enter something simple like "root" or "password." If anyone got ahold of this password, you'd be f*d. Leave NetInfo Manager open and get back to the Terminal.

Step 2

Log in as the Super User (Super!) by typing the following and pressing "return":

su

Password:

(enter the root password you just created)

You won't see much different at this point, but beware, you have great power.

Step 3

Enough jive, let's build the program. Type the following set of commands and press "return" after each. You will be going to a program folder, then writing the program, m'kay?

cd /usr/bin

(go to the hidden program folder)

vi backup

(create a program named "backup" using the text editor vi)

Copy (Ctrl + c) this code exactly from the "#!/bin/sh" all the way to the "exit 0."

#!/bin/sh

rsync -av --delete --stats -e ssh /Users/homefolder sshusername@domainname:~/myBackup > ~/Documents/backup.log

exit 0

Go back to the terminal and type the letter "i." You will see the word "Insert" at the bottom of the page. Paste (Ctrl + v) the code and it should appear at the top of the page. You'll have to edit the code to match your info using the arrow keys to move the cursor and the delete key to delete "homefolder," "sshusername" and "domainname." Be careful not to add extra spaces, or delete any other part of the script. Your homefolder name can be found by opening your hard drive/Users folder. See the little house? See it's name? That's your homefolder name. You should know your sshusername and domainname by now, right? If you used a backup folder name other than "myBackup," change that too. The last part of the script tells the program to display the action of the script in the file /Users/HOMEFOLDER/Documents/backup.log. If you want to watch the backup in progress, you can double-click this file while the backup script is running and the Console will open to display the files being checked and/or backed up. You can also choose to save the log in any directory.

When you are satisfied the information is correct, press the "Control" key and the "c" key. Now type a colon (:) and the letter "x" and press "return." Congratulations, you just wrote a program, sort of like how they made Photoshop, eh eh, right.

Rsync is a very flexible program and can do all sorts of things. It can act as your internal network wide backup system too (free and easy). You can push data from your client computers to your OSX server or you can pull data from all the computers on your network to the backup server. For example, if you wanted your OSX server to pull data from a computer with the sharing name "Pully," you'd setup rsync on the OSX server as follows (this will only work with Server version 10.4 or greater):

#!/bin/sh

rsync -avE --delete --stats -e ssh :~/myBackup /NetworkBackup > ~/Documents/backup.log

exit 0

Notice that the only difference is the "E" flag and that the source and destination have switched places. The E tag allows the transfer of Macintosh resource forks so that your Mac will know which program belongs to which file (only with OSX 10.4 or greater). Switching the source and destination makes the local computer pull data from the remote computer instead of push, as in the first example.

Make the program executable using the following command and pressing "return":

chmod +x backup

and exit to your regular safe user:

exit

Step 4

Let's deactivate the Super User now by going back to NetInfo Manager, clicking on the Security menu and clicking "Disable Root User." Click on the Security menu again and click "deauthenticate."

Step 5

You're now ready to make a decision. You can start your first backup now or you can configure the scheduler to run the backup at any recurring time you please. Either way, your first backup is going to take a long time (depending on how big your home folder is). A 2.1 gigabyte home folder took about 4 hours with cable internet. But hey, we have nothing but time right; just start the backup at 5:00pm one day and in the morning (or next week) you'll have your first backup complete. Once that's done, the subsequent backups will be almost instantaneous. If you want to execute your first backup now, read on. Instructions on automating the process are further down.

To start the backup manually at any time, all you have to do is open the Terminal and type:

backup

It will appear that nothing is happening. The cursor drops to another line. But if you open the file located in your Documents folder named "backup.log," you will see the action of the backup. The computer will build a list of new files and then send them to your website backup folder. That's it. Plan to leave your computer on and not let it sleep for the duration of the first backup. Just turn your monitor off and leave it be. As soon as the backup is complete, the command prompt will return.

Automation

Assuming "backup" ran successfully, and that you now have a copy of your important files on the host, it is time to automate the process. You have the option of using a cron job, a LaunchDaemon or an AppleScript. We prefer cron jobs, but we'll use OS X's calendar program "iCal" and AppleScript Editor to do this just for kicks.

If you want to setup a cron job, open the terminal as the regular user and type the following:

crontab -e

This will edit the local scheduling system. Now type the letter "i" once in order to edit the file. Type the following if you want to run the backup every day at midnight (assuming you named your backup script "backup"). Each character is separated by a tab.

#Backup my HOME folder every day at Midnight

0 0 * * * backup

Press and hold the Control Key on the keyboard and type the letter "c". Now type a semicolon (:) and then type wq and press return. Your cron job will be saved and activated. If you want to confirm this, type:

crontab -l

This command lists your active schedules. If you want to edit the schedule, type crontab -e

If you want to learn more about cron formats, click here

To use AppleScript and iCal, keep reading

Open the Script Editor in your hard drive/Applications/AppleScript folder. You'll see an untitled script window. Copy and paste this code into the window:

do shell script "backup"

Click on the "File" menu and click "Save As." Make sure the file format is "script," name it "BackupApp" and save it to a place you won't forget, move, or remove - the Applications folder would be apropo.

Open iCal in your Applications folder or on your dock. Double click on todays date. This will enter a new appointment. On the info pane to the right or left, you will see the appointment details, like this:

Change the title of the event from "New Event" to something like "Backup". Make the "from" and "to" times exactly the same. To the right of the word "alarm" click "None" and change it to "Run Script." Click the "None" below "Run Script" and click "Other." Go find where you put the "BackupApp" script, click on it and then click "select." Below that, change the 15 to 0 and click the pull down to "minutes before." Now, above that, click the "None" pull down menu to the right of the word "repeat." Select "every day" if you want to backup everyday, etc. Make sure the Backup appointment is set for everyday over the next few months (just to be sure) and then quit iCal.

You can also schedule the backup using the cron or launchd systems built into OSX. Click here for a good tutorial on using cron jobs, and click here to learn about the Launch Daemon.

That's all there is to it. Take a deep breath and pat yourself on the back for applying some OS X potential ... or shake your fist at the screen for not cooperating. If you have any problems with this process, please send us an email.

top

Transferring files between two computers using FireWire

You can use FireWire to connect your computer to another Mac and have one of the computers appear as an external hard disk on the other computer. This is sometimes called Target Disk Mode or T-Booting.

 To transfer files using FireWire:

    1.  Shut down the first computer and leave the second computer on.

    2.  Connect the two computers using a 6-pin to 6-pin FireWire cable. (If both computers have higher-speed FireWire 800 ports in addition to the standard FireWire 400 ports, you can use a 9-pin to 9-pin cable with the FireWire 800 ports to transfer data at higher speeds.)

    3.  Start up the first computer while holding down the T key.

 A disk icon for the first computer appears on the desktop on the second computer. Drag files to and from the disk to transfer them.

    4.   When you finish, eject the first computer's disk by dragging its icon to the Trash.

    5.   Push the power button on the first computer to shut it down and disconnect the FireWire cable.

 If the computer to which you connect your computer is running Mac OS 9, it must have FireWire version 2.3.3 or later installed. Use Apple System Profiler (located in Applications/Utilities) to check the version of FireWire installed. If it has an older version, get an update at the Apple FireWire website at www.apple.com/firewire.