Setting up Dirvish to connect via SSH Tunnel

By Steve Ramage

Jid: sjr (a.t) sjrx.net

Version 0.90

13-Oct-05

Introduction

One problem some people have is that they don't like the idea that there backup server has r00t access to all of the machines they are backing up. Until now the only real other option was to set up an Rsync Server, but then your data would go over the network in plain text. Well this guide tries to reconcile these, but allowing you to backup data from a machine, that you don't necessarily have root access to, but is also encrypted. This guide assumes that you have an SSH Server fully working, and are familiar with dirvish, it also assumes that you are familiar with Rsync, or can get it running. You may want to look at the Windows Dirvish Article that I wrote as they use to be the same guide. (Ignore the fact that its windows, you just need the rsyncd.conf file setup).

There are three steps to how this works:

Setting up Rsync

Assuming you have an rsync server up and running, you need to make a few changes to the server (/etc/rsyncd.conf). Again to be clear you need to know how to setup the shares, and also how to setup authentication, otherwise even with the following options any local user could rsync and get access to all your files. (Its times like this I wish, <blink> was still valid HTML)
uid = 0 
hosts allow = 127.0.0.1

Setting up SSH

On the target system, that is the machine that has the data that you would like to backup you need to do the following:

1) Create a user that we can login with: (The account won't have a shell, nor a password, and I believe if its a system user ssh will refuse interactive login )

adduser --system --disabled-password --shell /bin/false dirvish

2) Follow the instructions here on creating an SSH keypair, and being able to login without a password: http://edseek.com/~jasonb/articles/dirvish_backup/transport.html#transportssh. The trick here is that you won't be able to login and get shell access to cat the key. So just copy it over manually. In other words generate it on the backup server, open id_pub.rsa (on backup) copy it to the authorized_keys file (on target) (cut and paste works) on the other system. Make sure that the dirvish user owns the file and the permissions are set properly.

3) You can test this by ssh-ing to the server now. You should see something like the following:

[17:38:17] root@backup:~/.ssh$ssh dirvish@target
Linux target 2.6.10 #1 Fri Feb 11 18:26:49 PST 2005 i686 GNU/Linux

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Connection to target closed.
4) Now you might being saying that's all well and good, and if rsync can copy the files in the 1/10th of a second I'm connected I guess I'm set, thanks for wasting my time reading this page, #@$%@*%#. But wait don't close this article yet, if you had bothered to read the man page for ssh you'd be aware of the -N option which is used expressely for this purpose, to quote:

 -N      Do not execute a remote command.  This is useful for just forwarding ports (protocol version 2 only).
5) Now to actually test that it works, you can run the following commands:
[17:48:00] root@backup:~/.ssh$ssh -N  dirvish@target -L 3333:www.dirvish.org:80&
[1] 15837
[17:48:02] root@backup:~/.ssh$lynx http://localhost:3333 
If the site pops up you've successfully configured SSH.

Setting up Dirvish

Here is an example of a default.conf file:

client: dirvish@127.0.0.1
tree: :etc
xdev: 1
index: gzip
image-temp: latest
log: gzip
image-default: %Y%m%d
pre-server: /etc/dirvish/start-ssh-pascal.sh
post-server: /etc/dirvish/stop-ssh-pascal.sh
rsync-option:
        --port
        60066
exclude:

I'll go through the important ones, one by one

client: dirvish@127.0.0.1

This line tells dirvish that you are going to connect to the rsync server on the localhost as the dirvish user.

tree: :etc

This line tells dirvish to connect to download the etc share on the rsync server.

pre-server: /etc/dirvish/start-ssh-target.sh
post-server: /etc/dirvish/stop-ssh-target.sh 

These options tell dirvish to execute the following script, prior to backing up the data, and after backing up the data respectively. The contents of the files follow below.

rsync-option:
        --port
        60066

This option tells dirvish to pass the option --port 60066 to rsync when connecting. This has the effect of getting it to connect on port 60066 instead of the usual 873. Spacing is IMPORTANT. I know you might not think so, but they have to be exactly like they are above, you can't have them on the same line, long story. Note that the 60066 is the same as it is the start-ssh-target.sh file that is below:

start-ssh-target.sh:

#!/bin/bash
start-stop-daemon --start --b --quiet -m --pidfile /etc/dirvish/ssh-target.pid 
--exec /usr/bin/ssh  -- dirvish@target -N -L 60066:localhost:873
sleep 2

You can really just modify the target, and make user that /usr/bin/ssh exists, and copy the line. start-stop-daemon is a common utility used to start and stop things. I stole the usage out of the pure-ftpd script on debian. The options are VERY necessary and are explained below:

Note the port number here should be the same as in the default.conf file above. The second line, sleep 2, is necessary cause it can take time for the ssh connection to be connected. If you are on a LAN then 2 is okay, you might actually want to set it higher for connections over the internet.

stop-ssh-target.sh

#!/bin/bash
start-stop-daemon --stop --quiet --pidfile /etc/dirvish/ssh-target.pid

This one is easy, the only difference is the --stop, which I hope is explanatory.

Testing the Configuration

If all goes well you should initialize your vault as follows:

dirvish --vault vaultname --init

As with most things, no news is good news.

FAQ:

Q: Where can I get help?

A: Well as your basically running the Linux versions of software, so the Linux documentation is very handy. It's available on the Dirvish website at:

http://edseek.com/~jasonb/articles/dirvish_backup/

If this does not answer your question, try asking on the mailing list, and we should be able to help you out. To join the mailing list, follow the instructions here:

http://www.dirvish.org/mailman/listinfo/dirvish

You can also contact me via Jabber at sjr a.t sjrx.net. But I'd only be good at helping you with the contents of this guide, the mailing list is probably your best bet if you're stuck.

Q: What if I have more than one machine/more than one vault?

A: You should make another copy of the scripts, and rename them. The only other possible conflict is the port you use locally. Now my 'understanding' is that with Dirvish 1.2.1 all the backups are done sequentially, meaning one at a time, so as of right now it may not be a problem, but I'd recommend using different port numbers. The same should be said if you are backing up two different vaults on the machine. Reason being you could probably use the same port, but if they are running in parrallel one might kill the connection for the other.

Q: How much slower is it?

A: I just ran thru a dry run on setting dirvish to backup my /home drive on another linux machine. I'm initializing right now, and am getting about 3 MB/sec, over my LAN (100 Mbps). This isn't a great speed, but its not a bad speed either. My guess would be that you'd get a similar speed if you were to use the classic SSH method.

Q: How do I setup SSH on Windows?

A: Get CopSSH from: http://prdownloads.sourceforge.net/sereds/Copssh_1.3.5_Installer.zip and install it. Maybe I'll write a guide for setting it up.

Q: OMG MY SHADOW FILE IS OUT IN THE OPEN!!!!

A: That's not actually a question, but yes you should have setup rsync authentication properly.

Q: Your start-up scripts suck, I could do a better job.

A: Please do, I'm no expert by any means. I know nothing of shell scripting and didn't want to spend the time learning it to write this guide. I basically stole it out of the init.d script for proftpd (on debian), thanks author whoever you are.

Q: Are there any disadvantages to tunnel via port forwarding, as opposed to the classic method?

A: Yes one big one is that you can't run pre-client and post-client commands. This makes dynamically stopping and starting db's a pain. But this would require root access anyway.

Q: I noticed that this page was made in Frontpage...

A: Yes it was, I know you are all retching, but I have the web-designing capacity of an ant, which is ironic since I'm also a PHP developer. Long story short yes it was made in frontpage.

Q: Have you done anything else?

Lol, obligatory plug. I'm also very much active in the Jabber Community, mainly on the mailing lists but recently I've undertaken writing a PHP Library for Jabber, called XMPP4. Homepage is here. I also wrote the Windows Dirvish Guide, which I don't have a link for because it will change with the creation of this guide.

Special Thanks To: