CSYNCD User Guide

1. Introduction

csyncd is alpha software!

The daemon is still work in process. I’m using it daily but it could eat your data, you’ve been warned.

I recommend that you create backups of your synchronized folders with some snapshot backup tool like backintime and nothing fatal can happen.

At the moment the daemon only supports ssh with public/private key authentication.

2. Use Cases

Connect an USB flash drive to your PC and it gets synchronized automatically
- Gnome does mount it automatically
- When using KDE you have to enable automount or when the device notifier opens click on the black mount emblem alongside the device
- Be careful when using FAT file systems, better format your flash drive with ext2/ext3/ext4
- You can also use two partitions but when using a Windows PC you will need to flip the removable media bit that windows can recognize more than one partition: http://www.ghacks.net/2009/04/17/partition-usb-flash-drives/
Connect your laptop to your home network and your project folder gets automatically synchronized with your working PC
Synchronize a folder to an shared network folder via sftp:// or smb://

I have used Dropbox before but a) it is proprietary software and b) it needs an central server and all your data is transferred over the internet and I have only an slow internet connection (I have used encfs and later ecryptfs to secure my data).

Daemon running on both hosts not tested yet (which is not necessary until using inotify because we are doing a 2-way-sync).

3. Basics

How to get csyncd running.

3.1. Some Notes

You can start and stop the daemon and the gui independently from each other.
For a settings change to take effect you have to restart the daemon, this can simply be done by killall csyncd && csyncd.
If you are starting the daemon independently from the GUI (because you want so see more debug output) click on Save or hit Ctrl+S to save the settings.
If you want to see what the daemon is doing you can run it from the command line with csyncd --nofork or for more debug output with csyncd --nofork -v
You should also have a look at csync’s userguide

3.2. Configuration

~/.csync/csyncd.conf

An example configuration file which was created by kcsyncd, a KDE GUI for csyncd:

[default]
comment=
destination=/home/chris/csync/second
destination_id=
destination_uuid=
# dry_run, do not touch the filesystem
dry_run=1
# if this syncronisation task should be executed
enabled=1
exclude_file=/home/chris/.csync/default_exclude.conf
intervall_sync=1
source=/home/chris/csync/first
source_id=
source_uuid=
sync_intervall=1
sync_when_mounted=0
#option with_conflict_copys needs testing
with_conflict_copys=1

[to linuxbox]
comment=
destination=sftp://linuxbox.local:22/home/chris/csync
destination_id=
destination_uuid=
dry_run=0
enabled=1
exclude_file=/home/chris/.csync/to linuxbox_exclude.conf
intervall_sync=1
source=/home/chris/csync
source_id=
source_uuid=
# sync_intervall in minutes, sync every minute 2,3 .. 24*60
sync_intervall=5
sync_when_mounted=0
with_conflict_copys=0

[usb stick]
comment=
destination=/csync
destination_id=
# use ls -l /dev/disk/by-uuid/ to get the uuid
# and mount to get the mountpoint
# when the uuid is specified, the source and destination have to be relative paths
# from the mountpoint (but have to start with a /)
# we always use lower-case uuids
destination_uuid=c527ae39-55e1-4fe9-aa4a-3bf811a34f71
dry_run=0
enabled=1
exclude_file=
intervall_sync=1
source=/home/chris/csync
source_id=
source_uuid=
sync_intervall=5
# we will start syncing as soon as source and/or destination are mounted
# you only have to click inside your file manager (dolphin/nautilus) on the device so it gets mounted,
# or you have to setup automount, this also works with luks encrypted devices
sync_when_mounted=1
with_conflict_copys=0

images/kcsyncd.png

3.2.1. source_id and destination_id

Inside source and destination a folder named .csync with an file folder.id will be created (if it does not allready exist) which contains a number if this number is not the same as source_id (destination_id) we won’t sync. After the first sync this number will be added to this file (csyncd.conf), if source_id and/or destination_id are empty.

not yet implemented

3.2.2. New conflict resolution (with_conflict_copys)

Use the latest version of the two files under its original filename in both destinations (same as currently implemented).
Additionally, create a copy of the older file under a different name using the same file extension with appended marker (e.g. conflict) and timestamp in both destinations (e.g. textfile.odt → textfile_conflict-20100222-105000.odt)
At the Moment textfile_conflict-20100222-105000.odt is only created on the side where the conflict has occurred and gets synced to the other side on the second sync.

If files get changed on both sides between two syncs and you haven’t enabled with_conflict_copys you can loose some data.

3.3. Setup Avahi

source: http://fedoraproject.org/wiki/Docs/Drafts/ZeroconfLocalDomains

3.3.1. How to set up local domain names with Zeroconf

This paragraph describes how to set up a local network where each machine has a resolvable host name, without requiring a single server to be present. This is useful when for example only laptops are used in a home network, but none of them are always present. It can be done by setting up Avahi on each machine, which implements Zeroconf, and using the nss-mdns system to resolve hostnames using the Zeroconf multicast dns setup.

3.3.2. Configuration

Install the following packages:

$ sudo yum install avahi avahi-tools nss-mdns

The package adjusts /etc/nsswitch.conf so that the hosts line looks as follows:

hosts: files mdns4_minimal [NOTFOUND=return]  dns

This allows failure on domains ending with .local without checking DNS servers, but will only use mdns for non-.local domains if DNS fails.

Then you can edit /etc/avahi/avahi-daemon.conf and set the host-name to your desired hostname. This will combine with the domain which is set by default to .local (e.g. mytoaster.local. If the name was set before this step should not be necessary as the machine name is used instead.

Enable "Multicast DNS (mDNS) 5353/udp" in your firewall configuration:

$ sudo system-config-firewall

Finally run

$ sudo service avahi-daemon restart

to restart the avahi service.

You should then be able to resolve hostnames whenever machines are present on the network. You can also browse local services and hosts by running

$ avahi-browse -d local -a

3.3.3. Enable avahi sftp autodiscovery:

$ sudo vim /etc/avahi/services/sftp.service

<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name replace-wildcards="yes">%h SFTP</name>
<service>
<type>_sftp-ssh._tcp</type>
<port>22</port>
</service>
</service-group>

$ sudo service avahi-daemon restart

3.3.4. Try it out

[chris@thinkpad ~]$ sudo systemctl enable sshd.service
ln -s '/usr/lib/systemd/system/sshd.service' '/etc/systemd/system/multi-user.target.wants/sshd.service'
[chris@thinkpad ~]$ ssh chris@thinkpad.local
ssh: connect to host thinkpad.local port 22: Connection refused
[chris@thinkpad ~]$ sudo systemctl start sshd.service
[chris@thinkpad ~]$ ssh chris@thinkpad.local
The authenticity of host 'thinkpad.local (10.158.134.130)' can't be established.
RSA key fingerprint is 00:11:22:33:44:55:66:77:88:99:00:11:22:33:44:55.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'thinkpad.local,10.158.134.130' (RSA) to the list of known hosts.
Last login: Mon Sep  3 14:26:02 2012
[chris@thinkpad ~]$ exit
logout
Connection to thinkpad.local closed.
[chris@thinkpad ~]$

3.4. Setup SSH

3.4.1. Generating A SSH Key Public/Private Key Pair

Mind that if you leave the passphrase empty, anybody getting you private key /home/user/.ssh/id_rsa will be able to connect to your remote host. But at the moment this is the only way to let csyncd sync your files automatically, that’s why leave it empty.

$ ssh-keygen

By now, you should have id_rsa and id_rsa.pub in ~/.ssh directory. id_rsa is the so called private key. id_rsa.pub is the public key, the one you are going to upload on your server in order to be able to gain access to the remote machine using key authetication. You can define another filename to save your keys to. This become handy when you have a different set of key pairs to different hosts

Now set permissions on your private key:

$ chmod 700 ~/.ssh
$ chmod 600 ~/.ssh/id_rsa

3.4.2. Adding The Public Key To The Authorized Key

In the first place, we need to upload the key to the remote machine:

user@host:~$ scp ~/.ssh/id_rsa.pub remoteuser@remotehost:~/

Now, the public key is uploaded, let’s add it to the authorized keys. To do so, we are going to connect to remotehost as remoteuser and add the key at the end of file ~/.ssh/authorized_keys and delete it once added:

$ ssh remoteuser@remotehost
remoteuser@remotehost's password:
remoteuser@remotehost:~$ cat id_rsa.pub >> ~/.ssh/authorized_keys
remoteuser@remotehost:~$ rm id_rsa.pub
$ chmod 700 ~/.ssh
$ chmod 600 ~/.ssh/authorized_keys
remoteuser@remotehost:~$ exit

Maybe a better way, but I haven’t tested it yet: ssh-copy-id

Now, we need to configure the remote SSH server to accept authentication by key pair. This is usually enabled by default. If not, the next section will cover how to activate key based authentication.

3.4.3. Activating Key Based Authentication On The Server

$ ssh root@remotehost

Now open and edit /etc/ssh/sshd_config and make sure you have the following line:

RSAAuthentication yes
PubkeyAuthentication yes

Then reload your configuration:

# service sshd reload

Okay, now you should be able to connect to remoteuser@remotehost without supplying a password (but the passphrase of you private key if you supplied any) by simply typing the following:

user@host:~$ ssh remoteuser@remotehost
remoteuser@remotehost:~$

Or, if your private key file is not the standard ~/.ssh/id_rsa, you can inform ssh by using the -i switch as follow:

user@host:~$ ssh -i /path/to/private/key remoteuser@remotehost

Add an entry specifying the key file in your local ~/.ssh/config file:

Host linuxbox.local
  HostName linuxbox.local
  User chris
  IdentityFile /home/chris/.ssh/linuxbox_id_rsa

~/.ssh/config must have the following permissions:

$ chmod 600 ~/.ssh/config

Once you are sure that you can log into the remote host using your private key, we can safely disable the username/password authentication.

3.4.4. Disabling Authentication By Password

In order to disable authentication by password, we need to connect as root on the remote machine. On connected, go and edit /etc/ssh/sshd_config and make sure you have the following setting:

ChallengeResponseAuthentication no
PasswordAuthentication no

and reload SSH configuration file:

# service sshd reload

Now, open a new shell and connect the remote host using your private key:

user@host:~$ ssh remoteuser@remotehost
remoteuser@remotehost:~$

and check that you can’t connect without a key anymore:

$ cd ~/.ssh
$ mv id_rsa id_rsa.bck
$ ssh remoteuser@remotehost
Permission denied (publickey).
$ mv id_rsa.bck id_rsa

If you get rejected with Permission denied (publickey). it means it is all good and your ssh server is protected against brute-force attacks.

3.4.5. Disable Root Logins

SSH server settings are stored in the /etc/ssh/sshd_config file. To disable root logins, make sure you have the following entry:

# Prevent root logins:
PermitRootLogin no

and restart the sshd service:

service sshd restart

If you need root access, login as a normal user and use the su command.

3.4.6. Limit User Logins

SSH logins can be limited to only certain users who need remote access. If you have many user accounts on the system then it makes sense to limit remote access to only those that really need it thus limiting the impact of a casual user having a weak password. Add an AllowUsers line followed by a space separated list of usernames to /etc/ssh/sshd_config. For example:

AllowUsers alice bob

and restart the sshd service.

3.4.7. Disable Protocol 1

SSH has two protocols it may use, protocol 1 and protocol 2. The older protocol 1 is less secure and should be disabled unless you know that you specifically require it. Look for the following line in the /etc/ssh/sshd_config file, uncomment it and amend as shown:

# Protocol 2,1
Protocol 2

and restart the sshd service.

4. TODO

better logging so that you get a report which files have been updated and some statistics
inotify support like lsyncd
copy symlinks as symlinks
big warning and do not sync if more than 50% (xx%) of the files aren’t identical) (until the user approves that the sync should be done)
only transfer changed parts of a file like Dropbox or rsync (csync-devel)