Strange rsync error on Synology DSM_

I was chasing down random errors last weekend in an effort to cut down on the daily deluge of messages from cron(8) and I realized that it had been several months since the Synology NAS I have at work successfully backed up. It only runs once a week so the e-mails largely got overlooked and somewhat shamefully when I came across them I often suspected that the office Internet connection just dropped mid-transfer.

Sent by Cron Daemon:
Permission denied, please try again.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(235) [sender=3.1.2]
Failed to sync iTunes Library
Export confbkp file to : /volume1/archive/config.dss
Permission denied, please try again.
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(235) [Receiver=3.1.2]
Failed to backup /archive

A quick glance at the output hopefully helps you understand my assumption that it was just a random connectivity issue, connection unexpectedly closed sure seems like a network error, but the more astute reader will probably also see the random Permission denied, please try again. tossed in there. It took some screwing around to figure out where this was coming from, debugging the ssh(1) connection showed it being sent from the remote site (in this case a Synology NAS running DSM 6.2.2U4) but running the rsync server command from an interactive ssh connection seemed to work just fine.

I'll spare you the screwing around I did to finally find out the culprit and just come clean. It turns out that unless you have the rsync service enabled in the DSM web UI rsync will return the Permission denied, please try again. error if executed through a non-interactive ssh connection, but will work directly from the Synology's shell. 🤷

If your search brought you here because of the Permission denied error and you have solved your problem then feel free to skip the next part. If you happen to be interested in how I backup my Synology then go ahead and read on.

The following script actually does two things. The first thing it does is sync a copy of my iTunes library from the backup that is made from my MacBook Pro to my Synology so I can access it with my Mac Mini at work. The second thing it does is dump a copy of the Synology DSM configuration file to the share that I backup on the device and then finally it uses rsync to backup the share to my NAS at home. The very last thing it does it update a local touch file so a script can verify that a backup was completed.

#!/bin/sh
# backup-synology (c) 2018-2020 Matthew J Ernisse <matt@going-flying.com>
# All Rights Reserved
#
# Backup the synology and update the iTunes library therein.

set -e

if ! rsync -rltDq --delete --delete-during \
    -e "ssh -i /vol/backup/backup_sshkey" \
    --exclude "Album Artwork/" \
    --exclude "Movies/" \
    '/vol/media/media/music/iTunes Library/' \
    "root@nas01.work.example.com:/volume1/media/music/iTunes Library"
then
    echo "Failed to sync iTunes Library"
fi

# Backup the config to the volume to be backed up so we get it.
ssh -i /vol/backup/backup_sshkey root@nas01.work.example.com \
    rm -- /volume1/archive/config.dss

ssh -i /vol/backup/backup_sshkey root@nas01.work.example.com \
    /usr/syno/bin/synoconfbkp \
    export \
    --filepath=/volume1/archive/config.dss

if ! rsync -rltDq --delete --delete-during \
    -e "ssh -i /vol/backup/backup_sshkey" \
    --exclude "Software/" \
    "root@nas01.work.example.com:/volume1/archive/" \
    "/vol/backup/work.example.com/archive"
then
    echo "Failed to backup /archive"
    exit 1
fi

touch /vol/backup/work.example.com/.last-successful-backup

Since this runs as root, you want to make sure your public key for your ssh keypair is put on the Synology in /root/.ssh/authorized_keys, you may have to make the .ssh directory. Also note that synoconfbkp doesn't overwrite the configuration backup file so make sure you delete it before creating it.

This will plow along if the iTunes library fails to sync but will exit with an error if it cannot sync the /volume1/archive folder.

Hopefully the first part of this will save someone else the headache I got myself into troubleshooting this.

🍻