Apple Timemachine backups on Debian 8 (Jessie)
Upgrading Debian 7 (Wheezy) servers to Debian 8 (Jessie) proves (unexpectedly) quite rough around the edges.
That's what you get for using a version x.0, we should have known better .
And - of course - the release notes follow the common practice of not even mentioning any of the issues we encountered so far.
Ah, well, let's go through the first one:
In Debian 7 (Wheezy) there was netatalk 2.2.2 (packages link). Now during upgrades that package may or may not get removed. There is no netatalk in Debian 8 (Jessie) anymore. Duh. There is in sid (aka Debian unstable) (packages link) so we may see a backport some time. Or not. In any case this is still 2.2.5 at the time of writing and as Adrian Knoth put it in the three year old bug asking for a upgrade to Netatalk 3:
Let's not ship another release without netatalk3, it's embarrassing.
Yes. It is. Removing a working version and not even mentioning it in the release notes is even worse though. So no cookies there.
Luckily the absolutely awesome Debian and greater FLOSS community have sorted 90% of the problem out for us already:
A quick Google search turns up an excellent article on netatalk's wiki that details installing Netatalk 3.1.7 on Debian 8 Jessie. There are two shortcomings to this: First it doesn't compile to .debs but installs besides apt and friends. And second it compiles with Spotlight search, courtesy of Gnome tracker, which doesn't really work well on servers yet. Hence we're lucky that Adrian Knoth's debified install has not yet added the tracker dependencies. It does compile for systemd use (the default for Debian 8 Jessie). If you want to continue using SysVInit, you need to modify debian/rules
.
Compiling to .debs becomes as easy as:
apt-get install build-essential devscripts debhelper cdbs autotools-dev dh-buildinfo libdb-dev libwrap0-dev libpam0g-dev libcups2-dev libkrb5-dev libltdl3-dev libgcrypt11-dev libcrack2-dev libavahi-client-dev libldap2-dev libacl1-dev libevent-dev d-shlibs dh-systemd
# in case you want to try the tracker support (you need to ammend the debian/ build config as well)
# apt-get install tracker libtracker-sparql-1.0-dev libtracker-miner-1.0-dev
git clone https://github.com/adiknoth/netatalk-debian
cd netatalk-debian
debuild -b -uc -us
This should leave you with (at the time of writing this):
File | Function | md5 | sha1 |
---|---|---|---|
libatalk-dev_3.1.7-1_amd64.deb | Development files for the libatalk library (dev only) | e5a465e39a8560c919d8db85c8e5a83b | 0b924cf75f22ab42406289c6f18ae0243d6396a3 |
libatalk16_3.1.7-1_amd64.deb | libatalk library (needed) | 17a3d677ed0b3df1c2f4c1a8ab9045fd | 7345ed3edd442716c99c2fe979140703204c0826 |
netatalk_3.1.7-1_amd64.deb | netatalk daemons (needed) | c694abca7f3cdc0070b2b3e7d528324a | 932d1e3d5899958f29e79a7ba40e858d4ac272e8 |
Obviously you can download the files above if you run the AMD64 architecture and trust me enough to compile them for you.
If you need afppasswd and the depreciated, insecure uams (User Authentication Modules) that go with it, please check the update from 02.06.15 below.
Ok, whichever version of the debs you built or chose to download, now for installation:
# clean old netatalk 2.2.2 leftovers in case they have survived the Jessie upgrade
apt-get remove netatalk
# Install our newly built packages
dpkg -i libatalk16_3.1.7-1_amd64.deb netatalk_3.1.7-1_amd64.deb
Fortunately the netatalk devs decided to simplify the netatalk configuration as that was a mess in v2.
Unfortunately that means you need to adjust your configuration now.
Basically the whole configuration now happens in afp.conf
and extmap.conf
(which you can safely ignore usually). So afpd.conf
and AppleVolumes.default
and AppleVolumes.system
are all depreciated and you should move them out of /etc/netatalk
to a safe place.
For a timemachine share you need something like:
path = /backup/timemachine/laura
time machine = yes
# vol size limit is in kB
vol size limit = 500000
valid users = laura
which you can construct from your old AppleVolumes.default
and the helpful equivalence table from the netatalk upgrade documentation.
The files /etc/netatalk/afp_signature.conf
and /etc/netatalk/afp_voluuid.conf
now live in /var/**lib**/netatalk/
(for Debian). The netatalk docs say /var/netatalk
but that violates Debian's file system layout and has hence been fixed in the debianization mentioned above. So move them to /var/lib/netatalk/
.
With netatalk 3 the default way to store Apple's resource forks and assorted cruft has also changed from using .AppleDB
subdirectories to storing them in extended attributs (EAs) of the underlying filesystem. The conversion will be done on the fly or - if you feel like speeding things - up removing the .AppleDB
directories in your Apple vaults and then running:
NB: (adjust your path...)
We can enable the systemd units (that's what the init.d file equivalents are called now ) and run the daemons again:
systemctl enable netatalk
systemctl start avahi-daemon
systemctl start netatalk
Update
08.08.2015: systemd and perl in Jessie
I originally wrote "I'll cover more issues around perl and systemd in later blog posts." but I won't. For perl the reason is simple, we fixed the {useless|annoying|undocumented} perl 5.16+ depreciation (use of the defined function on aggregates) in all code very quickly. I was a major bummer for Jessie upgrade documentation to not warn of the incompatible perl version in the release notes, but that was it. See the glastree github for the simple sed -e "s/defined @/ @/"-type fix. Not worth writing a full blog post about it.
Now for systemd the reason is just the opposite. It is so broken in the Jessie incarnation that after battling with it for a few weeks, I decided to postpone using it on Debian systems for another release cycle. It needs to mature much more. Without systemd has good information on how to tame the beast in Jessie and Sid. Read from my friends at suckless what's conceptually wrong with systemd. Christoph Egger, one of the Debian maintainers, has written a piece and the bug list is so frightening and yet incomplete that I see no use in investing more time there right now. That said: check your watchdog. Yup, not running anymore if you upgraded from wheezy. If you want to use systemd I can recommend Fedora 22. It has its quirks there as well but they are nowhere near what you need to suffer through on Debian servers.
05.07.2015: logrotate script
Erik pointed out in the comments that the default afp.conf writes to /var/log/netatalk.log but netatalk does not install a logrotate configuration file for that.
So I'd drop the following into /etc/logrotate.d/netatalk :
rotate 13
minsize 10M
monthly
copytruncate
compress
delaycompress
notifempty
}
The important thing is the copytruncate as netatalk would continue writing to the old log file otherwise as a SIGHUP only causes a re-read of the configuration, not including a log file close and re-open as it should be.
02.06.2015: OpenSSL builds for backward compatibility with uams_dhx.so, uams_randnum.so and afppasswd
Niek's comment below made me aware of an issue for people that upgrade from Netatalk2 and have not used uams_pam.so before.
The old "Randnum exchange" and "2-Way Randnum exchange" uams (User Authentication Modules) have long been depreciated because they only encrypt passwords on the wire with 56 bit DES, so that's unsafe by any standards these days. So you should use the builds from the table above and authenticate your users against PAM (eg. via /etc/passwd, /etc/shadow or LDAP if you have that set up).
Please see the excellent Netatalk documentation on authentication options for an explanation of the options and how to enable safe password storage and usage.
That said, if you have a migration scenario and are sure to need the old uams and afppasswd to go with them, here you go:
File | Function | md5 | sha1 |
---|---|---|---|
libatalk-dev_3.1.7-1_amd64.deb (openssl build) | Development files for the libatalk library (dev only) | 4fd411d865820ce523697c31157fa6a8 | 3a4e9f0b6fd649518f499f52005a95739cadcba8 |
libatalk16_3.1.7-1_amd64.deb (openssl build) | libatalk library (needed) | ae1b665e0ea828c6822db4049e821d06 | eb8596ffac9138e06859bd3ddac02201ff6831b9 |
netatalk_3.1.7-1_amd64.deb (openssl build) | netatalk daemons (needed) | 42de18cc4572849c42b221c863ea2b3a | 66f924694de28c999a3287603115588e3388761b |
These have been built with openssl support (replacing the default libgcrypt). Thus they have uams_dhx.so, uams_randnum.so and afppasswd available after install.
Comments
Display comments as Linear | Threaded
Niek on :
Thanks for this! I'm pretty new to afp on Linux, but it seems these pacakges do not contain afppasswd, so i can't create users.
Daniel Lange on :
Thanks for the feedback Niek!
If you're new to afp you most probably don't want afppasswd because that is a depreciated way of storing passwords in a separate file. These days you'd want to use PAM (Pluggable Authentication Modules), the default way to authenticate on Linux systems.
With PAM your normal Linux users are also able to authenticate from a Mac to the afp shares. So you create your users in Linux and manage them only once, e.g. via useradd/usermod.
I have still updated the article and included openssl builds (that have afppasswd and the uams (User Authentication Modules) that go with it). Hopefully not for your new installation but for people that migrate from older installations and want to first upgrade and then improve the security of their installation in a second step.
Sean on :
Thanks for the post. Works like a charm on my (ARM) Qnap NAS running Jessie.
Eric on :
+1 thanks. You saved me a ton of time.
Just don't forget to add a logrotate.d rule too since it's not included and the sample afp.conf is not using syslog.
cope on :
Hey, just wanted to say thanks. It worked perfectly on my Debian Jessie.
The only thing I had to change was to reboot the server. Maybe I did something wrong
Cheers!
aimee on :
Hi compiled the deb's and installed as per instructions. It now works with my yosemite mac, yipee, but when I first clicked on the home directory i got this message...
An unexpected error occurred (error code 0).
Thanks for you efforts, its loads better than using ftp!!
Aimee
Ulf on :
Hi
Thanks for this superb cookbook. Now I have a timemachine backup system for my Macs with Jessie again.
Peter on :
+1
Works perfectly, thanks !
vincent on :
Thanks. It works perfectly. Now I have to do the same thing for Debian 7 because of xenserver 6.2 ( It not allow Debian Jessie ). I m looking for compiled packages of netatalk 3 for wheezy. Please, if you know where i can find it, say it
Thank you
Vincent
marty on :
Great read!
and working perfectly on Jessie.
didnt work without a reboot, not quite sure why. but, i can re-claim a USB3 removable hard disk now since dont need it for time machine. sweet. double bonus he he.
love your work!
Rene on :
Great post!
The rc file (/etc/init.d/netatalk) was out-of-date so I had to copy it from the git folder. I think it remained from the 2.x deb package.
cp netatalk-debian/distrib/initscripts/rc.debian /etc/init.d/netatalk chmod +x /etc/init.d/netatalk
Dmitry on :
Thanks for the guide Daniel, it was very helpful, i didn't even realize netatalk was gone in jessie.
Petri Laakso on :
Thank you! Now I've got Time Machine up and running!
Stéphane on :
Hi, How to manage this with multiple users ?
Thanks for the tuto!
Daniel Lange on :
You can copy the [TimeMachine_for_Laura] section into [TimeMachine_for_AnotherUser] and adjust "path" and "valid users" lines accordingly.
Or you can use the [Homes] special section, see
man afp.conf
.Roger Weihrauch on :
Howdy Daniel
1st: Thank you very much for your effort on this tutorial and creating the deb files; esp. with openssl support. Thank you.
2nd: Since, as you mentioned in the beginning, there was no netatalk installed/delivered before/with debian jessie: Which are the config steps on a (newly installed) debian jessie? (what has to be done where) How to do this config with fokus on a clean and conform Debian system layout?
3rd: Since I am not very conform with samba/afp/cifs/netatalk config, I do understand your steps, but I do not know how to perform all side effected files / configs. Could you help me in this matter, please? Or, could you tell me which manuals for which tools/programs are relevant to do an successful config?
Help is really appreciated in this case from everyone.
Thanks in advance, Roger
Berni Elbourn on :
I needed to add:-
afp listen =
to /etc/netatalk/afp.conf
before my mac mini would connect properly on my network.
Jon Frydensbjerg on :
Thanks, Daniel! This works wonderfully for Raspbian, as well.
Tip: By adding -j4 to debuild I was able to build all packages in about 15 minutes on a Raspberry Pi 3.
cw on :
on fresh jessie installation cannot compile
followed precisely steps above, please advise
Daniel Lange on :
You are building version 3.1.10 which is a new upstream version, Adi has imported into his git repository. This has changed the build dependencies. Run
dpkg-checkbuilddeps
from within thenetatalk-debian
directory and you should get a list of unsatisfied dependencies for building the package yourself.b on :
dpkg-checkbuilddeps: Unmet build dependencies: licensecheck
ok what next?
Daniel Lange on :
So many options...
Either install licensecheck from stretch or sid (it has been separated from devscripts as per https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=828872 ) or change debian/control to still include devscripts instead of licensecheck or install equivs and make yourself a licensecheck package that a stretch update will replace (so version below 2.0.9).
Or you can just use the 3.1.9 packages I provide as the bugfixes in 3.1.10 seem quite minor and are incomplete, see https://sourceforge.net/p/netatalk/bugs/636/.
Alexander on :
Hi, Daniel! I got some troubles installing netatalk on my Debian 8 server. After clean reinstalling Debian 8.7 all was OK some days. Today morning after powering on my MacBook says: "Backup is cancelled because backup disk is busy". Last backup was near midnight, at 01:37.
systemctl restart avahi-daemon systemctl restart netatalk
and backup started.
Have You met with this error?
TNX.
Daniel Lange on :
That usually happens when you update other network software on the Linux server and don't restart avahi afterwards. Avahi is quite fickle.
You could automate this by putting a file into /etc/apt/apt.conf.d/ with DPkg::Post-Invoke commands.
Alexander on :
Hm-m... Maybe it was happened after smbd reconfiguring. TNX!
Alexander on :
Hi Daniel! Something goes wrong again. I can't connect to volume TimeMachine from Mac.
After restart I can see only process /usr/sbin/netatalk started, there isn't /usr/sbin/afpd and /usr/sbin/cnid_metad.
Any ideas?
Daniel Lange on :
Well, it worked before and now it stopped working. So what you did between then and now makes the difference.
Check your logs
/var/log/{messages,syslog,daemon}
etc. orjournalctl
if you use systemd for clues. That should give you an indication of what is failing.Alexander on :
Wow... Something goes wrong, (maybe incorrect update of any package). Now netatalk is searching the afp.conf at /usr/local/etc instead of original /etc/netatalk.
Today I tried to rebulid netatalk from github, but got the same path to config.
Now OK, Time Machine is on. TNX!
Jose Mas on :
Thanks for the tuto. All went well but when I create more than 18 shares all of them dissapear. I can not have 30 clients for the Time Machine Server. Avahi daemon log says: "Record too large, cannot send" I have Time Machine server in Debian9 64bit Thanks for the help!!!
Daniel Lange on :
That is a common problem with Avahi and too many resources of one type in the announce packets. The packets then grow too large and Avahi can't split the MDNS properly. So it doesn't send announce of your time machine shares anymore and this is why you see them all disappear from Avahi announces once you grow too many.
You can still use them regardless, you just don't have the point-and-click convenience but need to use
tmutil setdestination -p afp://user@host/TimeMachine_for_Laura
or the like to get the backup share mounted.The most simple solution is to just use one (or two, or three) time machine volumes and share them among the users. You can encrypt the backups if keeping data secret between the users is an issue.
Alexander on :
Hi, Daniel!
3-4 years ago I made all according to Your tuto, great!
After upgrading to Big Sur I see very-very-very low speed of backing up: 100 Mb can be copied 10-20-30 minutes. Amazingly, the full backup (first) is copiing normally: about one hour for 70-80 Gb.
Unfortunately, the nevest netatalk for debian is from 12/08/20. Maybe something wrong in AppleTalk with the nevest Apple OS?
Can I fix this problem? Thanks.