APFS – A Backup Software Developer’s Perspective

When it was originally announced that Apple would be switching to a modern filesystem from HFS+ (based upon HFS which is more than 30 years old) my initial reaction was to expect roughly one thousand hours of work in order to get Mac Backup Guru working with it effectively. As in, basically totally rewriting it, since the first version of it took about a thousand hours itself. Getting basic backup software going is pretty trivial. Addressing all the nuances is harder than it sounds.

With that in mind I left it until the last minute and only started working on it a few days ago, which was only about a week before the public release of macOS 10.13, which will convert SSD’s to from HFS+ to APFS during the installation. My response was mostly due to the fact that the only official APFS documentation and code samples are just stubs, written back when APFS was in alpha. I was hoping that some updated information would be released. But it hasn’t, even now when the golden master candidates of the OS have been released. I’m still waiting for that despite the fact that, impressively, APFS has already been reverse engineered.

I also had a lot of reservations about switching up the filesystem so boldly. It is generally considered industry knowledge that it takes 10 years to properly develop a filesystem. It’s also generally accepted that any conversion from one filesystem to another will never be 100% safe.

Take a look at the tone (which, despite my own tone being different to, I simultaneously agree with) of a competitor’s backup software blog post to see what tends to run through an engineer’s mind when it relates to such a complex change.

When Apple changed the filesystem of all upgraded iOS devices from HFS+ to APFS with flawless execution I was impressed, but assumed it was due to iOS being a lot more restrictive about files in general. Since the OS does most of the file manipulation, and programmers interact with the filesystem through high level library calls, it seemed natural that converting the filesystem would be more standardized.

So, before installing 10.13, I started off by googling for any problems experienced by people installing and upgrading filesystems during the process using the public betas. To my surprise there were none. I then took the plunge and ran the installer – again with reservations. I know my filesystem is big and messy. Being a systems programmer it has had a lot of hard resets, and it has been a long time since I formatted and clean reinstalled. I assumed there were already some errors in the existing HFS+ filesystem. I made various backups using both Time Machine and my own Mac Backup Guru before I started. Now, several days later, and with extensive use (not forgetting that I’m a backup software developer), I’m quite stunned that APFS has been flawless both in transition and in general usage.

But more than that, its interoperability with HFS+ has been flawless too. In fact, for me coding wise, the transition from 10.11 -> 10.12 was more difficult than the transition from 10.12 -> 10.13 and that’s with a change of filesystem.

My software, Mac Backup Guru, uses rsync under the hood. Rsync is mature software with long release cycles, so it hasn’t been updated since 2015. It has the ability to to create hardlinks instead of copies of files if you provide it with the path to a previous backup when you get it to create a new backup. This has made it possible to provide incremental backups on HFS+ for the past few years. The most impressive part, to me, is this: Using the rsync binary compiled in 2015, I was able to create a backup from a bootable HFS+ disk to an APFS disk and then create an incremental snapshot of that backup on the APFS disk. Then, I was able to restore that incremental snapshot back on to an HFS+ disk, and boot from it. I thought there was no way that that was going to work out of the box. That is extreme attention to detail on APFS’s side. That would sometimes even fail when only using HFS+ disks, so APFS seems to implement HFS+ backwards compatibility better than HFS+ itself.

Rsync is also a fine piece of engineering. I have been involved with the low levels of the filesystem since 2002, when I wrote the engine what is still my bread and butter today, Mac Data Recovery Guru. After working on that and other projects it is precisely that which lead me to choose rsync rather than rolling my own file copier. It will adapt to any future changes and technology autonomously through community work.

Although too early to declare victory just yet, my anecdotal experiences combined with my research on the web does lead me to think that the mass transition from HFS+ to APFS is going to be another victory for Apple. This opinion differs from all others I have read from well informed people, many of who are predicting either catastrophe or at the very least some major stumbling blocks. In this case, time will tell, but I am bullish where most others are bearish. The only thing that bothers me is that I can’t understand how they managed to pull it off.