A complete overview and test of WM5 backup solutions

However much one of the reasons, depleted batteries, for your programs and data being deleted are no longer a concern under WM5, you may still need backup applications for the new operating system version, particularly if you often install new programs and/or use misbehaving applications that may irrecoverably crash your system, resulting in a need for a hard reset. (Yes, there're several programs like that; I've already run into some. I'll publish an article about them a bit later when I have scrutinized these all problems.)

Right now, there're three third-party applications that support backing up WM5 devices:

1. industry-leader Sprite Backup (still in beta; this also means betas are/have been a free download). Its final version will be released on the 7th of February and will be priced $29.95. Please read this thread for more information on the upgrade/lincensing conditions.
2. a "newcomer" in backup solutions (except for the industry-targeted and accordingly priced ($299) Spb Clone, which isn't, therefore, really suited for home use), Spb Backup (no official homepage as yet; it's, to my knowledge, only on MobileTechReview.com that the application has been mentioned, as far as frontpage news are concerned) and
3. SunnySoft Backup Manager (SBM for short; current, tested version: 3.7 build 01041).

In addition, OEM's may also deliver backup programs on WM5 PDA's. This is not a hard-and-fast rule: much as Dell does deliver one with the Dell Axim x51 series, built into the ROM, other hardware manufacturers don't necessarily follow suit. For example, HTC doesn't supply the old, well-known and, for example with my backup extractor utilities, well-supported Xbackup.

Please note that ActiveSync doesn't support backing up WM5 devices. Furthermore, not even the latest (2.3.24, as of the time of writing) version of SKTools is able to backup WM5 devices, unlike previous OS versions.

I've scrutinized the three commercial solutions (along with the built-in Dell one) to find out how well and reliably they work. Please note that two (Sprite and Spb Backup) of these applications are still in beta/Release Candidate stage; this means the final versions may be much different from their current state. Still, they are already worth reviewing because if you need a backup solution now, you may end up having to use a beta-stage product, particularly if your PDA lacks any kind of built-in backup application.

As, according to most user complaints, it's mostly Personal Information Manager (PIM) databases and, most importantly, (for example Short Message Service (SMS)) messages in WM5 that aren't backed up/restored by third-party tools, I've specifically set up my two WM5 devices, my Dell Axim x51v (English A06 ROM) and HTC Wizard (Qtek 9100, k-Jam, MDA Vario etc.) to have some e-mail (and, with the WM5 Phone Edition HTC Wizard, SMS) messages.

Additionally, I've paid special attention to checking the backup/restoration of files under the \Windows subdirectory and the registry under HKEY_LOCAL_MACHINE. These two areas in the file system/Registry are different from pre-WM5 OS versions in that they are a bit more protected: for example, with PC-based (that is, external) regsitry editors like MRE (see this registry editor roundup for more information on MRE), it's impossible to modify anything under HKEY_LOCAL_MACHINE (an example screenshot of this is here), or, with Total Commander and the WinCE FS plug-in running on the PC, it's no longer possible to "shadow" files that are in ROM and it's impossible to delete preinstalled link files from, say, \Windows\Startup.

The results

1. The current beta of Sprite Backup worked flawlessly on both of my WM5 Pocket PC's. It restored everything OK. I've never run into any problem.

   It's not only very fast, but also versatile (you can fine-tune what you want to backup/restore). The basic interface is really simple, you only need to click one of two buttons (to backup/restore), as can be seen in this screenshot. For advanced users, on the other hand, it offers excellent for example backup separation, as can be seen in this screenshot. Also, it's highly customizable.

   Of course, you will need to pay for this because, for example, all backup and restore operations are started/ended with a soft reset, which can be pretty time-consuming on devices like the HTC Wizard (it boots in in 70 seconds; fortunately, the x51v is much faster at booting in). Furthermore, it can't create executable files either, unlike SBM or Spb Backup. If you, however, keep the installation CAB file of Sprite Backup along with your backup files on your (backup) storage card, this problem won't be acute any more. (A word of caution: Please note that you should make the install CAB file read-only on some WM5 devices in order to avoid their being deleted after install. This applies to the HTC Wizard but not to the Axim x51v. This way, you can avoid having to make the entire card read-only (if it's a, say, SD card with a read-only switch) or moving/copying the CAB file around after every restore, which is prone to be forgotten, causing real problems if you can't access the Net to download it again.)

2. The current Release Candidate 1 version (build 1134; released at Jan 20, 2006) of Spb Backup is a bit less capable than Sprite Backup: there are not any kind of scheduling or remote backup capabilities. Much as its custom backup/restore interface allows for fine-tuning – in the category setting window, you can furher fine-tune the individual files to be saved/restored under \My Documents (in the My Documents customization dialog) and the entire file system (in the System Data customization dialog), unlike with Sprite or SMB, you can't individually backup/restore Windows CE databases. This is not a particularly acute problem, however – mostly, it's individual files that you may want to choose for restoration, not WinCE databases.

   It has a big advantage over Sprite backup (but not over SBM): it defaults to creating executable (.EXE) files; that is, you won't need to install the bacup application itself to be able to restore your backups.

   Unfortunately, as far as my thorough restoration tests are concerned, the application didn't fare particularly well. First, on the HTC Wizard, it was decidedly (about two times) slower than the current beta of Sprite Backup. (There weren't any speed difference on the x51v, though.)

   Second, on the x51v, it didn't restore the Hotmail mail body messages. Also, some subtleties (for example, the system-wide ClearType flag) weren't correctly backed up/restored either. However, not counting in the Hotmail problem, the restored Pocket PC worked pretty OK – much better after restoring the SBM or the Spb backup.

   Finally, on the HTC Wizard (the stumbling block for SBM too), it produced pretty bad results, at least as far as Messaging databases are concerned. It not only refused to restore any SMS messages (not only their body but also their headers!), but also deleted all my custom mail accounts, including the Hotmail account. It seems its Messaging database handling should be still worked on.

   Incidentally, during restoration on the HTC Wizard, it has asked permission for two CPL's (bkpmgr.cpl and DeviceConfig.cpl), unlike on the x51v and, at the end of restoration, it also displayed an error report. (In the error log, however, it only complained of about not having been able to create the file \restore_log.txt). The reset button doesn't work either on the HTC Wizard, unlike on the x51v.

3. The current version of the SunnySoft Backup Manager, on the other hand, seems to be pretty buggy. It has the following problems:
   • On the HTC Wizard, it wasn't ever able to restore anything. I've played a lot with the settings, including with trying to backup to/restore from the built-in flash ROM (\My Documents) instead of the memory card, switching off/on the compression, making EXE backups; all in vain. Of course, I've always tried to restore to an absolutely clean HTC Wizard (version data: IPL 1.01, SPL 1.06, GSM 01.01.10, OS 1.1.9.3). Upon starting to restore, I've always been greeted with the "Invalid file signature" error message, as can be seen in this screenshot.
   • On the Dell Axim x51v, it did work, but it royally messed up the Messaging databases. Upon clicking "restored" mails, I've always been presented a gwes.exe crash, as can be seen in this error message. This happened in all configuration I've tested, after every restoration.
   • Finally, the application executable file itself likes freezing / stucking. This screenshot is an example of that.

Verdict

The current beta of Sprite Backup seem to work flawlessly under WM5. If your Pocket PC doesn't have a built-in backup utility or you need something much better configurable (schedulable or with the ability to fine-tune exactly what to backup/restore) and/or much faster, give Sprite a try.

On the other hand, avoid the current (!) version of SunnySoft Backup Manager and Spb Backup. Hope later versions will be more bug-free and feature-rich.

Disclaimer

During writing this test, I've thoroughly tested these applications on two WM5 devices. I'm absolutely sure everything I've said is true - as far as the two Pocket PC models and their ROM version are concerned. Needless to say, I've run the tests on freshly hard reset Pocket PC's (too) to be absolutely sure there isn't any software incompatibility with third-party applications.

However, if you have other models / other ROM versions, you may find that, for example, the Sunnysoft manager or Spb Backup do work as expected. I can only recommend that you also make a test backup/restore to make sure these applications do work on your particular WM5 Pocket PC, preferably with some SMS messages (if it's a Phone Edition device) and/or standard E-mail in Messaging, before choosing a backup application.

Again, two of the tested applications are in beta stage. This means they may still change before they indeed get released. This is particularly topical with Spb Backup, which still has problems with WM5 devices.

Detailed comparison chart

Finally, a comparison chart – again, with a lot of comparable benchmark results and screenshots. Note that I've also tested and benchmarked the built-in Dell Axim x51v backup application so that you see in what ways third-party tools are better than OEM-shipped ones. (Note that it flawlessly restored everything and had no problems with anything.)

Backup app:x51v Built-inSpb Backup RC1Sprite Beta (latest as of February)Sunnysoft 3.7 (another screenshot here)
General overview, strictly for WM5 usersIf you don't need separate/ scheduled/ remote backup/ restore and you have an x51v, stick with this applicationHope it will be enhanced (for example, scheduling) and bugfixed (PIM data and generic reliability under WM5)Seems to be excellent and reliable even in beta. No problems encountered.I don't recommend it – if it runs at all, you will have problems restoring PIM data/mail.
Backup time (x51v / HTC Wizard) of a ~50M/~10M set1:49 / n/a~1:00 / 2:321:55 (compressed) / 0:55 (+2*1 minutes for the reboots)4:40 (compressed) / 2:10
Restore time (x51v / HTC Wizard)4:32 / n/a~4:00 / 2:174:08 / 0:503:40 (compressed) / unable to restore anything
Install CAB sizen/a802k3M; signed260k (WM5; it's said to be signed but it isn't in reality)
Generate executable files?-+ (default)++
Does is allow for compressing backup files? Backup size of a ~50M/~10M backup set? (x51v / HTC Wizard)-; 50.9M / no built-in backup+ (default)+; 29.5M / 7.1M+; 27.7M / 5.9M
PC client?--+ +
Any way to extract/restore files/WinCE DB's/registry entries on PC?No known toolNo known toolNo known tool. The file format is relatively simple though and comparatively easy to be restored with third-party tools (if someone writes one, that is).No known tool
File/registry/ WinCE database-level separation during backup/restore?You can backup either PIM data only or everything; no fine-tuning is possible+ (albeit unable to separately backup/ restore WinCE databases, unlike the other two commercial apps)++ / +
Scheduling on PDA?--+ ? it's greyed out
Network backup to an agent listening on a desktop computer?--+-

Finally, I'd like to thank Coeptum for pointing me to the source for the RC version of Spb Backup.

EDIT (9:31AM CET, Feb 2 2006): article linked from SmartMobileAssets frontpage.

EDIT (21:51 CET, Feb 2 2006): the new beta of Spb Backup has just been released; I'm testing it. Will publish the results in a few hours.

EDIT (6:46AM CET, Feb 3 2006): Please check out this blog entry for the review of the latest Spb Backup.