Everything you need to know about flushing the Registry and WindowsCE databases under Windows Mobile 5

I’m constantly asked about why I keep telling users to power off (suspend; “full” power off is an overkill and is not needed!) their Pocket PC’s after making some registry changes and before resetting their Pocket PC’s. As the answer also contains some really interesting stuff highly useful particularly to WM5 users that have a non-natively WM5 Pocket PC (the Dell Axim x50(v), the hx4700 or the WM2003SE-based HP iPAQ hx2xxx series), I devote an entire article to it.

(Executive) Summary

If you have a non-native WM5 device with WM5, you will really want to read at least the second part of this article so that you can make the “filesys” burden (the worst thing on all non-native WM5 Pocket PC’s) lesser.

If you’re a Pocket PC guru/hacker and (also) interested in how WM5 works, the first part of the article will be of definite interest to you too. Note that understanding it is not needed for casual WM5 users to implement the changes in the second part!

How does the Registry/WinCE databases work under WM5?

First, as you may know, everything is stored in the slow-to-read-and-even-slower-to-write Flash ROM under WM5 so that your settings, data and programs can survive a complete main battery depletion (or, which is equivalent to it, the full power off mode supported by almost all WM5 Pocket PC’s).

As access (both for reading, and, as we will soon see, writing) to the Registry and the WindowsCE databases are far more frequent than to the plain files in the file system (it’s possible the system doesn’t make any file writing operations for hours), the contents of both of these must be copied to the much faster dynamic RAM. Writing (modification) operations will, therefore, also take place in the RAM.

As we all know, the contents in the RAM are lost on when the battery is depleted or the device is completely shut down by the user (not only suspended) and, with the Registry, the changes are also lost, there must be an automatic way of writing (with the Flash ROM terminology, “flushing”) back the changes from the RAM to the storage Flash ROM so that they are permanent and also visible after a simple soft reset so common with Pocket PC’s (especially when you actively modify the Registry to tweak the device). Furthermore, it’s only at suspending the Pocket PC that it is flushed but not on resetting it.

This means Pocket PC manufacturers, along with Microsoft, must have found a way to periodically flush the contents of both the Registry and the WinCE databases to the storage ROM when there are changes. They, after all, can’t just tell users ‘make sure you suspend your device so that none of your changes are lost’.

The flushing algorithm of WM5

The flushing model Microsoft (MS for short) uses is as follows. MS has defined two registry keys under HKLM, System\ObjectStore\ for the Registry and the WinCE databases.

The two subkeys are common in that they have an ActivityThreshold value (and an ActivityName, which can be used to monitor the flushing from an external process, and, therefore, will be of no interest to us). They are as follows (first comes the Registry- and, then, the WinCE DB-related subkey):

[HKEY_LOCAL_MACHINE\System\ ObjectStore\RegFlush]
"ActivityThreshold"= dword:00000001
"ActivityName"= "$*@RegChanged#%&"

[HKEY_LOCAL_MACHINE\System\ ObjectStore\DBFlush]
"ActivityName"= "$*@DBChanged#%&"
"ActivityThreshold"= dword:00000001

ActivityThreshold in all of these both registry keys tells the system after how many registry/DB changes the changes should be written back to the system – at least this is how it should work and how the document linked at the bottom says them to work. In reality, however, as the write-back doesn’t happen immediately, I think it’s periodically done. This is where another possible registry value, FlushPeriod, comes into picture. (I think there’s a default value for this – some minutes – that WM5 implicitly uses when this value is not supplied by either the user or the programmer manually creating/importing this registry value.)

FlushPeriod, which can be optionally supplied but is present in the WM5 hx4700 registry (unlike on the HTC Wizard and the Dell Axim x51v) by default, tells the system how often it should check for registry/DB changes and accordingly flush it back to the storage Flash ROM when necessary. That is, it won’t flush when it’s not needed. (It’s only by explicitly creating a RegistryFlags value with the value ‘1‘ that flushing will inevitably take place even when there’re no changes, as far as the Registry is concerned. With the WinCE databases, it may be named DBFlags, but I’m not sure.)

It contains the flush period in milliseconds. For example, the dword:0000EA60 (60000) default value in hx4700 specifies a flushing every minute (60 seconds * 1000 milliseconds = 60000).

Note that, while the article linked below defines (“ActivityThreshold specifies the # of reg activity before we force a flush; [FlushPeriod:] Timeout period for a flush (flush occurs if there have been some changes during this period)... These settings cause the registry to flush when there are 256 changes or any changes within a second (which is probably too often.)”) an either-or relationship between these two registry values, I don’t think that, based on my experience, this is the case.

That is, in my opinion, the following relation is true:

“check for registry/WinCE DB changes periodically defined by the value FlushPeriod (if there is no such value defined, assume a some-minutes cycle), if the changes made exceed the value of ActivityThreshold, flush these changes to ROM”.

This explanation also makes clear why there are no immediate registry flushes – as opposed to what the article states.

Why is this all important? - you may ask. It’s very simple.

If you have a non-native WM5 device with constant filesys problems, either increasing the flush time period or increasing the threshold will result in less ROM flushing operations and, therefore, increased speed / less filesys overhead.

How often the Registry/WinCE DB are accessed for write, you may ask. WM5 constantly, every minute (but, fortunately, not more often) modifies the Registry even without any human interaction, updating [HKEY_LOCAL_MACHINE\ System\State\ DateTime\Time] to reflect the current clock time. This means that, if you stick to the original registry values, a flush operation will take place every (with the hx4700, where FlushPeriod is defined and set to one minute) minute. This is not necessarily what you want, especially not just because WM5 always wants to flush the registry back to ROM just because of the updated clock. Note that, unless you use ActiveSync or an application that writes to the Registry (or installing/ uninstalling programs), registry writing operations are very rare. Even more rare are WinCE DB writing operations – it’s being written to only when you, say, crate or modify an appointment, a task, a contact or work with Messaging – receiving, creating, answering etc. mail.

What should you do?

Just increase the value of HKEY_LOCAL_MACHINE\ System\ObjectStore\ RegFlush\ ActivityThreshold and HKEY_LOCAL_MACHINE\ System\ObjectStore\DBFlush\ ActivityThreshold from the default 1 to, say, 16. Then, on the hx4700, this will mean that, by default, the Registry will be flushed back to the ROM every 16 minutes (instead of the default 1 minute – again, because of the always-increasing internal clock.) Also, extensive WinCE database change operations (mass-importing contacts or downloading mail) will also result in far less ROM flushing than before.

Usability/the effects of this hack

Note that the effects of this hack aren’t as visible as, say, setting up a RAM cache or moving every cache items / filesys.exe volatile directory ([HKEY_LOCAL_MACHINE\ System\FileSys\ TempPath]) and, most importantly, shared DLL’s out of the main storage, deleting all the help, uninstall files from there and not letting the system install anything in there. That is, doing this hack is not so important as doing the above-mentioned file system-based relocation/deletion hacks. On the long run, however, this may also pay off.

(Speaking of .unloadfiles: under WM5, unlike with pre-WM5 operating systems where there were .unload files in \Windows, you can safely relocate the contents of \Windows\AppMgr. And, again, you can safely delete all *.html, *.jpg, *.bmp, *.2bp and *.gif files deletable (that is, installed by third-party apps) in \Windows. These are help files and their in-line images copied by your own programs to \Windows. You won’t need them. It’s only with very few apps that doing this may result in problems. You can find out more about the pre-WM5 .unload question in the article Fine-tune your memory consumption - relocating uninstall/update-time stuff (alternates: iPAQ HQ, AximSite, PPC Magazine, FirstLoox, PPCT).

Recommended links

An article (only for real gurus / programmers!) that discusses how you can force registry flushing to the storage under WM5.

Acknowledgements

Thanks goes to PDAMania forum member pdafun, who, by pointing out ActivityThreshold's being one of the factors in the storage flushing speed and frequency, made me further dive into this question.

UPDATE (01/22/2007): for example in here, I've elaborated on how you can make sure the registry / WinCE database changes on Pocket PC Phone Edition devices can be flushed back to flash ROM.