A lesson learned I believe worth passing on

While looking for backup software last year I was pointed towards Genie Timeline (GTL). Tried the free version then bought it for both my PCs. It was just what I wanted and have suggested it to others. One friend tried it, liked it, and bought two copies for her home XP PC and her work Windows 7 laptop. She also bought two portable external drives.

In Disaster Recovery mode GTL worked fine on the XP machine but on the Windows 7 it never seemed to complete. It would seemingly archive everything, then find another 3,000 items. It processed these though not apparently archiving them, then find another 3,000, and so one. And so it never completed.

When I had a look I tried various things but couldn't see what was failing. It did look as if the 3,000 chunks of files were mostly, if not all, in the C:\Windows\Winsxs folder. I changed the backup type to non-DR and that worked fine.

I logged a ticket and went through various diagnostic sessions - difficult as I could only get the laptop occasionally. Eventually a remote access session was set up for Wednesday this week for Genie to investigate. After about 90 minutes the tech.support person was able to find out that GTL was getting an error when it tried to create a folder on the external drive and asked me to CHKDSK it and possibly to reformat it.

I then tried to manually create a folder on the drive and I too got an error. Googling this error I found this link http://help.lockergnome.com/windows2/file-folder-lim... which includes

In reference to help.lockergnome.com/windows2/file-folder-limits--ftopict450749.html:

A FAT32 directory can have 65,536 directory entries. Each file and subdirectory takes from two to thirteen entries, depending on the length of its name, so those entries can disappear long before you think you've used them all up. Your total of 22,657 files could very easily use 65,000 entries.

So, given the number of files and folders in C:\Windows\Winsxs and the length of the folder and file names, FAT-32 would almost certainly have a problem holding this folder which, on the Win 7 PC, has in excess of 15,000 folders and a total of over 60,000 files. This I suggest was probably the root cause of the problem. Winsxs seems to be new in Vista and Windows 7, XP doesn�t have the folder so doesn't get the problem.

I tried to Convert the drive to NTFS, it failed after 15 minutes or so saying it was dirty. I tried CHKDSK /F which reported no errors. I copied the data from it (3 hours or more) and then tried reformatting it, it failed after several hours with no explanation. After trying a few other things I went into Disk Management from My Computer / Manage. Did Delete Volume, then New Volume (or similar), then Format NTFS. That worked!

Copied the data back on - another 3+ hours. Run up Timeline and this time it successfully completely ran a Disaster Recovery Timeline session. Hooray!

So the original problem was caused by the drive being formated in FAT-32 format, as many portable drives still seem to be, and GTL not fully recognising the situation and aborting or blacklisting or whatever the Winsxs folder.

Thinking further about the inability to NTFS convert or format I suspect the drive had a hidden partition which I understand can cause that sort of problem.

As often the case in hindsight it's obvious what the problem was. Nevertheless I feel it worth passing this on as others may have an analogous problem with very large folders, especially Vista or Windows 7, and FAT-32 drives.

First thing I did was format my external drive to NTFS because of the file size limit in FAT32 of 2gb. My current backup solution Paragon Backup and Restore uses 4gb files to store images which can be changed but the recommendation was to use NTFS.

As did I, however the average person with a home computer would not know what a file system let along the pros and cons of the different ones. Why are manufacturers still formatting their new drives in FAT-32? Is it to do with MAC compatibility?

Paucity of choice I think. FAT32 may well be patent free with no fee required for its use.

Ah, I didn't know that. It still begs the question of how the average user is supposed to cope with such things.

I don't believe that you have to pay a patent fee to format a drive. I suspect that it is just down to using the most widely applicable format.

Just one small correction. The maximum file size on a FAT32 volume is 4GB - 1 byte.

I stand corrected, well sit.

The average user isn't. Writers of software for payment should be aware of the limitations though.

In reply to a post by cheshire_man:

So, given the number of files and folders in C:\Windows\Winsxs

Eventually WinSxS is going to eat up the whole hard drive. It grows with every Win Update. Yet there is no way of thinning it down on Vista or 7, unlike XP where you can generally safely get rid of the equivalent $UnInstall folders after a month or two.