How does SmartDefrag optimise the files?

[Tester1]

As the title says: How does SmartDefrag optimise the files? I know it defragments them, but what about the "Defrag & Optimise"?

 

In the past I've said I would like to see SmartDefrag perform a "boot time defrag" of the system files, registry hives, paging file(s) etc. which can't done whilst Windows is running. If SmartDefrag could do this then nobody would need to purchase a commercial defragmenter - anyone agree? If they incorporate this "boot time defrag" into the program, I think SmartDefrag should alert the user this needs doing first if it detects any of these files being fragmented, when you open SmartDefrag. I know with other products they recommend you perform the “boot time defrag” (offline defrag) BEFORE doing a “normal” defrag (online defrag), if that makes sense?

 

Windows XP users can use PageDefrag to perform a “boot time defrag” (offline defrag) before running SmartDefrag.

 

I hope this message helps with improving SmartDefrag, I'm really impressed with it! Please keep up the good work.

[detailer]

Thank you Tester 1-very valuable link-especially considering that my 30 day free trial of Perfect Disk is about to expire-great day and God bless!! :-D

[Tester1]

That's okay

 

Glad to be of help.

 

But nobody's answered my original question!!

[solbjerg]

Hi Tester

Have you read enoskype's string?

Please read through it

http://forums.iobit.com/showthread.php?t=612

Cheers

solbjerg

 

 

Glad to be of help.

 

But nobody's answered my original question!!

[Tester1]

You haven't answered my original question.

 

See title.

[samr]

Hi tester,

Read this from the above mentioned thread:

 

Enable Smart Defrag

Intelligently organizes drive data for maximum program performance and long-lasting data contiguity. Smart Defrag is an advanced and patent-suspended technology of IObit. Files are organized by their creation and modified dates. The theory is that files that have not changed recently are less likely to change in the future. These older files are grouped together so that once Smart Defrag have placed them, future defragmentation passes are less likely to move them again. Thus, future defragmentation will work much better, keep hard disk healthy as well.

Cheers.

__________________

http://forums.iobit.com/image.php?u=14&type=sigpic&dateline=1201733803 enoskype http://www.egem.com.tr/downloads/enoskype.gif

 

samr.

[enoskype]

Tester1, solbjerg was trying to guide you to where you can find the answer to your original question and visually see what SD does.!!!

 

Cheers.

[verdy_p]

How to defragment the USN Journal?

 

I've currently seen absolutely no solution (not even in commercial applications or in offline defragmenters, or in defragmenters running rom Linux) about how to defragment the USN Journal on the NTFS partition containing the Windows installation. The number of fragments is growing ever, and now this is the only fragmented part that I can't optimize (this is true on Windows XP as well as on Vista).

 

It seems that the USN Journal is ONLY handled by the NTFS filesystem driver, and this driver shuold handle internally a background task to recollect all those fragments.

 

I've tried to disable the System Snapshots, and to clean up them with CleanMgr (including after a Failsafe boot to minimize the number of running processes or services, or booting from a BartPE live CD containing Windows XP, or a Linux kernel) however this just collects some fragments, but never most of them. For this reason, now the USN journal has more that 4000 fragments spread all over the disk, and this is certainly explaining why my disks have become so slow.

 

Do I have to reformat completely the NTFS partition and then restore the files from a backup?

 

This hevy fragmentation came after an installation of Service Pack 2 (that had failed completely the first time I tried it due to a bug in the installer of one of its storage drivers, requiring a preinstallation of another driver from the PC manufacturer to pass over this bug): I had to restore the system using the Windows builtin recovery from an archive, and since this time, my PC is dramatically slow.

 

Note: I don't want to reinstall everything from scratch: I'll loose several licences. If someone can indicate me how to backup the installed partition or rebuild it from a backup without also restoring the fragmentation of the USN journal, I would be very pleased.

 

Is there a Microsoft article somewhre speaking how to troubleshoot the USN journal?

 

Final note: CHKDSK does not detect any error in the USN journal. And I don't use the NT Filesystem Replication Service (NTFRS): Should I enable it to perform the transfer of partition?

 

Microsoft's PageDefrag CANNOT defragment the USN journal. I've tried to play with the defragmentation API, and visibly, you cannot move its clusters around. The USN journal is normally allocated ONCE after the NTFS volume is created, the first time Journaling is enabled on it. It normally never grows after it, the area is fixed on disk, even if the journal is used as a cyclic file (that's why it is marked as a "sparse" file, however it should only contain two fragments at most, unless the total USN journal size is grown).

 

I've not found any API to truncate the used part of the USN journal, and you can't even move the unused alllocated part of the journal as long it is active (and there's no way to make it completely inactive, once it has been created on a volume, because the NTFS driver maintains it constantly open, even when Journaling gets disabled; this effectively puts a permanent exclusive system lock on it, including when booting in safe mode or when booting from another partition, and there's visibly no way to mount an inactive partition without also having the NTFS driver also opening this journal, even if it keeps it idle when journaling is disabled on this volume).

 

If you look at a detailed map of the volume, you'll see these areas as unmoveable (for example with JkDefrag, another free, but open-sourcedn defragmenter: they appear in black).

 

So I see only one way to defragment the USN journal:

- disabling the journaling

- rebooting from another partition, in a system that DOES NOT use the Windows NTFS driver (this means using Linux or some DOS extender)

- in this system, emulating the NTFS driver and managing the NTFS structure completely (without using the Windows defragmentation API).

 

Note that Microsoft's Systernal PageDefrag uses the Windows kernel, and its NTFS driver as well as the Windows defragmentation API, but can defragment the special files like Paging, because it does it at boot time, when those special files are not needed, and still closed. Due to its design, it cannot defragment the USN journal, that must still be active for the NT Defragmentation API to work reliably.

 

 

My suggested workaround:

- delete the USN journal from the NTFS volume: open a command prompt and run "fsutil usn deletejournal /D C:" <press ENTER>

- recreate the USN journal immediately: run "fsutil usn createjournal m=1024 a=128 C:" <press ENTER>

(change "C:" according to the drive letter of the NTFS volume where you need to defragment the USN journal)

Caveat: the filesystem replication will stop working and you'll need to restart the volume replication from scratch (incremental updates are lost, it will take time and lots of I/O to resynchronize completely the replication, if you use NTFRS on a server, and you'll need admin authorization on the server to reenable the replication); this is not a problem if you are running a standalone desktop environment, but don't do it on a server relying on replication.

After this operation, make sure you create a new System Snapshot, using "rstrui.exe" (in Vista, open the System control panel to do it, in XP, use the shortcut in the "System Tools" in the "Accessories" menu), then cleanup the old snapshopts that are no longer usable: run "CleanMgr" and in the advanced panel, click the button to keep only the last snapshot just created.

Before doing all that, make sure your system was fully functional, because you won't be able to revert your system to a past snapshot.

[verdy_p]

Hi tester,

Read this from the above mentioned thread:

samr.

 

What the article does not says is where SmartDefrag takes the dates of creation/modification. apparently it takes it by looking at the file attributes in directories, however this method is not reliable, and maintaining those dates within directories is dramatically slow.

 

The NTFS filesystem has another reliable (and much faster) way to track the dates of change, as demonstrated in an MSDN article: you can use the "last USN" field that is present in ALL entries of the MFT.

 

Every file or directory on NT has an associated entry in the MFT, except when they are "resident", i.e. when they are stored in the MFT entry of their parent container: this occurs when a stream or attribute is small enough to fit within the 1KB record in the MFT describing a file or directory.

 

Normally, for all files, the NFS filename attribute is always resident, as well as the DOS 8.3 name, and (most of the time) the fragments location map to its content (except when the file is too fragmented: the file location map may require its own MFT record to store the whole list of fragments), and most named streams (such as the named stream that is added on files that were downloaded from the Internet to mark that they may be unsafe: these streams have only a few bytes of content when they are present), as well as the "standard" file attributes (compatible with DOS).

 

The file content may also be resident (not allocated separately) if it can fit in the file's MFT record, as well as the directory entries of a directory that only contains very few files (if more files get added to the directory, the resident content will be moved out of the MFT record to an external file).

 

But in all cases: all MFT records contain the value of the last USN assigned specifically to a version of a file; when a file or directory changes, and if journaling is enabled, the last USN entry gets updated to track the change. (Microsoft indicates that the last USN contains a timestamp, this may change in some future to use another method for linking a version to a file and timestamp, however this is still a usable timestamp in NTFS 5.0; apparently this has still not changed in Windows 7 Beta, and it is very unlikely that it will change for the next 6 years with Windows 8 or a major W7 Service Pack changing radically the way NTFS works...).

 

 

Final note: It seems that the installation of MSN Live 8 and of other Microsoft "Live" products enables the USN journal by default, however it does not specify appropriate parameters to maintain it in a stable condition: the USN journal can grow dramatically. However, it has absolutely NO use unless your system is connected with a server that archives a live replication of your filesystem (for its restoration). On a standalone PC, or if you don't have a Windows server with FRS enabled you should disable this unneeded USN journal or just create it with a minimum size (that will still allow system snapshots to work reliably). If your USN journal is too large, you'll immediately see that your volume gets fragmented at lightning speed and that you need much more frequent defragmentations to maintain its performance!

 

On a notebook, I really suggest you disable the USN journal completely, just delete it with this simple command from a command line window:

"fsutil usn deletejournal /D C:" <without the quotes, then press ENTER>

 

repeat this command for the other NTFS partitions you have outside C:

[Dalmation]

Defragment Algorithms.

 

I understand that Smart Defrag uses the file creation/modification date properties in it's algorithm.

 

But what about the last access property; that is primary when determining frequent used files.

 

In my case I have optimised my system by setting the NTFS last access property to disables.

 

Raxco Perfectdisk and Disckeeper Professional give several defragment algorithms to choose.

Does it give the option to defragment simply by folder / file name order?

 

Michael.

[enoskype]

Hi Michael,

 

SD doesn't have the option of defragmentation by folder/file name order.

 

I agree with you about the frequently used files' definition of using last access time.

 

Yes, AWC2 and ASC3 gives the option of disabling NTFS last access in System Optimization, (I agree that it takes some of the PC time) but since some of the other programs use this last access time, personally, I use the option of keeping that optimization in Ignore List.

 

Cheers.

[enoskype]

Verdy p ,

 

Very informative posts thank you. I will copy them to Think about Defragmentation thread in Lounge section.

 

FYI, Perfect Disk8 does defragment USN Journal on NTFS partition with Offline defrag when Master File Table, Metadata, Hibernation file and Paging File checkboxes are checked in Drive Properties. I have done that many times. (XP Pro sp3) (Never allowed fragmentation of it to grow to 4000 fragments though.)

 

Please find the related images attached.

 

-54 fragments of USN Journal are highlighted by Ultimate Defrag.

-An offline defrag is performed by Perfect Disk8 when there were 54 fragments (53 excess+1) of USN journal.

-Contiguous USN Journal (~10.51 MB) is highligted by Ultimate Defrag just after Offline defrag by Perfect Disk.

-Perfect Disk Report is shown with 0 (ZERO) fragmentation of the System files after offline defrag.

 

I hope this helps.

Cheers.