On Virtual Memory
Virtual memory was once an "overflow" space for main RAM. Back then, we had between 2MB and 16MB (depending on both platform and processor) and the MMU was maybe a seperate chip. I've got a 68851 knocking about somewhere.
An MMU is a bit of magic. It can remap addresses to absolutely anywhere. So if I have a patch of RAM at some forsaken longword hex address, I can use the MMU to throw that RAM out onto disk and, to the process which owns the RAM, not a thing has changed. But I now have some free RAM I can use.
That's swapping. You shouldn't really do it unless you want to do stuff that overflows your memory capacity. Bear in mind here that swapping came in on the desktop when the best CPU was about 33MHz. $200 might get you a 33MHz 68RC030 with built in MMU. RAM density was low and price was high. We were years away from getting 1MB for less than $1. Heck, we were barely getting 1MB for $40. Hard disks hadn't even hit 1MB/$1. CPUs were slow. To do a batch operation on ten 1024x768 images was a huge undertaking. To render a scene in Lightwave or 3DStudio was to leave your machine hard at work for days. Performance wasn't the issue. RAM capacity was. Swapping got us more RAM capacity and It Was Good. Sort of.
But swapping is wasteful, inefficient and tends to fragment memory (yes, this was a problem) as well as being very difficult to keep track of on a heavily multitasking system. So the OS had to defragment memory (pretty simple, just swap from one place in RAM to another) as well. The overhead built up pretty high. One of the reasons why Chicago was so damn unstable. When swapping, it's much better to have a stack of RAM and not have to swap at all.
The other option is paging. Rather than use the HD (often an entire partition or even an entire drive) as overflow space and stick chunks of RAM on it, you'd take that HD space and actually map it into the CPU's virtual addressing.
Remember how I said that the MMU can throw bits of RAM about? Well, this also applies to any address anywhere in the system as long as the CPU finds it addressable as memory. So we can move pages around rather than just chunks of RAM (a page may be a chunk of RAM, but it also may not be) to and from the HD. We can move those pages or assign them anywhere. Run a program and the PE (exe, dll, etc.) files it uses are actually in the CPU's address space under Win32. Physical RAM is in the CPU's address space. A portion of the HD (the page file) is in the CPU's address space. This is virtual addressing and how paging works. It isn't swapping. The OS can keep an eye on what page is where, how often it's used, what process it belongs to and move it to wherever's most appropriate. Heck, virtual memory can be allocated by a process without that memory existing anywhere at all. As soon as it's used it springs into existance. Unsure if NT does this or not, though.
Swapping either can't do these things or doesn't do them very well. Paging is not "overflow" RAM. You may well have several hundred MB of used page file with a gigabyte of free RAM. There are things in the page file which simply don't belong in RAM anyway. Things that should be addressible, but don't have to be in fast and at-a-premium main memory.
This is most of the reason why NT blows 95, 98 and ME clean out of the water when running applications. It just handles memory better by design. In order to do this, it has to have the full paging model. This includes a substantial page file.
Crapping on the page file just takes us back fifteen years and forces the OS to deal with only physical RAM. If you don't know or aren't sure, let the OS handle it. It knows what it's doing better than you. The default settings for 2k and XP are, like most other inner-working defaults, the best for 95% of everybody. The other 5% knows exactly what they're doing and are probably hacking the OS to bits anyway.
The Myth of Defragmentation
All file IO, unless it bypasses the file cache, is likewise done in chunks never larger than 192 kbytes and rarely larger than 64 kbytes. In between which chunks, your system is almost always moving the heads to access some other files. This is why it takes fairly unusual circumstances to see action performance impacts from fragmentation. And in fact, the more multitasking you're doing, the less effect fragmentation has.
It's easy to run tests (as all of the defragger vendors have done) that show otherwise, but these are invariably done with a single-threaded workload that's accessing just one file sequentially (that is, from end to end). That isn't how most of us use our machines any more.
A benefit you often DO get after a defrag run is that all the free space on the volume is contiguous, or much more so than it was before... resulting in all of the files being closer together than they were before. This cuts down on seek time among the parts of the various files. The fact that the files are contiguous isn't really what made things go faster, but it looks that way, and besides, it's a lot easier to explain and sell "defragmentation" than "file placement optimization".
So, the argument is that, except in extreme cases, fragmentation does not result in significantly more frequent seeks, nor significantly longer seeks.
The former can be measured by looking at the split IO rate as a fraction of the total IO rate to a given physical disk. If it doesn't get lower after defragmenting, defragmenting didn't help.
The latter can be measured by things like the IO request rate, the depth of the request queue to the disk, and the time per IO request (which is not just the inverse of the request rate), under high load periods. If these don't get better after defragmenting, defragmenting didn't help.
IMO defragmenting is overrated, and many users tend to think that defragmenting their drives will make their systems faster, when in reality only in extreme cases (and particularlt if you are running a server) this would be true. On other instances the difference will be so negligible, that the "myth" of defragmenting will become just a placebo effect. Cheers.
