Hello I have a 320 gb SATA 2 [wd3200aaks] on P4 2.66 with Mercury P4VM800M7 mobo
WHen I copy a movie in windows xp fat32 to fat 32, a programme called burst copy it shows speed of about 58 mbps [one partition to another]
when I copy same movie in pclinuxos 2007 (or debian4.0) i get speed of roughly 35 mbps, [XFS one partition to another XFS partition] why such a great difference in performance? and how can I speed up my harddisk under linux?
I have 2 gb ram
2 gb swap on same sata
no other harddisks
I have tested this number of times (and dont run any other software in between the copy process just to be sure of speed on both OS)

P.S does chosing syncronouse writting [all i/o/ synchronously] in XFS speed up the performance ? (because I think I had this feature unchecked (also if i check it will i get any performance penelty? and what its exact purpose btw, googled but didnt get something clearly )

If you are using Konqueror or Midnight Commander to measure the writing throughput, the bandwidth will worsen. I suggest use time with cp or mv to clock how fast or how long it will take. Then divide the type to the amount of bytes of the file. Any file system will have a bandwidth penalty when moving or copying data from one partition and then to the next partition. I suggest do the test with two hard drives on a different IDE channel or even better on a separate storage controller of the same model and brand.

On my notebook computer I get increase write and read performance using XFS compared to NTFS. Thanks to XFS buffering, I get more than 5 MB/sec compared to NTFS. My notebook computer has a Intel T7300 with 2 GB of DDR2.

My other computers also uses XFS and their throughput is high too. Again copying or moving data from partition to another partition cost performance.

I just play around with the format options.

The option async is faster than sync. The option sync will be slow because it tells the kernel and hard drive to flush cache to the medium. The async option will only flush the cache when the data has reached to a certain limit.

Try 'hdparm -Tt' to see what speed you get there.

Also look at 'hdparm -I' to see if something is fishy with the parameters for the drive.

hdparm -Tt :-

/dev/sda:
Timing cached reads: 826 MB in 2.00 seconds = 412.09 MB/sec
Timing buffered disk reads: 230 MB in 3.03 seconds = 76.00 MB/sec

--------------------------------------------------------------------
hdparm -I

/dev/sda:

ATA device, with non-removable media
Model Number: WDC WD3200AAKS-00VYA0
Serial Number: WD-WCARW1935518
Firmware Revision: 12.01B02
Transport: Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5
Standards:
Supported: 8 7 6 5
Likely used: 8
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors: 625142448
device size with M = 1024*1024: 305245 MBytes
device size with M = 1000*1000: 320072 MBytes (320 GB)
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, with device specific minimum
R/W multiple sector transfer: Max = 16 Current = 16
Recommended acoustic management value: 128, current value: 254
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
Power-Up In Standby feature set
* SET_FEATURES required to spinup after power up
SET_MAX security extension
Automatic Acoustic Management feature set
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
* General Purpose Logging feature set
* 64-bit World wide name
* Segmented DOWNLOAD_MICROCODE
* SATA-I signaling speed (1.5Gb/s)
* SATA-II signaling speed (3.0Gb/s)
* Native Command Queueing (NCQ)
* Host-initiated interface power management
* Phy event counters
DMA Setup Auto-Activate optimization
* Software settings preservation
* SMART Command Transport (SCT) feature set
* SCT Long Sector Access (AC1)
* SCT LBA Segment Access (AC2)
* SCT Error Recovery Control (AC3)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
unknown 206[12]
unknown 206[13]
Security:
Master password revision code = 65534
supported
not enabled
not locked
not frozen
not expired: security count
supported: enhanced erase
84min for SECURITY ERASE UNIT. 84min for ENHANCED SECURITY ERASE UNIT.
Checksum: correct

I tried using time, i get roughly same speed.
I am aware that If i copy data from one partition to another on same harddisk i face some performance penelty however I am tellling from my experience that in windows I was getting about 60 m/s from One partition partition to another on same harddisk ! how could it be while on linux i am getting virtuallly half of it

76 mb/s doesn't look too bad - I'd look into the acoustic management and the xfs options and into this.

Pls. don't get started on file systems and benchmarks there are enough threads flaming away already about it - just interesting benchmarks.

I am aware of performance penelties, but difference b/w speed of copying in windows (both partitions on same drive) is far higher (double) than xfs

hdparm -M /dev/sda

/dev/sda:
acoustic = 254 (128=quiet ... 254=fast)

i cant find any such helpful option for xfs

In copying between partitions on the same drive, the two big factors are:

1) How large are the chunks in which copying actually occurs. The OS may be buffering or caching to make the actual chunks larger than the chunks the copy program uses. In Linux, the dd program is available, which can copy in whatever chunk size you specify (subject to available memory).

2) How far apart are the partitions physically. Seek time is not linear with distance, but it does go up with distance. If I understood you correctly, four different partitions are involved: two for Windows and two for Linux. Maybe fdisk -l output would help us understand that aspect.

Sorry - got that all wrong.Thats what happens if you don't read the original post well.

First of all burstcopy uses RAM as buffer - wether that is a big factor depends on a lot of things.

Copy of oblivion.cab (1gb) ntfs->ntfs 42 sec with burstcopy
ntfs-> jfs 22 sec with Linux

In both cases that went from on disc to another with defragged M$ drives.

If you use partitions on the same disk or the drive is fragmented as hell I see how the buffer could speed up things a lot because it doesn't jump back and forth that much with an additional 64 mb or so buffer from ram.

Also disks are a lot slower the further the partition is to the inside of the platter because the relative speed is slower.

Sorry. I never heard of "burst copy" before reading the first post of this thread. I assume from its name it tries to do something intelligent regarding the buffer size. I have no idea what. But:

Any program copying from disk to disk uses ram as a buffer. How much ram it uses is usually a big factor.

But when going from one disk to another, the amount of ram isn't as big a factor as it would be from one partition to another on the same disk.

From one disk to another, it matters a lot whether something (OS or program) enables the overlap to start reading a second chunk before you're done reading the first. Given that overlap, even with unlimited ram, you wouldn't want the buffer to be a large fraction of the size of a large file.

With one disk and two partitions, you can't really start reading the second chunk before you're done writing the first. So with unlimited RAM, the fastest copy method would buffer the entire file in one chunk.

When copying a big file between partitions on one disk on a system with 2gig of ram, I hope the software is smart enough to use a lot more than 64mb for buffering. (I don't know it is smart enough. I just hope).

I didn't know that. Do you happen to have a URL for hard facts on that effect? (I almost never have a URL for the facts I mention in threads like this, but if you happen to, I'd appreciate it).

Long ago, drives used lower physical density on outer tracks so the temporal density wouldn't change very much. Obviously, electronics have improved faster than disk surface quality has improved, so consistent physical density now should be more important than consistent temporal density. So the effect you describe should exist. I just wonder how big it is.

I didn't know that. Do you happen to have a URL for hard facts on that effect? (I almost never have a URL for the facts I mention in threads like this, but if you happen to, I'd appreciate it).

Have a look here

That any program uses RAM as a disk I/O buffer is new to me.Thats what they put buffers on the disks for.

Softwarebuffering generally isn't needed except with bad dvd burners that run out of data to burn and produce coasters.

What gets read ahead and back and forth and so on depends to large part on the interface (scsi,sata,whatever).

The density is the same all over the platter - htat it gets slower is just simple physics.
You got always the same rpm's - on the outside it is faster than on the inside.It is a disk after all.

Disk /dev/sda: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 685 5502231 b W95 FAT32
/dev/sda2 686 33168 260919697+ 5 Extended
/dev/sda5 686 3279 20836273+ b W95 FAT32
/dev/sda6 3280 5876 20860371 b W95 FAT32
/dev/sda7 5877 12268 51343708+ b W95 FAT32
/dev/sda8 12269 12536 2152678+ 82 Linux swap / Solaris
/dev/sda9 12537 14392 14908288+ 83 Linux
/dev/sda10 14393 15501 8908011 83 Linux
/dev/sda11 15502 15757 2056288+ 83 Linux
/dev/sda12 15758 33168 139853826 83 Linux


For windows I was copying from SDA1 to SDA7
For linux I was copying from SDA11 to SDA12

oh yeah and
hdparm -M /dev/sda

/dev/sda:
acoustic = 254 (128=quiet ... 254=fast)

in these tests , the pci ide bus mastering was enabled from bios.
I dual boot to windows/linux using GRUB (linux is pclinuxos 2007)

I have just noticed that If is Disable pci bus mastering it takes grub like 3 seconds FASTER to load in constast to when pci bus mastering is Enabled.Though the copying speed difference wasent noticeable in linux (wether bus mastering is enabled or disabled)
I thought it might help telling you this

hdparm -Q /dev/sda
-Q: bad/missing queue-depth value (0..32)


and what does this mean? why isint it showing the DMA status for the device?