Chunk Size
Contents
Chunk Size in Contents Comparison
When comparing files using the Contents comparison method, FolderMatch processes data in chunks. The program reads a chunk of data from each file and compares these chunks. This continues until the program reaches the end of the files or finds a difference. The Chunk Size option allows you to specify the number of bytes in each chunk, ranging from one byte to ten million bytes. The default chunk size is 51,200 bytes.
Why is this necessary? Why should you care about Chunk Size?
One word: PERFORMANCE
To understand how chunk size affects performance, let's delve into the comparison process:
1.Reading Chunks from Disk
oThe program reads a chunk of data from each file and transfers it into memory.
2.Comparing Chunks in Memory
oOnce both chunks are in memory, the program compares them.
Performance Considerations
•Disk Read Speed
oThe speed at which data is read from the disk depends on the disk type and drive speed.
oLocal disks generally read data faster than external disks. For external disks, the speed of the connection also plays a role.
•Data Transfer Rate
oThe rate at which data is transferred to memory is influenced by whether the disk is local or external to the PC running FolderMatch.
•Memory Usage
oLarger chunk sizes take longer to compare. If the memory required exceeds available RAM, the operating system uses hard disk memory, further slowing down the process.
Choosing the Right Chunk Size
The simple rule of thumb is to avoid chunk sizes that overwhelm any part of this process. A chunk size that is too large will slow down the comparison.
Why is the Chunk Size Option Valuable?
There is no one-size-fits-all for optimal chunk size. The ideal chunk size varies based on your system configuration and the specific task. The best way to determine the optimal chunk size is through experimentation.
Setting Chunk Size
The chunk size can be changed on the Options window, under the Compare Folders tab. Adjust this setting based on your performance observations to find the optimal size for your needs.
Optimization Note: For optimal performance, choose a chunk size that is a multiple of 512 bytes due to disk geometry principles.
See Also: