Sunday, January 25, 2009

Memory mapped missunderstanding

I've become a stack overflow addict recently. Stack overflow is actually a really awesome programming community. It seems to be working very well when it comes to pushing the good and relevant answers to the top of the list so you don't need to spend time reading stupid, redundant or incorrect answers.
Regardless of that, reading programming forums is always stressing for me since sometimes people seem to be cooperatively clueless.

I recently had one of those experiences started by someone claiming one of my posts being stupid because I suggested using memory mapped files for large (4GB+) random file access. His motivation was that a memory mapped file that large would use all the address space of a 32 bit process. This is plain wrong, if anyone tells you that you need to map up the entire file in your address space, they don't know what they are talking about. You can choose how much of the file you want to map using one or multiple windows into the file. These are possible to move and manipulate in many cool ways. It's not for free, but it's definitely not a reason not to use memory mapped files!

Anyway I did some more search on postings on memory mapped files on the site and it seemed like people found this address space issue a reason not to use them!

The irony of the story is that the reason I started using memory mapped files in the first place was to prevent a problem caused by lack of (continous) address space in a 32 bit process :)
They are not only good for fast io, but also for throwing out large runtime data to disk and access it more like memory than files. 

No comments:

Post a Comment