Re: mlock() issues

[Matthew Mondor <mm_lists%pulsar-zone.net@localhost>]

On Fri, 22 Oct 2010 12:06:37 +0100
Sad Clouds <cryintothebluesky%googlemail.com@localhost> wrote:

> Well if you're allocating memory yourself, then you've just created your
> own application cache.

Say many files were mapped in the process's address space, the OS would
still be responsible of keeping frequently used ones pages active,
possibly swapping out long-unused ones, unless of course MAP_WIRED was
used.  A syscall per access would be eliminated however, i.e. read(2),
and I think that zero-copy could be used (with page loaning) if writing
64KB blocks out to a socket from a memory-mapped file.

> On the other hand if you mmap() those files
> directly, what happens if another process truncates some of those files
> while you're reading them?

I didn't do a test (it's definitely worth testing), but I think that a
SIGSEGV could occur if a previously available page disappeared unless
MAP_COPY, and file need to be remapped.

I could see a problem where a siginfo-provided address might need to be
easily matched with the file so that the process can efficiently know
which file to remap...  and for many files the current kqueue(2)
EVFILT_VNODE isn't very useful either to detect that a file was
recently modified, as it'd require too many open file descriptors :(

There was some discussion made years ago about a kqueue(2) filter that
could be set on a directory under which any modified file (possibly for
the whole involved filesystem for the superuser) would generate an
event with information about which file is modified by inode, but this
seems non-trivial and wasn't yet implemented.  There also are issues
with inode to file string lookup (multiple files could point to a
common destination, and a reverse name cache is needed).

Anyway, I like this kind of discussion and have nothing against NIH
personally (it fuels variety and competition, in fact), so thanks for
sharing your custom cache experiments and performance numbers.  If you
happen to do achieve interesting performance along the above
lines with mmap(2) as well, I'd also like to know how it went.

Thanks,
-- 
Matt