Re: Linux compat and swap

[tlaronde%polynum.com@localhost]

On Fri, Apr 24, 2020 at 12:23:14PM +1000, Paul Ripke wrote:
> On Thu, Apr 23, 2020 at 06:14:00PM +0200, tlaronde%polynum.com@localhost wrote:
> > On Thu, Apr 23, 2020 at 01:30:58PM +0100, Mike Pumford wrote:
> > > 
> > > 
> > > On 23/04/2020 11:48, tlaronde%polynum.com@localhost wrote:
> > > >[Please forgive me to give supplementary information gathered for all
> > > >answers and "before". And thank you to all for answering.]
> > > >
> > > >To be more precise:
> > > >
> > > >The node is not a desktop node but a server one (so there is no X11
> > > >running, so no firefox or whatever), with two main functions:
> > > >
> > > >1) A file server;
> > > >2) A processing node for data (GIS) manipulations, since there can be
> > > >huge amounts of data to process, so it is more efficient to process
> > > >where it is saved due to the network capabilities.
> > > >
> > > In my case the build machines main purpose is as a build server (hence the
> > > jenkins server which is a java process). I use it as a desktop when
> > > developing things as its my most powerful BSD build machine and its easier
> > > to develop when you can use a browser to check API references etc so its
> > > desktop use is intermittent. So they were just examples of memory hungry
> > > processes that got swapped out unexpectedly rather than the only ones.
> > > 
> > > >The node' uptime is 225 days (since some disk replacements) serving
> > > >during office time files to 2 to 6 nodes. And it is processing data
> > > >(sometimes during office time; generally outside office time).
> > > >
> > > >It has 3862 MB of avail memory.
> > > >
> > > >When using a program to convert coordinates, this program being a Linux
> > > >binary, the program bailed out, 3 times in a row, after processing 20M
> > > >records. Splitting the input data under this amount, I managed to
> > > >process everything.
> > > >
> > > Could you be running into the data size limits for the process?
> > 
> > No I think that you gave the answer in your previous mail. This is the
> > sysctl variable vm.execmax.
> > 
> > It is set to 30 (30%) and it was reaching the limit. Since there was
> > till now no need for more, the swap space is limited (only 128MB).
> > 
> > So the solution is indeed to play with the vm.* variables and here to
> > increase vm.execmax (and I will too add some swap since I have spare
> > partitions on some disks).
> 
> To the best of my knowledge, vm.execmax just defines the maximum amount
> of memory for mapped executable pages, and won't cause processes to be
> killed. Since mapped executable pages are generally read-only, they can
> just be dropped from RAM and paged in again from the executables as
> needed.
> 
> You probably just need to decrease vm.filemax, so that the file cache
> doesn't put so much pressure on anonymous memory, and allows processes
> to grow larger.

That makes indeed sense.

Thanks!
-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                     http://www.kergis.com/
                       http://www.sbfa.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C