Linux OS Settings
Linux Operating System Configuration
This section covers some settings which are specific to running AMPS on a Linux Operating System.
ulimit
The ulimit command is used by a Linux administrator to get and set user limits on various system resources.
ulimit -c
It is common for an AMPS instance to be configured to consume gigabytes of memory for large SOW caches. If a failure were to occur in a large deployment it could take seconds (maybe even hours, depending on storage performance and process size) to dump the core file. AMPS has a minidump reporting mechanism built in that collects information important to debugging an instance before exiting. This minidump is much faster than dumping a core file to disk. For this reason, it is recommended that the per user core file size limit is set to 0 to prevent a large process image from being dumped to storage.
ulimit -n
The number of file descriptors allowed for a user running AMPS needs to be at least double the sum of counts for the following: connected clients, SOW topics and pre-allocated journal files.
Minimum: 4096
Recommended: 32768, or the value recommended by AMPS in any diagnostic messages, whichever is greater
Transparent Huge Pages
Transparent huge pages is enabled by default for most linux distributions, which can add significant overhead to memory management. For this reason 60East recommends changing the setting to madvise, which requires applications to explicitly request use of transparent huge pages. Previously the recommendation was to use never, which is still acceptable, but is less flexible as it disables transparent huge pages for all applications running on the system.
To change the setting until the operating system is rebooted, the following command can be used:
To make a permanent change to this setting, add the above command to the startup scripts, or add the transparent_hugepage=madvise option to the kernel startup flags (see the documentation for your Linux distribution for details).
Recommended: madvise
/proc/sys/fs/aio-max-nr
Each AMPS instance requires AIO in the kernel to support at least 16384 plus 8192 for each SOW topic in simultaneous I/O operations. The setting aio-max-nr is global to the host and impacts all applications. As such this value needs to be set high enough to service all applications using AIO on the host.
Minimum: 65536
Recommended: 1048576
To view the value of this setting, as root you can enter the following command:
To edit this value, as root you can enter the following command:
This command will update the value for /proc/sys/fs/aio-max-nr and allow 1,048,576 simultaneous I/O operations, but will only do so until the next time the machine is rebooted. To make a permanent change to this setting, as a root user, edit the /etc/sysctl.conf file and either edit or append the following setting:
/proc/sys/fs/file-max
Each AMPS instance needs file descriptors to service connections and maintain file handles for open files. This number needs to be at least double the sum of counts for the following: connected clients, SOW topics and pre-allocated journal files. This file-max setting is global to the host and impacts all applications, so this needs to be set high enough to service all applications on the host.
Minimum: 262144
Recommended: 6815744
To view the value of this setting, as root you can enter the following command:
To edit this value, as root you can enter the following command:
This command will update the value for /proc/sys/fs/file-max and allow 6,815,744 concurrent files to be opened, but will only do so until the next time the machine is rebooted. To make a permanent change to this setting, as a root user, edit the /etc/sysctl.conf file and either edit or append the following setting:
/proc/sys/vm/min_free_kbytes
This parameter sets the minimum amount of memory to keep free in the system. Setting this value properly can help the operating system function more effectively in low-memory situations. If this value is set too low, the operating system can have difficulty reclaiming memory, which can lead to unnecessary out-of-memory events. If this value is set too high, overall system efficiency decreases as the operating system can spend more time than necessary reclaiming memory.
60East recommends setting this parameter to 1% of the physical memory on the system, rounding up to the nearest GB.
Notice that the units of this parameter are in kilobytes. For example, to set this value for a system that has 128GB of memory, you would calculate 1% of the physical memory (1.28 GB), round up to the nearest GB (2 GB) and then allocate 2000000 KB as the min_free_kbytes.
Minimum: 1000000 (1GB)
Recommended: 1% of physical memory, rounded up to the nearest GB
To edit this value, as root you can enter the following command:
Notice that this tuning recommendation is designed for a server-class machine with a reasonable amount of memory. For a small development machine or blade (for example, a system with less than 32GB of memory), leaving this parameter at the operating system default may be more appropriate.
/proc/sys/vm/max_map_count
AMPS makes extensive use of memory mapped files, and frequently modifies the maps. The /proc/sys/vm/max_map_count parameter sets the maximum number of maps that the Linux kernel will allow for a process. If the number of requested maps exceeds the number of maps in this parameter, memory allocation operations can fail even when there is sufficient memory available.
This setting is global to the host and applies to all applications, so this needs to be set high enough for the most map-intensive application on the host.
Minimum: 65530
Recommended: 500000
To edit this value, as root you can enter the following command:
This command will update the value for /proc/sys/vm/max_map_count and allow 500,000 maps to be created, but will only do so until the next time the machine is rebooted. To make a permanent change to this setting, as a root user, edit the /etc/sysctl.conf file and either edit or append the following setting:
/proc/sys/vm/swappiness
AMPS performs best when the data that it needs to retain is in memory. If the operating system needs to use swap because the system requires more memory than is available, performance degrades substantially.
60East recommends that, for systems that host performance-critical instances of AMPS, the vm.swappiness setting is set to 1. This will minimize swapping on this system, which will improve performance with the tradeoff of making it more likely for processes to be killed by the operating system in low-memory situations.
This setting is global to the host and applies to all applications.
Recommended: 1
To edit this value, as root you can enter the following command:
This command will update the value for /proc/sys/vm/swappiness and direct the operating system to avoid using swap space until the system is under severe memory pressure.
Using the command above will change the swappiness setting until the operating system is rebooted.To make a permanent change to this setting, as a root user, edit the /etc/sysctl.conf file and either edit or append the following setting:
/proc/sys/net/ipv4/tcp_frto
This option controls whether Forward RTO-Recovery (FRTO) is enabled for the TCP network. Enabling FRTO can be beneficial for overall network performance if a system is sending packets over wireless networks with substantial interference (for example, public WiFi in an urban area). However, this recovery algorithm can reduce performance in wired networks. While this option is enabled on most current Linux distributions by default, disabling the option can improve network performance.
60East recommends disabling this option unless the server is directly delivering traffic over a congested WiFi network.
This setting is global to the host and applies to all applications.
Recommended: 0
To edit this value, as root you can enter the following command:
This command will update the value for /proc/sys/net/ipv4/tcp_frto and direct the operating system to disable FRTO.
Using the command above will change the setting until the operating system is rebooted. To make a permanent change to this setting, as a root user, edit the /etc/sysctl.conf file and either edit or append the following setting:
Last updated