Nginx and sysctl configuration - Performance setting -

nginx acting reverse proxy adserver, receiving 20k requests per minute. response happens within 100ms adserver nginx

running on virtual machine configuration 128gb ram 4 vcpu 100gb hdd

considering above, setting of nginx , sysctl.conf

please keep in mind kernel tuning complex , requires lot of evaluation until correct results. if spots mistake please let me know can adjust own configuration :-)

also, memory quite high amount of requests if server running nginx, check how using during peak hours , adjust accordingly.

an important thing check amount of file descriptors, in situation set 65.000 cope 20.000+ requests per second. reason in normal situation require 4.000 file descriptors have 4.000 simultanious open connections (20.000 * 2 * 0.1). in case of issue end take 1 second or more load advertisement. in case amount of simultanious open connections higher:

20.000 * 2 * 1.5 = 60.000.  

so setting 65k in opinion save value.

you can check amount of file descriptors via:

cat /proc/sys/fs/file-max 

if below 65000 you'll need set in /etc/sysctl.conf:

fs.file-max = 65000 

also nginx you'll need add following in file: /etc/systemd/system/nginx.service.d/override.conf

[service] limitnofile=65000 

in nginx.conf file:

worker_rlimit_nofile 65000; 

when added need apply changes:

sudo systemctl -p sudo systemctl daemon-reload sudo systemctl restart nginx 

after these settings following settings started:

vm.swappiness = 0   # kernel swap avoid out of memory condition vm.min_free_kbytes = 327680 # kernel start swapping when memory below limit (300mb)  vm.vfs_cache_pressure = 125 # reclaim memory used caching of vfs caches vm.dirty_ratio = 15 # write pages disk when 15% of memory dirty vm.dirty_background_ratio = 10 # system can start writing pages disk when 15% of memory dirty 

additionally use following security settings in sysctl configuration in conjunction tunables above. feel free use them, for credits

# avoid smurf attack net.ipv4.icmp_echo_ignore_broadcasts = 1 # turn on protection bad icmp error messages net.ipv4.icmp_ignore_bogus_error_responses = 1 # turn on syncookies syn flood attack protection net.ipv4.tcp_syncookies = 1 # turn on , log spoofed, source routed, , redirect packets net.ipv4.conf.all.log_martians = 1 net.ipv4.conf.default.log_martians = 1 # no source routed packets here net.ipv4.conf.all.accept_source_route = 0 net.ipv4.conf.default.accept_source_route = 0 # turn on reverse path filtering net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 # make sure no 1 can alter routing tables net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.default.accept_redirects = 0 net.ipv4.conf.all.secure_redirects = 0 net.ipv4.conf.default.secure_redirects = 0 # don't act router net.ipv4.ip_forward = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.default.send_redirects = 0 # turn on execshild kernel.exec-shield = 1 kernel.randomize_va_space = 1 

as proxying request add following sysctl.conf file make sure not running out of ports, optional if running issues keep in mind:

net.ipv4.ip_local_port_range=1024 65000 

as evaluate default settings , adjust accordingly did not supply ipv4 , ipv4.tcp_ options. can find example below please do not copy , paste, you'll required reading before start tuning these variables.

# increase tcp max buffer size setable using setsockopt() net.ipv4.tcp_rmem = 4096 87380 8388608 net.ipv4.tcp_wmem = 4096 87380 8388608 # increase linux auto tuning tcp buffer limits # min, default, , max number of bytes use # set max @ least 4mb, or higher if use high bdp paths # tcp windows etc net.core.rmem_max = 8388608 net.core.wmem_max = 8388608 net.core.netdev_max_backlog = 5000 net.ipv4.tcp_window_scaling = 1 

the above parameters not should consider, there many more parameters can tune, example:

• set amount of worker processes 4 (one per cpu core).
• tune backlog queue.
• if not need acccess log turn off remove disk i/o.
• optionally: lower or disable gzip compression if cpu usage getting high.