BeStMan is highly scalable to the number of request handling per unit time, depending on the underlying file system and storage. Configurable options need to be adjusted depending on the local storage and file system capacity, and also the operating system control needs to be adjusted.
Adjust sysctl values on the node where bestman server runs. Put the values in /etc/sysctl.conf then run sysctl -p to apply them, or apply individually.
No need to reboot, and your OS kernel should be able to handle a lot more open connections.
Ref: http://fasterdata.es.net/tuning.html and http://fasterdata.es.net/TCP-tuning/linux.html
# General gigabit tuning: net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 net.ipv4.tcp_syncookies = 1 # this gives the kernel more memory for tcp # which you need with many (100k+) open socket connections net.ipv4.tcp_mem = 50576 64768 98152 net.core.netdev_max_backlog = 2500 # I was also masquerading the port comet was on, you might not need this net.ipv4.netfilter.ip_conntrack_max = 1048576
In addition to above, check and changes these:
/sbin/sysctl -w net.core.somaxconn=256 /sbin/sysctl -w net.core.netdev_max_backlog=2500
Also, check these values for long timeout values. If relatively short, keep them longer.
% sysctl \ net.ipv4.tcp_keepalive_time \ net.ipv4.tcp_keepalive_intvl \ net.ipv4.tcp_keepalive_probes net.ipv4.tcp_keepalive_time = 1800 net.ipv4.tcp_keepalive_intvl = 75 net.ipv4.tcp_keepalive_probes = 9
*1. Increase the maximum container thread pool size for the jetty server*
e.g. configure --with-max-container-threads=1024
*2. Have large number for jetty http connection acceptor threads for the server's channel connector*
e.g. configure --with-connection-acceptor-thread-size=8
*3. Have large size for jetty http connection waiting queue*
e.g. configure --with-connector-queue-size=512
*4. Increase the max java heap memory size*
e.g. configure --with-max-java-heap=4096
*5. Client side http connection timeout control - bestman clients (only)*
e.g. srm-copy -sethttptimeout 1800
*6. Each connection we make requires an ephemeral port, and thus a file descriptor*
By default this is limited to 1024. To avoid the Too many open files problem you'll need to modify the ulimit for your shell.
limit descriptors 65535
*7. This can be changed in /etc/security/limits.conf, but requires a logout/login.*
For now you can just sudo and modify the current shell.
(su back to your non-priv'ed user after calling ulimit if you don't want to run as root):
$ sudo bash # ulimit -n 999999 # erl