Prepare Linux OS for operation in a high-load cluster

To ensure maximum performance and stability of the Kubernetes system, optimize the operating system on which it is running. In this article, we will explore the installation of an optimized kernel using Ubuntu 20.04 as an example.

The optimized Ubuntu kernel is a specially adapted Linux kernel. It includes a set of changes and optimizations aimed at improving the efficiency of resource utilization and enhancing the processing of network requests.

Configuring the optimized Ubuntu kernel for Kubernetes involves two stages:

1. Install the optimized kernel
2. Optimize network parameters

Step 1. Install the optimized kernel

1. nstall the optimized Ubuntu kernel:

sudo apt-get install linux-lowlatency-hwe-20.04 -y

2. After successful installation, reboot the system:

sudo reboot

Step 2. Optimize network parameters

1. Create a tuned-sysctl.sh file and insert the following script:

#!/bin/bash
 
CONNTRACK_MAX_PER_CORE=131072
CONNTRACK_MIN=524288
 
CPU_NUM=`cat /proc/cpuinfo | grep -E '^processor\s+:\s+[0-9]+$' | wc -l`
CONNTRACK_BY_CPU=$(( $CPU_NUM * $CONNTRACK_MAX_PER_CORE ))
NF_CONNTRACK_MAX=$(( $CONNTRACK_BY_CPU > $CONNTRACK_MIN ? $CONNTRACK_BY_CPU : $CONNTRACK_MIN ))
MEM_NUM=`awk '/^MemTotal:/{print $2}' /proc/meminfo`
 
sysctl -w net.netfilter.nf_conntrack_max=$NF_CONNTRACK_MAX # set the maximum conntrack value
sysctl -w net.nf_conntrack_max=$NF_CONNTRACK_MAX
echo $(( $NF_CONNTRACK_MAX / 4 )) > /sys/module/nf_conntrack/parameters/hashsize # set proportional size hash table for conntrack search
 
# General optimization
sysctl -w vm.swappiness=0
sysctl -w net.core.somaxconn=1000
sysctl -w net.core.netdev_max_backlog=5000
sysctl -w net.core.rmem_max=$MEM_NUM
sysctl -w net.core.wmem_max=$MEM_NUM
sysctl -w net.ipv4.tcp_wmem="4096 87380 $MEM_NUM"
sysctl -w net.ipv4.tcp_rmem="4096 87380 $MEM_NUM"
sysctl -w net.ipv4.tcp_max_syn_backlog=8096
sysctl -w net.ipv4.tcp_no_metrics_save=1
sysctl -w net.ipv4.tcp_slow_start_after_idle=0
sysctl -w net.ipv4.tcp_tw_reuse=1
sysctl -w net.ipv4.ip_local_port_range="10500 65535"
sysctl -w net.ipv4.neigh.default.gc_thresh1=16384
sysctl -w net.ipv4.neigh.default.gc_thresh2=28672
sysctl -w net.ipv4.neigh.default.gc_thresh3=32768
sysctl -w net.bridge.bridge-nf-call-iptables=1 # kube-proxy requires this parameter
sysctl -w net.bridge.bridge-nf-call-arptables=1 # kube-proxy requires this parameter
sysctl -w net.bridge.bridge-nf-call-ip6tables=1 # kube-proxy requires this parameter
sysctl -w vm.dirty_ratio=80
sysctl -w vm.dirty_background_ratio=5
sysctl -w vm.dirty_expire_centisecs=12000
sysctl -w fs.file-max=1000000
sysctl -w vm.min_free_kbytes=131072
sysctl -w kernel.numa_balancing=0
sysctl -w fs.inotify.max_user_watches=524288
sysctl -w fs.inotify.max_user_instances=5120
sysctl -w kernel.pid_max=2000000
 
# kubelet parameters
sysctl -w vm.overcommit_memory=1
sysctl -w kernel.panic=10
sysctl -w kernel.panic_on_oops=1
 
# The nofile parameter sets the maximum number of opened files
echo -e "* hard nofile 4194304\n* soft nofile 4194304" >> /etc/security/limits.conf

2. Grant permissions to run the created file:

sudo chmod +x tuned-sysctl.sh

3. Run the file:

sudo ./tuned-sysctl.sh

4. Open the crontab file for editing:

sudo crontab -e

5. Add a command to execute the script after reboot:

@reboot sleep 180 && /path/to/tuned-sysctl.sh

Was this helpful?

Thanks for your feedback!

Please specify why:

Recommendations did not help meArticle is hard to understandDidn`t answer my questionContent does not match the topicOther

How we can improve it?

YesNo

Please specify whyRecommendations did not help meArticle is hard to understandDidn`t answer my questionContent does not match the topicOther

Found a typo? Highlight the text, press ctrl + enter and notify us