Kubernetes（k8s）v1.32.5高可用集群部署（一）基础准备篇

kubernetes部署是一个繁琐的过程，当然也可以使用第三方安装工具一键安装，但这不利于部署过程学习和自定义性，并且第三方工具或多或少都会有入侵性

节点规划

Master高可用规划

使用HAProxy+Keeplived的方式组建对Master节点的高可用

安装必要软件

dnf install tar epel-release wget net-tools vim rsync telnet nc ipvsadm ipset yum-utils device-mapper-persistent-data lvm2 psmisc make -y

修改主名和配置hosts文件

cat >> /etc/hosts << EOF
10.2.16.180 mvip
10.2.16.181 m181
10.2.16.182 m182
10.2.16.183 m183
10.2.16.184 n184
10.2.16.185 n185
10.2.16.186 n186
EOF

关闭防火墙和selinux

systemctl stop firewalld #关闭防火墙
systemctl disable firewalld
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

关闭swap交换空间

swapoff -a #临时关闭
sed -ri 's/.swap./#&/' /etc/fstab #永久关闭

时间同步

yum -y install chrony && timedatectl set-timezone Asia/Shanghai
vim /etc/chrony.conf
...
server ntp1.aliyun.com iburst
systemctl enable --now chronyd

配置limit

ulimit -SHn 65535 # 临时

cat >> /etc/security/limits.conf << EOF

• soft nofile 65535
• hard nofile 65535
• soft nproc 65535
• hard nproc 65535
• soft stack 65535
• soft core unlimited
• soft memlock unlimited
• hard memlock unlimited
  EOF

加载br_netfilter模块

modprobe br_netfilter && lsmod | grep br_netfilter # 临时

cat > /etc/modules-load.d/k8s.conf << EOF
overlay
br_netfilter
EOF

开启ipvs模块 （svc代理）

yum install ipvsadm ipset sysstat conntrack libseccomp -y

cat > /etc/modules-load.d/ipvs.conf <<EOF
ip_vs
ip_vs_rr
ip_vs_wrr
ip_vs_sh
nf_conntrack
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip
EOF

systemctl restart systemd-modules-load.service && lsmod | grep ip_vs

内核优化 （根据情况取舍）

cat > /etc/sysctl.d/kubernetes.conf << EOF
net.core.netdev_max_backlog=10000
net.core.somaxconn=65535
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.core.rps_sock_flow_entries=8192
net.ipv4.tcp_rmem=4096 12582912 16777216
net.ipv4.tcp_wmem=4096 12582912 16777216
net.ipv4.neigh.default.gc_thresh1=2048
net.ipv4.neigh.default.gc_thresh2=4096
net.ipv4.neigh.default.gc_thresh3=8192
net.ipv4.tcp_max_syn_backlog=8096
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_forward=1
net.ipv6.conf.all.disable_ipv6=1
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=524288
fs.file-max=52706963
fs.nr_open=52706963
kernel.threads-max=30058
kernel.core_pattern=core
vm.max_map_count=262144
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
EOF

sysctl --system

配置k8s的YUM仓库源

每个Kubernetes版本都有一个专用的包存储库

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.32/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.32/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF

查询可安装版本

yum list --showduplicates --disableexcludes=kubernetes | egrep kubeadm | sort -r

配置m181免密登录其它节点

只在主节点m181操作

ssh-keygen
for i in m181 m182 m183 n184 n185 n186; do ssh-copy-id -i .ssh/id_rsa.pub $i;done

测试登录：ssh m183

重启服务器与检验配置

for i in m182 m183 n184 n185 n186 m181; do ssh root@$i reboot;done

lsmod | grep --color=auto -e ip_vs -e nf_conntrack -e br_netfilter
sysctl --system