使用 kubeadm 部署 Kubernetes
Kubernetes 可以通过 kubeadm 进行安装部署。
安装流程
准备工作
- 关闭 swap,将
/etc/fstab中关于swap的行注释掉。 - 修改主机名:
hostnamectl set-hostname <hostname>。 - 内核开启 IP 转发:
echo 'net.ipv4.ip_forward=1' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
- 安装 containerd、runc。
- 你需要在每台机器上安装以下软件包:
- kubeadm:用来初始化集群的指令。
- kubelet:在集群中的每个节点上用来启动 Pod 和容器等。
- kubectl:用来与集群通信的命令行工具。
控制节点与工作节点
- 控制节点:通过
kubeadm init初始化集群。 - 工作节点:通过
kubeadm join加入已有集群。
kubeadm 安装
安装方法参见:kubeadm 安装文档
sudo apt-get update
# apt-transport-https 可能是一个虚拟包(dummy package);如果是的话,你可以跳过安装这个包
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
# 如果 `/etc/apt/keyrings` 目录不存在,则应在 curl 命令之前创建它,请阅读下面的注释。
# sudo mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.34/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
# 此操作会覆盖 /etc/apt/sources.list.d/kubernetes.list 中现存的所有配置。
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl # 锁定其版本
使用 kubeadm 创建集群
首先初始化控制平面节点:kubeadm init,建议机器内存资源要充足(16G 以上)。
# kubeadm 默认从 registry.k8s.io 拉取镜像,但该域名在国内(或某些网络环境)无法访问,因此需要从阿里云镜像仓库拉取。
root@slpc:~# kubeadm config images pull --image-repository=registry.aliyuncs.com/google_containers
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.34.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.34.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.34.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.34.3
[config/images] Pulled registry.aliyuncs.com/google_containers/coredns:v1.12.1
[config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.10.1
[config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.6.5-0
# 通过 kubeadm init 命令初始化集群,并指定使用阿里云镜像仓库,指定 pod 网络地址段(CIDR 格式)为 10.244.0.0/16
root@slpc:~# kubeadm init --image-repository=registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16
[init] Using Kubernetes version: v1.34.3
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local slpc] and IPs [10.96.0.1 192.168.232.128]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost slpc] and IPs [192.168.232.128 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost slpc] and IPs [192.168.232.128 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "super-admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests"
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 502.596944ms
[control-plane-check] Waiting for healthy control plane components. This can take up to 4m0s
[control-plane-check] Checking kube-apiserver at https://192.168.232.128:6443/livez
[control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz
[control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez
[control-plane-check] kube-controller-manager is healthy after 1.002103972s
[control-plane-check] kube-scheduler is healthy after 1.195871396s
[control-plane-check] kube-apiserver is healthy after 3.001422611s
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node slpc as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node slpc as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: zdd3y8.g6yz49h5q1ztz75y
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.232.128:6443 --token zdd3y8.g6yz49h5q1ztz75y \
--discovery-token-ca-cert-hash sha256:08377aeb0fa21dfceeecfece817b4263c6eddf11ee1a09e68c6f9f9ca7bc03a7
安装网络插件,这里选择 flannel 网络插件:
# 下载 flannel
wget https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
root@slpc:~# kubectl apply -f kube-flannel.yml
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
要使非 root 用户可以运行 kubectl,请运行以下命令:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
或者,如果是 root 用户,则可以运行:
export KUBECONFIG=/etc/kubernetes/admin.conf
然后通过 kubectl 查看集群状态:
root@slpc:~# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-flannel kube-flannel-ds-lb9mt 1/1 Running 0 58m
kube-system coredns-7cc97dffdd-5vvvk 1/1 Running 0 62m
kube-system coredns-7cc97dffdd-bzhww 1/1 Running 0 62m
kube-system etcd-slpc 1/1 Running 8 62m
kube-system kube-apiserver-slpc 1/1 Running 9 62m
kube-system kube-controller-manager-slpc 1/1 Running 0 62m
kube-system kube-proxy-68bfc 1/1 Running 0 62m
kube-system kube-scheduler-slpc 1/1 Running 10 62m
到目前为止,我们仅有一个节点,我们需要添加工作节点。在添加工作节点之前,需要准备如下:
- 每个要加入的工作节点都已安装 kubeadm 中所需的组件,例如 kubeadm、kubelet 和容器运行时。
- 一个正在运行的、由
kubeadm init命令所创建的 kubeadm 集群,且该集群的创建遵循使用 kubeadm 创建集群文档中所给的步骤。 - 你需要对节点拥有超级用户权限。
可参考 添加 Linux 工作节点
在控制节点执行生成工作节点加入 Kubernetes 集群的命令:
root@slpc:~# kubeadm token create --print-join-command
kubeadm join 192.168.232.128:6443 --token faro80.iwwif9f0duu8uv0k --discovery-token-ca-cert-hash sha256:08377aeb0fa21dfceeecfece817b4263c6eddf11ee1a09e68c6f9f9ca7bc03a7
工作节点需要安装并正确配置 containerd:
# 生成默认配置
sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml
# 启用 systemd cgroup 驱动(必须与 kubelet 一致)
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
# 重启 containerd
sudo systemctl restart containerd
sudo systemctl enable containerd
验证 CRI 是否正常:
crictl info # 正常情况返回 json 信息。
在工作节点执行如下:
root@node1:~# kubeadm join 192.168.232.128:6443 --token faro80.iwwif9f0duu8uv0k --discovery-token-ca-cert-hash sha256:08377aeb0fa21dfceeecfece817b4263c6eddf11ee1a09e68c6f9f9ca7bc03a7
[preflight] Running pre-flight checks
[preflight] Reading configuration from the "kubeadm-config" ConfigMap in namespace "kube-system"...
[preflight] Use 'kubeadm init phase upload-config kubeadm --config your-config-file' to re-upload it.
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/instance-config.yaml"
[patches] Applied patch of type "application/strategic-merge-patch+json" to target "kubeletconfiguration"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is healthy after 502.295181ms
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
镜像拉取失败问题(被墙限制)
# 拉取镜像(使用国内镜像加速,避免 registry.k8s.io 失败)
docker pull registry.aliyuncs.com/google_containers/pause:3.10
docker pull registry.aliyuncs.com/google_containers/kube-proxy:v1.34.3
docker pull docker.io/flannel/flannel:v0.25.6
docker pull docker.io/flannel/flannel-cni-plugin:v1.5.1-flannel2
# 重命名 tag 为 Kubernetes 期望的名称(关键!)
docker tag registry.aliyuncs.com/google_containers/pause:3.10 registry.k8s.io/pause:3.10.1
docker tag registry.aliyuncs.com/google_containers/kube-proxy:v1.34.3 registry.k8s.io/kube-proxy:v1.34.3
# 打包
docker save -o k8s-node-images.tar \
registry.k8s.io/pause:3.10.1 \
registry.k8s.io/kube-proxy:v1.34.3 \
docker.io/flannel/flannel:v0.25.6 \
docker.io/flannel/flannel-cni-plugin:v1.5.1-flannel2
scp k8s-node-images.tar root@192.168.232.137:/root/ # node1
scp k8s-node-images.tar root@192.168.232.138:/root/ # node2
# 在 node1 和 node2 上分别执行
sudo ctr -n=k8s.io images import /root/k8s-node-images.tar
# 重启 kubelet
sudo systemctl restart kubelet
查看集群节点信息:
root@slpc:~# kubectl get nodes
NAME STATUS ROLES AGE VERSION
node1 Ready <none> 18h v1.34.3
node2 Ready <none> 3h15m v1.34.3
slpc Ready control-plane 3d17h v1.34.2
安装 Kubernetes Dashboard
- 创建命名空间并部署:
# 部署 Dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml
- 验证安装:
kubectl get pods -n kubernetes-dashboard
kubectl get svc -n kubernetes-dashboard
- 如果镜像拉取有问题,可编辑 Deployment,将镜像地址改为国内镜像地址:
# 修改 dashboard 镜像
kubectl set image deployment/kubernetes-dashboard \
kubernetes-dashboard=registry.cn-hangzhou.aliyuncs.com/google_containers/dashboard:v2.7.0 \
-n kubernetes-dashboard
# 修改 metrics-scraper 镜像
kubectl set image deployment/dashboard-metrics-scraper \
dashboard-metrics-scraper=registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-scraper:v1.0.8 \
-n kubernetes-dashboard
kubectl set image会自动触发滚动更新,旧 Pod 被删除,新 Pod 使用新镜像启动。
- 启动 Kubernetes 代理:
kubectl proxy --address='0.0.0.0' --port=8001
- 使用 Token 登录 Dashboard:
# 创建一个名为 admin-user 的账号,并赋予其 cluster-admin 全局管理员权限。
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard
EOF
获取登录 Token:
root@slpc:~# kubectl -n kubernetes-dashboard create token admin-user
eyJhbGciOiJSUzI1NiIsImtpZCI6ImI0NmpHT3NnaUx2a0xVbno3Q2QtZklMdDdmNU1DYmVYVFRYRVQ2NjdZdFkifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNzY1ODk5OTI4LCJpYXQiOjE3NjU4OTYzMjgsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwianRpIjoiN2EwNDFkMzQtMzBmNi00NjZjLTg3MGQtMzhjNDMyNzMwZmUzIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsInNlcnZpY2VhY2NvdW50Ijp7Im5hbWUiOiJhZG1pbi11c2VyIiwidWlkIjoiY2Y3MDFlYzgtMzA2Yy00OWU2LTk1NWEtMWY1MWZmN2E5YzIxIn19LCJuYmYiOjE3NjU4OTYzMjgsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDphZG1pbi11c2VyIn0.bcwIpuhFDDK8fWlRv7yqT9MTJINq1tLujcRAjyjNdmU7ctJeUYy0WL54m152w8KcIc0TJ0IfXa8ZoE1ygqTtoR3DDrgkdPIMA08h5l11I3vQNpqUNptsjCLTqsTmE8FkoXoXfD2Aov3--obOiewykXsU72ppRHc0Fb0KlKDPLHJVzmObrzvLGngmfyiUU-D6qkAEtVpo6-DEZYMFn2KCZrboqgOztEpLH9hsEhOQkztC5QFMpqljwreVGqG3Px89GpQ97Zg3a3Qkkmjq_DBkoXbdV_2Qi8ypFB3F6LaDh7V1n_Szhr8e-ydjD8-7QFpJFRsHP0lxyj3b8k3tjS7Cqg
- 访问 Dashboard,在浏览器中打开:
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
输入上面的 token,点击登录。
安装 Metrics Server
Metrics Server 是 Kubernetes 官方推荐的轻量级资源指标聚合器(Resource Metrics API 实现),用于从集群中的节点(Node)和 Pod 收集 CPU 和内存使用率等核心资源指标。
wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
# 更改镜像地址为国内镜像
image: registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server:v0.8.0
# 编辑 Deployment,添加 --kubelet-insecure-tls 参数,跳过 TLS 证书校验
- --kubelet-insecure-tls
kubectl apply -f components.yaml
验证:
postgres@slpc:$ kubectl top nodes
NAME CPU(cores) CPU(%) MEMORY(bytes) MEMORY(%)
node1 36m 0% 616Mi 33%
node2 29m 0% 1003Mi 55%
slpc 198m 2% 6877Mi 88%
root@slpc:~# kubectl top pods -A
NAMESPACE NAME CPU(cores) MEMORY(bytes)
kube-flannel kube-flannel-ds-6lt2r 8m 12Mi
kube-flannel kube-flannel-ds-7pncp 7m 13Mi
kube-flannel kube-flannel-ds-plldn 7m 21Mi
kube-system coredns-7cc97dffdd-5vvvk 3m 17Mi
kube-system coredns-7cc97dffdd-bzhww 4m 15Mi
kube-system etcd-slpc 33m 47Mi
kube-system kube-apiserver-slpc 57m 254Mi
kube-system kube-controller-manager-slpc 28m 52Mi
kube-system kube-proxy-68bfc 1m 15Mi
kube-system kube-proxy-854cx 1m 26Mi
kube-system kube-proxy-b6jzz 1m 13Mi
kube-system kube-scheduler-slpc 9m 29Mi
kube-system metrics-server-587957767f-2wg97 5m 20Mi
kubernetes-dashboard dashboard-metrics-scraper-76c8db4-46f52 1m 12Mi
kubernetes-dashboard kubernetes-dashboard-7547fc74b4-nls72 1m 17Mi