共计 8301 个字符，预计需要花费 21 分钟才能阅读完成。
1 Containerd启动容器涉及的模块
前面我们梳理了docker、k8s、containerd、runc之间的渊源，Containerd早已脱离Docker成为了一个稳定可靠的容器运行时。
单从containerd的角度来看启动一个容器的过程大致是下图所示的流程:
我们可以使用Containerd的命令行工具ctr来启动一个容器，也可以在我们的应用中编写客户端程序为我们的应用加入启动容器的功能。
从containerd作为k8s的容器运行时来看，启动一个容器的过程大致是下图所示流程:
containerd实现了k8s的CRI接口，提供容器运行时核心功能，如镜像管理、容器管理等，当前containerd实现CRI接口的代码位于containerd代码的pkg/cri包下。 也就是说containerd同样是一个k8s CRI的实现，可以使用k8s提供的cri-tools中的crictl命令行工具与containerd的CRI实现交互。
理解了上面这两幅图中各个组件之间的交互关系，接下来就可以部署containerd相关组件。
2 二进制部署containerd
下载安装包
wget https://github.com/opencontainers/runc/releases/download/v1.0.0-rc93/runc.amd64
wget https://github.com/containerd/containerd/releases/download/v1.4.4/containerd-1.4.4-linux-amd64.tar.gz
wget https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.21.0/crictl-v1.21.0-linux-amd64.tar.gzrunc.amd64是runc的二进制文件
containerd-1.4.4-linux-amd64.tar.gz 包含 containerd 和它的 containerd-shim 以及命令行工具 ctr
crictl-v1.21.0-linux-amd64.tar.gz 包含 CRI 的命令行工具 crictl
下一步进行二进制文件的解压和安装:
mkdir /usr/local/containerd
mkdir /etc/containerd
mv runc.amd64 /usr/local/containerd/bin/runc
chmod +x /usr/local/containerd/bin/runc
tar -zxvf containerd-1.4.4-linux-amd64.tar.gz -C /usr/local/containerd
tar -zxvf crictl-v1.21.0-linux-amd64.tar.gz -C /usr/local/containerd/bin
ln -s /usr/local/containerd/bin/containerd /usr/local/bin/containerd
ln -s /usr/local/containerd/bin/ctr /usr/local/bin/ctr
ln -s /usr/local/containerd/bin/runc /usr/local/bin/runc
ln -s /usr/local/containerd/bin/crictl /usr/local/bin/crictl安装的目录接口如下:
tree /usr/local/containerd/
/usr/local/containerd/
└── bin
├── containerd
├── containerd-shim
├── containerd-shim-runc-v1
├── containerd-shim-runc-v2
├── crictl
├── ctr
└── runc下一步生成containerd的默认配置文件/etc/containerd/config.toml
containerd config default > /etc/containerd/config.toml查看一下配置文件(篇幅有限这里没有给出全部内容)
root@VM-16-16-centos ~]# cat /etc/containerd/config.toml
version = 2
root = "/var/lib/containerd"
state = "/run/containerd"
plugin_dir = ""
disabled_plugins = []
required_plugins = []
oom_score = 0
[grpc]
address = "/run/containerd/containerd.sock"
tcp_address = ""
tcp_tls_cert = ""
tcp_tls_key = ""
uid = 0
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
[ttrpc]
address = ""
uid = 0
gid = 0
[debug]
address = ""
uid = 0
gid = 0
level = ""创建 containerd 进程的 systemd 服务配置 containerd.service，containerd 源码的根目录中也给出了containerd.service这个文件的示例
cat <<EOF | tee /etc/systemd/system/containerd.service
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=1048576
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
EOF启动 containerd
# systemctl start containerd
# systemctl enable containerd
# systemctl status containerd
● containerd.service - containerd container runtime
Loaded: loaded (/etc/systemd/system/containerd.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2023-02-16 15:08:44 CST; 10s ago
Docs: https://containerd.io
Main PID: 13291 (containerd)
CGroup: /system.slice/containerd.service
└─13291 /usr/local/bin/containerd3 ctr 命令工具使用
3.1 打印服务端和客户端版本
[root@VM-16-16-centos ~]# ctr version
Client:
Version: v1.4.4
Revision: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
Go version: go1.15.8
Server:
Version: v1.4.4
Revision: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
UUID: f4d6ec71-79e8-44d2-ad92-333b369aae793.2 pull docker hub中的redis镜像
# pull镜像
ctr images pull docker.io/library/redis:alpine3.13
# 查看镜像
ctr i ls3.3 pull私有仓库的镜像
需要使用-u <username>:<password>给定镜像仓库的用户名和密码
ctr images pull -u user:password own.harbor.xxx.com/library/nginx:1.13.4 启动redis容器
ctr run -d docker.io/library/redis:alpine3.13 redis3.5 查看容器
ctr task ls
TASK PID STATUS
redis 14348 RUNNING
ctr container ls
CONTAINER IMAGE RUNTIME
redis docker.io/library/redis:alpine3.13 io.containerd.runc.v2 注意: 在 containerd 中，container 和 task 是分离的，container 描述的是容器分配和附加资源的元数据对象，是静态内容，task 是任务是系统上一个活动的、正在运行的进程。 task 应该在每次运行后删除，而 container可以被多次使用、更新和查询。这点和 docker 中 container 定义是不一样的
3.6 进入到容器中
ctr task exec -t --exec-id redis-sh redis sh
/data # redis-cli
127.0.0.1:6379> ping
PONG
127.0.0.1:6379>3.7 查看进程信息
ps -ef | grep runc | grep redis
/usr/local/containerd/bin/containerd-shim-runc-v2 -namespace default -id redis -address /run/containerd/containerd.sock可以看出containerd中是存在namespace概念的，这样可以将不同业务和应用进行隔离，例如k8s使用containerd和直接使用ctr创建的容器可以隔离开。查看一下当前的namespace
[root@VM-16-16-centos ~]# ctr ns ls
NAME LABELS
default不同namespace下pull的镜像也是隔离显示的，可以使用-n指定具体的namespace
ctr -n default i ls
ctr -n k8s.io i ls4 crictl命令行工具配置
crictl是k8s cri-tools的一部分，它提供了类似于docker的命令行工具，不需要kubelet就可以通过CRI跟容器运行时通信。 crictl是专门为k8s设计的，提供了Pod、容器和镜像等资源的管理命令，可以帮助用户调试容器应用或者排查异常问题。 crictl 可以用于所有实现了CRI接口的容器运行时
尝试使用ctictl命令行工具查看一下镜像，给了一个警告信息
[root@VM-16-16-centos ~]# crictl images
WARN[0000] image connect using default endpoints: [unix:///var/run/dockershim.sock unix:///run/containerd/containerd.sock unix:///run/crio/crio.sock]. As the default settings are now deprecated, you should set the endpoint instead.
ERRO[0002] connect endpoint 'unix:///var/run/dockershim.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded
IMAGE TAG IMAGE ID SIZE需要我们显示配置默认的endpoints
crictl config runtime-endpoint unix:///run/containerd/containerd.sock
crictl config image-endpoint unix:///run/containerd/containerd.sock上面的命令执行完成后将生成配置文件/etc/crictl.yaml
[root@VM-16-16-centos ~]# cat /etc/crictl.yaml
runtime-endpoint: "unix:///run/containerd/containerd.sock"
image-endpoint: "unix:///run/containerd/containerd.sock"
timeout: 0
debug: false
pull-image-on-create: false
disable-pull-on-run: false4.1 pull 镜像
crictl pull docker.io/library/redis:alpine3.13[root@VM-16-16-centos ~]# crictl images
IMAGE TAG IMAGE ID SIZE
docker.io/library/redis alpine3.13 1690b63e207f6 10.9MBcrictl pull的镜像实际上是在k8s.io namespace下，可以使用ctr -n k8s.io i ls查看
crictl不能像ctr那样通过参数给定用户名和密码的方式从开启认证的私有仓库中pull镜像。需要对containerd进行配置。 containerd提供的各种功能在其内部都是通过插件实现的，可以使用ctr plugins ls查看
root@VM-16-16-centos ~]# ctr plugins ls
TYPE ID PLATFORMS STATUS
io.containerd.content.v1 content - ok
io.containerd.snapshotter.v1 aufs linux/amd64 error
io.containerd.snapshotter.v1 btrfs linux/amd64 error
io.containerd.snapshotter.v1 devmapper linux/amd64 error
io.containerd.snapshotter.v1 native linux/amd64 ok
io.containerd.snapshotter.v1 overlayfs linux/amd64 ok
io.containerd.snapshotter.v1 zfs linux/amd64 error
io.containerd.metadata.v1 bolt - ok
io.containerd.differ.v1 walking linux/amd64 ok
io.containerd.gc.v1 scheduler - ok
io.containerd.service.v1 introspection-service - ok
io.containerd.service.v1 containers-service - ok
io.containerd.service.v1 content-service - ok
io.containerd.service.v1 diff-service - ok
io.containerd.service.v1 images-service - ok
io.containerd.service.v1 leases-service - ok
io.containerd.service.v1 namespaces-service - ok
io.containerd.service.v1 snapshots-service - ok
io.containerd.runtime.v1 linux linux/amd64 ok
io.containerd.runtime.v2 task linux/amd64 ok
io.containerd.monitor.v1 cgroups linux/amd64 ok
io.containerd.service.v1 tasks-service - ok
io.containerd.internal.v1 restart - ok
io.containerd.grpc.v1 containers - ok
io.containerd.grpc.v1 content - ok
io.containerd.grpc.v1 diff - ok
io.containerd.grpc.v1 events - ok
io.containerd.grpc.v1 healthcheck - ok
io.containerd.grpc.v1 images - ok
io.containerd.grpc.v1 leases - ok
io.containerd.grpc.v1 namespaces - ok
io.containerd.internal.v1 opt - ok
io.containerd.grpc.v1 snapshots - ok
io.containerd.grpc.v1 tasks - ok
io.containerd.grpc.v1 version - ok
io.containerd.grpc.v1 cri linux/amd64 ok 私有镜像仓库相关的配置在cri插件中，文档Configure Image Registry中包含了镜像仓库的配置。
关于私有仓库和认证信息配置示例如下，修改/etc/containerd/config.toml：
...
[plugins]
...
[plugins."io.containerd.grpc.v1.cri"]
...
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."harbor.my.org"]
endpoint = ["https://harbor.my.org"]
[plugins."io.containerd.grpc.v1.cri".registry.configs]
[plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.my.org".tls]
insecure_skip_verify = true
[plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.my.org".auth]
username = "username"
password = "passwd"
# auth = "base64(username:password)"
...配置完成后重启containerd，就可以使用crictl pull配置的私有仓库的镜像了
5 总结
我们已经部署了containerd，配置好了ctr和crictl工具，并使用ctr启动了第一个测试容器，但是此时容器还不具备网络能力，只能在容器内部自己玩。 下一步，将尝试手工配置containerd和cni及cni插件的集成，为容器加入基本的网络能力。
