搜索中...
🔍

未找到相关结果

Akemi

Buildah多架构镜像构建与合并多架构镜像

字数统计: 1.8k阅读时长: 9 min
2026/08/22
特性 说明
无守护进程 不需要 dockerd,构建过程独立运行
Rootless 非 root 用户也能构建镜像
无需 Dockerfile 可用 shell 脚本式构建(buildah runbuildah config
OCI 兼容 构建的镜像可直接用于 Docker/K8s
镜像外构建 构建工具不打包进镜像,体积更小
多阶段构建 支持,可显著减小最终镜像

我用它主要是想构建多架构的镜像,其实使用docker应该也可以,但是docker感觉更像一个临时性的工具,肯定是不如Buildah的

参考文档:Buildah

目前最新版是1.45.0,也可以通过release安装,应该是个rpm包或者deb包

部署

文档提到如果使用的是红帽9,直接通过yum就可以安装,不过我现在是WSL2环境,系统是ubuntu22.04 架构是x86(其实都可以直接安装

1
2
sudo apt-get update
sudo apt-get -y install buildah

主要功能

  • 构建镜像(dockerfile/交互式构建)
  • 镜像操作管理、容器实例管理(类似docker
  • 多架构构建 & Manifest 列表
  • 镜像合并成多架构(我当前的需求)
  • 镜像层操作、元数据操作
  • 权限与安全 可以使用rootless

镜像合并

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
现在我有两个镜像的tar包,位于/home/ws
csi-node-driver-registrar.tar.gz
csi-node-driver-registrar_arm.tar.gz

1.解压镜像
# 加载 x86 镜像
buildah pull docker-archive:///home/ws/csi-node-driver-registrar.tar.gz
# 此时本地标签: k8s-deploy/csi-node-driver-registrar:v2.8.0

# 给 x86 镜像额外打个本地标签,腾出原标签给 ARM
buildah tag k8s-deploy/csi-node-driver-registrar:v2.8.0 localhost/csi-x86:v2.8.0

# 加载 ARM 镜像(会覆盖原来的 v2.8.0 标签,但我们已经备份了)
buildah pull docker-archive:///home/ws/csi-node-driver-registrar_arm.tar.gz

# 给 ARM 镜像也打个本地标签
buildah tag k8s-deploy/csi-node-driver-registrar:v2.8.0 localhost/csi-arm64:v2.8.0

# 查看镜像
ws@DESKTOP-V15U79L:~$ buildah images | grep csi
localhost/k8s-deploy/csi-node-driver-registrar v2.8.0 83bf35b83767 3 years ago 21.8 MB
localhost/csi-arm64 v2.8.0 83bf35b83767 3 years ago 21.8 MB
localhost/csi-x86 v2.8.0 508684bf82d6 3 years ago 22.6 MB

2.创建 manifest list 与合并
buildah manifest create localhost/csi-multiarch:v2.8.0
buildah manifest add --arch amd64 localhost/csi-multiarch:v2.8.0 localhost/csi-x86:v2.8.0
buildah manifest add --arch arm64 localhost/csi-multiarch:v2.8.0 localhost/csi-arm64:v2.8.0

buildah manifest inspect localhost/csi-multiarch:v2.8.0
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 1891,
"digest": "sha256:47b5de1933101a10c98a29b7fbc7c0cfe7a43edeb499732a3aea218af2d3f9eb",
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 1891,
"digest": "sha256:3c6eac264cbebad6010a870a6065e7581bbba43645114932d7d437c0943fe9d4",
"platform": {
"architecture": "arm64",
"os": "linux"
}
}
]
}

3.重新tag与打包
buildah tag localhost/csi-multiarch:v2.8.0 k8s-deploy/csi-node-driver-registrar:v2.8.0
buildah images | grep 2.8
localhost/csi-multiarch v2.8.0 f9004fbb6c1e 2 minutes ago 1 KB
localhost/k8s-deploy/csi-node-driver-registrar v2.8.0 f9004fbb6c1e 2 minutes ago 1 KB
localhost/csi-arm64 v2.8.0 83bf35b83767 3 years ago 21.8 MB
localhost/csi-x86 v2.8.0 508684bf82d6 3 years ago 22.6 MB


buildah manifest push --all \
localhost/k8s-deploy/csi-node-driver-registrar:v2.8.0 \
oci-archive:/home/ws/csi-node-driver-registrar-v2.8.0-multiarch.tar

导出OCI标准的镜像,OCI标准才支持多架构,docker那个标准不支持多架构

# 解压与重打tag
buildah commit oci-archive-working-container csi-node-driver-registrar:v2.8.0
Getting image source signatures
Copying blob d0157aa0c95a skipped: already exists
Copying blob 6fbdf253bbc2 skipped: already exists
Copying blob 399826b51fcf skipped: already exists
Copying blob ff5700ec5418 skipped: already exists
Copying blob d52f02c6501c skipped: already exists
Copying blob e624a5370eca skipped: already exists
Copying blob 1a73b54f556b skipped: already exists
Copying blob d2d7ec0f6756 skipped: already exists
Copying blob 4cb10dd2545b skipped: already exists
Copying blob e9c86966ddc4 skipped: already exists
Copying blob 5f70bf18a086 done |
Copying config 15eff2b858 done |
Writing manifest to image destination
15eff2b858b83e7d9d641e6dd4c939626a6341f88370f21ef678621d810a6a8e

# 多架构直接推送
skopeo copy --tls-verify=false --multi-arch=all oci-archive:csi-node-driver-registrar-v2.8.0-multiarch.tar docker://xxx:8443/k8s-deploy/csi-node-driver-registrar:v2.8.0

多架构构建、交互式构建

多架构构建

之前用过Buildx,buildx需要一个builder,这个builder的镜像是moby/buildkit,需要代理拉取,而且用buildx构建的时候也需要代理,就非常恶心

而Buildah调用的是宿主机上已注册的binfmt_misc,内核看到ARM二进制文件时自动调用QEMU用户态解释器来执行。整个过程没有额外容器、没有额外镜像拉取

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# 原本的dockerfile,这个dockerfile是要做成两个架构的
cat Dockerfile
FROM alpine:3.18

# ===================== 构建代理(apk install 用) =====================
ARG HTTP_PROXY
ARG HTTPS_PROXY
ARG NO_PROXY

# ===================== 架构参数 =====================
ARG TARGETARCH=amd64

# 安装基础工具
RUN apk add --no-cache \
docker-cli \
coreutils \
grep

# 从本地 tarball 安装 nerdctl(构建前先下载对应架构的 tarball)
COPY nerdctl-2.1.2-linux-${TARGETARCH}.tar.gz /tmp/nerdctl.tar.gz
RUN tar -xzf /tmp/nerdctl.tar.gz -C /usr/local/bin/ nerdctl \
&& chmod +x /usr/local/bin/nerdctl \
&& rm /tmp/nerdctl.tar.gz \
&& nerdctl --version

# 复制清理脚本
COPY clean-disk.sh /scripts/clean-disk.sh
RUN chmod +x /scripts/clean-disk.sh

CMD ["/bin/sh", "/scripts/clean-disk.sh"]

交互式构建

交互式的真正价值不在于”能不能”,而在于调试和迭代效率(特别是在多架构场景下),因为构建的时候,不知道会出什么问题,比如可能apk add直接就报错了

构建 amd64

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
export HTTP_PROXY="http://192.168.10.238:7897"
export HTTPS_PROXY="http://192.168.10.238:7897"

# 创建 working container
buildah from --name disk-builder --arch amd64 docker.io/library/alpine:3.18

# 安装依赖
buildah run disk-builder apk add --no-cache docker-cli coreutils grep
buildah copy disk-builder nerdctl-2.1.2-linux-amd64.tar.gz /tmp/nerdctl.tar.gz
buildah run disk-builder sh -c "tar -xzf /tmp/nerdctl.tar.gz -C /usr/local/bin/ nerdctl && chmod +x /usr/local/bin/nerdctl && rm /tmp/nerdctl.tar.gz && nerdctl --version"

nerdctl version 2.1.2

# 放清理脚本
buildah copy disk-builder clean-disk.sh /scripts/clean-disk.sh
buildah run disk-builder chmod +x /scripts/clean-disk.sh

# 设置启动命令
buildah config --cmd '/bin/sh /scripts/clean-disk.sh' disk-builder

# 提交镜像
buildah commit disk-builder localhost/disk-cleaner:amd64

# 清理working container
buildah rm disk-builder

构建arm64

在构建之前,需要确认是否已经启用了解释器

我这里换了个系统,原本的almalinux9.6没法装解释器,换成了ubuntu 22.04 LTS

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# 确认解释器
cat /proc/sys/fs/binfmt_misc/status
enabled
ls /proc/sys/fs/binfmt_misc/
register status

# 可见没有解释器,需要安装
sudo apt update
sudo apt install -y qemu-user-static binfmt-support

ls /proc/sys/fs/binfmt_misc/
WSLInterop qemu-cris qemu-mips64 qemu-ppc64 qemu-sh4eb register
python3.10 qemu-hexagon qemu-mips64el qemu-ppc64le qemu-sparc status
qemu-aarch64 qemu-hppa qemu-mipsel qemu-riscv32 qemu-sparc32plus
qemu-alpha qemu-m68k qemu-mipsn32 qemu-riscv64 qemu-sparc64
qemu-arm qemu-microblaze qemu-mipsn32el qemu-s390x qemu-xtensa
qemu-armeb qemu-mips qemu-ppc qemu-sh4 qemu-xtensaeb

buildah from --name disk-builder --arch arm64 docker.io/library/alpine:3.18
buildah run disk-builder apk add --no-cache docker-cli coreutils grep
buildah run disk-builder apk add --no-cache nerdctl
buildah run disk-builder sh -c "nerdctl --version"
buildah config --cmd '/bin/sh /scripts/clean-disk.sh' disk-builder
buildah commit disk-builder localhost/disk-cleaner:arm64

buildah rm disk-builder

镜像合并

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
buildah images | grep disk
localhost/disk-cleaner arm64 4ae9def09560 About a minute ago 210 MB
localhost/disk-cleaner amd64 8d3f5c83e2c1 6 minutes ago 230 MB

# 创建manifest list
buildah manifest create localhost/disk-cleaner:latest
# 加入两个架构
buildah manifest add --arch amd64 localhost/disk-cleaner:latest localhost/disk-cleaner:amd64
buildah manifest add --arch arm64 localhost/disk-cleaner:latest localhost/disk-cleaner:arm64
# 验证
buildah manifest inspect localhost/disk-cleaner:latest
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"size": 992,
"digest": "sha256:b764074c41780efbc9b09455fabcbe5b93192030d91a8f4e959bd4ca2b2ae4a9",
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"size": 995,
"digest": "sha256:920f613a13e50cbef829e465eba8640309e88b96e601be69de937dfb45a62592",
"platform": {
"architecture": "arm64",
"os": "linux"
}
}
]
}
CATALOG
  1. 1. 镜像合并
  2. 2. 多架构构建、交互式构建
    1. 2.1. 多架构构建
    2. 2.2. 交互式构建
    3. 2.3. 镜像合并