Kubernetes中Nginx服務(wù)啟動失敗排查流程分析(Error:?ImagePullBackOff)
更新時間：2023年03月14日 14:27:20 作者：xybDIY
這篇文章主要介紹了Kubernetes中Nginx服務(wù)啟動失敗排查流程(Error:?ImagePullBackOff),本文給大家介紹的非常詳細(xì)，對大家的學(xué)習(xí)或工作具有一定的參考借鑒價值，需要的朋友可以參考下
?pod節(jié)點啟動失敗，nginx服務(wù)無法正常訪問，服務(wù)狀態(tài)顯示為ImagePullBackOff。
[root@m1 ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-f89759699-cgjgp 0/1 ImagePullBackOff 0 103m
??查看nginx服務(wù)的Pod節(jié)點詳細(xì)信息。
[root@m1 ~]# kubectl describe pod nginx-f89759699-cgjgp
Name: nginx-f89759699-cgjgp
Namespace: default
Priority: 0
Service Account: default
Node: n1/192.168.200.84
Start Time: Fri, 10 Mar 2023 08:40:33 +0800
Labels: app=nginx
pod-template-hash=f89759699
Annotations: <none>
Status: Pending
IP: 10.244.3.20
IPs:
IP: 10.244.3.20
Controlled By: ReplicaSet/nginx-f89759699
Containers:
nginx:
Container ID:
Image: nginx
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-zk8sj (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-zk8sj:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-zk8sj
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal BackOff 57m (x179 over 100m) kubelet Back-off pulling image "nginx"
Normal Pulling 7m33s (x22 over 100m) kubelet Pulling image "nginx"
Warning Failed 2m30s (x417 over 100m) kubelet Error: ImagePullBackOff
發(fā)現(xiàn)，獲取nginx鏡像失敗?？赡苁怯捎贒ocker服務(wù)引起的。
于是，檢查Docker是否正常啟動
systemctl status docker
發(fā)現(xiàn)，docker服務(wù)啟動失敗??，手動嘗試重新啟動。
systemctl restart docker
但是，重啟docker服務(wù)失敗，出現(xiàn)如下報錯信息。
[root@m1 ~]# systemctl restart docker Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
執(zhí)行systemctl restart docker命令失效。
接著，當(dāng)執(zhí)行docker version命令時，發(fā)現(xiàn)未能連接到Docker daemon
[root@m1 ~]# docker version Client: Docker Engine - Community Version: 20.10.17 API version: 1.41 Go version: go1.17.11 Git commit: 100c701 Built: Mon Jun 6 23:03:11 2022 OS/Arch: linux/amd64 Context: default Experimental: true Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
于是，再次通過執(zhí)行systemctl status docker命令，查看docker服務(wù)未能啟動，閱讀輸出報錯信息，如下所示。
[root@m1 ~]# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Fri 2023-03-10 10:28:16 CST; 4min 35s ago
Docs: https://docs.docker.com
Main PID: 2221 (code=exited, status=1/FAILURE)
Mar 10 10:28:13 m1 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Mar 10 10:28:13 m1 systemd[1]: docker.service: Failed with result 'exit-code'.
Mar 10 10:28:13 m1 systemd[1]: Failed to start Docker Application Container Engine.
Mar 10 10:28:16 m1 systemd[1]: docker.service: Service RestartSec=2s expired, scheduling restart.
Mar 10 10:28:16 m1 systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Mar 10 10:28:16 m1 systemd[1]: Stopped Docker Application Container Engine.
Mar 10 10:28:16 m1 systemd[1]: docker.service: Start request repeated too quickly.
Mar 10 10:28:16 m1 systemd[1]: docker.service: Failed with result 'exit-code'.
Mar 10 10:28:16 m1 systemd[1]: Failed to start Docker Application Container Engine.
[root@m1 ~]#通過上述輸出顯示，Docker 服務(wù)進(jìn)程的啟動失敗，狀態(tài)為 1/FAILURE。
?接下來，嘗試通過以下步驟來排查和解決問題：
1??查看 Docker 服務(wù)日志：使用以下命令查看 Docker 服務(wù)日志，以便更詳細(xì)地了解失敗原因。
sudo journalctl -u docker.service
2?? 通過輸出Ddocker日志分析，提取到了相關(guān)報錯信息片段，發(fā)現(xiàn)是配置daemon中的/etc/docker/daemon.json配置文件出錯導(dǎo)致的。
Mar 10 10:20:17 m1 systemd[1]: Starting Docker Application Container Engine... Mar 10 10:20:17 m1 dockerd[1572]: unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid character '"' after object key:value pair Mar 10 10:20:17 m1 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE Mar 10 10:20:17 m1 systemd[1]: docker.service: Failed with result 'exit-code'. Mar 10 10:20:17 m1 systemd[1]: Failed to start Docker Application Container Engine. Mar 10 10:20:19 m1 systemd[1]: docker.service: Service RestartSec=2s expired, scheduling restart. Mar 10 10:20:19 m1 systemd[1]: docker.service: Scheduled restart job, restart counter is at 2. Mar 10 10:20:19 m1 systemd[1]: Stopped Docker Application Container Engine.
3??此時，查看daemon配置文件/etc/docker/daemon.json是否配置正確。
[root@m1 ~]# cat /etc/docker/daemon.json
{
# 設(shè)置 Docker 鏡像的注冊表鏡像源為阿里云鏡像源。
"registry-mirrors": ["https://w2kavmmf.mirror.aliyuncs.com"]
# 指定 Docker 守護(hù)進(jìn)程使用 systemd 作為 cgroup driver。
"exec-opts": ["native.cgroupdriver=systemd"]
}咋一看，配置信息沒有什么問題，都是正確的，但仔細(xì)一看，就會發(fā)現(xiàn)應(yīng)該在"registry-mirrors"選項的結(jié)尾添加逗號。犯了缺少逗號（,）導(dǎo)致的語法錯誤，終于找到了問題根源。
??修改后：
[root@m1 ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://w2kavmmf.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"]
}
[root@m1 ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://w2kavmmf.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"]
}按下:wq報錯退出。
4?? 重新加載系統(tǒng)并重新啟動Docker服務(wù)
systemctl daemon-reload systemctl restart docker systemctl status docker
5??檢查docker版本信息是否輸出正常
[root@m1 ~]# docket version -bash: docket: command not found [root@m1 ~]# docker version Client: Docker Engine - Community Version: 20.10.17 API version: 1.41 Go version: go1.17.11 Git commit: 100c701 Built: Mon Jun 6 23:03:11 2022 OS/Arch: linux/amd64 Context: default Experimental: true Server: Docker Engine - Community Engine: Version: 20.10.17 API version: 1.41 (minimum version 1.12) Go version: go1.17.11 Git commit: a89b842 Built: Mon Jun 6 23:01:29 2022 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.6 GitCommit: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 runc: Version: 1.1.2 GitCommit: v1.1.2-0-ga916309 docker-init: Version: 0.19.0 GitCommit: de40ad0
[root@m1 ~]# docker info Client: Context: default Debug Mode: false Plugins: app: Docker App (Docker Inc., v0.9.1-beta3) buildx: Docker Buildx (Docker Inc., v0.8.2-docker) scan: Docker Scan (Docker Inc., v0.17.0) Server: Containers: 20 Running: 8 Paused: 0 Stopped: 12 Images: 20 Server Version: 20.10.17 Storage Driver: overlay2 Backing Filesystem: xfs Supports d_type: true Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc Default Runtime: runc Init Binary: docker-init containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1 runc version: v1.1.2-0-ga916309 init version: de40ad0 Security Options: seccomp Profile: default Kernel Version: 4.18.0-372.9.1.el8.x86_64 Operating System: Rocky Linux 8.6 (Green Obsidian) OSType: linux Architecture: x86_64 CPUs: 2 Total Memory: 9.711GiB Name: m1 ID: 4YIS:FHSB:YXRI:CED5:PJSJ:EAS2:BCR3:GJJF:FDPK:EDJH:DVKU:AIYJ Docker Root Dir: /var/lib/docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Registry Mirrors: https://w2kavmmf.mirror.aliyuncs.com/ Live Restore Enabled: false
至此，Docker服務(wù)重啟成功，pod節(jié)點恢復(fù)正常，Nginx服務(wù)能夠正常訪問。
[root@m1 ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-f89759699-cgjgp 1/1 Running 0 174m
查看pod詳細(xì)信息，顯示正常。
[root@m1 ~]# kubectl describe pod nginx-f89759699-cgjgp
Name: nginx-f89759699-cgjgp
Namespace: default
Priority: 0
Service Account: default
Node: n1/192.168.200.84
Start Time: Fri, 10 Mar 2023 08:40:33 +0800
Labels: app=nginx
pod-template-hash=f89759699
Annotations: <none>
Status: Running
IP: 10.244.3.20
IPs:
IP: 10.244.3.20
Controlled By: ReplicaSet/nginx-f89759699
Containers:
nginx:
Container ID: docker://88bdc2bfa592f60bf99bac2125b0adae005118ae8f2f271225245f20b7cfb3c8
Image: nginx
Image ID: docker-pullable://nginx@sha256:aa0afebbb3cfa473099a62c4b32e9b3fb73ed23f2a75a65ce1d4b4f55a5c2ef2
Port: <none>
Host Port: <none>
State: Running
Started: Fri, 10 Mar 2023 10:37:42 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-zk8sj (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-zk8sj:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-zk8sj
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal BackOff 58m (x480 over 171m) kubelet Back-off pulling image "nginx"
[root@m1 ~]# 
到此這篇關(guān)于Kubernetes中Nginx服務(wù)啟動失敗排查流程(Error: ImagePullBackOff)的文章就介紹到這了,更多相關(guān)Nginx服務(wù)啟動失敗內(nèi)容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！
相關(guān)文章
Nginx反向代理location和proxy_pass配置規(guī)則詳細(xì)總結(jié)
nginx代理訪問很好用,但是好多人不清楚location和proxy_pass組合在一起使用時訪問的url被代理的url真實地址是什么,下面這篇文章主要給大家介紹了關(guān)于Nginx反向代理location和proxy_pass配置規(guī)則的相關(guān)資料,需要的朋友可以參考下2022-09-09
nginx proxy_buffer_size解決后端服務(wù)傳輸數(shù)據(jù)過多,header過大問題
這篇文章主要介紹了nginx proxy_buffer_size解決后端服務(wù)傳輸數(shù)據(jù)過多,header過大問題,具有很好的參考價值,希望對大家有所幫助,如有錯誤或未考慮完全的地方,望不吝賜教2023-12-12
在現(xiàn)代的Web開發(fā)中,跨域訪問是一種常見的需求,由于瀏覽器的同源策略,不同域名之間的訪問存在一定的限制,本文將介紹如何使用Nginx來解決跨域訪問的問題,并通過一個完整的實例來展示,需要的朋友可以參考下2024-03-03
這篇文章主要介紹了Nginx大并發(fā)優(yōu)化實戰(zhàn)，文中通過示例代碼介紹的非常詳細(xì)，對大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價值，需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2019-10-10
這篇文章主要介紹了Nginx反向代理多個服務(wù)器的實現(xiàn)方法，文中通過示例代碼介紹的非常詳細(xì)，對大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價值，需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2021-02-02
使用Nginx的過程中,我們可能總是需要修改nginx配置文件,然后不停地啟動或者停止nginx服務(wù),本文就來介紹一下,感興趣的可以了解一下2023-11-11
深入探究Nginx體系化之虛擬主機(jī)分類及配置實現(xiàn)
Nginx,這款備受推崇的高性能 Web 服務(wù)器,以其強(qiáng)大的性能和靈活的配置而廣受歡迎,在實際應(yīng)用中,虛擬主機(jī)是一項重要的功能,允許我們在單個服務(wù)器上托管多個網(wǎng)站,本文將深入探討 Nginx 虛擬主機(jī)的分類和配置實現(xiàn),幫助您構(gòu)建一個高效多站點托管平臺2023-08-08
Nginx前端頁面刷新后出現(xiàn)404的原因與解決方案
這篇文章主要來為大家詳細(xì)介紹一下Nginx前端頁面刷新后出現(xiàn)404 Not Found的原因與對應(yīng)的解決方案,有需要的小伙伴可以根據(jù)情況參考一下2025-03-03
nginx是一個開源的，支持高性能，高并發(fā)的www服務(wù)和代理服務(wù)軟件。它是一個俄羅斯人lgor sysoev開發(fā)的，作者將源代碼開源出來供全球使用，下面小編給大家?guī)砹薲jango8.5?項目部署Nginx的操作步驟，感興趣的朋友一起看看吧2022-01-01
本文主要介紹了nginx代理postgresql的實現(xiàn)示例,文中通過示例代碼介紹的非常詳細(xì),對大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價值,需要的朋友們下面隨著小編來一起學(xué)習(xí)學(xué)習(xí)吧2023-10-10