Error loading cached network config network podman not found in cni cache

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line) /kind bug Description Failed to load rootfull container in ubuntu WSL2 with following error message: $ sudo podman run -dt ...

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Failed to load rootfull container in ubuntu WSL2 with following error message:

$ sudo podman run -dt --name webserver -p 8080:80 quay.io/libpod/banner

ERRO[0000] error loading cached network config: network "podman" not found in CNI cache
WARN[0000] falling back to loading from existing plugins on disk
ERRO[0000] Error tearing down partially created network namespace for container 4efd9483fda0dbce0391dca8b2defce4c37fbfb5824d26b7ece89093d2cee126: error removing pod webserver_webserver from CNI network "podman": running [/usr/sbin/iptables -t nat -D POSTROUTING -s 10.88.2.3 -j CNI-38c2f017994ad2e35c286de0 -m comment --comment name: "podman" id: "4efd9483fda0dbce0391dca8b2defce4c37fbfb5824d26b7ece89093d2cee126" --wait]: exit status 2: iptables v1.8.7 (nf_tables): Couldn't load match `comment':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.
Error: error configuring network namespace for container 4efd9483fda0dbce0391dca8b2defce4c37fbfb5824d26b7ece89093d2cee126: error adding pod webserver_webserver to CNI network "podman": running [/usr/sbin/iptables -t nat -C CNI-38c2f017994ad2e35c286de0 -d 10.88.2.3/24 -j ACCEPT -m comment --comment name: "podman" id: "4efd9483fda0dbce0391dca8b2defce4c37fbfb5824d26b7ece89093d2cee126" --wait]: exit status 2: iptables v1.8.7 (nf_tables): Couldn't load match `comment':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

$ sudo podman ps

CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES

Steps to reproduce the issue:

  1. Upgrade to Ubuntu 22.04 and install podman:
sudo apt-get -y install podman
sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 <user>
sudo chmod 4755 /usr/bin/newgidmap
sudo chmod 4755 /usr/bin/newuidmap
  1. run a rootfull container
sudo podman run -dt --name webserver -p 8080:80 quay.io/libpod/banner

Describe the results you received:

ERRO[0000] error loading cached network config: network "podman" not found in CNI cache
WARN[0000] falling back to loading from existing plugins on disk
ERRO[0000] Error tearing down partially created network namespace for container 4efd9483fda0dbce0391dca8b2defce4c37fbfb5824d26b7ece89093d2cee126: error removing pod webserver_webserver from CNI network "podman": running [/usr/sbin/iptables -t nat -D POSTROUTING -s 10.88.2.3 -j CNI-38c2f017994ad2e35c286de0 -m comment --comment name: "podman" id: "4efd9483fda0dbce0391dca8b2defce4c37fbfb5824d26b7ece89093d2cee126" --wait]: exit status 2: iptables v1.8.7 (nf_tables): Couldn't load match `comment':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.
Error: error configuring network namespace for container 4efd9483fda0dbce0391dca8b2defce4c37fbfb5824d26b7ece89093d2cee126: error adding pod webserver_webserver to CNI network "podman": running [/usr/sbin/iptables -t nat -C CNI-38c2f017994ad2e35c286de0 -d 10.88.2.3/24 -j ACCEPT -m comment --comment name: "podman" id: "4efd9483fda0dbce0391dca8b2defce4c37fbfb5824d26b7ece89093d2cee126" --wait]: exit status 2: iptables v1.8.7 (nf_tables): Couldn't load match `comment':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

Describe the results you expected:

$ docker ps

CONTAINER ID  IMAGE                         COMMAND               CREATED         STATUS             PORTS                 NAMES
XXXXXXXXXXXX  quay.io/libpod/banner:latest  nginx -g daemon o...  12 seconds ago  Up 12 seconds ago  0.0.0.0:8080->80/tcp  webserver

$ curl localhost:8080

<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

Additional information you deem important (e.g. issue happens only occasionally):

  • WSL2 Ubuntu 22.04 LTS version
  • Rootless command works:
$ podman run -dt --name webserver -p 8080:80 quay.io/libpod/banner

$ podman ps
CONTAINER ID  IMAGE                         COMMAND               CREATED        STATUS            PORTS                 NAMES
8002d44668c1  quay.io/libpod/banner:latest  nginx -g daemon o...  7 minutes ago  Up 7 minutes ago  0.0.0.0:8080->80/tcp  webserver

$ curl localhost:8080

<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

Output of podman version:

Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.17.3
Built:        Thu Jan  1 07:30:00 1970
OS/Arch:      linux/amd64

Output of podman info --debug:

sudo podman info --debug
host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - cpuset
  - cpu
  - cpuacct
  - blkio
  - memory
  - devices
  - freezer
  - net_cls
  - perf_event
  - net_prio
  - hugetlb
  - pids
  - rdma
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/bin/conmon'
    path: /usr/bin/conmon
    version: 'conmon version 2.0.25, commit: unknown'
  cpus: 4
  distribution:
    codename: jammy
    distribution: ubuntu
    version: "22.04"
  eventLogger: file
  hostname: myPortege
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.10.102.1-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: journald
  memFree: 282423296
  memTotal: 1505017856
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.17
      commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
      slirp4netns version 1.0.1
      commit: 6a7b16babc95b6a3056b33fb45b74a6f62262dd4
      libslirp: 4.6.1
  swapFree: 524288000
  swapTotal: 524288000
  uptime: 43m 34.14s
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries: {}
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 0
  BuiltTime: Thu Jan  1 07:30:00 1970
  GitCommit: ""
  GoVersion: go1.17.3
  OsArch: linux/amd64
  Version: 3.4.4

Package info (e.g. output of rpm -q podman or apt list podman):

$ apt list podman

Listing... Done
podman/jammy,now 3.4.4+ds1-1ubuntu1 amd64 [installed]

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

(3.4.4 is the latest version in the ubuntu apt package manager)

Additional environment details (AWS, VirtualBox, physical, etc.):

  • Windows 10 laptop
Microsoft Windows
Version 21H2 (OS Build 19044.1645)
  • WSL2 Ubuntu 22.04 LTS version
Distributor ID: Ubuntu
Description:    Ubuntu 22.04 LTS
Release:        22.04
Codename:       jammy

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

after installed podman version 3.3.1 on my archlinux, when run basic port mapping command like:

podman run -itd -p 8080:80 nginx

such error occurs:

ERRO[0000] error loading cached network config: network "podman" not found in CNI cache
WARN[0000] falling back to loading from existing plugins on disk
Error: error configuring network namespace for container caaac1893d98d09f60053f1c3f49c598ba77bbb7c5e54319bcf07c92688bfd1a: error adding pod pensive_dirac_pensive_dirac to CNI network "podman": unable to create chain CNI-HOSTPORT-SETMARK: running [/usr/sbin/iptables -t nat -C CNI-HOSTPORT-MASQ -m mark --mark 0x2000/0x2000 -j MASQUERADE --wait]: exit status 2: iptables v1.8.7 (legacy): Couldn't load match `mark':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

Steps to reproduce the issue:

1.install podman version 3.3.1

2.run podman run -itd -p 8080:80 nginx as root

Describe the results you received:
error described above occurs

Describe the results you expected:
container runs normally

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.22.3
  cgroupControllers:
  - cpuset
  - cpu
  - cpuacct
  - blkio
  - memory
  - devices
  - freezer
  - net_cls
  - perf_event
  - net_prio
  - hugetlb
  - pids
  - rdma
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: /usr/bin/conmon is owned by conmon 1:2.0.30-1
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: 2792c16f4436f1887a7070d9ad99d9c29742f38a'
  cpus: 8
  distribution:
    distribution: arch
    version: unknown
  eventLogger: file
  hostname: DESKTOP-C81EBT5
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.10.16.3-microsoft-standard-WSL2
  linkmode: dynamic
  memFree: 26536742912
  memTotal: 26827276288
  ociRuntime:
    name: crun
    package: /usr/bin/crun is owned by crun 1.0-2
    path: /usr/bin/crun
    version: |-
      crun version 1.0
      commit: 139dc6971e2f1d931af520188763e984d6cdfbf8
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/sbin/slirp4netns
    package: /usr/bin/slirp4netns is owned by slirp4netns 1.1.12-1
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 7516192768
  swapTotal: 7516192768
  uptime: 12m 28.9s
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 6
    paused: 0
    running: 1
    stopped: 5
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.3.1
  Built: 1630517266
  BuiltTime: Thu Sep  2 01:27:46 2021
  GitCommit: 4c5283fabff2de5145838f1847a5a7b2b1fbc0a5-dirty
  GoVersion: go1.17
  OsArch: linux/amd64
  Version: 3.3.1

Package info (e.g. output of rpm -q podman or apt list podman):

Name            : podman
Version         : 3.3.1-1
Description     : Tool and library for running OCI-based containers in pods
Architecture    : x86_64
URL             : https://github.com/containers/podman
Licenses        : Apache
Groups          : None
Provides        : None
Depends On      : cni-plugins  conmon  containers-common  device-mapper  iptables  libseccomp  crun  slirp4netns
                  libsystemd  fuse-overlayfs  libgpgme.so=11-64
Optional Deps   : podman-docker: for Docker-compatible CLI
                  btrfs-progs: support btrfs backend devices
                  catatonit: --init flag support
                  crun: support for unified cgroupsv2 [installed]
Required By     : None
Optional For    : None
Conflicts With  : None
Replaces        : None
Installed Size  : 71.85 MiB
Packager        : Morten Linderud <foxboron@archlinux.org>
Build Date      : Thu 02 Sep 2021 01:27:46 AM CST
Install Date    : Mon 27 Sep 2021 10:22:48 AM CST
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : SHA-256 Sum

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):
archlinux with wsl2 on windows.

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Running podman with sudo returns error loading cached network config: network "podman" not found in CNI cache
Steps to reproduce the issue:

  1. run sudo podman run -it fedora /bin/bash

Describe the results you received:

ERRO[0000] error loading cached network config: network "podman" not found in CNI cache 
WARN[0000] falling back to loading from existing plugins on disk 
ERRO[0000] Error tearing down partially created network namespace for container 687cab588473a65286b9f397bb03af56e5ff17995841b5667943099fef72e83d: CNI network "podman" not found 
Error: error configuring network namespace for container 687cab588473a65286b9f397bb03af56e5ff17995841b5667943099fef72e83d: CNI network "podman" not found

Describe the results you expected:
Podman should create container

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.16.8
Built:        Wed Dec  8 21:45:07 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 16
  distribution:
    distribution: fedora
    variant: silverblue
    version: "35"
  eventLogger: journald
  hostname: thinkxps-sb
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.15.8-200.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 28557361152
  memTotal: 33364307968
  ociRuntime:
    name: crun
    package: crun-1.3-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.3
      commit: 8e5757a4e68590326dafe8a8b1b4a584b10a1370
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 4m 37.41s
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /var/home/jaydoubleu/.config/containers/storage.conf
  containerStore:
    number: 213
    paused: 0
    running: 0
    stopped: 213
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/jaydoubleu/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 75
  runRoot: /run/user/1000/containers
  volumePath: /var/home/jaydoubleu/.local/share/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 1638999907
  BuiltTime: Wed Dec  8 21:45:07 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 3.4.4

Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.4.4-1.fc35.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

[jaydoubleu@thinkxps-sb ~]$ rpm-ostree status
State: idle
Deployments:
● fedora:fedora/35/x86_64/silverblue
                   Version: 35.20211218.0 (2021-12-18T00:47:07Z)
                BaseCommit: a5b83a07467600af1170e04dfa819ed2faddd1c130b05bcb40074504c0d013c6
              GPGSignature: Valid signature by 787EA6AE1147EEE56C40B30CDB4639719867C58F
           LayeredPackages: git kata-containers lorax virt-manager
[jaydoubleu@thinkxps-sb ~]$ sudo podman info --debug
host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 16
  distribution:
    distribution: fedora
    variant: silverblue
    version: "35"
  eventLogger: journald
  hostname: thinkxps-sb
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.15.8-200.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 28062343168
  memTotal: 33364307968
  ociRuntime:
    name: crun
    package: crun-1.3-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.3
      commit: 8e5757a4e68590326dafe8a8b1b4a584b10a1370
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 6m 6.9s
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 38
    paused: 0
    running: 0
    stopped: 38
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 19
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 1638999907
  BuiltTime: Wed Dec  8 21:45:07 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 3.4.4

[jaydoubleu@thinkxps-sb ~]$ sudo podman run --log-level=DEBUG -it fedora /bin/bash
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called run.PersistentPreRunE(podman run --log-level=DEBUG -it fedora /bin/bash) 
DEBU[0000] Merged system config "/usr/share/containers/containers.conf" 
DEBU[0000] Using conmon: "/usr/bin/conmon"              
DEBU[0000] Initializing boltdb state at /var/lib/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver overlay                   
DEBU[0000] Using graph root /var/lib/containers/storage 
DEBU[0000] Using run root /run/containers/storage       
DEBU[0000] Using static dir /var/lib/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/libpod                    
DEBU[0000] Using volume path /var/lib/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that metacopy is being used 
DEBU[0000] Cached value indicated that native-diff is not being used 
INFO[0000] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled 
DEBU[0000] backingFs=btrfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=true 
DEBU[0000] Initializing event backend journald          
DEBU[0000] configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument 
DEBU[0000] configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] Using OCI runtime "/usr/bin/crun"            
DEBU[0000] Default CNI network name podman is unchangeable 
INFO[0000] Setting parallel job count to 49             
DEBU[0000] Pulling image fedora (policy: missing)       
DEBU[0000] Looking up image "fedora" in local containers storage 
DEBU[0000] Trying "fedora" ...                          
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf" 
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/000-shortnames.conf" 
DEBU[0000] Trying "registry.fedoraproject.org/fedora:latest" ... 
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage ([overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae) 
DEBU[0000] Looking up image "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Trying "registry.fedoraproject.org/fedora:latest" ... 
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Found image "registry.fedoraproject.org/fedora:latest" as "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Found image "registry.fedoraproject.org/fedora:latest" as "registry.fedoraproject.org/fedora:latest" in local containers storage ([overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae) 
DEBU[0000] Looking up image "fedora" in local containers storage 
DEBU[0000] Trying "fedora" ...                          
DEBU[0000] Trying "registry.fedoraproject.org/fedora:latest" ... 
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage ([overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae) 
DEBU[0000] Inspecting image 3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Looking up image "fedora" in local containers storage 
DEBU[0000] Trying "fedora" ...                          
DEBU[0000] Trying "registry.fedoraproject.org/fedora:latest" ... 
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage ([overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae) 
DEBU[0000] Inspecting image 3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Inspecting image 3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae 
DEBU[0000] using systemd mode: false                    
DEBU[0000] Adding exposed ports                         
DEBU[0000] No hostname set; container's hostname will default to runtime default 
DEBU[0000] Loading seccomp profile from "/usr/share/containers/seccomp.json" 
DEBU[0000] Allocated lock 39 for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c 
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] created container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" 
DEBU[0000] container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" has work directory "/var/lib/containers/storage/overlay-containers/d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c/userdata" 
DEBU[0000] container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" has run directory "/run/containers/storage/overlay-containers/d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c/userdata" 
DEBU[0000] Handling terminal attach                     
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Made network namespace at /run/netns/cni-3342c823-7f24-71e6-c412-90ac04706be2 for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c 
DEBU[0000] Cached value indicated that metacopy is being used 
DEBU[0000] backingFs=btrfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=true 
DEBU[0000] overlay: mount_data=nodev,metacopy=on,lowerdir=/var/lib/containers/storage/overlay/l/3SBCDDB3TZMKMTS6VRBBO7WLZM,upperdir=/var/lib/containers/storage/overlay/4b964b2e2543e2ee68ce7160e2801637a3de9323f91355cdf2c0064035e398e0/diff,workdir=/var/lib/containers/storage/overlay/4b964b2e2543e2ee68ce7160e2801637a3de9323f91355cdf2c0064035e398e0/work,context="system_u:object_r:container_file_t:s0:c458,c576" 
DEBU[0000] mounted container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" at "/var/lib/containers/storage/overlay/4b964b2e2543e2ee68ce7160e2801637a3de9323f91355cdf2c0064035e398e0/merged" 
DEBU[0000] Created root filesystem for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c at /var/lib/containers/storage/overlay/4b964b2e2543e2ee68ce7160e2801637a3de9323f91355cdf2c0064035e398e0/merged 
INFO[0000] Got pod network &{Name:nifty_khayyam Namespace:nifty_khayyam ID:d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c NetNS:/run/netns/cni-3342c823-7f24-71e6-c412-90ac04706be2 Networks:[{Name:podman Ifname:eth0}] RuntimeConfig:map[podman:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]} 
DEBU[0000] Default CNI network name podman is unchangeable 
INFO[0000] Got pod network &{Name:nifty_khayyam Namespace:nifty_khayyam ID:d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c NetNS:/run/netns/cni-3342c823-7f24-71e6-c412-90ac04706be2 Networks:[{Name:podman Ifname:eth0}] RuntimeConfig:map[podman:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]} 
ERRO[0000] error loading cached network config: network "podman" not found in CNI cache 
WARN[0000] falling back to loading from existing plugins on disk 
DEBU[0000] Default CNI network name podman is unchangeable 
ERRO[0000] Error tearing down partially created network namespace for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c: CNI network "podman" not found 
DEBU[0000] unmounted container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" 
DEBU[0000] Network is already cleaned up, skipping...   
DEBU[0000] Cleaning up container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c 
DEBU[0000] Network is already cleaned up, skipping...   
DEBU[0000] Container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c storage is already unmounted, skipping... 
DEBU[0000] ExitCode msg: "error configuring network namespace for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c: cni network "podman" not found" 
Error: error configuring network namespace for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c: CNI network "podman" not found

jaydoubleu@thinkxps-sb ~]$ sudo podman --log-level=DEBUG network ls
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called ls.PersistentPreRunE(podman --log-level=DEBUG network ls) 
DEBU[0000] Merged system config "/usr/share/containers/containers.conf" 
DEBU[0000] Using conmon: "/usr/bin/conmon"              
DEBU[0000] Initializing boltdb state at /var/lib/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver overlay                   
DEBU[0000] Using graph root /var/lib/containers/storage 
DEBU[0000] Using run root /run/containers/storage       
DEBU[0000] Using static dir /var/lib/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/libpod                    
DEBU[0000] Using volume path /var/lib/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that metacopy is being used 
DEBU[0000] Cached value indicated that native-diff is not being used 
INFO[0000] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled 
DEBU[0000] backingFs=btrfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=true 
DEBU[0000] Initializing event backend journald          
DEBU[0000] configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument 
DEBU[0000] configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] Using OCI runtime "/usr/bin/crun"            
DEBU[0000] Default CNI network name podman is unchangeable 
INFO[0000] Setting parallel job count to 49             
NETWORK ID  NAME        VERSION     PLUGINS
DEBU[0000] Called ls.PersistentPostRunE(podman --log-level=DEBUG network ls) 


Description


morgan read



2021-08-29 09:30:58 UTC

Created attachment 1818815 [details]
output of terminal sessons leading to errors

Created attachment 1818815 [details]
output of terminal sessons leading to errors
[I probably included more info than necessary - useful dnf update info starts at l 698 and useful podman info at l 3458]

Description of problem:
dnf upgrade seems to have broken podman

Version-Release number of selected component (if applicable):
podman-3.3.1-3.module_el8.5.0+911+f19012f9.x86_64
podman-3.3.0-0.17.module_el8.5.0+874+6db8bee3.x86_64

How reproducible:
Always broken since upgrade

Steps to Reproduce:
1. dnf upgrade
2. # podman container start X
   # podman pod start Y
3.
ERRO[0000] error loading cached network config: network "podman" not found in CNI cache 
WARN[0000] falling back to loading from existing plugins on disk 
ERRO[0000] Error tearing down partially created network namespace for container 2d2be3756348540e90e48474cd47006e5ac901af50ef30434d1795a5169c41ee: CNI network "podman" not found 
Error: unable to start container "2d2be3756348540e90e48474cd47006e5ac901af50ef30434d1795a5169c41ee": error configuring network namespace for container 2d2be3756348540e90e48474cd47006e5ac901af50ef30434d1795a5169c41ee: CNI network "podman" not found

ERRO[0000] error loading cached network config: network "podman" not found in CNI cache 
WARN[0000] falling back to loading from existing plugins on disk 
ERRO[0000] Error tearing down partially created network namespace for container c2785c5e43b0b9829de0e318655041683b5e46c4caf81d601e58c828b1963266: CNI network "podman" not found 
Error: error starting container c2785c5e43b0b9829de0e318655041683b5e46c4caf81d601e58c828b1963266: error configuring network namespace for container c2785c5e43b0b9829de0e318655041683b5e46c4caf81d601e58c828b1963266: CNI network "podman" not found
Error: error starting container 15d9fdbe53daea84929b76c4d325803341962f32b9832c250c140207e200751d: a dependency of container 15d9fdbe53daea84929b76c4d325803341962f32b9832c250c140207e200751d failed to start: container state improper
Error: error starting container 54fc0cc8f0829c30b17dcb500ad5060247808be2c2f21c3525586d24f85a57b4: a dependency of container 54fc0cc8f0829c30b17dcb500ad5060247808be2c2f21c3525586d24f85a57b4 failed to start: container state improper

Actual results:
As above

Expected results:
containers start

Additional info:
I came back to following this howto:
https://blog.while-true-do.io/podman-systemd-container-management/
From 'Recreate' - after an absence of a week or two - and thought to do a dnf upgrade before proceeding further (it seems from the attached output that I may have been prompted by wanting to install gedit for some reason - perhaps I initially wanted to carry out the investigation I did on the dnf errors with meld, with gedit instead)
Unfortunately, doing the dnf upgrade seems to have broken podman - which has been a bit of an anticlimax...  Especially as podman is responsible for playing all the audio in the house...


Comment 1


morgan read



2021-08-29 09:48:13 UTC

[root@frontserver ~]# podman info --debug
host:
  arch: amd64
  buildahVersion: 1.22.3
  cgroupControllers:
  - cpuset
  - cpu
  - cpuacct
  - blkio
  - memory
  - devices
  - freezer
  - net_cls
  - perf_event
  - net_prio
  - hugetlb
  - pids
  - rdma
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: conmon-2.0.29-1.module_el8.5.0+890+6b136101.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: 84384406047fae626269133e1951c4b92eed7603'
  cpus: 2
  distribution:
    distribution: '"centos"'
    version: "8"
  eventLogger: file
  hostname: frontserver.lan
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 4.18.0-305.12.1.el8_4.centos.plus.x86_64
  linkmode: dynamic
  memFree: 582135808
  memTotal: 3350163456
  ociRuntime:
    name: runc
    package: runc-1.0.2-1.module_el8.5.0+911+f19012f9.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.2
      spec: 1.0.2-dev
      go: go1.16.7
      libseccomp: 2.5.1
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_NET_RAW,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.8-1.module_el8.5.0+890+6b136101.x86_64
    version: |-
      slirp4netns version 1.1.8
      commit: d361001f495417b880f20329121e3aa431a8f90f
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.1
  swapFree: 2145632256
  swapTotal: 2147479552
  uptime: 12h 1m 29.09s (Approximately 0.50 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 4
    paused: 0
    running: 0
    stopped: 4
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 6
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.3.1-dev
  Built: 1630096035
  BuiltTime: Fri Aug 27 21:27:15 2021
  GitCommit: ""
  GoVersion: go1.16.7
  OsArch: linux/amd64
  Version: 3.3.1-dev

[root@frontserver ~]#


Comment 2


Daniel Walsh



2021-08-29 13:26:35 UTC

I have the same situation on my Fedora 34 system.


Comment 3


Paul Holzinger



2021-08-30 11:18:52 UTC

I can reproduce this as well. Can you run `sudo rm /var/lib/containers/storage/libpod/defaultCNINetExists`, this should fix it for now.


Comment 4


morgan read



2021-08-30 12:32:09 UTC

Thanks Paul, works for now.


Comment 5


Matthew LeSieur



2022-01-06 13:40:04 UTC

I ran into this bug too on a RHEL 8.5 server.  The bug likely happened when podman was upgraded to 3.3.1-9, but I'm not certain because the container on this server is rarely used. I resolved the problem by removing /var/lib/containers/storage/libpod/defaultCNINetExists as per https://github.com/containers/podman/issues/12651#issuecomment-997394699

# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.5 (Ootpa)

# rpm -qa --last | grep podman
podman-3.3.1-9.module+el8.5.0+12697+018f24d7.x86_64 Wed 08 Dec 2021 11:31:41 AM EST
podman-catatonit-3.3.1-9.module+el8.5.0+12697+018f24d7.x86_64 Wed 08 Dec 2021 11:31:39 AM EST


Comment 6


Paul Holzinger



2022-02-24 13:17:34 UTC

This should be fixed permanently with podman v4.0 since it will always have the default network in memory.
I am not sure how exactly this happened but it I think this was a packaging bug with podman v3.3. It should also work with 3.4.


Comment 12


Alex Jia



2022-03-03 02:59:18 UTC

This bug has been verified on podman-4.0.2-1.module+el8.6.0+14379+4ec2a99a.


[root@sweetpig-21 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.6 Beta (Ootpa)

[root@sweetpig-21 ~]# rpm -q podman runc crun criu systemd kernel
podman-4.0.2-1.module+el8.6.0+14379+4ec2a99a.x86_64
runc-1.0.3-2.module+el8.6.0+14379+4ec2a99a.x86_64
crun-1.4.2-1.module+el8.6.0+14379+4ec2a99a.x86_64
criu-3.15-3.module+el8.6.0+14379+4ec2a99a.x86_64
systemd-239-58.el8.x86_64
kernel-4.18.0-369.el8.x86_64

[root@sweetpig-21 ~]# podman create registry.access.redhat.com/ubi8 bash
Trying to pull registry.access.redhat.com/ubi8:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob 8dfe9326f733 done
Copying blob 0d875a68bf99 done
Copying config 52de04277b done
Writing manifest to image destination
Storing signatures
50128433a31387c23b02ea2b589107d6aeb53899b9b27f23fea9b37702f24ab3

[root@sweetpig-21 ~]# podman ps -a
CONTAINER ID  IMAGE                                   COMMAND     CREATED        STATUS      PORTS       NAMES
50128433a313  registry.access.redhat.com/ubi8:latest  bash        5 seconds ago  Created                 silly_mestorf

[root@sweetpig-21 ~]# podman container start 50128433a313
50128433a313

[root@sweetpig-21 ~]# podman ps -a
CONTAINER ID  IMAGE                                   COMMAND     CREATED        STATUS                    PORTS       NAMES
50128433a313  registry.access.redhat.com/ubi8:latest  bash        8 minutes ago  Exited (0) 8 seconds ago              silly_mestorf

[root@sweetpig-21 ~]# podman pod create
f5cdee4a0f513638f5be5c17f019d48b5cb1d6d91abcf0a593b6e4890d541c02

[root@sweetpig-21 ~]# podman pod ps
POD ID        NAME           STATUS      CREATED         INFRA ID      # OF CONTAINERS
f5cdee4a0f51  cranky_mclean  Created     23 seconds ago  41321b9c7f78  1

[root@sweetpig-21 ~]# podman pod start f5cdee4a0f51
f5cdee4a0f513638f5be5c17f019d48b5cb1d6d91abcf0a593b6e4890d541c02

[root@sweetpig-21 ~]# podman pod ps
POD ID        NAME           STATUS      CREATED             INFRA ID      # OF CONTAINERS
f5cdee4a0f51  cranky_mclean  Running     About a minute ago  41321b9c7f78  1


Comment 14


errata-xmlrpc



2022-05-10 13:27:31 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: container-tools:rhel8 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1762

Prerequisites

I am trying to migrate from docker to podman for production use of some containers. Some of my containers are in need to be assigned IP adresses from a DHCP server in my network, as such I used docker-net-dhcp.

Now, as I understand, while it cannot be used in docker to do this, the macvlan driver in podman can.

Steps to reproduce

Here are the steps I have done so far:

  1. Ensure my (physical) network interface exists:
    ip addr show enp7s0
    2: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group 
     default qlen 1000
    link/ether 50:e5:49:36:fd:ee brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.115/24 brd 10.0.1.255 scope global dynamic enp7s0
       valid_lft 59635sec preferred_lft 59635sec
    inet 10.0.1.22/24 brd 10.0.1.255 scope global secondary dynamic enp7s0:1002
       valid_lft 59635sec preferred_lft 59635sec
    [ipv6 ommitted]
    
  2. Create the podman network:
    sudo podman network create -d macvlan -o parent=enp7s0 newnet
    

    and verify

    sudo podman network inspect newnet
    [
        {
            "name": "newnet",
            "id": "884e74728f045cb5209bbaf5e5cf35f1ab7640b11e74ec988fc08ddf97575cf7",
            "driver": "macvlan",
            "network_interface": "enp7s0",
            "created": "2022-06-19T16:45:29.660485744+02:00",
            "ipv6_enabled": false,
            "internal": false,
            "dns_enabled": false,
            "ipam_options": {
                "driver": "dhcp"
            }
        }
    ]
    
  3. Try setting up a test container from a minimal image, and attach it to the network newnet and ask for its ip address:
    sudo podman run -it --rm --net newnet alpine ip addr
    WARN[0000] Failed to load cached network config: network newnet not found in CNI cache, falling back to loading network newnet from disk
    WARN[0000] 1 error occurred:
     * plugin type="macvlan" failed (delete): cni plugin macvlan failed: error dialing DHCP daemon: dial unix /run/cni/dhcp.sock: connect: no such file or directory
    
    Error: plugin type="macvlan" failed (add): cni plugin macvlan failed: error dialing DHCP daemon: dial unix /run/cni/dhcp.sock: connect: no such file or directory
    

Problem Statement

While I obviously can see what went wrong here, I don’t have any idea how to fix it. Can you point me in the right direction?

Additional Information

Debian version

sudo lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 11 (bullseye)
Release: 11
Codename:       bullseye

Installed Packages (shortened to relevant ones)

sudo apt list --installed
Listing... Done
containerd.io/bullseye,now 1.6.6-1 amd64 [installed]
containernetworking-plugins/unknown,now 100:1.1.1-1 amd64 [installed]
containernetworking/unknown,now 100:1.1.1-1 amd64 [installed,automatic]
containers-common/unknown,now 100:0.48.0-1 amd64 [installed,automatic]
podman/unknown,now 100:4.1.1-1 amd64 [installed]
python3-podman-compose/unknown,now 100:1.0.3-1 all [installed]
python3/stable,now 3.9.2-3 amd64 [installed]

Podman

sudo podman --version
podman version 4.1.1

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Running podman with sudo returns error loading cached network config: network "podman" not found in CNI cache
Steps to reproduce the issue:

  1. run sudo podman run -it fedora /bin/bash

Describe the results you received:

ERRO[0000] error loading cached network config: network "podman" not found in CNI cache 
WARN[0000] falling back to loading from existing plugins on disk 
ERRO[0000] Error tearing down partially created network namespace for container 687cab588473a65286b9f397bb03af56e5ff17995841b5667943099fef72e83d: CNI network "podman" not found 
Error: error configuring network namespace for container 687cab588473a65286b9f397bb03af56e5ff17995841b5667943099fef72e83d: CNI network "podman" not found

Describe the results you expected:
Podman should create container

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.16.8
Built:        Wed Dec  8 21:45:07 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 16
  distribution:
    distribution: fedora
    variant: silverblue
    version: "35"
  eventLogger: journald
  hostname: thinkxps-sb
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.15.8-200.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 28557361152
  memTotal: 33364307968
  ociRuntime:
    name: crun
    package: crun-1.3-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.3
      commit: 8e5757a4e68590326dafe8a8b1b4a584b10a1370
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 4m 37.41s
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /var/home/jaydoubleu/.config/containers/storage.conf
  containerStore:
    number: 213
    paused: 0
    running: 0
    stopped: 213
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/jaydoubleu/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 75
  runRoot: /run/user/1000/containers
  volumePath: /var/home/jaydoubleu/.local/share/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 1638999907
  BuiltTime: Wed Dec  8 21:45:07 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 3.4.4

Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.4.4-1.fc35.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

[[email protected] ~]$ rpm-ostree status
State: idle
Deployments:
● fedora:fedora/35/x86_64/silverblue
                   Version: 35.20211218.0 (2021-12-18T00:47:07Z)
                BaseCommit: a5b83a07467600af1170e04dfa819ed2faddd1c130b05bcb40074504c0d013c6
              GPGSignature: Valid signature by 787EA6AE1147EEE56C40B30CDB4639719867C58F
           LayeredPackages: git kata-containers lorax virt-manager
[[email protected] ~]$ sudo podman info --debug
host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 16
  distribution:
    distribution: fedora
    variant: silverblue
    version: "35"
  eventLogger: journald
  hostname: thinkxps-sb
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.15.8-200.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 28062343168
  memTotal: 33364307968
  ociRuntime:
    name: crun
    package: crun-1.3-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.3
      commit: 8e5757a4e68590326dafe8a8b1b4a584b10a1370
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 6m 6.9s
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 38
    paused: 0
    running: 0
    stopped: 38
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 19
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 1638999907
  BuiltTime: Wed Dec  8 21:45:07 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 3.4.4

[[email protected] ~]$ sudo podman run --log-level=DEBUG -it fedora /bin/bash
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called run.PersistentPreRunE(podman run --log-level=DEBUG -it fedora /bin/bash) 
DEBU[0000] Merged system config "/usr/share/containers/containers.conf" 
DEBU[0000] Using conmon: "/usr/bin/conmon"              
DEBU[0000] Initializing boltdb state at /var/lib/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver overlay                   
DEBU[0000] Using graph root /var/lib/containers/storage 
DEBU[0000] Using run root /run/containers/storage       
DEBU[0000] Using static dir /var/lib/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/libpod                    
DEBU[0000] Using volume path /var/lib/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that metacopy is being used 
DEBU[0000] Cached value indicated that native-diff is not being used 
INFO[0000] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled 
DEBU[0000] backingFs=btrfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=true 
DEBU[0000] Initializing event backend journald          
DEBU[0000] configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument 
DEBU[0000] configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] Using OCI runtime "/usr/bin/crun"            
DEBU[0000] Default CNI network name podman is unchangeable 
INFO[0000] Setting parallel job count to 49             
DEBU[0000] Pulling image fedora (policy: missing)       
DEBU[0000] Looking up image "fedora" in local containers storage 
DEBU[0000] Trying "fedora" ...                          
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf" 
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/000-shortnames.conf" 
DEBU[0000] Trying "registry.fedoraproject.org/fedora:latest" ... 
DEBU[0000] parsed reference into "[[email protected]/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage ([[email protected]/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae) 
DEBU[0000] Looking up image "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Trying "registry.fedoraproject.org/fedora:latest" ... 
DEBU[0000] parsed reference into "[[email protected]/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Found image "registry.fedoraproject.org/fedora:latest" as "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Found image "registry.fedoraproject.org/fedora:latest" as "registry.fedoraproject.org/fedora:latest" in local containers storage ([[email protected]/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae) 
DEBU[0000] Looking up image "fedora" in local containers storage 
DEBU[0000] Trying "fedora" ...                          
DEBU[0000] Trying "registry.fedoraproject.org/fedora:latest" ... 
DEBU[0000] parsed reference into "[[email protected]/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage ([[email protected]/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae) 
DEBU[0000] Inspecting image 3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Looking up image "fedora" in local containers storage 
DEBU[0000] Trying "fedora" ...                          
DEBU[0000] Trying "registry.fedoraproject.org/fedora:latest" ... 
DEBU[0000] parsed reference into "[[email protected]/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage 
DEBU[0000] Found image "fedora" as "registry.fedoraproject.org/fedora:latest" in local containers storage ([[email protected]/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae) 
DEBU[0000] Inspecting image 3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] Inspecting image 3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae 
DEBU[0000] using systemd mode: false                    
DEBU[0000] Adding exposed ports                         
DEBU[0000] No hostname set; container's hostname will default to runtime default 
DEBU[0000] Loading seccomp profile from "/usr/share/containers/seccomp.json" 
DEBU[0000] Allocated lock 39 for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c 
DEBU[0000] parsed reference into "[[email protected]/var/lib/containers/storage+/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] exporting opaque data as blob "sha256:3059bef432ebb91a6a51d8f5cf20b033041dbddb3cab79628c1eb3412cbde0ae" 
DEBU[0000] created container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" 
DEBU[0000] container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" has work directory "/var/lib/containers/storage/overlay-containers/d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c/userdata" 
DEBU[0000] container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" has run directory "/run/containers/storage/overlay-containers/d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c/userdata" 
DEBU[0000] Handling terminal attach                     
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Made network namespace at /run/netns/cni-3342c823-7f24-71e6-c412-90ac04706be2 for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c 
DEBU[0000] Cached value indicated that metacopy is being used 
DEBU[0000] backingFs=btrfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=true 
DEBU[0000] overlay: mount_data=nodev,metacopy=on,lowerdir=/var/lib/containers/storage/overlay/l/3SBCDDB3TZMKMTS6VRBBO7WLZM,upperdir=/var/lib/containers/storage/overlay/4b964b2e2543e2ee68ce7160e2801637a3de9323f91355cdf2c0064035e398e0/diff,workdir=/var/lib/containers/storage/overlay/4b964b2e2543e2ee68ce7160e2801637a3de9323f91355cdf2c0064035e398e0/work,context="system_u:object_r:container_file_t:s0:c458,c576" 
DEBU[0000] mounted container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" at "/var/lib/containers/storage/overlay/4b964b2e2543e2ee68ce7160e2801637a3de9323f91355cdf2c0064035e398e0/merged" 
DEBU[0000] Created root filesystem for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c at /var/lib/containers/storage/overlay/4b964b2e2543e2ee68ce7160e2801637a3de9323f91355cdf2c0064035e398e0/merged 
INFO[0000] Got pod network &{Name:nifty_khayyam Namespace:nifty_khayyam ID:d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c NetNS:/run/netns/cni-3342c823-7f24-71e6-c412-90ac04706be2 Networks:[{Name:podman Ifname:eth0}] RuntimeConfig:map[podman:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]} 
DEBU[0000] Default CNI network name podman is unchangeable 
INFO[0000] Got pod network &{Name:nifty_khayyam Namespace:nifty_khayyam ID:d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c NetNS:/run/netns/cni-3342c823-7f24-71e6-c412-90ac04706be2 Networks:[{Name:podman Ifname:eth0}] RuntimeConfig:map[podman:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]} 
ERRO[0000] error loading cached network config: network "podman" not found in CNI cache 
WARN[0000] falling back to loading from existing plugins on disk 
DEBU[0000] Default CNI network name podman is unchangeable 
ERRO[0000] Error tearing down partially created network namespace for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c: CNI network "podman" not found 
DEBU[0000] unmounted container "d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c" 
DEBU[0000] Network is already cleaned up, skipping...   
DEBU[0000] Cleaning up container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c 
DEBU[0000] Network is already cleaned up, skipping...   
DEBU[0000] Container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c storage is already unmounted, skipping... 
DEBU[0000] ExitCode msg: "error configuring network namespace for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c: cni network "podman" not found" 
Error: error configuring network namespace for container d416479c14ad9125c4ab896975dd04ca0d03654a4e8e95d9fe4a19ce3a95e77c: CNI network "podman" not found

[email protected] ~]$ sudo podman --log-level=DEBUG network ls
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called ls.PersistentPreRunE(podman --log-level=DEBUG network ls) 
DEBU[0000] Merged system config "/usr/share/containers/containers.conf" 
DEBU[0000] Using conmon: "/usr/bin/conmon"              
DEBU[0000] Initializing boltdb state at /var/lib/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver overlay                   
DEBU[0000] Using graph root /var/lib/containers/storage 
DEBU[0000] Using run root /run/containers/storage       
DEBU[0000] Using static dir /var/lib/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/libpod                    
DEBU[0000] Using volume path /var/lib/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that metacopy is being used 
DEBU[0000] Cached value indicated that native-diff is not being used 
INFO[0000] Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled 
DEBU[0000] backingFs=btrfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=true 
DEBU[0000] Initializing event backend journald          
DEBU[0000] configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument 
DEBU[0000] configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] Using OCI runtime "/usr/bin/crun"            
DEBU[0000] Default CNI network name podman is unchangeable 
INFO[0000] Setting parallel job count to 49             
NETWORK ID  NAME        VERSION     PLUGINS
DEBU[0000] Called ls.PersistentPostRunE(podman --log-level=DEBUG network ls) 

It is Solved!!!!! After reboot, it is fixed!

Hi, I am new here and I met a problem when I ran podman run podman run httpd with root. The error below:

WARN[0000] Failed to load cached network config: network podman not found in CNI cache, falling back to loading network podman from disk 
WARN[0000] 1 error occurred:
        * plugin type="bridge" failed (delete): cni plugin bridge failed: running [/usr/bin/iptables -t nat -D POSTROUTING -s 10.88.0.8 -j CNI-f90cda40ce3493a528f90a9a -m comment --comment name: "podman" id: "82c2803a67ee60b7ae1f7e1bf4e9df8e7fdffd262f7d2aaa36125508ab742d09" --wait]: exit status 2: iptables v1.8.7 (legacy): Couldn't load match `comment':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

 
Error: plugin type="bridge" failed (add): cni plugin bridge failed: running [/usr/bin/iptables -t nat -C CNI-f90cda40ce3493a528f90a9a -d 10.88.0.8/16 -j ACCEPT -m comment --comment name: "podman" id: "82c2803a67ee60b7ae1f7e1bf4e9df8e7fdffd262f7d2aaa36125508ab742d09" --wait]: exit status 2: iptables v1.8.7 (legacy): Couldn't load match `comment':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

And I run podman network ls get

NETWORK ID    NAME        DRIVER
2f259bab93aa  podman      bridge

Run podman -v get podman version 4.0.1
Run uname -a get Linux hasee 5.15.25-1-MANJARO #1 SMP PREEMPT Wed Feb 23 14:44:03 UTC 2022 x86_64 GNU/Linux

I have no idea how to fix it, I google it and many say it is a bug in 3.x on Fedora, how should I do.
Thanks in advance!

View previous topic :: View next topic  
Author Message
Dr.Willy
Guru
Guru

Joined: 15 Jul 2007
Posts: 545
Location: NRW, Germany

PostPosted: Mon Apr 25, 2022 4:59 pm    Post subject: [solved] podman, podman-compose and iptables Reply with quote

Hey guys, im trying to get podman to work on gentoo, but I’m stuck on an issue.

I’m using podman-compose, which is just a python-script that runs the appropriate podman commands. Since it helpfully prints out the commands it runs I can confirm that running those commands manually produces the same results.

Code:
~ # podman-compose up

[‘podman’, ‘—version’, »]

using podman version: 4.0.3

** excluding:  set()

podman volume inspect myproj_pgdata || podman volume create myproj_pgdata

[‘podman’, ‘volume’, ‘inspect’, ‘myproj_pgdata’]

[‘podman’, ‘network’, ‘exists’, ‘myproj_default’]

podman create —name=myproj_db_1 —label io.podman.compose.config-hash=123 —label io.podman.compose.project=myproj —label io.podman.compose.version=0.0.1 —label com.docker.compose.project=myproj —label com.docker.compose.project.working_dir=/home/w/myproj —label com.docker.compose.project.config_files=docker-compose.yml —label com.docker.compose.container-number=1 —label com.docker.compose.service=db -e POSTGRES_PASSWORD=_DUMMY_PW_ -v myproj_pgdata:/var/lib/postgresql/data:rw —net myproj_default —network-alias db -p 5432 —restart unless-stopped postgres:13

podman start -a myproj_db_1

WARN[0000] Failed to load cached network config: network myproj_default not found in CNI cache, falling back to loading network myproj_default from disk

WARN[0000] 1 error occurred:

   * plugin type=»firewall» failed (delete): cni plugin firewall failed: could not initialize iptables protocol 0: exec: «iptables»: executable file not found in $PATH

Error: unable to start container 38f565465c9797f93004630f61d56286c8c0f8ad8fd0817f9d10903886d8f54a: plugin type=»bridge» failed (add): cni plugin bridge failed: failed to locate iptables: exec: «iptables»: executable file not found in $PATH

So some package seems to be missing an iptables dependency somewhere.

After emerging it manually I end up with:

Code:
~ # podman-compose up

[‘podman’, ‘—version’, »]

using podman version: 4.0.3

** excluding:  set()

podman volume inspect myproj_pgdata || podman volume create myproj_pgdata

[‘podman’, ‘volume’, ‘inspect’, ‘myproj_pgdata’]

[‘podman’, ‘network’, ‘exists’, ‘myproj_default’]

podman create —name=myproj_db_1 —label io.podman.compose.config-hash=123 —label io.podman.compose.project=myproj —label io.podman.compose.version=0.0.1 —label com.docker.compose.project=myproj —label com.docker.compose.project.working_dir=/home/w/myproj —label com.docker.compose.project.config_files=docker-compose.yml —label com.docker.compose.container-number=1 —label com.docker.compose.service=db -e POSTGRES_PASSWORD=_DUMMY_PW_ -v myproj_pgdata:/var/lib/postgresql/data:rw —net myproj_default —network-alias db -p 5432 —restart unless-stopped postgres:13

podman start -a myproj_db_1

WARN[0000] Failed to load cached network config: network myproj_default not found in CNI cache, falling back to loading network myproj_default from disk

WARN[0000] 1 error occurred:

   * plugin type=»bridge» failed (delete): cni plugin bridge failed: running [/sbin/iptables -t nat -D POSTROUTING -s 10.89.0.8 -j CNI-09e02578e23c8563453d468a -m comment —comment name: «myproj_default» id: «38f565465c9797f93004630f61d56286c8c0f8ad8fd0817f9d10903886d8f54a» —wait]: exit status 2: iptables v1.8.7 (legacy): Couldn’t load match `comment’:No such file or directory

Try `iptables -h’ or ‘iptables —help’ for more information.

Error: unable to start container 38f565465c9797f93004630f61d56286c8c0f8ad8fd0817f9d10903886d8f54a: plugin type=»bridge» failed (add): cni plugin bridge failed: running [/sbin/iptables -t nat -C CNI-09e02578e23c8563453d468a -d 10.89.0.8/24 -j ACCEPT -m comment —comment name: «myproj_default» id: «38f565465c9797f93004630f61d56286c8c0f8ad8fd0817f9d10903886d8f54a» —wait]: exit status 2: iptables v1.8.7 (legacy): Couldn’t load match `comment’:No such file or directory

Try `iptables -h’ or ‘iptables —help’ for more information.

exit code: 125

Now I’ve never worked with iptables so I’m at a loss what is going on here (hence: Networking & Security).

Since my google-fu has failed me, can someone explain to me why this iptables command fails?
_________________
gentoo repos: kakoune | oil | hyper-v

Last edited by Dr.Willy on Mon Apr 25, 2022 6:59 pm; edited 1 time in total

Back to top

View user's profile Send private message

alamahant
Advocate
Advocate

Joined: 23 Mar 2019
Posts: 3304

PostPosted: Mon Apr 25, 2022 5:54 pm    Post subject: Reply with quote

What is the output of

Code:

ls -l /sbin/iptables-legacy

ls -l /sbin/iptables



Do they point to

xtables-legacy-multi

?
_________________
:)

Back to top

View user's profile Send private message

Dr.Willy
Guru
Guru

Joined: 15 Jul 2007
Posts: 545
Location: NRW, Germany

PostPosted: Mon Apr 25, 2022 6:06 pm    Post subject: Reply with quote

alamahant wrote:
What is the output of

Code:

ls -l /sbin/iptables-legacy

ls -l /sbin/iptables



Do they point to

xtables-legacy-multi

?

Yes, they do.

Also:

Code:
~ # eselect iptables list

Available iptables symlink targets:

  [1]   xtables-legacy-multi *


_________________
gentoo repos: kakoune | oil | hyper-v

Back to top

View user's profile Send private message

alamahant
Advocate
Advocate

Joined: 23 Mar 2019
Posts: 3304

PostPosted: Mon Apr 25, 2022 6:11 pm    Post subject: Reply with quote

Do you have

Code:

CONFIG_NETFILTER_XT_MATCH_COMMENT=m



in /usr/src/linux/.config

?

Code:

grep CONFIG_NETFILTER_XT_MATCH_COMMENT /usr/src/linux/.config


_________________
:)

Back to top

View user's profile Send private message

Dr.Willy
Guru
Guru

Joined: 15 Jul 2007
Posts: 545
Location: NRW, Germany

PostPosted: Mon Apr 25, 2022 6:58 pm    Post subject: Reply with quote

alamahant wrote:
Code:
grep CONFIG_NETFILTER_XT_MATCH_COMMENT /usr/src/linux/.config



Bingo, that was missing!

Added CONFIG_NETFILTER_XT_MATCH_COMMENT (…and CONFIG_NETFILTER_XT_MATCH_MULTIPORT) and we’re up and running.

Thank you alamahant! :)
_________________
gentoo repos: kakoune | oil | hyper-v

Back to top

View user's profile Send private message

Display posts from previous:   

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

I just git clone from the reposirotyr, and config all environment, finally after I make install,my podman version printf this:

root@PS-03:/opt/go/src/github.com/containers/libpod# podman version
ERRO[0000] Error loading CNI config file /etc/cni/net.d/99-loopback.conf: error parsing configuration: missing 'type' 
Version:      3.2.0-dev
API Version:  3.2.0-dev
Go Version:   go1.16.3
Git Commit:   d6ec38f2eec6c9ac9e11db5cbf949a357fa13f28
Built:        Sat May  1 16:21:00 2021
OS/Arch:      linux/amd64

and my /etc/cni/net.d/99-loopback.conf content is:

root@PS-03:/opt/go/src/github.com/containers/libpod# more /etc/cni/net.d/99-loopback.conf 
{
  "cniVersion": "0.4.0",
  "name": "podman",
  "plugins": [
    {
      "type": "bridge",
      "bridge": "cni-podman0",
      "isGateway": true,
      "ipMasq": true,
      "hairpinMode": true,
      "ipam": {
        "type": "host-local",
        "routes": [{ "dst": "0.0.0.0/0" }],
        "ranges": [
          [
            {
              "subnet": "10.88.0.0/16",
              "gateway": "10.88.0.1"
            }
          ]
        ]
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    },
    {
      "type": "firewall"
    },
    {
      "type": "tuning"
    }
  ]
}

can somebody help me? tks.

Понравилась статья? Поделить с друзьями:
  • Error loading bmp texture half life model viewer
  • Error load balancing failed to find a valid mailbox database
  • Error loading attackinfo json
  • Error lnk2026 небезопасный модуль для образа safeseh
  • Error loading asgi app could not import module main