Failed to start daemon error initializing network controller

• This is a bug report
• This is a feature request
• [ x]I searched existing issues before opening this one

Expected behavior

Actual behavior

Steps to reproduce the behavior

Output of docker version:

Docker version 19.03.12, build 48a66213fe

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 19.03.12
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1127.19.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.7GiB
 Name: XXXMachine
 ID: UDX4:5ZLP:Z3R5:NS3G:IPSF:4HCA:7HXQ:IRER:SV63:LJJ4:IUUW:MGGK
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.)
KVM Open Nebula — OS Centos 7

I have suddenly Docker stop working and cannot start,

Journalctl -xe

-- Unit docker.service has finished shutting down.
Sep 13 20:51:10 XXXMachine systemd[1]: Closed Docker Socket for the API.
-- Subject: Unit docker.socket has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit docker.socket has finished shutting down.
Sep 13 20:51:10 XXXMachine systemd[1]: Stopping Docker Socket for the API.
-- Subject: Unit docker.socket has begun shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit docker.socket has begun shutting down.
Sep 13 20:51:10 XXXMachine systemd[1]: Starting Docker Socket for the API.
-- Subject: Unit docker.socket has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit docker.socket has begun starting up.
Sep 13 20:51:10 XXXMachine systemd[1]: Listening on Docker Socket for the API.
-- Subject: Unit docker.socket has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit docker.socket has finished starting up.

Found this at /var/log/messages

Sep 13 20:47:37 XXXMachine dockerd: time="2020-09-13T20:47:37.358504531+07:00" level=warning msg="Runnin
g modprobe bridge br_netfilter failed with message: , error: exit status 1"
Sep 13 20:47:37 XXXMachine dockerd: time="2020-09-13T20:47:37.360959332+07:00" level=warning msg="Runnin
g iptables --wait -t nat -L -n failed with message: `iptables v1.4.21: can't initialize iptables table `
nat': Table does not exist (do you need to insmod?)nPerhaps iptables or your kernel needs to be upgrade
d.`, error: exit status 3"
Sep 13 20:47:37 XXXMachine dockerd: time="2020-09-13T20:47:37.416045671+07:00" level=info msg="stopping 
event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
Sep 13 20:47:37 XXXMachine dockerd: failed to start daemon: Error initializing network controller: error
 obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N D
OCKER: iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to ins
mod?)
Sep 13 20:47:37 XXXMachine dockerd: Perhaps iptables or your kernel needs to be upgraded.
Sep 13 20:47:37 XXXMachine dockerd: (exit status 3)

I did :
yum update -y

yum upgrade -y

Then started again docker with systemd, but same issue happen, finally I tried to reboot the VM after it, I started again
then docker went start dan run again.

Why it can be happen in my case? Please addvice.

Error starting daemon: Error initializing network controller: list bridge addresses failed: no available network #35121

Description

I have physical machine with Gentoo as host OS for Docker containers. I have compiled kernel using instructions on page https://wiki.gentoo.org/wiki/Docker#Kernel and I have installed Docker from Gentoo repository (see on the section Additional environment details (AWS, VirtualBox, physical, etc.)). I have set following USE flags:

I have emerged Docker and added it to boot level default in OpenRC init system. After compiling kernel and Docker I wanted to check if Docker is working so I typed docker info in terminal and I got error. I have decided to check what is wrong and I need your help with solving issue.

Steps to reproduce the issue:

1. Issue the docker version command.
2. Try get Docker system-wide informations using docker info .
3. Check Docker daemon status.
4. Check Docker logs.

Describe the results you received:
In the output of docker version (see below) you can see error Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? . The same message appears if I try get Docker system-wide informations. The same error appears if I try run the same command prepending by sudo, so this error applies to daemon. I tried to check if there a mistake in Docker daemon privileges. Based on these messages I am able to say that maybe Docker daemon not running. I checked daemon status to make sure. Docker daemon is crashed. To see the reason, I looked at the logs.

Output of cat /var/log/docker.log :

Describe the results you expected:

docker info should return Docker system-wide informations instead of Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? .

Expected output of docker version :

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version :

Output of docker info :

Output of sudo docker info :

Output of sudo service docker status :

Additional environment details (AWS, VirtualBox, physical, etc.):
I am using Gentoo as Host OS for Docker containers. I have compiled kernel using instructions on page https://wiki.gentoo.org/wiki/Docker#Kernel and I have installed Docker from Gentoo repository.

Host system informations:

I have disabled iptables and ip6tables because firewall is not actually properly configured. I am connecting to internet through VPN and I am using 8.8.8.8 and 8.8.4.4 DNS providers. I have running Tor and Privoxy daemons and I am using OpenRC init system.

The text was updated successfully, but these errors were encountered:

Источник

docker-failed to start daemon: Error initializing network controller: error obtaining controller instance #1105

Expected behavior

Actual behavior

Steps to reproduce the behavior

Output of docker version :

Output of docker info :

Additional environment details (AWS, VirtualBox, physical, etc.)
KVM Open Nebula — OS Centos 7

I have suddenly Docker stop working and cannot start,

Found this at /var/log/messages

I did :
yum update -y

Then started again docker with systemd, but same issue happen, finally I tried to reboot the VM after it, I started again
then docker went start dan run again.

Why it can be happen in my case? Please addvice.

The text was updated successfully, but these errors were encountered:

Im having the same issue? any helpers?

Hi , I searched also in several documentations,
Please check your firewalld, if it stop, you could try to activated using the systemd. «systemctl start firewalld»

In my case , it dosn’t work, but several documentations work with this. thanks

🤔 How is the status for this issue together with current software functionality?

If you run Debian try:
sudo update-alternatives —set iptables /usr/sbin/iptables-legacy

ensure that /etc/sysctl.conf includes:
net.ipv4.ip_forward = 1

Third guess: Are you using openvpn?
If so, create the bridge yourself:
`sudo apt-get install bridge-utils

sudo brctl addbr docker0

sudo ip addr add 10.1.0.1/24 dev docker0

sudo ip link set dev docker0 up

ip addr show docker0

sudo systemctl restart docker

sudo iptables -t nat -L -n`

If you run Debian try:
sudo update-alternatives —set iptables /usr/sbin/iptables-legacy

ensure that /etc/sysctl.conf includes:
net.ipv4.ip_forward = 1

Third guess: Are you using openvpn?
If so, create the bridge yourself:
`sudo apt-get install bridge-utils

sudo brctl addbr docker0

sudo ip addr add 10.1.0.1/24 dev docker0

sudo ip link set dev docker0 up

ip addr show docker0

sudo systemctl restart docker

sudo iptables -t nat -L -n`

I don’t use any VPN so the first 2 steps worked perfectly. Thank you very much @wlanboy

Источник

Error starting daemon: Error initializing network controller: list bridge addresses failed: no available network #31546

i am building docker from source on gentoo. pulling from app-emulation/docker. building runs fine. no kernel config or use flag issues. running contrib/check-config.sh checks out OK, just missing AUFS_FS and some zfs stuff which is fine.

when trying to start the systemd docker service, i receive the following error.

i am currently running

docker-1.13.1-r2
docker-runc-1.0.0_rc2_20170201
docker-proxy-0.8.0_p20161111
containerd-0.2.5-r1
systemd-233

i am not sure how to resolve the list bridge addresses failed issue. it seems like this could just be an environment issue, do i need to create my own network bridge for docker? is the ebuild install process on gentoo missing something?

i’m just at a loss here. any help would be great.

The text was updated successfully, but these errors were encountered:

Is your host connected to VPN ?
If so, because of the gateway redirect route, docker won’t be able to elect one of its default address pools to assign to the docker0 interface.

@aboch yea i saw that, i am not using a vpn though

I’m wondering if there is an undocumented use flag or kernel option necessary for use with systemd. or possibly i overlooked a bug already reported else where. Could it be a missing dependency, or maybe some arbitrary environment issue? This one’s got me scratching my head.

If you are blocked by this, a workaround is to manually create the docker0 bridge interface, then start the docker daemon. Or start the docker daemon passing an explicit bridge ip/subnet (ex. dockerd —bip 192.168.100.1/24 )

PTAL at #30295 although you should not be affected since you are already on 1.13.1

@aboch yea i was confused, i thought upgrading would have resolved these issues for me. thanks for the replies. i’ll give the proposed solutions a go. probably just write up a script to create the docker0 bridge using bridge-utils.

@aboch alright, manually creating the docker0 bridge resolved my issue. Apparently systemd is having some bugs with cgroups, so i also had to add systemd.legacy_systemd_cgroup_controller=yes to my kernel boot options.

here is the bash script i used to create the docker0, actually just slightly modified from the build your own bridge user guide.

i’m going to go ahead and close this one up.

after you added bridge
can docker image be run?

Источник

dockerd fails to start if there are routes for all private networks although a custom bip is configured #33925

Description

All our hosts have configured routes for the following networks:

So the default networks that dockerd tries to use for the docker0 bridge are already in use. Therefore I specify a custom bip which docker should use instead.

Steps to reproduce the issue:

1. Configure routes for the networks given in the description on a host
2. Start dockerd with a bip which isn’t part of one of the networks

Describe the results you received:

Relevant part of the logs:

If I configure the docker0 bridge manually dockerd starts sucessfully.

Describe the results you expected:

dockerd shouldn’t check for available networks if a custom bip is specified.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version :

Output of docker info :

The text was updated successfully, but these errors were encountered:

Today a colleague of mine asked me if I had changed something on the network because his Docker configuration was suddenly giving a lot of problems. At first I did not know what he was talking about but after some questions, it slowly became clear to me that he had problems starting his docker environment when his VPN connection to the office was online. I looked at the error message that he received and I saw the following: «Error initializing network controller: list bridge addresses failed: no available network». This was very strange because the network he had configured in his daemon.yaml looked like this:
<
«default-address-pools»:
[
<«base»:»10.10.0.0/16″,»size»:24>
]
>

In our corporate network we have a lot of RFC1918 networks, a few in the 10.x.x.x/8 range, a lot in the 172.16.0.0/12 and 192.168.0.0/16 ranges. But nothing that collides with above ranges, and even if something would collide, it was all local on his workstation where he was developing and testing some monitoring systems, and he is completely free to use whatever network he wants to use locally, as long as he doesn’t interfere with the corporate network. On the VPN router I have a default set of routes set for RFC1918 networks pointing towards the corporate routers, so everyone can reach the internal corporate networks without having to worry about anything. The firewalls will take care of the rest.

I started debugging the error message and did some Google searches and I found a lot of people complaining about exactly this same problem. Some example tickets:
docker/for-linux#123
#35121
#33925 (this ticket, currently open, so I leave a comment here.)

At first the error didn’t make any sense to me because:

• a network is available
• the configured network is not directly connected so docker is not able to say that it should not use it.
• even if an overlapping network is used somewhere else, a more specific route would be configured locally and this should prevent any routing issues.

But then I thought about something. What if the docker code, searching for free networks, takes the local routing table and checks the configured network against EVERY route in the routing table. If something matches or overlaps the route in the routing table it gives this error. At first I thought that this couldn’t be true because this would always fail because a default route of 0.0.0.0/0 would always match. But what if this default route is filtered out in the code for this specific reason. Then this hypothesis could be the truth.

I started testing locally on my own system, first I reproduced the error:

• Setup my docker daemon with the same configuration
• Had my normal local routing table without VPN.
• Started docker and this worked fine.

The resulting routing table:
default via 192.168.178.1 dev enp62s0u1u1 proto static metric 1024
10.10.0.0/24 dev docker0 proto kernel scope link src 10.10.0.1 linkdown
192.168.178.0/24 dev enp62s0u1u1 proto kernel scope link src 192.168.178.74 metric 100

Then I started my VPN. The result was 3 extra routes:
10.0.0.0/8 via 192.168.2.1 dev tap0 proto static metric 50
172.16.0.0/12 via 192.168.2.1 dev tap0 proto static metric 50
192.168.0.0/16 via 192.168.2.1 dev tap0 proto static metric 50

I then stopped my docker daemon and tried to start it again, and indeed I received the same error. So I could reproduce the problem, now for my hypothesis: «Does the code check EVERY route in the routing table, filtering out the default route.»

To test this I did the following:
I removed the default route and replaced it by 2 more specific routes that are together the whole internet:
0.0.0.0/1 via 192.168.178.1 dev enp62s0u1u1
128.0.0.0/1 via 192.168.178.1 dev enp62s0u1u1

My routing table then looks like this:
0.0.0.0/1 via 192.168.178.1 dev enp62s0u1u1
128.0.0.0/1 via 192.168.178.1 dev enp62s0u1u1
192.168.178.0/24 dev enp62s0u1u1 proto kernel scope link src 192.168.178.74 metric 100

The only difference between this state and a clean state of my system, is not having a default route, but having two routes that are together the default route of my system. Now I tried to start the docker daemon again. If the daemon starts fine my hypothesis is wrong and I have to continue my search. If the daemon fails then my hypothesis must me correct because the default route is the only difference in my local configuration.

And indeed, I received the same error again. Now I’m sure there is absolutely no reason to give this error because:

• I don’t have the 10.10.0.0/16 network anywhere in my home network
• I have a routing table that only routes for 192.168.178.0/24 and the internet

This also proves my hypothesis that every route in the routing table is being checked against the configured network, filtering out the default route. If any route matches the configured network, the configuration is rejected.

This is a bug in the docker code. The code should be changed to only match routes with «scope link» because these routes are directly connected and would be a problem when you start a docker daemon with an overlapping network configuration. Any route that is not «scope link» should be ignored because those routes could be:

• Injected by DHCP
• Injected by a routing protocol
• Injected by a VPN config.
• Less specific behind a router somewhere remote

There is one corner case where you could give a warning or maybe an error. This is when there is an equal or more specific route that is not «scope link». Because this could result in routing issues to other systems. But even then, I would make it configurable because it could very well be that this is intentional.

I’m not a developer but a network and systems engineer, so I am not able at the moment to provide a patch for this problem, but one of my colleagues thought that he had already found some parts of the code. So maybe .

The version I have tested this with is: Docker version 19.03.13, build 4484c46d9d

Источник