Docker iptables error

• This is a bug report
• This is a feature request
• I searched existing issues before opening this one

Expected behavior

Create a docker network or use a docker-compose including a network creation.

Actual behavior

Error response from daemon: Failed to program FILTER chain: iptables failed: iptables --wait -I FORWARD -o br-ad0b6136ca3d -j DOCKER: iptables v1.8.2 (nf_tables):  RULE_INSERT failed (Invalid argument): rule in chain FORWARD
 (exit status 4)

Steps to reproduce the behavior

docker network create test

Output of docker version:

Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea838
 Built:             Wed Nov 13 07:25:38 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea838
  Built:            Wed Nov 13 07:24:09 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 13
  Running: 6
  Paused: 0
  Stopped: 7
 Images: 41
 Server Version: 19.03.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.4.182-mainline-rev1
 Operating System: Debian GNU/Linux 10 (buster)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.62GiB
 Name: DEDIHUGO
 ID: OCER:S7KK:KZVH:WFZT:MSUH:Q5YE:N6GH:UNTQ:CAZ3:73ID:VCC6:K2HO
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.)
Already done : Reinstall docker

Iptables: No chain/target/match by that name is an error while running a Docker container.

Confused about how running a Docker caused an iptables error?

This is because Docker works in correlation with iptables.

At Bobcares, we get requests to fix docker iptables errors, as a part of our Server Management Services.

Today, let’s have a look at how our Support Engineers fix this error.

Docker and iptables

Docker is one of the popular container software. It allows both Developers and Sysadmins to develop, setup, and run applications. Whereas, iptables is the built-in firewall for Linux based systems.

So how does a Docker relates to the Linux firewall iptables?

Usually, on Linux, Docker modifies or creates iptables rules. And the purpose is to provide network isolation. Iptables add these rules to the DOCKER chain.

While running the Docker, these rules come into action. Iptables allow all the rules predefined by Docker. This works fine until both Docker and firewall work in correlation. Let’s now see how this leads to an error.

Docker error iptables no chain/target/match by that name

Many of our customers approach us with Docker iptables errors. But, this error shows up when the user is running a Docker.

While users execute the command to run the Docker, it checks for the firewall status. When the firewall is down, it shows up the error. A Docker iptables error in laravel-nginx server appears as,

This is the complete error message. Here the message says that the driver has difficulty in connecting to the endpoint. Let’s see how our Support Engineers fix this error.

Fix for Docker error iptables no chain/target/match

This error indicates a missing firewall chain while the Docker is running.

That is, sometimes users delete the DOCKER chain from iptables. Otherwise, users restart the firewall service while the Docker is running. Hence, removing the iptables rules created by Docker.

In both cases, Docker loses its correlation with iptables. And results in Docker error.

Our customers often approach us with this error. Firstly, we check if the firewall service status using

systemctl status iptables.service

If the service is down we restart the service.

Then, we check the iptables rules using the command

iptables -L

The docker firewall rules were missing thus it shows the error.

To resolve the error our Support Engineers restart the docker service. For instance, to restart the docker we use the command,

service docker restart

While restarting the Docker, it automatically creates the firewall rules. And we ensure to enable the firewall before restarting the docker.

Similarly, executing rules without specifying table-name can result in the same error message.

For example, consider the command,

iptables -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-ports 8443

Here the PREROUTING chain belongs to the NAT table. As the user hasn’t specified the name, it considers it as default table. And this results in an error.

In this case, our Support Team asks the customers to execute the command by specifying table-name.

iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-ports 8443

Hence this fixes the error.

[Need help in fixing Docker iptables error? – We’ll help you.]

Conclusion

In short, iptables no chain/target/match by that name Docker error occur due to non-correlation. In Linux Docker creates iptables rules. The error shows up if the firewall is not in correlation with the Docker. Today, we saw how our Support Engineers fix this error.

var google_conversion_label = «owonCMyG5nEQ0aD71QM»;

Добрый день!
Сразу скажу, что админ я не очень хороший (и докером пользуюсь 2-ой час), и скорее всего жестко туплю, но Docker ругается на Iptable и не хочет прокидывать 8080 порт.

Система: CentOS 7
Kernel:

Linux ****** 3.10.0-327.4.5.el7.x86_64 #1 SMP Mon Jan 25 22:07:14 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

docker -v:

Docker version 1.9.1, build a34a1d5

docker info:

Containers: 12
Images: 14
Server Version: 1.9.1
Storage Driver: devicemapper
Pool Name: docker-253:1-131382-pool
Pool Blocksize: 65.54 kB
Base Device Size: 107.4 GB
Backing Filesystem:
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 2.454 GB
Data Space Total: 107.4 GB
Data Space Available: 17.51 GB
Metadata Space Used: 3.293 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.144 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.107-RHEL7 (2015-10-14)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.10.0-327.4.5.el7.x86_64
Operating System: CentOS Linux 7 (Core)
CPUs: 1
Total Memory: 993.1 MiB
Name: *****
ID: *****
WARNING: IPv4 forwarding is disabled
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

iptables:

# sample configuration for iptables service
# you can edit this manually or use system-config-firewall
# please do not ask us to add additional ports/services to this default configuration
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state —state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state —state NEW -m tcp —dport 22 -j ACCEPT

-A INPUT -p tcp -m state —state NEW -m tcp —dport 80 -j ACCEPT
-A INPUT -p tcp -m state —state NEW -m tcp —dport 8080 -j ACCEPT

-A INPUT -j REJECT —reject-with icmp-host-prohibited
-A FORWARD -j REJECT —reject-with icmp-host-prohibited
COMMIT

Запускаю я это все:
docker run -d -p 8080:80 node_test_app node ./

Ответ:

WARNING: IPv4 forwarding is disabled. Networking will not work.
*****
Error response from daemon: Cannot start container *****: failed to create endpoint high_mcnulty on network bridge: COMMAND_FAILED: '/sbin/iptables -w2 -t nat -A DOCKER -p tcp -d 0/0 --dport 8080 -j DNAT --to-destination ****.***.****.****:80 ! -i docker0' failed: iptables: No chain/target/match by that name.

When I try to publish container’s ports to the host, docker fails to add iptables rules:

$ sudo docker run --rm -p 8080:80 nginx
docker: Error response from daemon: driver failed programming external 
connectivity on endpoint:

iptables failed:
  iptables --wait -t nat -A DOCKER -p tcp -d 0/0
           --dport 8080 -j DNAT --to-destination 172.17.0.2:80 ! -i docker0

 iptables: No chain/target/match by that name. (exit status 1)).

What confuses me is that the DOCKER chain seems to exist:

$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy DROP)
target     prot opt source               destination         
DOCKER-USER  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain DOCKER (2 references)
target     prot opt source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target     prot opt source               destination         
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-ISOLATION-STAGE-2 (2 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Chain DOCKER-USER (1 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere            

… and also here:

$ sudo iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
DOCKER     all  --  anywhere             anywhere             ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
DOCKER     all  --  anywhere            !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  172.17.0.0/16        anywhere            
MASQUERADE  all  --  172.18.0.0/16        anywhere            

Chain DOCKER (2 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            

Maybe relevant:ip a shows that docker0 is down:

$ ip a
...
74: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:01:52:de:7d brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

I tried to stop docker, remove the broken docker0 device, and restart docker again. Same error:

sudo systemctl stop docker.service
sudo ip link del docker0
sudo systemctl start docker.service

ip a still shows that docker0 is down. Running sudo docker run --rm -p 8080:80 nginx
also runs in the identical error message as above.

System information:

$ docker --version
Docker version 18.05.0-ce, build f150324782

$ uname -a
Linux amd8-arch 4.16.7-1-ARCH #1 SMP PREEMPT Wed May 2 21:12:36 UTC 2018 x86_64 GNU/Linux

I am using Arch Linux.

After recently running pacman -Syu, then installing some unrelated packages, on a computer running Arch Linux, exposing ports of a new container using --publish produces a strange error.

Here’s an example of the error produced:

docker: Error response from daemon: driver failed programming external connectivity on endpoint bsp-devkit (7a47f30eaa0e104d6b9a7b75003a768e734182e20e057e51d33cca23da7dbf02):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 10809 -j DNAT --to-destination 172.17.0.2:10809 ! -i docker0: iptables v1.8.2 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).

Here’s a Docker run command that’s failing in this way:

docker run 
 --publish=2223:22 
 --publish=8000:8000 
 --publish=69:69/udp --publish=10809:10809 
 --detach=true --privileged 
 --hostname=bsp-devkit --name=bsp-devkit 
 -v /sys/fs/cgroup:/sys/fs/cgroup:ro 
 -v $MIRRORDIR:/home/devel/mirror 
 -it $IMAGE

This is very weird to me, because it seems like a regression caused by a newer version of iptables, but the iptables version has «legacy» written after it (so why would that version be updated with a breaking change?) and I also haven’t found anyone else online asking about this specific error so I must have messed something up really badly.

The packages I installed included the hex editor Bless, but I don’t remember if I installed anything else.

Additional Notes
— I have never configured a firewall on this system
— This computer is on a VPN, but this hasn’t been an issue before

Updates:

• Updating Docker does not fix the issue
• Downgrading the iptables package to v1.8.0 from the pacman cache doesn’t fix it…
• iptables daemon doesn’t seem to be running; logs say /etc/iptables/iptables.rules is missing
• The problem persists after creating iptables.rules and starting the daemon