Skip to main content

Locking Down Docker Networks with SPR

· 8 min read
Alex Radocea

Envision a homelab scenario with a feature-rich router that's suitable as a container host with storage and memory. Locking down the router's container network policy is surprisingly difficult to set up and manage.

SPR makes it easy with secure by default network controls. Instead of worrying about IP ranges and interfaces, join the interfaces to the groups of devices they can communicate with and set internet access policy.

First Consider Inbound Access To The Container​

Let's assume the container has a webserver running on a typical port ( 8000), which is exposed to the host network on the same port (8000) in the host network. But we don't want any client device to get access to it, only localhost.

Okay so launch docker with an IP to bind on, and set a firewall rule to restrict access right and drop other address inputs? docker run -p 127.0.0.1:8000:8000 ....

Well it turns out that even if you apply standard rules to drop incoming packets: docker makes it impossibly hard to actually firewall correctly when Docker inserts in its own firewall rules. To do this more correctly, docker recommends inserting rules into the DOCKER-USER chain. A default-deny policy works best here.

Note that attackers that are only a single-hop away can forward packets to the direct IP of the container and the docker host will forward the traffic, by default. This applies to all docker hosts, by the way, not just hosts that are also routers, for how docker exposes ports. Setting a source address in docker has no impact against one-hop attacks, and if a DOCKER-USER chain is applied it must block interfaces with spoofing outright or have some way to authenticate the IP Addresses to be filtered by the firewall.

To better understand how the single hop attacks work, consider the firewall rules docker creates when starting a container with: docker run -p 192.168.1.2:8000:8000 -it ubuntu bash. The container has an internal IP on the docker bridge of 172.17.0.2.

info

The rules permissively forward traffic to 172.17.0.2, from all interfaces, and the host to bind on is mainly for dnat. It does not constrain access to the interface the 192.168.1.2 IP belongs to.

To leverage this an attacker one hop away could run something like:

# arp -i wlan1 -s 172.17.0.2 44:44:44:44:44:44 #where 44:44:44:44:44:44 is the MAC ADDR of the docker host
# ip route add 172.17.0.2 dev wlan1
# nc 172.17.0.2 8000
Connection to 172.20.0.2 8000 port [tcp/*] succeeded!

This also holds true if the container launch had specified -p 127.0.0.1:8000:8000

And adding an INPUT rule to drop packets on port 8000 does not help either because forwarding is in play, rather than input. The below rule does not block the attack.

# iptables -A INPUT -p tcp --dport 8000 -j DROP
# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
DROP       tcp  --  anywhere             anywhere             tcp dpt:8000
...

Again, DOCKER-USER should be used instead.

See the full rules below for what docker typically establishes


net.ipv4.conf.all.forwarding=1
net.ipv4.conf.docker0.forwarding=1

iptables -n -L -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-USER  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-ISOLATION-STAGE-1  0    --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     0    --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     0    --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     6    --  !docker0 docker0  0.0.0.0/0            172.17.0.2           tcp dpt:8000

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  0    --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       0    --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0  


iptables -n -L -t nat -v
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination         
 115 19578 DOCKER     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination         
   0     0 DOCKER     0    --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination         
   0     0 MASQUERADE  0    --  *      !docker0  172.17.0.0/16        0.0.0.0/0           
   0     0 MASQUERADE  6    --  *      *       172.17.0.2           172.17.0.2           tcp dpt:8000

Chain DOCKER (2 references)
pkts bytes target     prot opt in     out     source               destination         
   0     0 RETURN     0    --  docker0 *       0.0.0.0/0            0.0.0.0/0           
   0     0 DNAT       6    --  !docker0 *       0.0.0.0/0            192.168.1.2          tcp dpt:8000 to:172.17.0.2:8000:::

Next Consider The Container's Outbound Access​

Typically the containers also have unrestricted outbound access. The container would have full network access to all client devices, as well as all of the upstream networks including any private subnets, in addition to DNS and the internet.

Consider the following setup. A HomeLab router may be meant for running docker containers, and networking IOT gadgets. Upstream from the HomeLab router is the main router router that handles the trusted network devices. The container would be able to route packets upstream to all devices on the main network too.

Custom firewall rules can help here.

Browsers are routers too! (sort of)​

One last thing to keep in mind is that, increasingly, browsers are a springboard for breaking into insecure NAS, Routers, and IOT devices. Malicious websites and web-ads can make requests to the internal network to try to compromise devices with default credentials and vulnerabilities.

When a container is hosted on a router, malicious websites visited by trusted device on the network would be able to potentially access the container too.

Google's Chrome employs Private Network Access controls but not all browsers support this hardening. Even then, it's not perfect and there are limiting factors to how effective these controls are. See this ctf challenge which bypasses chrome's private network access to exploit a frontier speaker

If the Docker Host is reachable from the user's device, malicious websites are also able to communicate with any reachable container ports.

How SPR Helps​

Out of the box: SPR as a router solves these challenges. SPR establishes routing and firewall policy for containers as well as interfaces in general.

By default, containers on the docker bridge only have outbound internet access & DNS. They can not talk to client devices on the network, and furthermore they are blocked from accessing private networks upstream.

If a user wants to set policy for the container to access client devices, they can choose to do so. Create a docker network bridge for your container service, and use the built-in firewall to set policy. Join the interface to a group to give it access. Similarly, policy can be set to completely block outbound network access altogether.

#docker-comopse Example

networks:
  containernet:
    driver_opts:
      com.docker.network.bridge.name: spr-containernet

How This Makes Integrating Cloud VPNs More Secure​

We take advantage of this in our tailscale integration. It's possible to connect Tailscale's overlay network to your home network with SPR as the bridge with fine grained access control. Policy ensures not all devices on the network see Tailscale devices and vice versa.

SPR's firewall:

  1. Prevents the entire tailscale overlay from accessing the local devices without explicit policy, even if tailscale's access control is broken or misconfigured
  2. Prevents the local devices from accessing the tailscale overlay devices without explicit policy

Normally this kind of fine grained access control is only possible with devices directly connecting into Tailscale but leveraging SPR it can be enforced by the router's firewall policy too.

Looking to use this on SPR?​

Get started with the interface rules on SPR see the User Manual's Firewall Page

firewall-custom-interface-rule-add