Wider-world-to-container doesn't apply to traffic coming from within a container

[pitkley]

Scenario:

1. Create a container providing some service under a non-published port, e.g. with the following docker-compose.yml:

   version: '2'
   
   services:
       whoami:
           image: containous/whoami:latest
           command: "--port 8081"

   Start the service using docker-compose --project-name whoami up -d.

2. Add the following to your WWTC-configuration:

   [[wider_world_to_container.rules]]
   network = "whoami_default"
   dst_container = "whoami_whoami_1"
   expose_port = "8081/tcp"

3. Signal DFW to reapply the rules (e.g. by restarting it).

4. Try to cURL the above container from within another container on the same host:

   # docker container run --rm curlimages/curl -v http://<your host or IP>:8081
   Status: Downloaded newer image for curlimages/curl:latest
   *   Trying <IP>:8081...
   * connect to <IP> port 8081 failed: Connection refused
   * Failed to connect to <your host or IP> port 8081: Connection refused
   * Closing connection 0
   curl: (7) Failed to connect to <your host or IP> port 8081: Connection refused

Since the container is exposed to the wider-world, I'd expect the container to also be reachable from other containers.

(Thanks to @cybermcm for reporting this.)

[pitkley]

@cybermcm I just released 1.2.0-rc.5 (which you'll be able to retrieve via Docker Hub under pitkley/dfw:1.2.0-rc.5 once it is done building). Maybe you can retest your use-case? 🙂

If your issue isn't fixed, please feel free to reopen this ticket!

[cybermcm]

@pitkley Thanks for this version but it breaks Docker internal DNS (at least for me)
Tested 1.2.0-rc.4 -> within container nslookup google.com -> working
1.2.0-rc.5 -> within container nslookup google.com -> no result (refused)
I'm currently reverting back to rc.4

[pitkley]

@cybermcm hm, while I can't directly confirm the DNS-issue, I can see that outgoing traffic overall seems to be a bit wonky. Sorry that I didn't catch that.

I'll reopen this issue and look at it again. 👍 Thanks so much for all your testing efforts!

[pitkley]

@cybermcm just for your information: I have reverted the change and created rc.6 (which is effectively identical to rc.4). For the time being the required workaround is to publish the port.

(I'm planning to release v1.2.0 proper some time this week, potentially today.)

I have postponed this issue into v1.3.0 and I'm looking into how this could be handled.

---

Some background, both for myself and in case anyone is interested: DFW decides how/when to route traffic based on the port requested and on the interface the traffic is arriving from. If the interface is not specified (which is what rc.5 did), this also affected traffic that would have eventually leave the box, because the prerouting stage (which is what DFW needs to use) does not yet know if the packet it is looking at is inbound or outbound relative to the machine.

This circumstance makes it tough to decide if a packet for a specific port is targeting "us" or the outside world. My current idea is to allow the user to specify (or auto-determine) the IP-addresses traffic can ingress on and filtering on that, but I'm still exploring if there is maybe some metadata on a packet that can filtered on.

[pitkley]

Note: iptables can match -m addrtype --dst-type local where local identifies "destinations [...] assigned to this host". nftables seems to have fib (with kernel >=4.10), which supposedly does the same thing.

This could be the type of match we need.