Issue with DNS client in KVM guest over multiple network interfaces

[actionmystique]

Hello guys,

On "Linux msi-ge60-ubuntu 4.2.0-23-generic #28-Ubuntu SMP Sun Dec 27 17:47:31 UTC 2015 x86_64", I'm trying to solve an issue or misconfiguration of DNS client over multiple network interfaces inside a KVM virtual machine:

# NAT interface - IP address allocated in the 192.168.122.0/24 subnet
auto eth0
iface eth0 inet dhcp

# Isolated network internal interface - IP address allocated in the 172.19.100.0/24 subnet
auto eth1
iface eth1 inet dhcp

# Isolated network internal interface - IP address allocated in the 172.20.100.0/24 subnet
auto eth2
iface eth2 inet dhcp

Now, the goal is 4-fold:

• to dismiss the ISP DNS server (returned by their DHCP server),
• to avoid the host (on which this VM is running) DNS resolver, otherwise the VM sends a query to each DNS server (local & remote) and closes the previous opened port each time it generates a new query, thus generating an ICMP port unreachable as soon as it receives the response to the first query - this sounds like a bug to me or I'm missing a special tweak somewhere
• to always use Google's DNS server,
• to send all DNS queries over the main interface eth0 only.

To achieve that, we need to configure resolvconf; when we look at resolvconf & its configuration manual online, it points to openresolv package instead of resolvconf. It seems that the latter has been replaced by the former in the last versions of Ubuntu; we cannot have both packages at the same time.
However, my host had only resolvconf package installed because it is a dependency of ubuntu-minimal. So in order to completely uninstall resolvconf, I had to uninstall ubuntu-minimal as well, which is not advised. So the only way out is to uninstall resolvconf, and remove all its configuration files manually.
All this could be have been avoided if ubuntu-minimal depended on openresolv instead of resolvconf...

Anyway, here's the /etc/resolvconf.conf I've come up with:

##########################################################################################
# Configuration for resolvconf(8)
# See resolvconf.conf(5) for details
# Changes to this file should be updated by "resolvconf -u" to apply the new configuration.
##########################################################################################
resolv_conf=/etc/resolv.conf

# Setting the DNS server
name_servers=8.8.4.4

# Ignoring the DNS server(s) advertised by local host DHCP server
name_server_blacklist=192.168.*

# Removing "search lan" and other unneeded entries from resolv.conf - virtual-bridge-1 & virtual-bridge-2 are the last 2 host internal interfaces used by the VM
replace="search/lan/ search/virtual-bridge-1/ search/virtual-bridge-2/ domain/virtual-bridge-1/ domain/virtual-bridge-2/"

# Mirroring the Debian package defaults for the below resolvers
# so that resolvconf integrates seamlessly.
dnsmasq_resolv=/var/run/dnsmasq/resolv.conf
pdnsd_conf=/etc/pdnsd.conf
unbound_conf=/var/cache/unbound/resolvconf_resolvers.conf

The result is not a success; the /etc/resolv.conf contains:

# Generated by resolvconf
nameserver 127.0.0.1

But a single DNS tentative resolution from the VM (named Ubuntu-Gnome-Server) leads to multiple unsuccessful DNS network queries due to the bug already mentioned earlier in the "4-fold goal/2":

ping github.com

Wireshark traffic trace

However, if I remove the last 2 interfaces and leave only the NAT interface on that VM, DNS works almost perfectly with the same "/etc/resolvconf.conf", although there are 2 queries instead of one; one from the host, and one from the guest VM and Google's DNS server responds to each of them:

ping ubuntu.org

Wireshark traffic trace

Any advice will be appreciated.

[The Cog]

This rather brutal approach will probably do what you want, if you don't find a better way to fix resolvconf: Remove the symlink to /run/resolvconf/resolv.conf and create your own resolv.conf file instead:

sudo mv /etc/resolv.conf /etc/resolv.conf.linky
sudo sh -c 'echo nameserver 8.8.4.4 > /etc/resolv.conf'

[actionmystique]

Thanks for your answer.
Yes, that would be an option, but that would prevent the VM from using a local DNS cache.

I have found a solution/workaround by modifying one of my initial requirements: using the host DNS forwarder cache with the following /etc/resolvconf.conf & keeping the symlink /etc/resolv.conf:

########################################################################################### Configuration for resolvconf(8)
# See resolvconf.conf(5) for details
# Changes to this file should be updated by "resolvconf -u" to apply the new configuration.
##########################################################################################
resolv_conf=/etc/resolv.conf

# We are not DNS server and we dismiss all local DNS resolvers that do not come from the main local network interface
name_server_blacklist="127.0.0.1 172.*"

# Removing "search lan" and other unneeded entries from resolv.conf - virtual-bridge-1 & virtual-bridge-2 are the last 2 host internal interfaces used by the VM
replace="search/lan/ search/virtual-bridge-1/ search/virtual-bridge-2/ domain/virtual-bridge-1/ domain/virtual-bridge-2/"

# Mirrorring the Debian package defaults for the below resolvers
# so that resolvconf integrates seemlessly.
dnsmasq_resolv=/var/run/dnsmasq/resolv.conf
pdnsd_conf=/etc/pdnsd.conf
unbound_conf=/var/cache/unbound/resolvconf_resolvers.conf

The resolv.conf generated is:

# Generated by resolvconf
nameserver 192.168.122.1

That solution assumes that the host is configured correctly to bypass the ISP DNS and to use Google DNS server.

And it works pretty well: the number of DNS queries are low and directed to the host DNS cache; when the answers are available there, it minimizes the network traffic and the delays.

The ideal configuration would be to define a local DNS cache in the VM itself, but adding 127.0.1.1 to the name_servers and updating resolv.conf with "resolvconf -u" makes the whole DNS process fail. That's too bad but the previous settings are good enough.

EDIT:

• The name-servers directive is not necessary since we already receive that DNS server address from DHCP.
• The configuration was incomplete:we have to remove the DNS servers received by DHCP for the secondary interfaces (name_server_blacklist="172.*");
• this has to be modified to suit your own interfaces configuration