18.04.1 Server all DNS resolution broken, no luck troubleshooting so far

[ericbowers]

I am wondering if I can get some tips on fixing my dns resolution for my 18.04 server machine. I use this machine as an OpenVPN server and a Samba file server, in addition to using it for light web browsing as a desktop. DNS resolution is neither successful from the Desktop interface nor from my OpenVPN remote client.


I have been attempting to troubleshoot for several days with no success, and I see that many others have had issues with 18.04’s DNS. I am hoping to get the original bug repaired (no reverting to broken DNS resolution on every reboot) and to also access the outside Internet from both my remote OpenVPN client and the web browser on the home machine (in my house).


I’m relatively novice to running a linux system, only starting with this Ubuntu server less than one year ago.


I can ping 8.8.8.8 successfully, but not google.com.



“ping google.com
ping: google.com: Name or service not known”


I am pasting below some of my various settings.


-----------------------------------------------------------------------



$ cat /etc/netplan/01-netcfg.yaml
# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
version: 2
renderer: networkd
ethernets:
enp2s0:
dhcp4: true


-----------------------------------------------------------------------



cat /etc/systemd/resolved.conf
[Resolve]
DNS=192.168.1.1
#FallbackDNS=
#Domains=
#LLMNR=no
#MulticastDNS=no
#DNSSEC=no
#Cache=yes
#DNSStublistener=yes
DNSStubHandler=no


-----------------------------------------------------------------------



cat /etc/network/interfaces
# The loopback network interface
auto lo eth0
iface lo inet loopback


# The primary network interface
auto eth0
iface eth0 inet dhcp


-----------------------------------------------------------------------



cat /etc/resolvconf/resolv.conf.d/head
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
-----------------------------------------------------------------------



cat /etc/dnsmasq.d/lxd
# Tell any system-wide dnsmasq instance to make sure to bind to interfaces
# instead of listening on 0.0.0.0
# WARNING: changes to this file will get lost if lxd is removed.
bind-interfaces
except-interface=lxdbr0


-----------------------------------------------------------------------



cat /etc/dnsmasq.d-available/lxd
# Tell any system-wide dnsmasq instance to make sure to bind to interfaces
# instead of listening on 0.0.0.0
# WARNING: changes to this file will get lost if lxd is removed.
bind-interfaces
except-interface=lxdbr0


-----------------------------------------------------------------------


I would appreciate any tips and advice - as I’ve been trying to troubleshot both from similar threads on this forum, and others - for nearly a week now with no success.

[SeijiSensei]

First, let's see if you can reach Google's server at all. Try


host ubuntuforums.org 8.8.8.8

You should see



$ host ubuntuforums.org 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:

ubuntuforums.org has address 91.189.94.16


Do you?

[ericbowers]

Hi SeijiSensei,

Yes, appears so. I get the following:


host ubuntuforums.org 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:


ubuntuforums.org has address 91.189.94.16

[SeijiSensei]

What if you use 8.8.8.8 here?


cat /etc/systemd/resolved.conf


[Resolve]
DNS=192.168.1.1

[ericbowers]

No dice it seems. Not just yet.

[darkod]

1. /etc/network/interfaces should be irrelevant now if you are using netplan, right?

2. I see in netplan that you are using dhcp. That opens further questions:
2a. Have you tried with static setup in netplan (static IP, DNS)? Servers should have static setup anyway, not relying on dhcp.
2b. How come ../resolv.conf.d/head has 8.8.8.8 inside if using dhcp? Does your dhcp server transmit 8.8.8.8 as primary dns? Because you also have 192.168.1.1 in resolved.conf.

[ericbowers]

1. /etc/network/interfaces should be irrelevant now if you are using netplan, right?

2. I see in netplan that you are using dhcp. That opens further questions:
2a. Have you tried with static setup in netplan (static IP, DNS)? Servers should have static setup anyway, not relying on dhcp.
2b. How come ../resolv.conf.d/head has 8.8.8.8 inside if using dhcp? Does your dhcp server transmit 8.8.8.8 as primary dns? Because you also have 192.168.1.1 in resolved.conf.


Hi Darkod -

I'm sure I've made a couple settings worse in trying my own haphazard troubleshooting this past week.

For 2a - I will attempt to set up static DNS. To my knowledge this server (from a 16.04 install and later upgraded, has always been on DHCP - perhaps due to my own lack of technical acuity in running a private server).

2b - I went back and rechecked - my /etc/resolvconf/resolv.conf.d/head now only shows the following




# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN ...I may have already changed that back after realizing it was a faulty setting.

[darkod]

Can you post the output of the following (I saw dnsmasq mentioned so we should know if it's installed):

apt-cache policy dnsmasq
sudo netstat -plunt | grep 53

[ericbowers]

Can you post the output of the following (I saw dnsmasq mentioned so we should know if it's installed):

apt-cache policy dnsmasq
sudo netstat -plunt | grep 53

darkod - I see the following:


$ apt-cache policy dnsmasq
dnsmasq:
Installed: (none)
Candidate: 2.79-1
Version table:
2.79-1 500
500 http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages
500 http://us.archive.ubuntu.com/ubuntu bionic/universe i386 Packages



and...


$ sudo netstat -plunt | grep 53
[sudo] password for X?X?:
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 1822/systemd-resolv
udp 6912 0 0.0.0.0:5353 0.0.0.0:* 1303/avahi-daemon:
udp 0 0 127.0.0.53:53 0.0.0.0:* 1822/systemd-resolv
udp6 25856 0 :::5353 :::* 1303/avahi-daemon:

[darkod]

So dnsmasq is not installed, that is OK. And there seems to be something listening on port 53 but I can't figure out if that's local resolvconf process or not (don't have a 18.04 yet to try compare).

Did you have opportunity to set up static IP in netplan and check how it goes? I would start from there... Don't forget when saying static IP we actually mean the rest of networking parameters that are necessary (netmask, gateway, dns server...).

[ericbowers]

Darko, can you suggest a terminal prompt I should enter to determine my netmask, gateway, and DNS server settings? (I am still rather n00b-ish on these kinds of settings).


I am pasting my current settings as I work toward getting the static IP set up (and bearing in mind that my /etc/network/interfaces and dnsmasq are probably irrelevant.

-----------------------------------------------------------------------

$ sudo cat /etc/netplan/01-netcfg.yaml
# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
version: 2
renderer: networkd
ethernets:
enp2s0:
dhcp4: no
dhcp6: no
addresses: [192.168.1.1/24]
gateway4: 192.168.0.1
nameservers:
addresses: [8.8.8.8,8.8.4.4]


-----------------------------------------------------------------------

$ sudo cat /etc/systemd/resolved.conf
[Resolve]
DNS=127.0.1.1
#FallbackDNS=
#Domains=
#LLMNR=no
#MulticastDNS=no
#DNSSEC=no
#Cache=yes
#DNSStublistener=yes
DNSStubHandler=no

-----------------------------------------------------------------------

cat /etc/dnsmasq.d/lxd
# Tell any system-wide dnsmasq instance to make sure to bind to interfaces
# instead of listening on 0.0.0.0
# WARNING: changes to this file will get lost if lxd is removed.
bind-interfaces
except-interface=lxdbr0

-----------------------------------------------------------------------

cat /etc/dnsmasq.d-available/lxd
# Tell any system-wide dnsmasq instance to make sure to bind to interfaces
# instead of listening on 0.0.0.0
# WARNING: changes to this file will get lost if lxd is removed.
bind-interfaces
except-interface=lxdbr0

-----------------------------------------------------------------------

sudo cat /etc/network/interfaces
# The loopback network interface
auto lo eth0
iface lo inet loopback


# The primary network interface
auto eth0
iface eth0 inet dhcp

-----------------------------------------------------------------------
Thank you.

[darkod]

If the server IP is 192.168.1.1/24 that covers the subnet 192.168.1.1-192.168.1.254. If the gateway is really 192.168.0.1, it would be in different subnet and the server can't reach it.

If you put it back to dhcp and run ifconfig (if it works in 18.04) it would show you the IP. And 'route -n' will show you the gateway. You can work from there to figure out your LAN subnet. And make sure you assign free static IP from it, no conflicts.

In a small setup you would have simple /24 subnet, like 192.168.1.x or 192.168.0.x.

[ericbowers]

Darko - I have set my netplan back to the following...


cat /etc/netplan/01-netcfg.yaml
# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
version: 2
renderer: networkd
ethernets:
enp2s0:
dhcp4: true



My ifconfig now shows...


$ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.96 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 2605:a601:225:3f00:1e1b:dff:feff:25ae prefixlen 64 scopeid 0x0<global>
inet6 fe80::1e1b:dff:feff:25ae prefixlen 64 scopeid 0x20<link>
ether 1c:1b:0d:ff:25:ae txqueuelen 1000 (Ethernet)
RX packets 393 bytes 43769 (43.7 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 331 bytes 46049 (46.0 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0


lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 43 bytes 5241 (5.2 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 43 bytes 5241 (5.2 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0


tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500
inet 10.8.0.1 netmask 255.255.255.255 destination 10.8.0.2
inet6 fe80::54ad:1dd4:5b6b:9a6d prefixlen 64 scopeid 0x20<link>
unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 100 (UNSPEC)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6 bytes 288 (288.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0



and my route -n shows...


$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
10.8.0.0 10.8.0.2 255.255.255.0 UG 0 0 0 tun0
10.8.0.2 0.0.0.0 255.255.255.255 UH 0 0 0 tun0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eth0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0




...trying your suggestion now. Thx.

[ericbowers]

Edited below code about one hour after first posting:

Darko and others -

My current netplan is set to



network:
version: 2
renderer: networkd
ethernets:
enp2s0:
dhcp4: no
dhcp6: no
addresses: [192.168.1.2/24]
gateway4: 192.168.1.1
nameservers:
addresses: [8.8.8.8,8.8.4.4]





and my current /etc/systemd/resolved.conf is set to:



cat /etc/systemd/resolved.conf
[Resolve]
DNS=8.8.8.8
#FallbackDNS=
#Domains=
#LLMNR=no
#MulticastDNS=no
#DNSSEC=no
#Cache=yes
#DNSStublistener=yes
DNSStubHandler=no



Still not getting outside connection (I know practically nothing). Thanks again.


LABOR DAY LATE NIGHT EDIT:
"Cannot find device enp2s0" -- this must be a lot of my problem. Could I have suggestions on why this is not detectable to my system. That is the usual "device."

[darkod]

I see you also have vpn connection. It might be related. Disable the vpn temporarily, then check if the server is working on its own.

Unless the 192.168.1.2 is used be another device (IP conflict), it should work.

Try pinging in order:
192.168.1.1 (to check conn to gateway)
8.8.8.8 (to check internet)
google.com (to check dns)

[ericbowers]

Darko - maybe there is some sort of IP conflict.

Pinging 192.168.1.1 - success
Pinging 8.8.8.8 - success
Pinging google.com - Name or service not known

...however, I see my server machine's local IP shows to be 192.168.1.2 according to my router settings and also the following...


ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 1c:1b:0d:ff:25:ae brd ff:ff:ff:ff:ff:ff
inet 192.168.1.2/24 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever
inet 192.168.1.96/24 brd 192.168.1.255 scope global secondary eth0
valid_lft forever preferred_lft forever
inet6 2605:a601:225:3f00:1e1b:dff:feff:25ae/64 scope global dynamic mngtmpaddr noprefixroute
valid_lft 3596sec preferred_lft 3596sec
inet6 fe80::1e1b:dff:feff:25ae/64 scope link
valid_lft forever preferred_lft forever
3: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100
link/none
inet 10.8.0.1 peer 10.8.0.2/32 scope global tun0
valid_lft forever preferred_lft forever
inet6 fe80::2909:91bd:3648:bec7/64 scope link stable-privacy
valid_lft forever preferred_lft forever


for some reason I now have two separate local IP addresses to the exact same machine/device. Any tips? I may well have messed up something further prior while attempting a fix in this DNS resolution.

[darkod]

Ok, so ip connectivity and routing is ok. You only have dns issues. Lets see which dns are you using:
nslookup google.com

The second ip I think is from the /etc/network/interfaces setup. You said this was an upgrade? Not sure what it does during upgrades, netplan should have taken over. Unless you put the config in the interfaces file yourself manually.

[darkod]

I just found this: https://askubuntu.com/questions/1021884/no-internet-after-upgrade-from-16-04-to-18-04

Look at the answer by Paul. Even though his answer is not the most voted one, I like what he says. Looks like permanent solution if it works for you. And it explains from where the 127.0.0.53 listener is coming. I didn't know that when we saw it in your netstat output because it is new in 18.04.

If his method doesn't work you can still try the most voted answer by Zoltan.

[ericbowers]

Still nothing. I've tried all of those fixes. Now my OpenVPN tunnel will not even authenticate even though I have made sure to turn the OpenVPN service back on.

Are there any other settings I can post to help us narrow down the problem?

[ericbowers]

I've done something causing it to now be impossible to shell into the system externally. I must do it only from inside my home network, behind my router.

[darkod]

How were you accessing externally? Over the vpn?

Otherwise router port forwarding usually needs static IP and you were using dhcp.

I wouldn't worry about this too much right now, once you get connectivity as it should be, things will work out again. It's probably just different IP now so the external traffic is not forwarded to it.

Again, I still haven't got the opportunity to do 16.04 -> 18.04 upgrade and I don't know if /etc/network/interfaces keeps living or it gets "destroyed" because netplan should take over the network setup. Did you have to do any manual actions in netplan or /etc/network/interfaces?

[ericbowers]

below are results of "sudo netplan --debug apply"



sudo netplan --debug apply
[sudo] password for ericbowers:
** (generate:4191): DEBUG: 13:44:48.499: Processing input file //etc/netplan/01-netcfg.yaml..
** (generate:4191): DEBUG: 13:44:48.499: starting new processing pass
** (generate:4191): DEBUG: 13:44:48.499: eth0: setting default backend to 1
** (generate:4191): DEBUG: 13:44:48.499: Generating output files..
** (generate:4191): DEBUG: 13:44:48.499: NetworkManager: definition eth0 is not for us (backend 1)
DEBUG:netplan generated networkd configuration exists, restarting networkd
DEBUG:no netplan generated NM configuration exists
DEBUG:device lo operstate is unknown, not replugging
DEBUG:netplan triggering .link rules for lo
DEBUG:device eth0 operstate is up, not replugging
DEBUG:netplan triggering .link rules for eth0
DEBUG:device tun0 operstate is unknown, not replugging
DEBUG:netplan triggering .link rules for tun0



Does this mean anything? "NetworkManager: definition eth0 is not for us (backend 1)"

[ericbowers]

How were you accessing externally? Over the vpn?

Otherwise router port forwarding usually needs static IP and you were using dhcp.

I wouldn't worry about this too much right now, once you get connectivity as it should be, things will work out again. It's probably just different IP now so the external traffic is not forwarded to it.

Again, I still haven't got the opportunity to do 16.04 -> 18.04 upgrade and I don't know if /etc/network/interfaces keeps living or it gets "destroyed" because netplan should take over the network setup. Did you have to do any manual actions in netplan or /etc/network/interfaces?


Yes, most of the time tunneling into the server through my VPN. Once I began having the DNS resolution problems, I could also shell into the system remotely through my phone's hotspot or a third-party VPN I use. As of now though it also seems I cannot shell into the system externally to my own home network. Strange. I will check my router settings now.

[darkod]

Did you try removing eth0 from /etc/network/interfaces? I am asking because I believe it gets confused between the two different types of setting up networking. You have config for eth0 in interfaces file (old way) and netplan (the new way for ubuntu 18.04).

[ericbowers]

Thanks so far but still no luck, still the exact same problem. I have tried removing the "eth0" from "/etc/network/interfaces", and I have also just tried completely deleting the "/etc/network/interfaces" file. Still the same lack of connectivity.

When I have this attempted configuration, my OpenVPN will not connect. When I revert back to the DHCP settings in "/etc/network/interfaces" and in the /etc/netplan/01-netcfg.yaml," my OpenVPN tunnel will still authenticate so I can mount my file system remotely, albeit still with no Internet connectivity when routed through the VPN tunnel.

I am pasting my attempted settings for static netplan again as they are right now - if anyone sees an error or can think of something additional I should look into, please let me know.

/etc/network interfaces completed deleted for now (I have a backup copy to reinstate if necessary).

Worth noting also that normally my internal IP for my server (inside my home router's network) would be 192.168.1.96. That is set in my Google Fiber network box as a static IP for the server machine to have open ports accessible while I am away. For now, that above local address will not work. Only 192.168.1.2 will route to my server and allow me to login now while inside my home network. This is all very strange and new to me.


sudo cat /etc/netplan/01-netcfg.yaml
[sudo] password for ericbowers:
# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
version: 2
renderer: networkd
ethernets:
eth0:
dhcp4: no
dhcp6: no
addresses: [192.168.1.2/24]
gateway4: 192.168.1.1
nameservers:
addresses: [8.8.8.8,8.8.4.4]



sudo cat /etc/systemd/resolved.conf
[Resolve]
DNS=8.8.8.8
FallbackDNS=8.8.4.4
#Domains=
#LLMNR=no
#MulticastDNS=no
#DNSSEC=no
#Cache=yes
#DNSStublistener=yes
#DNSStubHandler=yes


-- Below - could my entire trouble be these settings for DNS in my router/network box set-up? I had them that way because I thought I was doing myself a favor in case my home IPv4 address ever changes on its own. I may have been entirely wrong.

https://c2.staticflickr.com/2/1848/43764294604_55eb2a2309_b.jpg
https://www.flickr.com/photos/ericbowers/43764294604.jpg
https://www.flickr.com/photos/ericbowers/43764294604.jpg

[darkod]

Actually the netplan config looks good to me. And you can see in resolved.conf you now have both DNS and FallbackDNS which makes sense (both servers you put in the netplan dns section).

The server IP should now be .2 and anything you had relating to .96 should not work, that is normal. What is strange is if dns resolution is not working still from the server. It has static setup to use google dns servers.

Try again with nslookup to check which server it uses, it will list first the server it uses and then the reply (if there is one):

nslookup google.com

That will tell you if using 8.8.8.8.

About your router question, I can't know much from that screenshot. it is for you to investigate. But usually you can freely put google dns nameservers in the router too. I prefer it instead of using ISP/router dns.

However these settings should make no difference for your server, because it has static setup.

[ericbowers]

Here's what I have for that nslookup -


nslookup google.com
;; connection timed out; no servers could be reached


...and can still ping successfully to 8.8.8.8 but not to google.com.

[darkod]

This is really very weird... I forgot to ask, now that you removed the interfaces file, does it still give the same debug output as above when running netplan apply? Has that changed at least?

PS. And also, after these recent changes, did you reboot the server to give it chance to boot from scratch? Maybe it will change something...

[ericbowers]

Darko - when I run 'sudo netplan apply' and everything is correct with my /etc/netplan/01-netcfg.yaml settings, no debug into shows - and this is the same whether or not the /etc/network/interfaces file is active, commented out, or deleted.

I have indeed been restarting/rebooting the machine every time I make these changes but still nothing. I am pretty sure I must have done something, somewhere to a file or setting while I was making my own ill-advised troubleshooting attempts last week.

Could it still be something to do with ifup/ifdown?

or...


/etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile


[ifupdown]
managed=true


[device]
wifi.scan-rand-mac-address=no

[darkod]

In the server version NM is not used (not by default anyway). If you look at the yaml, the line renderer = networkd means that. networkd will be used, not NM.

As far as I know NM is still used in the desktop version in 18.04.

So the above file content even though it's there, is probably not used. I say probably because you can't be sure what you changed during your attempts.

Actually if you can trace back those changes (or most of them) it would probably be helpful.

[darkod]

PS. Actually, I should have probably suggested this much earlier. Start checking the logs in /var/log (everything that looks related). Start with syslog for example. Try the nslookup and then check the main log files that look related. It might have some clues. We didn't get very far otherwise... :(

[Bashing-om]

darkod -

I am trying to follow along - I am presently booting 18.04 as a core install with xfce as the DE.


sysop@x1804mini:~$ dpkg -l network-manager
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-=================================
ii network-manage 1.10.6-2ubun amd64 network management framework (dae

cat /etc/netplan/01-network-manager-all.yaml
# Let NetworkManager manage all devices on this system
network:
version: 2
renderer: NetworkManager
sysop@x1804mini:~$


Will show what I have.

[darkod]

Yeah, for a desktop system I expected NM to handle networking. Some insight about the DNS system would be useful. :)

Unfortunately I still don't have my 18.04 server. Have been delaying re-installing my home server for months. Planned to do it next weekend...

[darkod]

One more test. What is the result of the following:

nslookup google.com 8.8.8.8

That should query google dns directly. Just to make sure it works that way.

PS. Some people report success when recreating the link to resolv.conf (here among other https://askubuntu.com/questions/1048641/18-04-ignoring-dhcp-provided-dns-server):

sudo mv resolv.conf resolv.conf-old
sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf

[darkod]

And another thing. When I ask for nslookup output please post the whole result, not just the final line. The more important part is above it, telling you which dns server it is querying. I don't have much experience with 18.04 but from what I have seen so far in your case:
1. Network communication is working OK, you can ping the gateway and 8.8.8.8.
2. The dns server queried by default is the stub 127.0.0.53 which is new in 18.04 (not directly the nameservers you have specified).
3. This stub 127.0.0.53 actually does not know how to pass the queries to 8.8.8.8 (it is either corrupted, wrongly configured, or whatever).

I don't know yet how you can try reinstalling the stub. Maybe reinstalling the package systemd-resolve (or something similar)???

[darkod]

I have found more comments online saying that linking resolv.conf to /run/ fixed the issue for them. It seems that way you avoid using the stub 127.0.0.53 which is exactly the part that is giving you problems. So if you try the commands from post #34 with little luck you might get it resolved.

You have the correct values in netplan and systemd resolve.conf, so just creating that link should be enough...

[ericbowers]

I'm sure we're getting closer but for now nothing on the DNS resolution.

I ran the following per post 34 in this thread:


sudo mv /etc/resolv.conf /etc/resolv.conf-old
sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf



I see this which could help us narrow this down (Too many nameservers?)...


cat /etc/resolv.conf
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients directly to
# all known uplink DNS servers. This file lists all configured search domains.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.


nameserver 8.8.8.8
nameserver 192.168.1.1
nameserver 8.8.4.4
# Too many DNS servers configured, the following entries may be ignored.
nameserver fe80::f6f5:e8ff:fe8c:7029%2



Here is my nslookup:


nslookup google.com 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8#53


Non-authoritative answer:
Name: google.com
Address: 172.217.11.142
Name: google.com
Address: 2607:f8b0:4009:815::200e


There was nothing else returning to paste in those prior nslookups of just google.com.

[ericbowers]

Here is a snippet from my /var/log/syslog :



p 5 20:30:22 server systemd[1]: Started Wait until snapd is fully seeded.
Sep 5 20:30:22 server systemd[1]: Started Disk Manager.
Sep 5 20:30:22 server udisksd[1283]: Acquired the name org.freedesktop.UDisks2 on the system message bus
Sep 5 20:30:22 server udisksd[1283]: mountpoint /media/ericbowers/Ubuntu-Server 17.10 amd64 is invalid, cannot recover the canonical path
Sep 5 20:30:22 server udisksd[1283]: Cleaning up mount point /media/ericbowers/Ubuntu-Server 17.10 amd64 (device 8:49 is not mounted)
Sep 5 20:30:23 server ModemManager[1298]: <info> Couldn't check support for device at '/sys/devices/pci0000:00/0000:00:1c.0/0000:02:00.0': not supported by any plugin
Sep 5 20:30:24 server systemd-networkd[1488]: eth0: Gained carrier
Sep 5 20:30:24 server avahi-daemon[1297]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.1.2.
Sep 5 20:30:24 server NetworkManager[1492]: <info> [1536197424.1370] device (eth0): carrier: link connected
Sep 5 20:30:24 server avahi-daemon[1297]: New relevant interface eth0.IPv4 for mDNS.
Sep 5 20:30:24 server systemd-networkd-wait-online[1635]: ignoring: lo
Sep 5 20:30:24 server avahi-daemon[1297]: Registering new address record for 192.168.1.2 on eth0.IPv4.
Sep 5 20:30:24 server systemd-networkd-wait-online[1635]: ignoring: lo
Sep 5 20:30:24 server systemd-timesyncd[1086]: Network configuration changed, trying to establish connection.
Sep 5 20:30:24 server kernel: [ 13.052208] r8169 0000:02:00.0 eth0: link up
Sep 5 20:30:24 server kernel: [ 13.052214] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Sep 5 20:30:24 server sh[1497]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8 (xid=0xc177d970)
Sep 5 20:30:24 server sh[1497]: DHCPREQUEST of 192.168.1.96 on eth0 to 255.255.255.255 port 67 (xid=0x70d977c1)
Sep 5 20:30:24 server sh[1497]: DHCPOFFER of 192.168.1.96 from 192.168.1.1
Sep 5 20:30:24 server dhclient[1595]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8 (xid=0xc177d970)
Sep 5 20:30:24 server dhclient[1595]: DHCPREQUEST of 192.168.1.96 on eth0 to 255.255.255.255 port 67 (xid=0x70d977c1)
Sep 5 20:30:24 server dhclient[1595]: DHCPOFFER of 192.168.1.96 from 192.168.1.1
Sep 5 20:30:24 server dhclient[1595]: DHCPACK of 192.168.1.96 from 192.168.1.1
Sep 5 20:30:24 server sh[1497]: DHCPACK of 192.168.1.96 from 192.168.1.1
Sep 5 20:30:24 server avahi-daemon[1297]: Registering new address record for 192.168.1.96 on eth0.IPv4.
Sep 5 20:30:24 server systemd[1]: Stopping Network Name Resolution...
Sep 5 20:30:24 server systemd[1]: Stopped Network Name Resolution.
Sep 5 20:30:24 server systemd[1]: Starting Network Name Resolution...
Sep 5 20:30:24 server systemd-resolved[1851]: Positive Trust Anchors:
Sep 5 20:30:24 server systemd-resolved[1851]: . IN DS 19036 8 2 49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1 cdde32f24e8fb5
Sep 5 20:30:24 server systemd-resolved[1851]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457 104237c7f8ec8d
Sep 5 20:30:24 server systemd-resolved[1851]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal intranet lan local private test
Sep 5 20:30:24 server systemd-resolved[1851]: Using system hostname 'server'.
Sep 5 20:30:24 server systemd[1]: Started Network Name Resolution.
Sep 5 20:30:24 server dhclient[1595]: bound to 192.168.1.96 -- renewal in 41820 seconds.
Sep 5 20:30:24 server sh[1497]: bound to 192.168.1.96 -- renewal in 41820 seconds.
Sep 5 20:30:24 server ifup[1517]: /bin/run-parts --exit-on-error --verbose /etc/network/if-up.d
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/000resolvconf
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/avahi-autoipd
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/avahi-daemon
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/ethtool
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/ip
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/openssh-server
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/openvpn
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/postfix
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/sendmail
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/vlan
Sep 5 20:30:24 server ifup[1517]: run-parts: executing /etc/network/if-up.d/wpasupplicant
Sep 5 20:30:24 server systemd[1]: Started Raise network interfaces.
Sep 5 20:30:24 server systemd[1]: Reached target Network.
Sep 5 20:30:24 server systemd[1]: Started Unattended Upgrades Shutdown.
Sep 5 20:30:24 server systemd[1]: Starting OpenVPN service...
Sep 5 20:30:24 server systemd[1]: Starting Fail2Ban Service...
Sep 5 20:30:24 server systemd[1]: Starting A high performance web server and a reverse proxy server...
Sep 5 20:30:24 server systemd[1]: Started OpenVPN service.
Sep 5 20:30:24 server systemd[1]: Started Fail2Ban Service.
Sep 5 20:30:24 server systemd[1]: Started A high performance web server and a reverse proxy server.

[darkod]

Damn, it all looks good... You have the resolv.conf correct, nslookup works when querying 8.8.8.8 directly.

One thing I noticed is that you still receive dhcp with .96 address. Didn't you remove all dhcp setup and use only static .2? How come syslog is showing dhcprequest and dhcpoffer for .96 and later bound to it.

Did you restart the server after doing the ln command? Just in case it helps...

I literary have no more ideas because all I have seen is looking good... :(

[ericbowers]

Well, I think I'm still getting closer. Today the GUI update installer on my server set something about updates involving a DNS/name resolution issue. I cannot download it of course because of my problem we've been working on here. I cannot find the update online to download via "wget." Any hints?

One other positive development - I can now use my VPN to access the server remotely with my netplan static file configured. I had to enter the following command:

sudo ip addr add 192.168.1.2 dev eth0
ip link set dev eth0 up




which I took from here.... https://help.ubuntu.com/lts/serverguide/network-configuration.html.en

In my prior attempts at a solution before opening this thread, I may have entered some sort of ill-advised terminal command that only turned out to make my problems worse in the near-term.

Is there any way I can look up what this software update was that's supposed to deal with a code bug in 18.04.1 Server's DNS resolution?

Thanks for all the help so far.

[ericbowers]

Scratch the part about my VPN working outside of my home network while the server's Netplan is set to static. Apparently it still doesn't. Ill keep looking for a way to get that DNS resolver update installed and then see what happens.

[darkod]

This thread got so long that I don't remember now if it was discussed. Did you check the /etc/resolv.conf and does it look correct? If not, how about manually creating it and putting 8.8.8.8 nameserver inside?
But the problem might be that this new stub 127.0.0.53 might be ignoring it...

[darkod]

I just checked few posts back, resolv.conf is ok. Since I suspect the stub is not working correctly for you, how about trying to stop it?

sudo systemctl stop systemd-resolved
sudo netstat -plunt | grep 53

The second command will show you if 127.0.0.53 is still listening on port 53. If it's gone, check nslookup and dig to see if they resolve correctly.

PS. Is resolvconf installed?

apt-cache policy resolvconf

[Bashing-om]

Guys, -

I really do not want to appear as stupid, but -

release 18.04 and the interface is still identified as eth0 ?
https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
where I would expect a name such as " enp1s9" and to apply on a server install as well .



hey, it's a thought

[darkod]

You have a point. If I remember correctly this is an upgrade from 16.04 but even then the new interface naming existed.
Not sure what interface name to expect if it's upgrade all the way back from 14.04...

[ericbowers]

Bashing-Om - yessir! I used to have "enp2s0" until this debacle but it seems to have gone away. Everything (ifconfig, route -n) only shows eth0?

Any ideas on how I get "enp2s0" to reappear? Whenever I mention it to the terminal, I'm told something like "Unknown interface."

Darko and Bashing - do you know where I can find that giant online library of Ubuntu subfolders containing individual programs? What would really be helpful is if I had a way of downloading some sort of update that flashed on my GUI today about "DNS nameserver bug fixes yadda yadda something or other." Sounded perfect. If I had the actual link to this in a repository, I could hopefully just use "wget" and substitute the ip address of the US server with the rest of the URL.

[Bashing-om]

ericbowers; Hummmmm ...

All I can do is poke and stroke . My netplan skills still reek to this time. I am willing to learn !

What does the system expect for the name ?


ls /sys/class/net

Any activity on the enp2s0 interface ?


ls -al /sys/class/net/enp2s0/statistics/


In my mind we need to bring that enp2s0 interface up.



just a thought

[ericbowers]

Bashing -


ls /sys/class/net
eth0 lo tun0


and...


ls -al /sys/class/net/enp2s0/statistics/ls -a
ls: cannot access '/sys/class/net/enp2s0/statistics/ls': No such file or directory

[ericbowers]

Can I have instructions on how to install "firmware.xml.gz.asc"? I've downloaded it but I don't know how to make the machine install it.

https://c1.staticflickr.com/2/1867/44526298701_cefd2701d7_b.jpg

[darkod]

Unfortunately at this point it might be best to consider reinstalling clean new 18.04. How complicated would be for you to restore the necessary configurations if you did a clean install?

This looks like too many things have broken... This is not only about dns not working.

Considering the time you already spent troubleshooting and how much more you might spend without any guarantee of success, I would consider reinstalling. It will be faster and you get a good, clean system.

[darkod]

Just FYI, I installed my home server with 18.04.1 this weekend (clean install) and apart from some minor issues due to compatibility with the new Ryzen 5 2400G CPU, all the rest went just fine.

The DNS stub 127.0.0.53 is created automatically and listening on port 53. The netplan file was also created based on the info I put in during installation. The /etc/network/interfaces file is empty.

I don't have any problems with either network/internet connectivity or DNS resolving.

Like I said before, it might be a good moment for you to evaluate whether a new clean install will get you faster to a fully working system...

[ericbowers]

Hey Darko - indeed; I decided I agree with you. I backed up my most important config files for various stuff and then took a blow torch to the old installation.

At first I reinstalled a fresh 18.04.1 - still had DNS problems. I'm not entirely confident in the viability of 18.04 yet for the use of a trouble-free server machine. A lot of things have not gone very smoothly since I upgraded my prior install (which actually looks to have been 17.10 - not the LTS). And I see a lot of similar problems with DNS on 18.04 on the forums this year.

So I ended up flashing the 16.04 LTS and installed that system. I've got most of my important stuff back to working properly. I'll be working on configuring my static DNS for this 16.04.5 installation fairly soon, but for now it works at least so I can tunnel into my machine while away and still get work done.

Thanks for bearing with me through all of my problems last week.

[sioxs]

Had the same issue upgrading from 16.04 to 18.10 - DNS could not resolve addresses although ping did reach network ip addresses - therefore no repositories could be reached to complete the upgrade process. Normally I would never attempt doing this with wifi & always used ethernet cable for installs. For the sake of testing & curiosity I choose to try it this way.

The solution was quite simple after searching what others had attempted running into the same issues.

Edit /etc/NetworkManager/NetworkManager.conf with your favorite editor

Change
dns=dnsmasq

to

dns=systemd-resolved

Restart NetworkManager

sudo systemctl restart NetworkManager

You should now have DNS resolving properly.
Other solutions can be found https://www.ubuntugeek.com/how-to-fix-dns-problems-after-upgrading-ubuntu-17-10-from-ubuntu-17-04-16-10-16-04.html