16.04 Server, SSH problems with OpenVPN, Firewall Problems With OpenVPN

[ericbowers]

Hi all - browsed around for a day on this one but couldn't find anything that quite applied to this situation.

I changed my 16.04 server configuration to static ip as shown below:


# The loopback network interface
auto lo
iface lo inet loopback


# The primary network interface
auto enp2s0
iface enp2s0 inet static
address 192.168.1.3
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 8.8.8.8 8.8.4.4

# This is an autoconfigured IPv6 interface
# iface enp2s0 inet6 auto


My OpenVPN client can authenticate into the server with its .ovpn config file, but I have no SSH connectivity to the server from the client over SSH. For instance, always getting the "connection refused" message, and I cannot mount my remote file system to my Mac client with SSHFS.

I did some prior debugging to make sure the router sends SSH connections to port 22 on the server - it does, and my firewall is allowing incoming connections - but I cannot shell into the system with my own VPN and mount the file system.

Any help is appreciated.

[darkod]

SSH is very sensitive to IP changes because they might signal foul play. If you already accessed the old ip from that client, it keeps a record in known_hosts. If now new ip has the same ssh signature from the server, it will stop and warn you when trying to connect. Do you get such warning?

To remove the old entry on the client do:

ssh-keygen -R <old_ip>

After that it should let you connect.

[The Cog]

Connection refused is a very clear message. The server has received your connection request and replied with a reset packet, so it's not a firewall issue either. It means that the server you are connecting to is not acepting connections on that port (22 for ssh by default).
Your ssh server is probably not running, or perhaps it is configured to listen specifically on the old address. But it's not listening on your new address.

Tools to check this:
From the client, try the command nc -vz 192.168.1.3 22 which I expect will confirm that the connection is being refused.

On the server, use the command netstat -lnt which lists the ports (and addresses) that the server is listening on. Both 0.0.0.0 and ::: mean it's listening on all addresses for that particular port.

On the server, use the command pgrep -a sshd which lists the process id and name if it is running.

P.S. If you're sure the server is listening on port 22, consider the possibility that you now have a duplicate IP address and it is a different box that is rejecting the call.

[ericbowers]

Darko and Cog - thanks for the replies. I ended up seeing that I changed a router setting to make incoming SSH connections route to a Raspberry Pi I put on my home network. I thought I had checked these settings over and over, but once I finally saw this culprit and got my router back sending incoming SSH requests to the correct machine and not the Pi, the issue was resolved.

Sure enough though another problem has come up (don't know how I attract so many technical difficulties - still too much of a n00b I guess). Now I have some kind of UFW/firewall problem. The UFW service has to be disabled in order for my OpenVPN connection to establish. Been troubleshooting this myself for a couple days without success (as per usual).

I tried flushing all of the iptables settings and beginning from scratch - problem persisted. Also uninstalled and reinstalled ufw, to no effect. I'm seeing frequent port-scanned, brute force login attempts that somehow don't get filtered by Denyhosts, so I would like to have some ability to manually block certain IPs. But so far even having the firewall enabled at all will prevent my OpenVPN tunnel from connecting.

Are there some system settings I can paste over that could help troubleshoot? I'm at a loss. The VPN would tunnel through the firewall with no problems before. I didn't change any settings to the best of my knowledge.

[The Cog]

Make sure your server can resolve names with DNS (udp port 53 outgoing)
OpenVPN uses a single port (tcp or udp depending on configuration) - make sure that is allowed both in and out. The other end may be on a different port number.
Make sure that traffic in/out of the vpn interface (tun0?) is allowed.

Does OpenVPN show the connection being established in its logs? If not then you are probably blocking the VPN protocol/port.
Does it appear to connect and then lock up? If so, maybe the route to the far end is being redirected down the tunnel itself, causing a loop.

[ericbowers]

Looks like I am also back to the same problem of no SSH capability to the server from inside my VPN tunnel (OpenVPN works as long as my UFW firewall is disabled; I will deal with that part after I have fixed my SSH-from-OpenVPN problem). This has been recurring intermittently. Sometimes I have it fixed (though not entirely sure how, but in less than 24 hours the problem begins again).



SSH is very sensitive to IP changes because they might signal foul play. If you already accessed the old ip from that client, it keeps a record in known_hosts. If now new ip has the same ssh signature from the server, it will stop and warn you when trying to connect. Do you get such warning?
To remove the old entry on the client do:

ssh-keygen -R <old_ip>
After that it should let you connect.

I did this on client side after the problem began again but no effect unfortunately. My current error message is 'timed-out"


ssh: connect to host 10.8.0.1 port 22: Operation timed out


Any hints as to where I should begin?

[darkod]

Did you also try to ping 10.8.0.1 from the vpn client? If you have no firewall on the vpn server ping should also work.

If you add more verbocity to ssh it will also show you more debug messages while trying to connect. Something like ssh -vv or -vvv.

[ericbowers]

Pinging from VPN client results in timeouts -


ping 10.8.0.1
PING 10.8.0.1 (10.8.0.1): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1


Here's what I'm seeing when trying to login with -vv


ssh -vv MYUSERNAME@10.8.0.1
OpenSSH_7.6p1, LibreSSL 2.6.2
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 10.8.0.1 port 22.
ssh: connect to host 10.8.0.1 port 22: Operation timed out



Question... above refers to OpenSSH, but that is not installed on my system for SSH. I have something called OpenBSD, for whatever reason. Could this be a culprit?

[ericbowers]

ssh: connect to host 10.8.0.1 port 22: Operation timed out


When attempting to connect by SSH from inside my OpenVPN client, the above response keeps happening. However I can still SSH into the machine from any other connection external to my OpenVPN arrangement. I notice above that Port 22 times out when receiving the SSH request from my VPN client, which is successfully connecting to the same machine on port 1194 with UDP.

[darkod]

It looks to me like your own firewall is blocking you. Can you share more details about the FW and post the rules you have in place about SSH?

[ericbowers]

Not sure if it could be the firewall right now; although I definitely do have problems with my firewall (start of thread), I have it deactivated for now until I can that other problems sorted out.

Below is my firewall status right now, even as I still cannot use SSH to access my system from inside my VPN.


service ufw status
● ufw.service - Uncomplicated firewall
Loaded: loaded (/lib/systemd/system/ufw.service; enabled; vendor preset: enab
Active: inactive (dead) since Mon 2018-10-01 23:42:37 CDT; 2 days ago
Process: 27765 ExecStop=/lib/ufw/ufw-init stop (code=exited, status=0/SUCCESS)
Process: 27723 ExecStart=/lib/ufw/ufw-init start quiet (code=exited, status=0/
Main PID: 27723 (code=exited, status=0/SUCCESS)


Oct 01 23:42:23 server systemd[1]: Starting Uncomplicated firewall...
Oct 01 23:42:23 server systemd[1]: Started Uncomplicated firewall.
Oct 01 23:42:37 server systemd[1]: Stopping Uncomplicated firewall...
Oct 01 23:42:37 server ufw-init[27765]: Skip stopping firewall: ufw (

[ericbowers]

Okay I solved this one nagging issue of my server rejecting my SSH attempts from inside my OpenVPN client tunnel.

Here's what happened - checked the following:


sudo iptables -L INPUT
revealed the following...

DROP all -- 10.8.0.6 anywhere

I deleted that entry by way of

sudo iptables -D INPUT 2
...where '2' means the second line from sudo iptables -L INPUT


....on to the remaining set of problems I've probably created for myself. I'm only at the stage of knowing Linux only enough that I can be very dangerous to my own configurations.

I'm curious as to why any of this makes a difference given that my firewall (ufw) has been disabled while I work to make sure I can even access my rig remotely?

[darkod]

The bad thing with ufw is that it is only a frontend for iptables (not very good, too). So in the background there might be iptables rules that you are not even aware of, or remains...

I remember 5-6 years ago it looked to me too that ufw is easier to use than iptables. I used it once and realized that in fact iptables is much simpler and cleaner. It shows you the applied rules without being worried that something else might be hidden somewhere.

Now that the SSH is resolved, do you have other issues too?

[ericbowers]

Darko - are there alternatives to UFW? For now I have Denyhosts running also but there are some brute-force botnet SSH attempts that don't get filtered by Denyhosts for some reason. Would it be just as well to simply keep using Denyhosts and manually adjust IPtables rules while leaving UFW disabled or even uninstalling the program?

[darkod]

I would simply disable ufw. You don't really need to remove the package. Then you simply control iptables rules, and that's your FW.

I am not familiar with Denyhosts. If you have lots of intents you can use fail2ban (read on it, I don't use it myself).

Any server on the public intenet would get hack attempts today. That doesn't mean they can get in. You can also set openssh to use only keys and disable password login. Keys are (almost) impossible to break.