Server Finally Hacked

[starz677]

Ubuntu 10.04
Standard install with GUI
Setup as LAMP server

Hello,

I do believe my server finally got hacked, or at least I finally found evidence.

The evidence came from the Fail2ban Jail.conf file

An unknown IP address had apparently been added to the Ignore list on each of the jail configurations.

I have one other IP address there and that was a local IP address.

My passwords were complex and I blocked all countries other than the US and even all the proxy server I could find.

Updates were up to date.

What was the likely vector they used to gain access?

SSH is disabled and there is no remote access configured.

Thx

[prshah]

An unknown IP address had apparently been added to the Ignore list on each of the jail configurations.

As a start, why don't you check which area the IP address belongs to? You can check it using (for example) www.ip2location.com

Of course, that's assuming it's a public ip address and not an internal one (Eg 10.0.0.1, etc)

[Ms. Daisy]

Agree with prshah. POST the evidence and we may be able to actually help.

[starz677]

Hi,

I have the physical address where the IP address was assigned or used, the ISP, the date and time and other information.. However, that won't rule out proxy use, hijacked PC's or other vectors that would render that info useless.
Someone wanted to stop fail2ban from blocking their hacking so they edited the file manually.

It's not a guess.

The server was hacked. I should not have said "I believe".

Fact: No OS is hack proof. Admin experience makes the job harder for hackers. But never impossible.


Thanks for the offer of help, but as I investigate this more, that would be an exercise in futility. The time would be better spent starting over and trying to sharpen my skills to make the new build less vulnerable. I wish I was savvy enough to install a vanilla Ubuntu server but I lack the skills to do everything from the command line and I need GUI's.

[starz677]

Here is one of the IP addresses that appears to be involved....

90.199.29.86

5ac71d56.bb.sky.com
.
.
.
.

[cariboo]

The IP address you provided proves nothing. How do you know someone edited the fail2ban configuration file? Was there a none route-able private ip address inserted into the file?

What other indication do you have that your server has been cracked?

[Ms. Daisy]

OK, so you're looking to harden the new installation now, correct? We need information to help with that.

What security measures did you have in place in the last installation? What services are you running on it? What services are facing the internet?

BTW, no time like the present to learn how to use the command line.

[prshah]

Of course finding an IP address and ISP does not rule out proxy, etc, but it's a starting point. Given that the IP address was "unblocked" in the fail2ban conf file indicates confidence that this ip address may be used again, probably indicating a static ip.

Most static ip's can then be traced to used to trace their parent source, and an email to webmaster or administrative contact can alert them to misuse if any.

Can you also check the "/var/log/auth.log" (or auth.log.1, .2.gz, .3.gz...) at about the time you believe the server was hacked? It may indicate the entry point that you are looking for.

I don't think there is any entry point that bypasses auth.log; if you can't find something suitable in auth.log, maybe you need to look at your local system for some rouge programs or "user error" / social engineering.

Suggestions only. I'm by no means a security or server expert.

[starz677]

The IP address you provided proves nothing. How do you know someone edited the fail2ban configuration file? Was there a none route-able private ip address inserted into the file?

What other indication do you have that your server has been cracked?

I want to make sure I understand you crystal clear. You say that the clear and indisputable alteration of a fail2ban configuration file and the addition of an IP address into that file which I never added is not evidence of hacking? Explain that please. I think it makes absolutely no difference what External IP address was added (but for the record, it was a static IP address in a nearby state). I was able to identify the domain location and even a physical address. However, it is also possible that that IP address at the time was being used as a proxy and was not the ORIGIN of the attack.

[starz677]

OK, so you're looking to harden the new installation now, correct? We need information to help with that.

What security measures did you have in place in the last installation? What services are you running on it? What services are facing the internet?

BTW, no time like the present to learn how to use the command line.

How I would love to. I can use the command line (to some degree) and what I dont know I can find, but an even bigger issue to me is how to monitor a headless server for malicious activity. On my desktop server I can watch ipblock as it blocks domains and I can see the domain name. with one click I can see any IP addresses tagged by fail2ban. I can use etherape to see a graphical real time representation of the bandwitdth being used by a connection.... How would I do all that on a headless server? thx