Page 1 of 2 12 LastLast
Results 1 to 10 of 16

Thread: I got hacked twice on Ubuntu 18.04, don't want third time

  1. #1
    Join Date
    May 2021
    Beans
    5

    I got hacked twice on Ubuntu 18.04, don't want third time

    Hosting gave me fresh installation of Ubuntu 18.04. I changed password for root, created other user (but never used it), installed Apache, PHP, MySQL, created remote user for Mysql and uploaded my website. After few days hosting shut down the server, because it was attempting to connect to other servers.

    I reinstalled whole thing. And few days I was shut down again. But I don't know how I'm getting hacked.

    -------

    "kthreaddi", which is some crypto miner. Could this help me? Check which user created it, what rights it has (if they somehow compromise root or what)? I still have infected server running here (disconnected from internet), any input is very welcome.


  2. #2
    Join Date
    Jul 2008
    Location
    The Left Coast of the USA
    Beans
    Hidden!
    Distro
    Kubuntu

    Re: I got hacked twice on Ubuntu 18.04, don't want third time

    I recommend you don't use root at all. Delete the account's password and you will effectively disable the account. Use your other account as a super-user and make sure your password is complex.

    Do you use SSH to log in? Don't allow root login, don't use password login. Use public/private keys. Allow SSH only over a high-numbered port.

    You currently have SSH on port 22 (the standard) and you are using a root account. That means a hacker need only brute-force your root password to get in. He has the port and the user already.

    How frequently do you review your logs?
    Last edited by QIII; May 22nd, 2021 at 07:54 PM.
    Please read The Forum Rules and The Forum Posting Guidelines

    A thing discovered and kept to oneself must be discovered time and again by others. A thing discovered and shared with others need be discovered only the once.
    This universe is crazy. I'm going back to my own.

  3. #3
    Join Date
    May 2021
    Beans
    5

    Re: I got hacked twice on Ubuntu 18.04, don't want third time

    Quote Originally Posted by QIII View Post
    I recommend you don't use root at all. Delete the account's password and you will effectively disable the account. Use your other account as a super-user and make sure your password is complex.

    Do you use SSH to log in? Don't allow root login, don't use password login. Use public/private keys. Allow SSH only over a high-numbered port.

    You currently have SSH on port 22 (the standard) and you are using a root account. That means a hacker need only brute-force your root password to get in. He has the port and the user already.

    How frequently do you review your logs?

    First time I was hacked after about 7 months. But second time I was hacked within 24 hours. Second password was "karel4029euplavby" - not the best, but how do you brute force it in 24 hours, especially it being non-interesting target?

    I understand that better login security is a way to go. But before I wipe out infected server is there something more I can do to identify where did it come from?

  4. #4
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: I got hacked twice on Ubuntu 18.04, don't want third time

    Quote Originally Posted by andymyvps View Post
    First time I was hacked after about 7 months. But second time I was hacked within 24 hours. Second password was "karel4029euplavby" - not the best, but how do you brute force it in 24 hours, especially it being non-interesting target?

    I understand that better login security is a way to go. But before I wipe out infected server is there something more I can do to identify where did it come from?
    You will need to review logs, but if you don't have a firewall in place that logs outbound traffic, it's unlikely you'll find anything useful outside of checking which processes are running.

    In the OP, you didn't really say what you did to lock your server down or what you were running on it - only that you had a database and web server with PHP. Are you running Wordpress or another CMS?
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

  5. #5
    Join Date
    May 2021
    Beans
    5

    Re: I got hacked twice on Ubuntu 18.04, don't want third time

    Quote Originally Posted by CharlesA View Post
    You will need to review logs, but if you don't have a firewall in place that logs outbound traffic, it's unlikely you'll find anything useful outside of checking which processes are running.

    In the OP, you didn't really say what you did to lock your server down or what you were running on it - only that you had a database and web server with PHP. Are you running Wordpress or another CMS?

    Hosting disconnected my server once they got reports about it trying to connect to other servers.

    Website made on Laravel. I only installed Apache, PHP, MySQL, Composer.

  6. #6
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 18.04 Bionic Beaver

    Re: I got hacked twice on Ubuntu 18.04, don't want third time

    Quote Originally Posted by andymyvps View Post
    Hosting disconnected my server once they got reports about it trying to connect to other servers.

    Website made on Laravel. I only installed Apache, PHP, MySQL, Composer.
    If you don't have access to the server, your best bet would be to wipe the instance and start from scratch.

    There are some good guides for hardening your server - most of them include steps to lock down SSH access and enable key-based authentication for SSH.

    Here are a couple examples:
    https://www.linode.com/docs/guides/s...g-your-server/
    https://www.digitalocean.com/communi...h-ubuntu-20-04

    FWIW, I have SSH locked down at both the firewall level and via SSH keys on my VPSes.

    You can't get attacked if it isn't open to the world. If the firewall fails for some reason, the attacker would still need my private key in order to connect.
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

  7. #7
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: I got hacked twice on Ubuntu 18.04, don't want third time

    I doubt someone is cracking your password in that time, but you really shouldn't be using passwords at all - use ssh-keys. There are thousands of how-tos for that.

    I think these are the problem: Apache, PHP, MySQL. It is easy to not configure those to be secure and crap php code is the most likely problem.

    For fun, bring up the server, don't enable apache, put all your stuff onto it, but don't allow apache to run. I bet it doesn't get hacked. Out of the box, Ubuntu Server doesn't get cracked unless you are using passwords that are better for luggage - 123456. But really, any password should be randomly generated by your password manager for all enabled accounts and you should use ssh-key-based authentication. Also, run fail2ban, so any brute force attempts are automatically blocked. Validate that too. Late 1 night, try to connect with bogus passwords until you are locked out - a dynamic firewall rule will be enabled. I think the default period is 1 hour. The next morning, you'll be able to get in.

    Re-look at your settings and security posture for php, apache and mysql. Do you block access for each of those so that only the end-users who need access can gain access? If you are in the UK and don't expect any paying clients from Latin America, perhaps some firewall rules to block that access are needed. Any country that shouldn't have access ... er ... shouldn't have access to anything on the server. If only you and a few IPs that you know should have access to ssh in, don't let anywhere else have access.

    And setup versioned backups, so you can compare the system between when it was installed and when it gets hacked. Wouldn't that be helpful to determine exactly what you've done wrong?

    It should be obvious, but we can't really help you. We don't have logins (and don't want any) on this system. Get paranoid. Look up "best practices" for each tool/server you install. Follow the best practices. Look up how to crack each tool/service you install too and ensure you don't allow those methods. I suspect you have assumed that some guide installs everything in a secure manner, which just isn't true. They are interested in a working system, not a secure system, for the most part.

    There is a mod_security for apache which can ruin your day/week too. I don't use it. Tried, but there were just too many settings. In the end, I setup a VPN and blocked all access to the servers that didn't connect through the VPN first. I have a low confidence in being able to secure any php webapp myself, so I don't allow php webapps to be accessed over the internet.

  8. #8
    Join Date
    May 2021
    Beans
    5

    Re: I got hacked twice on Ubuntu 18.04, don't want third time

    Quote Originally Posted by TheFu View Post
    I doubt someone is cracking your password in that time, but you really shouldn't be using passwords at all - use ssh-keys. There are thousands of how-tos for that.

    I think these are the problem: Apache, PHP, MySQL. It is easy to not configure those to be secure and crap php code is the most likely problem.

    For fun, bring up the server, don't enable apache, put all your stuff onto it, but don't allow apache to run. I bet it doesn't get hacked. Out of the box, Ubuntu Server doesn't get cracked unless you are using passwords that are better for luggage - 123456. But really, any password should be randomly generated by your password manager for all enabled accounts and you should use ssh-key-based authentication. Also, run fail2ban, so any brute force attempts are automatically blocked. Validate that too. Late 1 night, try to connect with bogus passwords until you are locked out - a dynamic firewall rule will be enabled. I think the default period is 1 hour. The next morning, you'll be able to get in.

    Re-look at your settings and security posture for php, apache and mysql. Do you block access for each of those so that only the end-users who need access can gain access? If you are in the UK and don't expect any paying clients from Latin America, perhaps some firewall rules to block that access are needed. Any country that shouldn't have access ... er ... shouldn't have access to anything on the server. If only you and a few IPs that you know should have access to ssh in, don't let anywhere else have access.

    And setup versioned backups, so you can compare the system between when it was installed and when it gets hacked. Wouldn't that be helpful to determine exactly what you've done wrong?

    It should be obvious, but we can't really help you. We don't have logins (and don't want any) on this system. Get paranoid. Look up "best practices" for each tool/server you install. Follow the best practices. Look up how to crack each tool/service you install too and ensure you don't allow those methods. I suspect you have assumed that some guide installs everything in a secure manner, which just isn't true. They are interested in a working system, not a secure system, for the most part.

    There is a mod_security for apache which can ruin your day/week too. I don't use it. Tried, but there were just too many settings. In the end, I setup a VPN and blocked all access to the servers that didn't connect through the VPN first. I have a low confidence in being able to secure any php webapp myself, so I don't allow php webapps to be accessed over the internet.
    You actually solved it!

    Due to limited experience with running servers I was convicted I did something wrong and it has to do something with server/Ubuntu. It didn't. It was website itself, which I didn't analyze until you mentioned it.

    Pulling some PHP error logs I was able to find out, that it was due to vulnerable Laravel package https://github.com/facade/ignition/issues/350

    Thanks!

  9. #9
    Join Date
    Mar 2011
    Location
    U.K.
    Beans
    Hidden!
    Distro
    Ubuntu 20.04 Focal Fossa

    Re: I got hacked twice on Ubuntu 18.04, don't want third time

    I also use Laravel 8 for php coding.
    It is a fine framework (there are others) but you have to configure correctly for production usage.
    As the link above explains ..

    • never set APP_DEBUG=true in production
    • you could disable runnable solutions completely by setting enable_runnable_solutions to false in the ignition.php config file

    Looking at a typical Laravel 8 app I created recently on localhost (for testing) I read ..

    'enable_runnable_solutions' => env('IGNITION_ENABLE_RUNNABLE_SOLUTIONS', null),

    So clearly in production mode you need to tighten security.


    [Later edit]
    Found security reports..

    https://www.cybersecurity-help.cz/vd...rable%20system.

    https://www.ambionics.io/blog/laravel-debug-rce
    Last edited by dragonfly41; May 23rd, 2021 at 11:38 AM. Reason: Added links

  10. #10
    Join Date
    May 2021
    Beans
    5

    Re: I got hacked twice on Ubuntu 18.04, don't want third time

    Quote Originally Posted by dragonfly41 View Post
    I also use Laravel 8 for php coding.
    It is a fine framework (there are others) but you have to configure correctly for production usage.
    As the link above explains ..




    Looking at a typical Laravel 8 app I created recently on localhost (for testing) I read ..

    'enable_runnable_solutions' => env('IGNITION_ENABLE_RUNNABLE_SOLUTIONS', null),

    So clearly in production mode you need to tighten security.


    [Later edit]
    Found security reports..

    https://www.cybersecurity-help.cz/vd...rable%20system.

    https://www.ambionics.io/blog/laravel-debug-rce

    I read in new projects (Laravel 8) is not an issue anymore, as it uses Ignition version where it is fixed. Unfortunately my server is running project in Laravel 7, so I followed similar steps that you mentioned. Solved. I guess maybe I could just update Ignition instead. I'm glad we figured it out.

    I was surprised that you can seriously infect server this way. I would expect that you need root access. No. Apparently one wrongly implemented file_get_contents (or what was the underlying problem) can turn server into a crypto mining machine that is attempting to connect to other server and trying to infect them too.

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •