Page 1 of 2 12 LastLast
Results 1 to 10 of 17

Thread: Practical advices from experienced administrtors wanted.

  1. #1
    Join Date
    Jan 2013
    Location
    Norrkoping, Sweden
    Beans
    144
    Distro
    Ubuntu 13.04 Raring Ringtail

    Practical advices from experienced administrtors wanted.

    I'm new as a Linux administrator and I would like to ask you all what to think about in the daily work with my server? I run Ubuntu 12.04.2. It has installed Samba 4.0.5 with AD DC and internal DNS, it has isc-dhcp-server, and later on quota, cups and Postfix mail server.


    • What is the first thing you do when you log on to your server?
      • I do apt-get update && apt-get upgrade. Is this a good thing to do?

    • What logs are you following and what commands do you run?
      • I do tail -f /var/log/syslog, but it shows only named and dhcp leases etc.

    • How often do you update your server and what makes you decide to update or not?
      • I run updates as soon as possible, but it can sometimes be a wrong desicion, or?

    • What are the pit falls as a linux admin, and what do I have to be careful of?
      • I know about the root account and the danger in having it enabled.



    Any advice is valuable to me. Thanks.

    //Jn.Pson

  2. #2
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Practical advices from experienced administrtors wanted.

    There are as many ways to admin a Linux server as there are Linux admins. If you admin fewer than 5 servers, doing everything manually, like MS-Windows admins are taught, might be just fine. Soon, any Linux admin should learn that being lazy is the key to becoming a good admin.

    Quote Originally Posted by JnPson View Post
    I'm new as a Linux administrator and I would like to ask you all what to think about in the daily work with my server? I run Ubuntu 12.04.2. It has installed Samba 4.0.5 with AD DC and internal DNS, it has isc-dhcp-server, and later on quota, cups and Postfix mail server.
    I would not run any email on the same server as I run DHCP or DNS or AD. Email is too large a security risk to run on a system running anything else, IMHO. When it comes to the most hacked services on UNIX/Linux systems, DNS and email are the top 2. Those really need to be run on different machines (even a different VM would be fine) and inside a chroot environment for both.

    Quote Originally Posted by JnPson View Post
    [*]What is the first thing you do when you log on to your server?
    • I do apt-get update && apt-get upgrade. Is this a good thing to do?
    I avoid logging into servers directly. If I do need to login at all, that means I've failed in some other way. System logs should be pushed to a centralized system where other tools monitor for issues, not me. I don't have time to review every line of every log file from every server. I need to have monitoring and alarming systems point out issues that need my direct attention.

    I definitely do not manually install updates. Those are all scripted - though I do manually run that centralized script every Saturday morning. Further, I use dist-upgrade to ensure the new kernels are installed.

    Quote Originally Posted by JnPson View Post
    [*]What logs are you following and what commands do you run?
    • I do tail -f /var/log/syslog, but it shows only named and dhcp leases etc.
    logwatch and some custom scripts help me stay on top of logs. If I had a few more systems, I'd get splunk working.

    Quote Originally Posted by JnPson View Post
    [*]How often do you update your server and what makes you decide to update or not?
    • I run updates as soon as possible, but it can sometimes be a wrong desicion, or?
    I update weekly, during an approved maintenance window. This minimizes downtime for other users and provides a time period where I can correct any issues - including rebuilding a VM if necessary - to resolve a problem.

    Quote Originally Posted by JnPson View Post
    [*]What are the pit falls as a linux admin, and what do I have to be careful of?
    • I know about the root account and the danger in having it enabled.
    Root access is a necessary part of our jobs. Use it when you need it, but try to use sudo most of the time so the command is logged.
    The most important thing I do as an admin is have backups, versioned backups. If a server was hacked 30 days ago, I can restore the backup from 31 days ago and be relatively happy. Backups, backups, backups. Of course, if I don't know how to rebuild the VM/machine from that backup, then all that effort was a complete waste of time. Testing restores is mandatory. Don't THINK you know how to restore, KNOW that you KNOW how to restore.


    During the week, my systems are up and available - PERIOD. Ok, there is a 5 minute window nightly during which the email IMAP server is not available ... during backups, but the rest of the time, it is up and available. The email gateway server is always available, unless it needs to be rebooted for a kernel update.

    If I can't perform patching on the Saturday morning as planned, then patching waits for the following Saturday. Also, if I'm going to be out of town, I will not patch a stable system within 5 days of the trip.

    Many large scale admins will talk about rebuilding servers from the ground up using tools like Rex or Chef or Puppet. Monitoring tools like nagios, nms, zabbix, and others (can't think of the best one for simple setups now, sorry) are critical for environments with many servers too. These tools are very important when you have more servers and need to automate administration tasks. Log analysis can be very time consuming. Learn a few different tools and become a regex master.

    There are GUI tools like webmin/mysqladmin/etc that some admins swear by. I've never used them, but when I was new, I did find their appeal very enticing. Fortunately, they all sucked 20 yrs ago, so I never bothered. These days I see those admin-GUI tools as a liability. From time to time, a security whole will be found in a tool that provides admin access to anyone on the internet. If you do use those tools, please, please, please, please lock down the remote IP addresses from which logins are allowed. Please.

    Learn at least 1 scripting language to help automate tasks. Python is probably the best all-around tool for this today, but some knowledge of bash, ksh, sh, perl and ruby would be a good idea too. Don't fear the shell.

    ssh rocks. Know it, love it, use it.
    Avoid running plain FTP anywhere. Avoid running any services on any interface where it isn't needed.

    It is hard to dump 20+ yrs of admin experience into a forum. I can't wait to see what others with slightly different experiences say on this topic too.
    Last edited by CharlesA; April 18th, 2013 at 07:19 PM. Reason: Removed text list tag that broke CSS for some reason.

  3. #3
    Join Date
    Jan 2013
    Location
    Norrkoping, Sweden
    Beans
    144
    Distro
    Ubuntu 13.04 Raring Ringtail

    Re: Practical advices from experienced administrtors wanted.

    Wow I'm gonna read the answers in your post and the others, like a bible.
    Much, much appreciated advices. Thank you.

  4. #4
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Practical advices from experienced administrtors wanted.

    Quote Originally Posted by JnPson View Post
    Wow I'm gonna read the answers in your post and the others, like a bible.
    Much, much appreciated advices. Thank you.
    There are many different opinions on administration techniques. You need to find the best solution to manage the risks that your systems experience. Bible is too strong a term. There are definitely areas where I can still improve.

    I've been relatively lucky over the years - only had 2 systems hacked in all that time and nothing in the last decade. Does that mean I'm a better admin than anyone else? Nope. It just means that my systems were hardened enough and not something that someone really wanted into. When I see some admin claiming that all the hardening I do is excessive - clearly they are experts since their system(s) have never been hacked - I have to wonder how much they really understand. Any system can be hacked. ANY SYSTEM. I firmly believe it. It doesn't matter what the OS is, if it is connected to a network, it can be hacked.

    The best security training that I've ever had was being hacked and figuring out who, how and why they did it. In my research, the reason was always "because they could." The systems were not targeted, they just happened to have a weakness in the running program or configuration.

    I've had conversations with CEOs explaining that their web site would be hacked and there was nothing we could do about it. Nothing, except to have a good plan to restore service and make the data available. The plan wouldn't allow interactive access for a few hours or days, but the data would be available. Thankfully, we've never been hacked, but we have a plan and the ability to execute that plan to a completely different service provider, if necessary. Service would be restored. Clearly, backups are central to the plan.

    BTW, I have a blog where I write about mostly Linux, networking and virtualization stuff. I've been a little busy with life the last few months, so my post frequency has dropped, but used to get 1 article a week out covering all sorts of things. Most of the articles were just a way to keep notes for myself. jdpfu.com

  5. #5
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 22.04 Jammy Jellyfish

    Re: Practical advices from experienced administrtors wanted.

    Quote Originally Posted by TheFu View Post
    There are as many ways to admin a Linux server as there are Linux admins. If you admin fewer than 5 servers, doing everything manually, like MS-Windows admins are taught, might be just fine. Soon, any Linux admin should learn that being lazy is the key to becoming a good admin.
    +1. Shell scripts (and other scripts) are your friends. Various monitoring tools are also your friends.

    I would not run any email on the same server as I run DHCP or DNS or AD. Email is too large a security risk to run on a system running anything else, IMHO. When it comes to the most hacked services on UNIX/Linux systems, DNS and email are the top 2. Those really need to be run on different machines (even a different VM would be fine) and inside a chroot environment for both.
    Agreed. I would run an email server on a VM to seperate it from everything else. That way if the email server is owned, the whole server isn't owned.

    I avoid logging into servers directly. If I do need to login at all, that means I've failed in some other way. System logs should be pushed to a centralized system where other tools monitor for issues, not me. I don't have time to review every line of every log file from every server. I need to have monitoring and alarming systems point out issues that need my direct attention.
    I do the same at work and on my VPS. The only reason I actually log in to those is if I am testing something (on my VPS) or if I need to do an update.

    I definitely do not manually install updates. Those are all scripted - though I do manually run that centralized script every Saturday morning. Further, I use dist-upgrade to ensure the new kernels are installed.
    I have both my home server, VPS and Debian box at work running unattended-upgrades set for security updates only and I have apticron running to notify me of any non security updates that need to be applied. As far as when I apply updates, it depends. On my home box and VPS, I do them whenever I get a notification they are avalible, but on the work machine, I leave it alone until the rest of IT does their 'patch day'

    Granted the Debian box at work is only running a few services - apache, mysql and postfix, I rarely check the logs on it because it is only accessible internally and most of the people that use it access it via web browser.

    logwatch and some custom scripts help me stay on top of logs. If I had a few more systems, I'd get splunk working.
    I've been using logwatch on my home server and VPS, and it really helps to cut down on the time spent parsing logs. It might not catch everything, but it'll tell you if someone is trying to bruteforce your SSH server.

    The most important thing I do as an admin is have backups, versioned backups. If a server was hacked 30 days ago, I can restore the backup from 31 days ago and be relatively happy. Backups, backups, backups. Of course, if I don't know how to rebuild the VM/machine from that backup, then all that effort was a complete waste of time. Testing restores is mandatory. Don't THINK you know how to restore, KNOW that you KNOW how to restore.
    +9000. I do versioned backups via a custom bash script.

    There are GUI tools like webmin/mysqladmin/etc that some admins swear by. I've never used them, but when I was new, I did find their appeal very enticing. Fortunately, they all sucked 20 yrs ago, so I never bothered. These days I see those admin-GUI tools as a liability. From time to time, a security whole will be found in a tool that provides admin access to anyone on the internet. If you do use those tools, please, please, please, please lock down the remote IP addresses from which logins are allowed. Please.
    Yes, please lock those tools down so they aren't accessible from the entire internet. I used to use webmin when I was using 9.04 or 9.10, but I've moved to SSH only now. Less services to worry about and you don't have to worry about another vector of attack.

    Learn at least 1 scripting language to help automate tasks. Python is probably the best all-around tool for this today, but some knowledge of bash, ksh, sh, perl and ruby would be a good idea too. Don't fear the shell.
    Agreed. I learned BASH, but I really want to learn more python.

    ssh rocks. Know it, love it, use it.
    Avoid running plain FTP anywhere. Avoid running any services on any interface where it isn't needed.
    SSH = awesome. I love it and the fact that I can use scp or sftp instead of regular ftp when transferring files.

    Quote Originally Posted by TheFu View Post
    I've been relatively lucky over the years - only had 2 systems hacked in all that time and nothing in the last decade. Does that mean I'm a better admin than anyone else? Nope. It just means that my systems were hardened enough and not something that someone really wanted into. When I see some admin claiming that all the hardening I do is excessive - clearly they are experts since their system(s) have never been hacked - I have to wonder how much they really understand. Any system can be hacked. ANY SYSTEM. I firmly believe it. It doesn't matter what the OS is, if it is connected to a network, it can be hacked.
    So far I have been lucky, but I only have 1 box "out in the wild" so that one is pretty much locked down. My home server is only accessible via ssh from certain IPs, and that is the only service that is actually accessible via the Internet.
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

  6. #6

    Re: Practical advices from experienced administrtors wanted.

    Fix more than you break.

    20 years.
    Windows assumes the user is an idiot.
    Linux demands proof.

  7. #7
    Join Date
    Jan 2013
    Location
    Norrkoping, Sweden
    Beans
    144
    Distro
    Ubuntu 13.04 Raring Ringtail

    Re: Practical advices from experienced administrtors wanted.

    I will definitely learn python or bash or some other scripting language, I've seen the strength of scripted installations when Toxic64 helped me with the installation of Samba 4.0.5 GIT, and backups is the most important piece in the puzzle. No backups is like saying, "I never make mistakes" or "no one or nothing can hurt me".

    Can you point me to a good tutorial/howto on scripting languages?

    It would also be a good thing for me to learn linux file/folder permissions and security and how it's built with the numbered stuff and what the different numbers means (like 0750 and drwxr-xr-x).
    Code:
    drwxrwsr-x+ 2 3000000 users 4096 Apr 17 11:20 test
    This below is also a little confusing to me. I understand the principle but have no deeper understanding of the meaning of each of the lines. I would like to know what I'm doing and with that knowledge, make the the most efficient and safest server I can.
    Code:
    mkdir -m 770 /Users chmod g+s /Users chown root:users /Users
    Code:
    [Users]
            directory_mode: parameter = 0700
            read only = no
            browseable = no
            path = /Users
    Toxic64 helped me with the scripts. I added 'browseable = no' and then I needed to add users roaming profiles so I just copied and pasted the lines. But then again, I don't have the deeper understanding what each line means, and if it is safe or the best way of creating shares.
    Code:
    mkdir -m 770 /Profiles chmod g+s /Profiles chown root:users /Profiles
    Code:
    [Profiles]
            directory_mode: parameter = 0700
            read only = no
            browseable = no
            path = /Profiles
    You see what I mean?

    If you know how this stuff works or have a nice tutorial please point me there.

  8. #8
    Join Date
    Jan 2013
    Location
    Norrkoping, Sweden
    Beans
    144
    Distro
    Ubuntu 13.04 Raring Ringtail

    Re: Practical advices from experienced administrtors wanted.

    Hi again.
    I found several bash-scripting guides and I've downloaded them for offline browsing. I hope these guides are up-to-date and still valid, hopefully nothing important has changed. If so, I might run in to scripting problems later on.

    http://tldp.org/LDP/abs/html/ Advanced Bash-Scripting Guide
    http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html BASH Programming - Introduction How-To
    http://www.tldp.org/LDP/Bash-Beginners-Guide/html/ BASH Guide for beginners

    //JnPson

  9. #9
    Join Date
    Nov 2012
    Location
    Halloween Town
    Beans
    Hidden!
    Distro
    Xubuntu Development Release

    Re: Practical advices from experienced administrtors wanted.


  10. #10
    Join Date
    Oct 2009
    Beans
    Hidden!
    Distro
    Ubuntu 22.04 Jammy Jellyfish

    Re: Practical advices from experienced administrtors wanted.

    Don't forget the Bash Pitfalls page!
    http://mywiki.wooledge.org/BashPitfalls
    Come to #ubuntuforums! We have cookies! | Basic Ubuntu Security Guide

    Tomorrow's an illusion and yesterday's a dream, today is a solution...

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •