All Linux HowTo's

Do this every time you log into a Linux server

This is a list of things i do every time i log into a Linux server. It’s habit now and something you might consider doing too. Essentially we want to know who’s on the server, what the server state is and how different it is to normal.

Check who’s on the server

Check who else is on the server is the “w” command. You can then broadcast a message to those other technicians to let them know they have company.

[agixuser2@www.example.com ~]$ w
 12:21:18 up 402 days, 23:34,  4 users,  load average: 0.00, 0.00, 0.00
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
agixuser1 pts/6    192.168.1.2  03:38    8:19m  0.28s  0.45s sshd: agixuser1 [priv]
agixuser2 pts/7    192.1678.1.6  12:21    0.00s  0.13s  0.09s w

You can see from the above that we have two users on that server including myself.

Say hi to the other logged-in technicians

I will let all other users (in this case just agixuser1) know i am here.

[agixuser2@www.example.com ~]$ wall "I'm glad to see you working on the weekend."

Check the servers uptime

Check how long the server has been up and what the load averages are:

[agixuser2@www.example.com ~]$ uptime
 12:36:31 up 402 days, 23:49,  2 users,  load average: 0.00, 0.00, 0.00

We can see the uptime is “402” days (not bad) and the load averages are all zeros. Load averages are based on server “load” 1 minute ago, 5 minutes ago and 15 minutes ago. The “load” is not a percentage but a calculated number based on all kinds of information. The lower, the better. If you see numbers above 10, your server is working very hard.

Check the disk space

Next we should check the disk space and memory utilisation.

[agixuser2@www.example.com ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg00-root
                       39G   18G   20G  48% /
tmpfs                 4.0G     0  4.0G   0% /dev/shm
/dev/sda1             243M   47M  184M  21% /boot
/dev/mapper/vg00-var  5.0G  3.6G  1.1G  77% /var
/dev/mapper/vg00-tmp  2.0G  109M  1.8G   6% /tmp

The disk space is fine. Anything near 90% “Use%” or where a filesystem is more used than i remember is cause for alarm.

Check the servers memory

Now check the memory using the “free” command. You can use “-m” to show the results in MB. Use “-G” as well for GB.

[agixuser2@www.example.com ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          8001       7538        463         25        638       5449
-/+ buffers/cache:       1450       6551
Swap:         4095         56       4039

The above server has “463” MB RAM free. Not cause for alarm given that it’s not swapping out (not using much swap space). If it was using more than a few hundred B of swap, i’d be concerned and up the memory on this server.

Check the servers CPU load

Next check the CPU utilisation and see if there is anything hogging resources.

top

The “top” command shows:

drakes-top

Nothing strange there. Tip, when you start “top”, press the number “1” key to seperate the CPU’s. This doesn’t work on a Mac/BSD server (as far as i know) but does on Linux.

Check the servers historic load

Finally we check the CPU and load history. We use the “sar” command.

[agixuser2@www.example.com ~]$ sar
Linux 2.6.32-358.14.1.el6.x86_64 (www.example.com) 	18/09/16 	_x86_64_	(2 CPU)

00:00:01        CPU     %user     %nice   %system   %iowait    %steal     %idle
00:10:01        all      5.30      1.39      1.40      0.20      0.00     91.70
00:20:01        all      0.64      0.00      0.48      0.02      0.00     98.85
00:30:01        all      0.63      0.00      0.32      0.01      0.00     99.04
00:40:01        all      0.60      0.00      0.31      0.01      0.00     99.08
00:50:01        all      0.62      0.00      0.33      0.01      0.00     99.03
01:00:01        all      0.63      0.00      0.35      0.02      0.00     99.00
01:10:01        all      1.16      0.00      0.50      0.07      0.00     98.28
...

The output goes over a few pages starting from midnight through to now with 10 minute intervals. The details are pretty self-explanatory but worth going over a little. The useful columns to us at a glance are “Time”, “%iowait” and “%idle”. We need to know the time to know when the server was under particular load. The i/o is important to know if the server is waiting on the underlying hardware to catch up which would indicate the disk hardware is a cause of performance degradation. Finally we can see how much of the time the server was idle and therefore how much it was under load.

Check for malicious scripts

One last thing i do is check what “perl” scripts are running. In my experience, a compromised server is likely to be used as a relay of some kind and “perl” is an excellent tool. So run the following to find out if any perl processes are running.

[agixuser2@www.example.com ~]$ ps aux | grep perl

At this point we have a great idea of how the server is looking. I hope this help. It does seem like a lot to do each time you log in and you don’t need to do this every time. But worth doing when you haven’t been on the given server for a while.

Similar Posts:

Leave a Reply

Your email address will not be published. Required fields are marked *