Checklists – System is Hacked – Part 2 – Preventive Steps for Infra (OS Hardening)

In last article we described List of Checks which can determine if system is compromised or hacked .  In this article we will talk about preventive steps (specially infra related) can be taken care to avoid hacking or to make system more secure  . There are many directions in which we can secure our application  as follows : 

  • OS hardening (Infra Level Security)
  • Secure Coding guidelines
  • Encryption Of Sensitive Data  . 
  • Ensure No Vulnerability exists in system . 

In this Blog we will be concerned about OS hardening (Infra Level Security) in Linux systems(CentOS/Redhat). We will Cover Other parts in Future Blogs .

Now Let’s go to the System Part. It has following things to be taken care of : 

  • SSH Configuration :  
    • In linux based system SSH default port is 22 . This Defaut port should be changed to some unused port to enhance security .  
    • Use SSH Protocol 2 Version
    • Ensure SSH X11 forwarding is disabled
  • Port Configuration at Firewall :  Generally , in any application there are many applications running on set of servers and each running on some different ports , Say for example : 
    • Application server at  8080 port
    • Database Server at 5432 port

So,  as in above Case Users need to login through 8080 port so only this port should be opened for public as Database needs to interact generally with application server so 5432 port should be allowed from Application Server’s IP  . 

  • Multi Factor Authentication for SSH should be enabled   —  For setting up Google Authentication on CentOS or Redhat you can follow the link
  • Root login for any server must be disabled 
  • Server Login Policies 
    • Ensure password expiration is 365 days or less 
    • Ensure minimum days between password changes is 7 or more 
    • Ensure password expiration warning days is 7 or more 
    • Ensure inactive password lock is 30 days or less 
    • Ensure Password should be strong enough when user change its password
  • Application and Database should be on different Servers  :  this is because of that if due to some vulnerability  application hacked than acces to database in that case is protected  . 
  • Regular package updates   :  Configure Auto update or regularly update packages on all configured servers .
  • Tune Network Kernel Parameters :
    • IP forwarding should be disabled on all servers  
      • Do the following entry in sysctl.conf 
        • net.ipv4.ip_forward = 0
    • Packets Redirecting  should be diabled on all servers . 
      • Do the following entry in sysctl.conf 
        • net.ipv4.conf.all.send_redirects = 0
        • net.ipv4.conf.default.send_redirects = 0
  • Selinux should be enabled and configured . 
  • Antivirus must  be installed on all servers . 

All Above are basic minimum checklists which should be applied to all the servers in any production environment . For implementing in-depth OS Hardening specially for CentOS based Systems , one need to follow the latest CIS CentOS Benchmarklatest CIS

You can also check the below benchmark list from CIS for CentOS hardening : Below doc also explain how to implement things on CentOS .

For Other Operating Systems/Technologies follow the CIS benchmark link.  

In Our Future blog we will explain other parts like Secure Code guidelines , Encryption , VAPT scan etc  to make system more secure . 

Stay tuned . 

Checklists – System is Compromised or Hacked – Part 1

Introduction

As in my previous Blog where i explained how i came to know if my system is hacked or compromized (link here). Here in this blog i will explain what basic things we can check on our system when we have doubt if our system is compromized .

This Blogs have 3 parts

  • List of Checks which can determine if system is compromised or hacked – Part 1
  • List of checks which can give a direction how system is compromised or hacked – Part 2
  • What preventive steps (specially infra related) can be taken care to avoid hacking or to make system more secure – Part 3

Here , i am assuming system is Linux system with Centos installed .

List of Checks which can determine if system is compromised or hacked

  • Generally when hacker break into a linux system it is high chance that it will alter you main packages like openssh,kernel etc.. , So first if of please check if these packages are altered or there are some changes in the files or binaries provided by these packages . Following are commands to check on Centos
    • sudo rpm -qa | grep openssh | xargs -I '{}' sudo rpm -V '{}'
    • If therr are files shown by above command in which you did not change anything then it means there is high chance your system is compromised
  • Run rootkit Hunter to check if you system is compromised
    • Download rkhunter.tar.gz
    • copy it in /root and goto /root
    • tar zxvf rkhunter-1.4.2.tar.gz
    • cd rkhunter-1.4.2/
    • sh installer.sh --layout default --install
    • changes in /etc/rkhunter.conf ENABLE_TESTS="all" DISABLE_TESTS="none" HASH_CMD=SHA1 HASH_FLD_IDX=4 PKGMGR=RPM 7
    • /usr/local/bin/rkhunter --propupd
    • /usr/local/bin/rkhunter --update
    • /usr/local/bin/rkhunter -c -sk 10.
    • note output or check and copy /var/log/rkhunter.log
    • you can also check the link for using rkhunter
  • Check /var/log/secure to check if there are many authentication failure requests and someone trying brute force to enter in to system
    • following will be the comand :
      • [root@localhost ~]# less /var/log/secure | grep 'authentication failures'
    • and output will be something like :
      • Apr 25 12:48:46 localhost sshd[2391]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.29.14 user=root
      • Apr 25 12:49:33 localhost sshd[2575]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.29.14 user=root
    • In above output you can see the rhost from where login attempt are made . If you see lots of entries like this then also check if at some point of time some login attempt will be successful from any of attempting rhosts . In secure logs accepted logs will looks something like as follows :
      • Apr 25 12:53:10 localhost sshd[3551]: Accepted password for root from 192.168.29.14 port 36362 ssh2

  • Check in Processes if some unusual process is running and consuming high CPU using top and ps commands .
    • Command to list all process running in system : ps aux | less
    • Also check using top command if some unusual process trying to utiize high cpu
  • Check if there is some unusual entry in crontab of all users made on system
    • crontab -u <user> -l . by default user is root
  • Check if in id_rsa.pub , if some attacker has somehow made its entry in .ssh folder in every users’s home directory .

This was the Part 1 of the Blog , In later Parts i will explain some further checklist to ensure that you system will remain less hackable .

Thankyou .

Linux Machine Compromized/Broken -Power of Observation

Introduction :

In Debugging any issue or any dealing any problem or circumstance two things are important

  • Observation — Observation not only at the time of issue but in general times also .
  • Combining your general observations and Observations at the time of issue to conclude something .

In this Blog , I will explain the following :

  • What was happening on my machine
  • How i came to know my machine is broken into — Power of observation

What was happening on my machine

  • Load on my machine is going very high
  • On top command one process ./kwsapd0 is consuming around 3000% cpu

From here we get to know that kswapd is consuming process , The process kswapd0 is the process that manages virtual memory . So I thought that may be our some process is consuming more RAM and Virtual Memory is being used due to which kswapd process is doing its work but after hours of debugging we found no process is consuming RAM and around 80% RAM was free .

How i came to know my machine is broken into — Power of observation

There were two general observation which i observed and helped my geeting know what was the issue

  • 1st is kswapd process looks in top command like [kswapd] not ./kswapd
  • Kswapd0 can only consume 100% as it uses only one core in the machine .

From there I got to know that this kswapd0 is something unusual . On further debugging

I found ./.configrc/a/kswapd0 in root users directory .

Contents of this directory was :

$ find .configrc -type f
.configrc/dir2.dir
.configrc/a/kswapd0
.configrc/a/dir.dir
.configrc/a/a
.configrc/a/bash.pid
.configrc/a/run
.configrc/a/stop
.configrc/a/init0
.configrc/a/.procs
.configrc/a/upd
.configrc/cron.d
.configrc/b/sync
.configrc/b/dir.dir
.configrc/b/a
.configrc/b/run
.configrc/b/stop

There was also an entry in cron to run this .

So , From all of this i got to know that my system was compromised .

Yet I was unable to find out how my system was broken into . But in my future Blog i will explain what things one can check if your system is compromised and how it is compromised and what all security we can apply to our system to make it less hackable .