Subscribe by RSS RSS Icon
Follow me on Twitter Twitter Icon

Quick and dirty log analysis script

I wrote the script to automate the process of doing analysis on Apache and SSH log for Linux machine. Nothing fancy just a simple script to get the source IP address of the attacker based on certain keyword you specify. Just copy the script and name it with .sh extension for e.g. log-analysis.sh. To run the script type ./log-analysis.sh and follow the steps. Btw, I'm not a coder so the script may look like an amateur work ;)




#!/bin/bash
#LOG Analysis script written by dgodam@gmail.com

#Global Function
probeip(){
while :
do
  read -p "Probe suspicious IP address (y/n)? " CONT
  if [ "$CONT" == "y" ]; then
    read -e -p "Enter source IP address: " -i "8.8.8.8" PROBEIP
    cat $FILEPATH/$LOGFILE | grep -i $PROBEIP | more
  else
     break
   fi
done
}

#BEGIN
read -e -p "What type of attack do you want to analyse (web/auth/ddos)? " -i "web" LOG
if [ $LOG  == "web" ]; then
    accesslog
  elif [ $LOG == "auth" ]; then
    authlog
  elif [ $LOG == "ddos" ]; then
   ddos
 else
    exit;
fi

#Web
print_access(){
read -p "Print result (y/n)? " CONT
if [ "$CONT" == "y" ]; then
  cat $FILEPATH/$LOGFILE | grep -i $KEYWORD | awk {'print $1,$7,$9'} | grep " 200" | awk '{print $1}' | uniq -c | sort -rn > 1.tmp
  echo -e "begin\ncountrycode" > 2.tmp
  cat 1.tmp | awk '{print $2}' | sort -u >> 2.tmp
  echo "end" >> 2.tmp
  netcat whois.cymru.com 43 < 2.tmp > 3.tmp
  echo -e "Unique Source IP Count" > accesslog_result.txt
  cat < 1.tmp >> accesslog_result.txt
  cat < 3.tmp >> accesslog_result.txt
  rm *.tmp
  echo "Your output file: accesslog_result.txt"
else
  exit;
fi
}

accesslog(){
read -e -p "Enter path to the log folder: " -i "/var/log/apache2" FILEPATH
read -e -p "Enter name of the log file: " -i "access.log" LOGFILE
cat $FILEPATH/$LOGFILE | more
read -e -p "Enter keyword: " -i "select" KEYWORD
cat $FILEPATH/$LOGFILE | grep -i $KEYWORD | awk '{print $1,$7,$9}' | more
read -p "Include HTTP 200 successful connection only (y/n)? " CONT
if [ "$CONT" == "y" ]; then
   cat $FILEPATH/$LOGFILE | grep -i $KEYWORD | awk '{print $1,$7,$9}' | grep " 200" | more
   probeip
else
  probeip
fi
print_access
}

#Ddos
print_ddos(){
read -p "Print result (y/n)? " CONT
if [ "$CONT" == "y" ]; then
  cat $FILEPATH/$LOGFILE | grep -i $KEYWORD | awk {'print $1'} | sort -rn | uniq -c > 1.tmp
  echo -e "begin\ncountrycode" > 2.tmp
  cat 1.tmp | awk '{print $2}' |sort -u >> 2.tmp
  echo "end" >> 2.tmp
  netcat whois.cymru.com 43 < 2.tmp > 3.tmp
  echo -e "Unique Source IP Count" > ddos_result.txt
  cat < 1.tmp >> ddos_result.txt
  cat < 3.tmp >> ddos_result.txt
  rm *.tmp
  echo "Your output file: ddos_result.txt"
else
  exit;
fi
}

ddos(){
read -e -p "Enter path to the log folder: " -i "/var/log/apache2" FILEPATH
read -e -p "Enter name of the log file: " -i "access.log" LOGFILE
cat $FILEPATH/$LOGFILE | more
read -e -p "Enter keyword: " -i "select" KEYWORD
cat $FILEPATH/$LOGFILE | grep -i $KEYWORD | awk '{print $1}' | sort -rn | uniq -c | more
print_ddos
}

#Auth
print_auth(){
read -p "Print result (y/n)? " CONT
if [ "$CONT" == "y" ]; then
  cat $FILEPATH/$LOGFILE | grep -i $KEYWORD | awk {'print $11'} | uniq -c | sort -rn > 1.tmp
  echo -e "begin\ncountrycode" > 2.tmp
  cat 1.tmp | awk '{print $2}' |sort -u >> 2.tmp
  echo "end" >> 2.tmp
  netcat whois.cymru.com 43 < 2.tmp > 3.tmp
  echo -e "Unique IP Count" > auth_result.txt
  cat <  1.tmp >> auth_result.txt
  cat < 3.tmp >> auth_result.txt
  rm *.tmp
  echo "Your output file: auth_result.txt"
else
  exit;
fi
}

authlog(){
read -e -p "Enter path to the log folder: " -i "/var/log" FILEPATH
read -e -p "Enter name of the log file: " -i "auth.log" LOGFILE
cat $FILEPATH/$LOGFILE | more
read -e -p "Enter keyword: " -i "accepted" KEYWORD
cat $FILEPATH/$LOGFILE | grep -i $KEYWORD | awk '{print $1,$2,$3,$5,$11}' | more
probeip
print_auth
  }



Linux Log Files under var/log Directory

The following are the 20 different log files that are located under /var/log/ directory. Some of these log files are distribution specific. For example, you’ll see dpkg.log on Debian based systems (for example, on Ubuntu).
  1. /var/log/messages – Contains global system messages, including the messages that are logged during system startup. There are several things that are logged in /var/log/messages including mail, cron, daemon, kern, auth, etc.
  2. /var/log/dmesg – Contains kernel ring buffer information. When the system boots up, it prints number of messages on the screen that displays information about the hardware devices that the kernel detects during boot process. These messages are available in kernel ring buffer and whenever the new message comes the old message gets overwritten. You can also view the content of this file using the dmesg command.
  3. /var/log/auth.log – Contains system authorization information, including user logins and authentication machinsm that were used.
  4. /var/log/boot.log – Contains information that are logged when the system boots
  5. /var/log/daemon.log – Contains information logged by the various background daemons that runs on the system
  6. /var/log/dpkg.log – Contains information that are logged when a package is installed or removed using dpkg command
  7. /var/log/kern.log – Contains information logged by the kernel. Helpful for you to troubleshoot a custom-built kernel.
  8. /var/log/lastlog – Displays the recent login information for all the users. This is not an ascii file. You should use lastlog command to view the content of this file.
  9. /var/log/maillog /var/log/mail.log – Contains the log information from the mail server that is running on the system. For example, sendmail logs information about all the sent items to this file
  10. /var/log/user.log – Contains information about all user level logs
  11. /var/log/Xorg.x.log – Log messages from the X
  12. /var/log/alternatives.log – Information by the update-alternatives are logged into this log file. On Ubuntu, update-alternatives maintains symbolic links determining default commands.
  13. /var/log/btmp – This file contains information about failed login attemps. Use the last command to view the btmp file. For example, “last -f /var/log/btmp | more”
  14. /var/log/cups – All printer and printing related log messages
  15. /var/log/anaconda.log – When you install Linux, all installation related messages are stored in this log file
  16. /var/log/yum.log – Contains information that are logged when a package is installed using yum
  17. /var/log/cron – Whenever cron daemon (or anacron) starts a cron job, it logs the information about the cron job in this file
  18. /var/log/secure – Contains information related to authentication and authorization privileges. For example, sshd logs all the messages here, including unsuccessful login.
  19. /var/log/wtmp or /var/log/utmp – Contains login records. Using wtmp you can find out who is logged into the system. who command uses this file to display the information.
  20. /var/log/faillog – Contains user failed login attemps. Use faillog command to display the content of this file.
Apart from the above log files, /var/log directory may also contain the following sub-directories depending on the application that is running on your system.
  • /var/log/httpd/ (or) /var/log/apache2 – Contains the apache web server access_log and error_log
  • /var/log/lighttpd/ – Contains light HTTPD access_log and error_log
  • /var/log/conman/ – Log files for ConMan client. conman connects remote consoles that are managed by conmand daemon.
  • /var/log/mail/ – This subdirectory contains additional logs from your mail server. For example, sendmail stores the collected mail statistics in /var/log/mail/statistics file
  • /var/log/prelink/ – prelink program modifies shared libraries and linked binaries to speed up the startup process. /var/log/prelink/prelink.log contains the information about the .so file that was modified by the prelink.
  • /var/log/audit/ – Contains logs information stored by the Linux audit daemon (auditd).
  • /var/log/setroubleshoot/ – SELinux uses setroubleshootd (SE Trouble Shoot Daemon) to notify about issues in the security context of files, and logs those information in this log file.
  • /var/log/samba/ – Contains log information stored by samba, which is used to connect Windows to Linux.
  • /var/log/sa/ – Contains the daily sar files that are collected by the sysstat package.
  • /var/log/sssd/ – Use by system security services daemon that manage access to remote directories and authentication mechanisms.
Instead of manually trying to archive the log files, by cleaning it up after x number of days, or by deleting the logs after it reaches certain size, you can do this automatically using logrotate.

Credit to: www.thegeekstuff.com

Armitage and Metasploit Training Videos

Raphael Mudge has made a six-part training series on Armitage and Metasploit that introduce the penetration testing process and walkthrough for each step. You'll learn how to break into hosts, carry out post-exploitation activities, develop more access from your initial foothold, and you'll do this in a team environment. 

Introduction 
This is part 1 of a 6 part series showing you the method to the madness behind Armitage for Metasploit. You'll learn how to use Armitage by following each step of the network penetration testing process.

http://vimeo.com/26638955


Metasploit Overview
This is part 2 of a 6 part series showing you the method to the madness behind Armitage for Metasploit. You'll learn how to use Armitage by following each step of the network penetration testing process.



Gaining Access
This is part 3 of a 6 part series showing you the method to the madness behind Armitage for Metasploit. You'll learn how to use Armitage by following each step of the network penetration testing process.



Post Exploitation
This is part 4 of a 6 part series showing you the method to the madness behind Armitage for Metasploit. You'll learn how to use Armitage by following each step of the network penetration testing process.



Maneuver
This is part 5 of a 6 part series showing you the method to the madness behind Armitage for Metasploit. You'll learn how to use Armitage by following each step of the network penetration testing process.



Team Tactics
This is part 6 of a 6 part series showing you the method to the madness behind Armitage for Metasploit. You'll learn how to use Armitage by following each step of the network penetration testing process.

 
Copyright Info.

Only for my personal reference. I do not own any of these materials here. Use it at your own risk!

XHTML/CSS validations
Valid XHTML 1.0 Transitional Valid CSS!