testyouresuchageek

Sunday, November 11, 2012

Howto : Guide to SNORT IDS in Debian based system with Barnyard2, Mysql and Pullpork

If you are interested in advanced Opensource security tools, you probably already know about Snort Intrusion Prevention System, and if you don't let's follow my guide to help easily getting a full working Snort installation running in Intrusion Prevention System mode.

I will begin with some explanations:

- Snort can operate in various mode, as a Network sniffer, as an Intrusion Detection System (IDS) and as an Intrusion Prevention System (IPS)

- Snort IDS is Opensource but rules are not (anymore) and can be delivered in 2 ways by Sourcefire :

* For free if you register to Snort, you will get 1 month old rules version and an Oinckcode if you want to use third party tools to automatically update rules

* With charge if you want to get a subscription and get always up to date verified rules

- Snort does not support anymore database logging, all activity will logged to files, so you need a third party tool to insert Snort events into a database (Mysql recommended) : Barnyard2

- Snort does not offer itself any frontend or GUI, you will need a third party tool : most known "Base" (Basic And Security Engine)

- Snort won't update itself its detection rules, using the 3rd party tool "Pulledpork" will automatically update your rules. (you need to register on Snort site and generate an Oinckcode, see later)

My guide is based on following documentation, with some adaptations:
http://www.snort.org/assets/167/deb_snort_howto.pdf
http://www.snort.org/assets/158/snortinstallguide293.pdf

I will assume you already have a working Linux installation, which is really far way from the objective of this guide :-)

Summary of steps:

Step 1: Prepare your system
1.1: Install Apache2, Mysql and system dependencies

1.2: Install Snort dependencies (libpcap, libdnet, daq)

1.3: Update Library Path

Step 2: Install and configure Snort

2.1: Installation

2.2: Test Snort

Step 3: Install Barnyards

Step 4: Setup Mysql Databases

4.1: Connection to Mysql Server

4.2: Databases and Privileges creation

4.3: Populate database with Snort Structure

4.4: Launch Snort and Barnyard for second testing

Step 5: Init script

Step 6: Cleanup and start services

Step 7: Install Pullpork and update your rules

7.1: Register to Snort and get your Oinckcode

7.2: Install and configure Pullpork

7.3: Install Pullpork and update your rules

7.4: Plan auto updates using cron

Step 8: Configure Apache2 and BASE

8.1: Configure php.ini

8.2: Install Perl requirements

8.3: Activate Apache SSL

8.4: Install and configure BASE

Step 9: Testing Snort IDS

Step 1: Prepare your system

The ideal configuration would be to have 2 networking interfaces, one used as the management interface and an other that will act as the collector interface. (eg. eth0: Management Int, eth1: Collector Int)

Therefore you can off course install and use Snort with only one networking interface system that will be used for both normal networking traffic and data collecting.

I will assume you only have one Networking Interface for all traffic, but this can be easily adapted to your needs.

Step 1.1: Configure your Network Interfaces

First, ensure your Network IP Address are statically fixed, or reserved in your DHCP Server.

Disable ”Large Receive Offload” and ”Generic Receive Offload” on the collector interface

# ethtool -K eth0 gro off
# ethtool -K eth0 lro off

Explanation from Snort guide, chapter 1.5:

Some network cards have features named ”Large Receive Offload” (lro) and ”Generic Receieve Offload”
(gro). With these features enabled, the network card performs packet reassembly before they’re processed by
the kernel.
By default, Snort will truncate packets larger than the default snaplen of 1518 bytes. In addition, LRO and
GRO may cause issues with Stream5 target-based reassembly. We recommend that you turn off LRO and
GRO. On linux systems, you can run:
$ ethtool -K eth1 gro off
$ ethtool -K eth1 lro off

Step 1.1: Install Apache2, Mysql and system dependencies

Update:

As always, update apt:

# sudo apt-get update

Install Apache2 Web Server:

If you don't already have Apache2 installed and running:

# sudo apt-get install apache2 apache2-utils apache2.2-bin apache2.2-common libapache2-mod-php5

Install Mysql server:

Your Mysql server can be local or remote.

Local Mysql-server installation:

# sudo apt-get install mysql-client mysql-common mysql-server

Note: Installation wizard will ask to enter your mysql-server admin password

In case of remote mysql-server, just install mysql client and some dependencies:

# sudo apt-get install mysql-client

Install Dependencies:

Install various dependencies:

# sudo apt-get install g++ make autoconf automake libtool flex bison gcc libnet1 libnet1-dev libapache2-mod-php5 libcrypt-ssleay-perl libpcre3 libpcre3-dev libmysqlclient-dev libphp-adodb libssl-dev libtool libwww-perl libmysqlclient-dev mysql-common mysql-client ntp php5-cli php5-gd php5-mysql php-pear

Step 1.2: Install Snort dependencies : (libpcap, libdnet, daq)

Install libpcap:

# cd /usr/src && wget http://www.tcpdump.org/release/libpcap-1.3.0.tar.gz 
# tar -zxf  libpcap-1.3.0.tar.gz && cd libpcap-1.3.0 
# ./configure --prefix=/usr --enable-shared && make && make install

Install libdnet:

# cd /usr/src && wget http://libdnet.googlecode.com/files/libdnet-1.12.tgz 
# tar -zxf libdnet-1.12.tgz && cd libdnet-1.12 
# ./configure --prefix=/usr --enable-shared && make && make install

Install libdnet:

# cd /usr/src && wget http://www.snort.org/dl/snort-current/daq-1.1.1.tar.gz 
# tar -zxf daq-1.1.1.tar.gz && cd daq-1.1.1 
# ./configure && make && make install

Step 1.3: Update library path

Do:

# echo >> /etc/ld.so.conf /usr/lib 
# echo >> /etc/ld.so.conf /usr/local/lib && ldconfig

Step 2: Install Snort

Step 2.1: Installation

Download, configure compile and install Snort:

# cd /usr/src && wget http://labs.snort.org/snort/2931/snort.conf -O snort.conf
# wget http://www.snort.org/dl/snort-current/snort-2.9.3.1.tar.gz -O snort-2.9.3.1.tar.gz
# tar -zxf snort-2.9.3.1.tar.gz && cd snort-2.9.3.1
# ./configure --enable-sourcefire && make && make install
# mkdir /etc/snort /etc/snort/rules /var/log/snort /var/log/barnyard2 /usr/local/lib/snort_dynamicrules
# touch /etc/snort/rules/white_list.rules /etc/snort/rules/black_list.rules
# groupadd snort && useradd -g snort snort
# chown snort:snort /var/log/snort /var/log/barnyard2
# cp /usr/src/snort-2.9.3.1/etc/*.conf* /etc/snort
# cp /usr/src/snort-2.9.3.1/etc/*.map /etc/snort
# cp /usr/src/snort.conf /etc/snort

Edit "/etc/snort/snort.conf": (using vi to search for a line : esc then ":N" where N is the line number)

Line #45 - ipvar HOME_NET 172.26.12.0/22 – make this match your internal (friendly) network
Line #48 - ipvar EXTERNAL_NET !$HOME_NET
Line #104 - var RULE_PATH ./rules
Line #113 - var WHITE_LIST_PATH ./rules
Line #114 - var BLACK_LIST_PATH ./rules
Line #297 - add this to the end after “decompress_depth 65535” max_gzip_mem 104857600
Line #538 - add this line output unified2: filename snort.log, limit 128
Line #554 - delete or comment out all of the “include $RULE_PATH” lines except “local.rules”

Step 2.2: First test of Snort

Temporarily and for testing purposes, add this to "/etc/snort/rules/local.rules"

alert icmp any any -> $HOME_NET any (msg:"ICMP test"; sid:10000001;)

This will be a local testing rule using ICMP request, when running Snort in console and pinging our host from another computer, Snort shall detect it:

Start Snort:

# /usr/local/bin/snort -A console -q -u snort -g snort -c /etc/snort/snort.conf -i eth0

Ping your Snort host, you should see Snort logging it:

02/09-11:29:43.450236 [**] [1:10000001:0] ICMP test [**] [Priority: 0] {ICMP} 172.26.12.1 -> 172.26.12.2
02/09-11:29:43.450251 [**] [1:10000001:0] ICMP test [**] [Priority: 0] {ICMP} 172.26.12.2 -> 172.26.12.1

If you see this kind of output, then your Snort installation is functional ^^

If not, you may have a configuration issue, stop all and carefully restart from the beginning!

Enter crtl+c to stop Snort.

Step 3: Install Barnyard2

Download, configure, compile and install:

# cd /usr/src && wget https://nodeload.github.com/firnsy/barnyard2/tarball/master
# tar -zxf master && cd firnsy-barnyard2-*
autoreconf -fvi -I ./m4 && ./configure --with-mysql && make && make install
# mv /usr/local/etc/barnyard2.conf /etc/snort
# cp schemas/create_mysql /usr/src

Edit "/etc/snort/barnyard2.conf":

Line #215 change to output alert_fast

At the end of the file, add the Database configuration line, adapt:

- "<mypassword>" to the value of the database snort password you want to set (not the admin password database, but the password you are planing to use for Snort database)

- "localhost" : If your Mysql-server will local, then let localhost, if your Mysql server is remote then enter its IP Address

output database: log, mysql, user=snort password=<mypassword> dbname=snort host=localhost

Step 4: Setup Mysql Databases

Here comes the Mysql configuration, we will:

- Create a main database called "snort" where Snorts events are going to be logged by the barnyards daemon
- Create a secondary database called "archive" to rotate Snort events
- Create a user "snort" with all privileges to these databases

Step 4.1: Connect to your Mysql server

If Mysql-server local, enter:

# mysql -u root -p

If Mysql-server remote, enter: (adapt IPAddress)

# mysql -u root -p -h IPAddress

Then enter your admin password, you should get the mysql prompt:

mysql>

Step 4.2: Create Databases and privileges

Create Databases:

mysql>

create database snort;
create database archive;

Create Snort user and privileges: (adapt <mypassword> to the Snort database user password value of your choice)

If Mysql-server local: (adapt IPAddress with Mysql Server value)

mysql>

grant usage on snort.* to snort@192.168.1.104;
grant usage on archive.* to snort@192.168.1.104;
set password for snort@192.168.1.104=PASSWORD('snort');
grant all privileges on snort.* to snort@192.168.1.104;
grant all privileges on archive.* to snort@192.168.1.104;
flush privileges;

If Mysql-server Remote: (adapt IPAddress with Mysql Server value)

mysql>

grant usage on snort.* to snort@IPAddress;
grant usage on archive.* to snort@IPAddress;
set password for snort@IPAddress=PASSWORD('<mypassword>');
grant all privileges on snort.* to snort@IPAddress;
grant all privileges on archive.* to snort@IPAddress;
flush privileges;

Step 4.3: Populate Database with Snort structure

Do:

mysql>

use snort;
source /usr/src/create_mysql
show tables; # you should see the list of new tables you just imported.
exit

Step 4.4: Launch Snort and Barnyes for second testing purposes

Launch Snort and Barnyard :

Do:

# /usr/local/bin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 &

# /usr/local/bin/barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.log -w /etc/snort/bylog.waldo -G /etc/snort/gen-msg.map -S
/etc/snort/sid-msg.map -C /etc/snort/classification.config &

If Barnyard successfully connects to the Snort database, you should get such an output:

database: compiled support for (mysql)                                                                                        
database: configured to use mysql                                                                                             
database: schema version = 107                                                                                                
database:           host = 192.168.1.100                                                                                      
database:           user = snort                                                                                              
database:  database name = snort                                                                                              
database:    sensor name = raspberrypi:NULL                                                                                   
database:      sensor id = 1                                                                                                  
database:     sensor cid = 12275                                                                                              
database:  data encoding = hex                                                                                                
database:   detail level = full                                                                                               
database:     ignore_bpf = no                                                                                                 
database: using the "log" facility                                                                                            
                                                                                                                              
        --== Initialization Complete ==--                                                                                     
                                                                                                                              
  ______   -*> Barnyard2 <*-                                                                                                  
 / ,,_  \  Version 2.1.10 (Build 313)                                                                                         
 |o"  )~|  By Ian Firns (SecurixLive): http://www.securixlive.com/                                                            
 + '''' +  (C) Copyright 2008-2012 Ian Firns <firnsy@securixlive.com>                                                         
                                                                                                                              
Using waldo file '/etc/snort/bylog.waldo':                                                                                    
    spool directory = /var/log/snort                                                                                          
    spool filebase  = snort.log                                                                                               
    time_stamp      = 1352226216                                                                                              
    record_idx      = 24326                                                                                                   
Opened spool file '/var/log/snort/snort.log.1352226216'                                                                       
Closing spool file '/var/log/snort/snort.log.1352226216'. Read 24326 records                                                  
Opened spool file '/var/log/snort/snort.log.1352301110'                                                                       
Waiting for new data

Any error preventing Barnyard2 from connecting to the Mysql Database will be shown in ouput messages (bad username/password, host unreacheable, etc...)

If successful, then congratulation you have a working Snort and Barnyard2 installation ^^

Let's continue!

Step 5: Init script

Add and enable an init script for Snort and Barnyard2.

NOTE: Change "eth0" to your interface name if your collector Interface is different from eth0.

Create the new file "/etc/init.d/snortbarn" and add this content:

#! /bin/sh
#set -x
#
### BEGIN INIT INFO
# Provides: snortbarn
# Required-Start: $remote_fs $syslog mysql
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# X-Interactive: true
# Short-Description: Start Snort and Barnyard
### END INIT INFO
. /lib/init/vars.sh
. /lib/lsb/init-functions
do_start()
{
log_daemon_msg "Starting Snort and Barnyard" ""
/sbin/ifconfig eth0 up
/usr/local/bin/snort -q -u snort -g snort -c /etc/snort/snort.conf -i eth0 &
/usr/local/bin/barnyard2 -q -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.log -w /etc/snort/bylog.waldo -G /etc/snort/gen-msg.map –S /etc/snort/sid-msg.map -C /etc/snort/classification.config 2> /dev/null &
log_end_msg 0
return 0
}
do_stop()
{
log_daemon_msg "Stopping Snort and Barnyard" ""
kill $(pidof snort) 2> /dev/null
kill $(pidof barnyard2) 2> /dev/null
log_end_msg 0
return 0
}
case "$1" in
start)
do_start
;;
stop)
do_stop
;;
restart)
do_stop
do_start
;;
*)
echo "Usage: snort-barn {start|stop|restart}" >&2
exit 3
;;
esac
exit 0

Authorisation:

# chmod +x /etc/init.d/snortbarn

Activate:

# update-rc.d snortbarn defaults

Step 6: Cleanup and start services

Stop existing processes and clean up various files:

# rm /var/www/index.html
# chmod 755 /var/www/base
# pkill snort && pkill barnyard2
# rm -rf /var/log/snort/* /var/log/barnyard2/*

Deactivate the local test rules and activate standard rules:

Edit "etc/snort/rules/local.rules" and comment out the test rule line
Edit "/etc/snort/snort.conf" – Line 553: add: include $RULE_PATH/snort.rules

Start Snort and Barnyard2:

# sudo service snortbarn start

Step 7: Install Pullpork and update your rules

Step 7.1: Register to Snort and get your Oinckcode

First, go to www.snort.org and register (for free).

When your accound had been created, go in:

- "My Account"
- Then "Subscriptions and Oinckcodes"
- "Oinkcodes"

And generate your Oinckcode, copy it somewhere it will be required to set up Pulleporked.

Step 7.2: Install and configure Pulledporked

Do:

# cd /usr/src && wget http://pulledpork.googlecode.com/files/pulledpork-0.6.1.tar.gz
# tar -zxf pulledpork-0.6.1.tar.gz && cd pulledpork-0.6.1
# cp pulledpork.pl /usr/local/bin && cp etc/*.conf /etc/snort

Edit "/etc/snort/pulledpork.conf":

Comment out lines 22 & 26

Line 20: enter your “oinkcode” where appropriate or comment out the line if you didn’t get one above
Line 23: leave alone (uncommented) to use the Emerging Threats rule set
Line 71: change to: rule_path=/etc/snort/rules/snort.rules
Line 86: change to: local_rules =/etc/snort/rules/local.rules
Line 89: change to: sid_msg=/etc/snort/sid-msg.map
Line 112: change to: config_path=/etc/snort/snort.conf

Line 124: change to: distro=Debian-Lenny

Line 171: Uncomment and change to: enablesid=/etc/snort/enablesid.conf
Line 173: Uncomment and change to: disablesid=/etc/snort/disablesid.conf
Line 174: Uncomment and change to: modifysid=/etc/snort/modifysid.conf

Disable Blocking rules:

Do:

# echo pcre:fwsam >> /etc/snort/disablesid.conf # disables all block (fwsam) rules

Step 7.3: Start Pulledporked and update your rules set

Do:

# /usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -T -l

If everything is fine, you should get this kind of output:

# /usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -T -l                                          
                                                                                                                              
    http://code.google.com/p/pulledpork/                                                                                      
      _____ ____                                                                                                              
     `----,\    )                                                                                                             
      `--==\\  /    PulledPork v0.6.1 the Smoking Pig <////~                                                                  
       `--==\\/                                                                                                               
     .-~~~~-.Y|\\_  Copyright (C) 2009-2011 JJ Cummings                                                                       
  @_/        /  66\_  cummingsj@gmail.com                                                                                     
    |    \   \   _(")                                                                                                         
     \   /-| ||'--'  Rules give me wings!                                                                                     
      \_\  \_\\                                                                                                               
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                               
                                                                                                                              
Checking latest MD5 for snortrules-snapshot-2931.tar.gz....                                                                   
        They Match                                                                                                            
        Done!                                                                                                                 
Prepping rules from snortrules-snapshot-2931.tar.gz for work....                                                              
        Done!
Reading rules...                                                                                                              
Reading rules...                                                                                                              
Processing /etc/snort/enablesid.conf....                                                                                      
        Modified 0 rules                                                                                                      
        Done                                                                                                                  
Processing /etc/snort/disablesid.conf....                                                                                     
        Modified 0 rules
        Done                                                                                                                  
Setting Flowbit State....                                                                                                     
        Enabled 10 flowbits                                                                                                   
        Enabled 1 flowbits
        Done
Writing /etc/snort/rules/snort.rules....                                                                                      
        Done
Generating sid-msg.map....                                                                                                    
        Done
Writing sid_msg....                                                                                                           
        Done
Writing /var/log/sid_changes.log....                                                                                          
        Done                                                                                                                  
Rule Stats....                                                                                                                
        New:-------0                                                                                                          
        Deleted:---0                                                                                                          
        Enabled Rules:----3511                                                                                                
        Dropped Rules:----0                                                                                                   
        Disabled Rules:---11787                                                                                               
        Total Rules:------15298                                                                                               
        Done                                                                                                                  
Please review /var/log/sid_changes.log for additional details                                                                 
Fly Piggy Fly!

Step 7.4:Plan auto updates using cron

Edit your crontab and add a line for Pullpork, example with an auto update every Sunday at 2.30 AM:

Edit your crontab:

# crontab -e

Add:

30 2 0 * * /usr/local/bin/pulledpork.pl -c /etc/snort/pulledpork.conf -T -l >/var/log/snort/pulledpork_crontab.log

The auto update result will be available in the log file.

Step 8: Configure Apache2 and install BASE

To use BASE as a Snort front-end which is Web GUI you off course need a running Web Server, as installed before we will use Apache2.

Configuring Apache is far away from this guide objective, the default configuration will allow us to access to BASE with no issue.

Step 8.1: Configure php.in

Edit "/etc/php5/apache2/php.ini", look for the line "error_reporting" and change it to:

error_reporting = E_ALL & ~E_NOTICE

Step 8.2: Install Perl requirements

Do:

# pear config-set preferred_state alpha && pear channel-update pear.php.net && pear install --alldeps Image_Color Image_Canvas Image_Graph

Step 8.3: If you are using Apache2 default configuration, activate SSL

Do:

# a2enmod ssl
# service apache2 restart

Step 8.4: Install and configure BASE

Do:

# cd /usr/src && wget http://sourceforge.net/projects/secureideas/files/BASE/base-1.4.5/base-1.4.5.tar.gz
# tar -zxf base-1.4.5.tar.gz && cp -r base-1.4.5 /var/www/base
# chown -R www-data:www-data /var/www/base

Open your Web Browser and navigate to "https://IPAddress/base", you should get the BASE configuration page.

Do as follows:

Path to adodb: /usr/share/php/adodb
Click Continue
Database Name: snort
Database Host: localhost or your Mysql Server IP if remote
Database Port: leave blank
Database User Name: snort
Database Password: <mypassword>

Activate Database Archive and configure as above. (database name "archive")

Put in values for the authentication system and click submit.
Click "create baseag" which extends the DB to support BASE.

And you should get access to BASE ^^

Step 8: Testing SNORT IDS

Here are some tools i recommend:

1. Nessus

Nessus is a well known security scan tool, you can get for a free a personal user key and use the software to scan your sites or hosts.

See:
http://www.tenable.com/

Installation is really simple, then simply launch a Web Application scan and wait for the report, this should generate a lot of events in Snort!

2. Nikto

Also very powerful and simple to use, Nikto will carefully scan and launch various attacks/exploit against your Web Server.

See:
http://cirt.net/

Example of use:

Launching a scan in http:

perl nikto.pl -h HOST -p PORT

Launching a scan in https:

perl nikto.pl -h HOST -p PORT -ssl

Testing Web Server with Nikto should also generate new events in Snort.

Saturday, November 10, 2012

Howto - Apache2 ModSecurity - Enhance your Web Server and Applications security with an Opensource Web Application Firewall (WAF)

Last Update: 11/10/2012

The Goal:

ModSecurity is an OpenSource Web Application Firewall, today stable when used with Apache Web server, it will allow to drastically increase your Web security by analyzing in real time any kind of traffic and applying various analysis rules.

This tutorial is based on Ubuntu and Debian based system, bu can be easily adapted to any Unix / Linux based OS.

Major sources:

Official ModSecurity site:
http://www.modsecurity.org/

ModSecurity Rules:
http://spiderlabs.github.com/owasp-modsecurity-crs/

Various interesting sources:
http://www.thefanclub.co.za/how-to/how-install-apache2-modsecurity-and-modevasive-ubuntu-1204-lts-server
http://www.ebelair.fr/2011/06/07/installer-et-configurer-modsecurity/

Summary of steps:

Step 1: Prepare your systemStep 2: Download components - ModSecurity Sources and Rules

Step 3: Compile and install ModSecurity Module
Step 4: Configure ModSecurity module and rules
Step 5: Main ModSecurity Configuration
Step 6: Test your configuration, Simulate a generic attack and malicious code upload
Step 7: Whitelist configuration, test your Web sites and add whitelist exceptions if required
Step 8: Advanced vulnerabilities and attack tools

FAQ & issues

I will assume you already have a running OS and Apache installation.

To implement ModSecurity for Apache on Debian / Ubuntu based systems, you can use official OS Repositories or download, compile and configure latest official versions of ModSecurity sources and rules.

This tutorial is based on the second solution because we wan't to take benefit of very last versions.

Step 1: Prepare your system

Install various dependencies:

$ sudo apt-get install g++ make autoconf automake libtool flex bison gcc apache2-threaded-dev libxml2-dev libcurl4-gnutls-dev libapr1 libapr1-dev libpcre3 libpcre3-dev libxml2 libxml2-dev

ModSecurity can be combined with an Antivirus if you want to scan for malicious code any incoming file, this can be a great idea to secure your Web Server.
Clamav is an Opensource Antivirus and will be used to achieve this.

Install Clamav:

$ sudo apt-get install clamav clamav-base clamav-freshclam

An Apache module called "unique_id" is required to use ModSecurity.
This module should be installed by default.

$ sudo a2enmod unique_id

Step 2: Download components - ModSecurity Sources and Rules

I would advise to download and keep files into "/usr/local/src/modsecurity" but this is up to you to change it.

Download required components:

ModSecurity sources:

Go to http://www.modsecurity.org/download/ and download last version "modsecurity*apache*.tar.gz"

ModSecurity Core Rules:

Go to http://spiderlabs.github.com/owasp-modsecurity-crs/ and download last modsecurity*apache*.tar.gz

Step 3: Compile and install ModSecurity Module

Extract Modsecurity Sources:

$ tar -xvzf <replace with Modsecurity Sources archive version name>
$ cd <extracted directory>

Configure, compile and install:

$ ./autogen.sh
$ ./configure
$ make
$ sudo make install

The module core will be installed by default in "/usr/local/modsecurity/lib".

Step 4: Configure ModSecurity module and rules

Create module loading file in Apache:

Create a new file "/etc/apache2/mods-available/security2_module.load" and add:

LoadFile libxml2.so
LoadFile liblua5.1.so
LoadModule security2_module /usr/local/modsecurity/lib/mod_security2.so

Extract Modsecurity Core Rules:

$ tar -xvzf <replace with Modsecurity Core Rules archive version name>
$ sudo mv <extracted directory> /usr/local/src/modsecurity-crs

Prepare your Apache configuration:

$ sudo mv <extracted directory> /usr/local/src/modsecurity-crs
$ sudo mkdir -p /etc/apache2/modsecurity/conf
$ sudo mkdir -p /etc/apache2/modsecurity/activated_rules
$ sudo cp /usr/local/src/modsecurity-crs/*.conf.example /usr/local/src/modsecurity-crs/modsecurity.conf
$ sudo ln -s /usr/local/src/modsecurity-crs/modsecurity.conf /etc/apache2/modsecurity/conf/modsecurity.conf

Activate recommended rules by creating symlinks:

$ for f in `ls base_rules/`; do sudo ln -s /usr/local/src/modsecurity-crs/base_rules/$f /etc/apache2/modsecurity/activated_rules/$f; done
$ for f in `ls optional_rules/ | grep comment_spam`; do sudo ln -s /usr/local/src/modsecurity-crs/optional_rules/$f /etc/apache2/modsecurity/activated_rules/$f; done

Add symlink to "util" directory:

$ sudo ln -s /usr/local/src/modsecurity-crs/util /etc/apache2/modsecurity/util

Add module launch configuration file:

Create a new file "/etc/apache2/conf/modsecurity.conf" and add:

<IfModule security2_module>
 Include /etc/apache2/modsecurity/conf/*.conf
 Include /etc/apache2/modsecurity/activated_rules/*.conf
<IfModule>

Activate modsecurity module:

$ sudo a2enmod security2_module

Step 5: Main ModSecurity Configuration

Now, let's create the module configuration file, based on recommended version by modsecurity:
https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual

Create the main Modsecurity configuration file:

/etc/apache2/mods-available/security2_module.conf :

Modifications i recommend:

# Line 7: Change "SecRuleEngine DetectionOnly" to "SecRuleEngine On" if you want active response from the module.
In detection mode only, no actions but logging will be done by modsecurity module.

# Line 167: Comment out and change debug log file, change "#SecDebugLog /opt/modsecurity/var/log/debug.log" to "SecDebugLog /var/log/apache2/modsec_debug.log"

# Line 187: Change main modsec log from"SecAuditLog /var/log/modsec_audit.log" to "SecAuditLog /var/log/apache2/modsec_audit.log"

# Line 147: Change "#SecUploadDir /opt/modsecurity/var/upload/" to "/var/cache/www-upload" (will be created later)

# Line 32/33: Depending on your needs, you have set the maximum allowed file size for uploads in your sites, there is an hard limit 1GB, if you want this, then set:
SecRequestBodyLimit 1073741824
SecRequestBodyNoFilesLimit 1073741824

If you want to use Clamav to scan uploaded files:

#Line 160: add the following line:
SecRule FILES_TMPNAMES "@inspectFile /etc/apache2/modsecurity/util/runav.pl" "id:159,phase:2,t:none,log,deny,msg:'Malicious Code Detected, access denied'"

/etc/apache2/mods-available/security2_module.conf :

# -- Rule engine initialization ----------------------------------------------

# Enable ModSecurity, attaching it to every transaction. Use detection
# only to start with, because that minimises the chances of post-installation
# disruption.
#
SecRuleEngine On

# -- Request body handling ---------------------------------------------------

# Allow ModSecurity to access request bodies. If you don't, ModSecurity
# won't be able to see any POST parameters, which opens a large security
# hole for attackers to exploit.
#
SecRequestBodyAccess On

# Enable XML request body parser.
# Initiate XML Processor in case of xml content-type
#
SecRule REQUEST_HEADERS:Content-Type "text/xml" \
"id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"

# Maximum request body size we will accept for buffering. If you support
# file uploads then the value given on the first line has to be as large
# as the largest file you are willing to accept. The second value refers
# to the size of data, with files excluded. You want to keep that value as
# low as practical.
#
SecRequestBodyLimit 1073741824
SecRequestBodyNoFilesLimit 1073741824

# Store up to 128 KB of request body data in memory. When the multipart
# parser reachers this limit, it will start using your hard disk for
# storage. That is slow, but unavoidable.
#
SecRequestBodyInMemoryLimit 131072

# What to do if the request body size is above our configured limit.
# Keep in mind that this setting will automatically be set to ProcessPartial
# when SecRuleEngine is set to DetectionOnly mode in order to minimize
# disruptions when initially deploying ModSecurity.
#
SecRequestBodyLimitAction Reject

# Verify that we've correctly processed the request body.
# As a rule of thumb, when failing to process a request body
# you should reject the request (when deployed in blocking mode)
# or log a high-severity alert (when deployed in detection-only mode).
#
SecRule REQBODY_ERROR "!@eq 0" "id:'200001', phase:2,t:none,log,deny, \
status:400,msg:'Failed to parse request body.',logdata:'%{reqbody_error_msg}',severity:2"

# By default be strict with what we accept in the multipart/form-data
# request body. If the rule below proves to be too strict for your
# environment consider changing it to detection-only. You are encouraged
# _not_ to remove it altogether.
#
SecRule MULTIPART_STRICT_ERROR "!@eq 0" \
"id:'200002',phase:2,t:none,log,deny,status:400, \
msg:'Multipart request body failed strict validation: \
PE %{REQBODY_PROCESSOR_ERROR}, \
BQ %{MULTIPART_BOUNDARY_QUOTED}, \
BW %{MULTIPART_BOUNDARY_WHITESPACE}, \
DB %{MULTIPART_DATA_BEFORE}, \
DA %{MULTIPART_DATA_AFTER}, \
HF %{MULTIPART_HEADER_FOLDING}, \
LF %{MULTIPART_LF_LINE}, \
SM %{MULTIPART_MISSING_SEMICOLON}, \
IQ %{MULTIPART_INVALID_QUOTING}, \
IP %{MULTIPART_INVALID_PART}, \
IH %{MULTIPART_INVALID_HEADER_FOLDING}, \
FL %{MULTIPART_FILE_LIMIT_EXCEEDED}'"

# Did we see anything that might be a boundary?
#
SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" "id:'200003',phase:2,t:none,log,deny, \
status:400,msg:'Multipart parser detected a possible unmatched boundary.'"

# PCRE Tuning
# We want to avoid a potential RegEx DoS condition
#
SecPcreMatchLimit 1000
SecPcreMatchLimitRecursion 1000

# Some internal errors will set flags in TX and we will need to look for these.
# All of these are prefixed with "MSC_". The following flags currently exist:
#
# MSC_PCRE_LIMITS_EXCEEDED: PCRE match limits were exceeded.
#
SecRule TX:/^MSC_/ "!@streq 0" \
"id:'200004',phase:2,t:none,deny,msg:'ModSecurity internal error flagged: %{MATCHED_VAR_NAME}'"

# -- Response body handling --------------------------------------------------

# Allow ModSecurity to access response bodies.
# You should have this directive enabled in order to identify errors
# and data leakage issues.
#
# Do keep in mind that enabling this directive does increases both
# memory consumption and response latency.
#
SecResponseBodyAccess On

# Which response MIME types do you want to inspect? You should adjust the
# configuration below to catch documents but avoid static files
# (e.g., images and archives).
#
SecResponseBodyMimeType text/plain text/html text/xml

# Buffer response bodies of up to 512 KB in length.
SecResponseBodyLimit 524288

# What happens when we encounter a response body larger than the configured
# limit? By default, we process what we have and let the rest through.
# That's somewhat less secure, but does not break any legitimate pages.
#
SecResponseBodyLimitAction ProcessPartial

# -- Filesystem configuration ------------------------------------------------

# The location where ModSecurity stores temporary files (for example, when
# it needs to handle a file upload that is larger than the configured limit).
#
# This default setting is chosen due to all systems have /tmp available however,
# this is less than ideal. It is recommended that you specify a location that's private.
#
SecTmpDir /tmp/

# The location where ModSecurity will keep its persistent data. This default setting
# is chosen due to all systems have /tmp available however, it
# too should be updated to a place that other users can't access.
#
SecDataDir /tmp/

# -- File uploads handling configuration -------------------------------------

# The location where ModSecurity stores intercepted uploaded files. This
# location must be private to ModSecurity. You don't want other users on
# the server to access the files, do you?
#
SecUploadDir /var/cache/www-upload

# By default, only keep the files that were determined to be unusual
# in some way (by an external inspection script). For this to work you
# will also need at least one file inspection rule.
#
#SecUploadKeepFiles RelevantOnly

# Uploaded files are by default created with permissions that do not allow
# any other user to access them. You may need to relax that if you want to
# interface ModSecurity to an external program (e.g., an anti-virus).
#
#SecUploadFileMode 0600
SecRule FILES_TMPNAMES "@inspectFile /etc/apache2/modsecurity/util/runav.pl" "id:159,phase:2,t:none,log,deny,msg:'Malicious Code Detected, access denied'"

# -- Debug log configuration -------------------------------------------------

# The default debug log configuration is to duplicate the error, warning
# and notice messages from the error log.
#
SecDebugLog /var/log/apache2/modsec_debug.log
#SecDebugLogLevel 3

# -- Audit log configuration -------------------------------------------------

# Log the transactions that are marked by a rule, as well as those that
# trigger a server error (determined by a 5xx or 4xx, excluding 404,
# level response status codes).
#
SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"

# Log everything we know about a transaction.
SecAuditLogParts ABIJDEFHZ

# Use a single file for logging. This is much easier to look at, but
# assumes that you will use the audit log only occasionally.
#
SecAuditLogType Serial
SecAuditLog /var/log/apache2/modsec_audit.log

# Use concurrent logging
#SecAuditLogType Concurrent
#SecAuditLog "|/opt/modsecurity/bin/mlogc /opt/modsecurity/etc/mlogc.conf"

# Specify the path for concurrent audit logging.
#SecAuditLogStorageDir /opt/modsecurity/var/audit/

# -- Miscellaneous -----------------------------------------------------------

# Use the most commonly used application/x-www-form-urlencoded parameter
# separator. There's probably only one application somewhere that uses
# something else so don't expect to change this value.
#
SecArgumentSeparator &

# Settle on version 0 (zero) cookies, as that is what most applications
# use. Using an incorrect cookie version may open your installation to
# evasion attacks (against the rules that examine named cookies).
#
SecCookieFormat 0

# Specify your Unicode Code Point.
# This mapping is used by the t:urlDecodeUni transformation function
# to properly map encoded data to your language. Properly setting
# these directives helps to reduce false positives and negatives.
#
#SecUnicodeCodePage 20127
#SecUnicodeMapFile unicode.mapping

Create upload dir cache:

$ sudo mkdir /var/cache/www-upload
$ sudo chown www-data:wwwdata /var/cache/www-upload

Create whitelist file (blank for now):

$ sudo touch /etc/apache2/modsecurity/conf/whitelist.conf

Were're done with configuration, restart Apache2:

$ sudo service apache2 restart

Step 6: Test your configuration, Simulate a generic attack and malicious code upload

Test your configuration:

1. Main log file

Upon Apache2 start-up, you should see a new file "/var/log/apache2/modsec_audit.log", to see its content in real time:

$ tail -f /var/log/apache2/modsec_audit.log

2. Simulate a simple attack

Open your Web Browser and simulate a generic attack (adapt to your configuration, name and port):
http://localhost/test.php?secret_file=/etc/passwd

If you have activated SecRuleEngine to On, then your get an refused error from Apache.
Otherwise, an attack should have been logged in main audit log.

If ok:


Forbidden


You don't have permission to access / on this server.

In log you should see something like this:

Message: Access denied with code 403 (phase 2). Pattern match "(?:\\b(?:\\.(?:ht(?:access|passwd|group)|www_?acl)|global\\.asa|httpd\\.conf|boot\\.ini)\\b|\\/etc\\/)" at ARGS:secret_file. [file "/etc/apache2/modsecurity/activated_rules/modsecurity_crs_40_generic_attacks.conf"] [line "205"] [id "950005"] [rev "2"] [msg "Remote File Access Attempt"] [data "Matched Data: /etc/ found within ARGS:secret_file: /etc/passwd"] [severity "CRITICAL"] [ver "OWASP_CRS/2.2.6"] [maturity "9"] [accuracy "9"] [tag "OWASP_CRS/WEB_ATTACK/FILE_INJECTION"] [tag "WASCTC/WASC-33"] [tag "OWASP_TOP_10/A4"] [tag "PCI/6.5.4"]
Action: Intercepted (phase 2)
Apache-Handler: application/x-httpd-php
Stopwatch: 1352556406954555 3620 (- - -)
Stopwatch2: 1352556406954555 3620; combined=2104, p1=546, p2=1462, p3=0, p4=0, p5=95, sr=121, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.0 (http://www.modsecurity.org/); OWASP_CRS/2.2.6.
Server: Apache
Engine-Mode: "ENABLED"

3. Try to upload a malicious test file

Create locally a file "testeicar.txt" with following content and try to upload it to your Web Server:

X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*

Upload should fail and you should see this kind of message in modsec log:

Message: Access denied with code 403 (phase 2). File "/tmp//20121110-143758-UJ5Ytn8AAQEAAFjiESIAAAAI-file-pCUZqH" rejected by the approver script "/etc/apache2/modsecurity/util/runav.pl": 0 clamscan: Eicar-Test-Signature [file "/etc/apache2/mods-enabled/security2_module.conf"] [line "160"] [id "159"] [msg "Malicious Code Detected, access denied"]
Action: Intercepted (phase 2)
Apache-Handler: application/x-httpd-php
Stopwatch: 1352554678179590 9450540 (- - -)
Stopwatch2: 1352554678179590 9450540; combined=9447023, p1=562, p2=9446396, p3=0, p4=0, p5=64, sr=117, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.0 (http://www.modsecurity.org/); OWASP_CRS/2.2.6.
Server: Apache
Engine-Mode: "ENABLED"

Step 7: Whitelist configuration, test your Web sites and add whitelist exceptions if required

You could probably encounter actions refused based on rules policies with your Web Sites and Applications, so it is recommended to test and add exceptions when required.

As far en example, in my case using Ajaxplorer and uploading files won't possible in default configuration because some rules policies will reject it.

The way to deal with that is almost simple, keep a terminal view on modsec log and test normal user action, each rejected action will be associated to a unique id, we will add exceptions to these ids to allow normal users actions.

Example with ajaxplorer and file upload:

Without whitelist exception, upload fails with following message log:

Message: Access denied with code 400 (phase 2). Match of "eq 0" against "MULTIPART_UNMATCHED_BOUNDARY" required. [file "/etc/apache2/mods-enabled/security2_module.conf"] [line "80"] [id "200003"] [msg "Multipart parser detected a possible unmatched boundary."]
Action: Intercepted (phase 2)
Apache-Handler: application/x-httpd-php
Stopwatch: 1352555512125594 4755100 (- - -)
Stopwatch2: 1352555512125594 4755100; combined=1441, p1=1276, p2=65, p3=0, p4=0, p5=99, sr=336, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.0 (http://www.modsecurity.org/); OWASP_CRS/2.2.6.
Server: Apache
Engine-Mode: "ENABLED"

This is the first policy blocking our normal user action, so we will had an exception associated with this id:

Edit "/etc/apache2/modsecurity/conf/whitelist.conf" and add:

<LocationMatch "/ajaxplorer/">
SecRuleRemoveById 200003
</LocationMatch>

Then reload or restart Apache2 (sudo service apache2 reload) and restart your normal user action, if it still fails then re-check your modsec log, note the id and add it to exception.

Finally, for Ajaxplorer/owncloud example, i had to deal with these exceptions:

<LocationMatch "/ajaxplorer/">
SecRuleRemoveById 200003 960024 960915
</LocationMatch>

Proceed the same way with all your sites and applications, and you will be done!

Note:

If you have any problem with whitelist by location like exception not been working, try to apply your exceptions directly in the Apache configuration file.

Behind the directive DocumentRoot, append your exceptions like this:

SecRuleRemoveById 200003 960024 960915

And restart/reload Apache.

Step 8: Advanced vulnerabilities and attack tools

Now that your Apache installation has been secured, you should think about testing your site with advanced vulnerabilities tools.

Here are some tools i recommend:

1. Nessus

Nessus is a well known security scan tool, you can get for a free a personal user key and use the software to scan your sites or hosts.

See:
http://www.tenable.com/

Installation is really simple, then simply launch a Web Application scan and wait for the report, you should see many output in log showing ModSecurity activity.

2. Nikto

Also very powerful and simple to use, Nikto will carefully scan and launch various attacks/exploit against your Web Server.

The result will instantly visible in ModSecurity main log with thousands of events logs, i would recommend to to launch 2 scan, a first scan with ModSecurity activated and second with ModSecurity deactivated and just compare results ^^

See:
http://cirt.net/

Example of use:

Launching a scan in http:

perl nikto.pl -h HOST -p PORT

Launching a scan in https:

perl nikto.pl -h HOST -p PORT -ssl

3. Web-sorrow

Another tool, under hard tests :-)

http://code.google.com/p/web-sorrow/

FAQ & Issues

How to activate / deactivate ModSecurity Module

Nothing more simple:

Activating:

$ sudo a2enmod security2_module
$ sudo service apache2 restart

Deactivating:

$ sudo a2enmod security2_module
$ sudo service apache2 restart

How to partially deactivate ModSecurity for a location

You may want to partially deactivate ModSecurity for a specific location, to do this add a such section into your whitelist:

<LocationMatch "/application/">
SecRuleEngine Off
</LocationMatch>

And restart / reload Apache to apply.

How to deactivate ModSecurity for a full Apache instance

You may also want to deactivate ModSecurity by instance, to do so edit the instance Apache configuration file and this behind the DocumentRoot line:

SecRuleEngine Off