Mining Firewall Logs for Nuggets

The firewall is a barrier between your organization and outside Internet traffic that tries to enter your company’s network. The firewall determines which attempts gain entry, and also logs all successful and unsuccessful connection attempts, such as harmful scans, probes and intrusions.

These logs can also clarify whether your link is being accessed for file-sharing traffic. They can notify administrators if spyware has infected company desktops, and inform them if the system has been down and even how long the stoppage lasted.

Unfavorable data might paint a dismal picture of the network, causing many network or security administrators to ignore firewall log data or even discard it. Don’t worry. Here’s what you should do with your logs.

Supersize That Log

The broader the scope of incoming and outgoing connection attempts covered, the more likely that intrusion attempts will be caught and trouble spots exposed. Track questionable activities that pop up on the log. For example, configuring the firewall to log all “permits” and “denies” will result in a collection of data you can compare against a list of known spyware. Catalogued data uncovers trends in activity, and you can use it to develop better firewall rules.

Figure 1

A typical red flag is a pattern of numerous failed administrator login attempts at the firewall. This could indicate something as innocuous as a forgotten password by the system administrator or something more critical, such as unauthorized personnel inside or outside your organization attempting to access firewall configuration settings (see Figure 1).

Excessive network and bandwidth usage are other red flags. Analyzing firewall traffic logs is fundamental to understanding overall network usage. Excess traffic and bandwidth could indicate viruses or other malware, and may uncover blatant policy violations, including employees installing and using peer-to-peer file-sharing software such as LimeWire, BitTorrent or Gnutella.

The ManageEngine Firewall Analyzer by AdventNet can do this type of bandwidth analysis and is compatible with firewall devices from SonicWall, Check Point Software Technologies, Cisco Systems and others.

A freeware program called the DShield distributed intrusion detection system (dshield.org) collects, correlates and analyzes log data. If you use Microsoft products, you will find Log Parser (microsoft.com/technet/scriptcenter/tools/logparser/default.mspx) a versatile tool for generating custom text-based output for a variety of specific log data.

Administrators should classify, sort and examine any changes in the network boundaries (interfaces), firewall policies and administrator accounts — including deletions or additions. Review compromises to the system, messages within the host operating system log and all connections (both permitted and rejected/failed attempts) to and from the network. Check their particulars, such as time of day, Internet Protocol address, protocol and user names.

Determine the sources of IP addresses if they appear to be rejected and dropped. Veritable IP addresses can be resolved with the “Who Is” database, and you can query their owners about the reasons a user at their site is probing your ports. If the owner is an Internet service provider, it will be able to determine whether the prober is one of its customers.

Repeated rejected login attempts from the same domain are suspect. The administrator should first determine the IP address and whether it is being spoofed. If the administrator elects to continue to reject connections from that IP address, write a rule to drop all connection attempts from it.

Hackers often target unused ports for installing Trojans, probing specific ports to see whether they are in use. If your logs reveal such repeated attempts, it might indicate hacker activity. A common backdoor Trojan horse known as BackOrifice, for example, uses port 31337. Look up known Trojan programs and the ports they use.

Suspicious outbound connections originating from your public Web server may indicate that a hacker is using your server to instigate an attack on another system. Administrators should determine that connections from their organizations to other systems are authorized and examine source-routed packets. An internal address may be being spoofed if a packet with a source address internal to your network originates from outside your network. Spoofing in this way allows an unauthorized user to access your internal network.

Get a Grep on Logs

Having firewall log files to review is great. Sifting through their data is not. To search for specific entries, consider Grep, a Unix, Linux and Mac utility that searches files for text and/or text strings and detects specific activity patterns. There is also a Windows version at www.wingrep.com.

To extract information from log files, Grep them as follows:

• grep -v excludes the greped word or phrase.
• Use egrep “word1|word2|word3” for multiple words; egrep -i is case insensitive.

If you want to determine which host is going offline, for example:

• egrep “online|offline” /data/log/watchdog.log

Is the detector offline? Look at the last entry of:

• egrep “online|offline” /data/log/watchdog.log | grep detector

Was there a retry error?

• grep retry /data/log/det_api_workstation.log

When was the door or tray ajar?

• grep door_or_tray_ajar /data/log/watchdog.log

When were images ready? This will list all of them:

• grep image_ready /data/log/watchdog.log

For a faster, better way for listing recently collected images:

• tac /data/log/watchdog.log | grep image_ready | less

Now you should be able to sift through firewall log files with little pain, and get the most benefit from their contents.