Logs are ubiquitous in IT – they are semi-structured pieces of information about the behavior of a system and its users. Many standards, regulations and best practices assume and require the existence of logs. Consequently, many systems collect those logs and make use of them for various purposes. Too often organizations have just one tool that collects all logs in hope that the chosen tool covers all purposes for collecting logs. This is often wrong but unfortunately reinforced by the tools’ marketing materials.
Let’s look at several types of tools, several types of logs and several purposes for collecting logs. The tools can be:
- SIEM
- Log collector
- UEBA
- Audit trail solution
- General data aggregation tools
The types of logs (as discussed in another article about audit trail) are usually:
- application logs
- network logs
- an audit trail (who did what)
- OS logs
- Database logs
- Investigating application behavior and problems
- Detecting anomalous network traffic
- Detecting anomalous user behavior and fraud
- Legal purposes (including proof of compliance and court cases)
- Forensics
- Business process analysis
- Analytics
- application development department
- information security department
- network operations department
- compliance department
- risk management department
And the above technologies are good with some parts of the puzzle, but not with everything:
- SIEMs are good with network traffic and the related anomalies
- Log collectors and SIEMs are good for application log investigations
- UEBAs are good with user behavior analytics based on multiple data points (not just logs)
- Dedicated audit trail solutions are good with compliance and legal cases
- UEBA and Audit trail are good for helping business process analysis
- Most of the tools do anomaly detection on some aspect of the data, but rarely having just one is enough to cover the whole spectrum of anomalies and potential fraud
- Data aggregation tools are good for analytics
- All of the above is useful and needed for forensics
Some solutions grow to be in more than one category. SIEMs are now increasingly incorporating UEBA functionality, “gather everything” systems like Splunk or possibly Datadog can, in theory, cover everything and in practice are too complicated to set up for all different use-cases and cover them partially.
That makes it a complex 4-dimensional matrix and therefore simply collecting logs in whatever tool is available doesn’t cut it. It usually covers the needs of one or two departments. Usually, it’s the technical departments that get to choose the technology, leaving other use-cases only partially covered.
Below you can see a sample diagram of different strengths of different solutions. It’s more complicated than a Venn diagram can show, but it does illustrate the log collection ecosystem:
Maybe because of that, dedicated audit trail solutions are underrepresented. Neither of the other solutions provides proper audit trail functionality. For example, even having several tools may leave an organization blind to some parts of its IT infrastructure. Because they don’t nudge the IT team to think in terms of an audit trail.
That’s why we believe SentinelTrails fits nicely into most organizations’ tech stacks, to cover compliance and legal use-cases not properly covered by the tools currently deployed. To put it bluntly, having a SIEM, an application log collector and a data analytics platform don’t mean you have a proper audit trail. And compliance departments are getting more and more aware of that.
And if you need a SIEM that covers all of the above requirements, including the strong audit log integrity guarantees, our LogSentinel SIEM, which includes the SentinelTrails module, is the right tool for the job.
Bozhidar Bozhanov is a senior software engineer and solution architect with 15 years of experience in the software industry. Bozhidar has been a speaker at numerous conferences and is among the popular bloggers and influencers in the technical field. He’s also a former government advisor on e-government, transparency, and information security.
