Logs – every system has them, but companies don’t usually pay much attention to them. At least not until a problem occurs. Log aggregation solutions come handy in many scenarios – tracing production issues, alerting on service degradation, fixing bugs, forensics, fraud detection.
We’ve argued, however, that logs have a dual nature – on the one hand, they contain data about the functioning of the system; data, that’s useful for developers to diagnose and fix issues. On the other hand, they are effectively an audit trail of everything that has happened.
Log collection vendors may tell you that you can use the same product for both of these natures of logs. But that has many issues:
- Differentiating events with business meaning from system/application logs. It’s hard for an auditor or a business person to look through logs where 80% of the data is “we’ve initialized service X”, “exception in service Y”, “database connection pools size is approaching the configured limit”, and so on, and only 20% is related to business events – “who did what”
- Investigating behavior – application logs make it hard to drill down into the behavior of a particular user, role or department.
- Security – typical logs lack any additional security, i.e. anyone with admin access can modify them, back-date them or delete them without being detected.
- Compliance – simply having logs is not compliant with standards and laws like ISO 27001, PCI-DSS, HIPAA, PSD2, as we’ve previously argued
Below you can find a feature comparison between LogSentinel and major log collection vendors, as well as vendor of “integrity guarantee” and compliance packages. Obviously, when we are making the comparison, there are many dimensions to select from, but we do believe that these shown here are the most important when it comes to audit logs, security, and compliance.
Certainly a simple table cannot represent all the complexity on the market. For example, log collecting solutions claim to be PCI-DSS, ISO 27001 and HIPAA compliant, and many certification bodies do consider that okay. In our interpretation of the standards, that’s wrong and gives companies a false sense of security – that’s why the table has a yellow color for these rows.
Furthermore, products like GuardTime and Tierion aren’t necessarily audit log solutions – they can be used as such, but they are more focused on the integrity aspect, which means they lack the analytical features (they don’t usually care about the content of the logs, as long as their integrity is guaranteed).
And finally, the products listed below are not mutually exclusive. The feature comparison below is based on the “secure audit trail” use case, but log collection has a wider scope. As mentioned above, you always have applications and system logs and they are distinct from the audit logs. You can have your log collector deployment and only forward a subset of these logs to LogSentinel, thus getting the best of both worlds – secure and compliant audit trail and rich analytical capabilities for logs.
Real information security is rarely about installing a product. Even almost never about the warm feeling that “fact sheets” generate, claiming compliance and endless possibilities. Information security is about a set of measures and tools, properly applied to the problem at hand. And when it comes to audit logs, we believe LogSentinel is the best tool for the job.
Features of main Audit log collectors: Comparison
The following table compares the main features of the audit log collector that are often the main ones companies look for.
- SaaS – all of the listed audit log collectors (except for Oracle Audit Vault) – LogSentinel, Loggly, Splunk, Logz.io, Tierion, GuardTime – follow the Software-as-a-Service business model. Each of them has a subscription plan and some offer freemium options. However, the Tierion business model is on-premise focused but they also offer some subscription plans
- On-premise – some log collectors such as Loggly, Logz.io, and Tierion can’t be used by companies looking for on-premise solutions. For businesses in highly regulated industries, the decision might already be made for them as to whether to house their applications on-premise.
- Tamper-evident logs – tamper-evident logs can be used as legit digital evidence in court, and for auditing purposes. Therefore some of the log collectors support time-stamped tamper-evident logs ensuring that no one can delete or modify them. Such log collectors are LogSentinel and GuardTime
- Timestamp (RFC 3161) and Signing – while most of the log collectors do not support timestamp and signing, LogSentinel and GuardTime have this functionality, which is crucial when it comes to logs of actions which should have qualified time stamping and signing. This technique can be successfully applied in e-voting, online transactions, real estate deals, etc.
- Searchable encryption – the search in encrypted data allows companies to make maximum use of their data while retaining complete confidentiality. This also ensures no sensitive data can leak through the logs. From the listed log collectors only LogSentinel has implemented such technology to date
- Easy Integration – one of the main reasons for bigger organizations to avoid updates is that the integration of such log collectors is a time-consuming process involving many stakeholders. Some log collectors, however, have developed easy, intuitive technology saving time and effort. We evaluate that such log collectors are LogSentinel, Loggly, Splunk, Logz.io. Oracle Audit Vault can be easy to integrate with other Oracle products
- Fraud Detection – logs can be used for detecting different types of anomalous activity. Sometimes this activity can be considered fraud. Log collectors like LogSentinel, Loggly, Splunk, Logz.io and Oracle Audit Vault use advanced, AI-driven methods to provide detailed information on any such activities detected and ensure high quality of the processes and protection of sensitive records
- Rich dashboard – supporting a rich dashboard is not a must-have in terms of collection. Rich dashboards, however, provide full visibility over the processes and their interactions. Dashboards provide high-level real-time reporting which can be useful to the C-level team, but at the same time can be used by Data Protection Officers in their everyday tasks. For this reason, most of the log collectors do have such a dashboard. From the listed only Oracle Audit Vault, Tierion and GuardTime don’t have such
- Alerts – we already mentioned how important fraud and anomaly detection can be. Real-time alerts are the complementary functionality that comes naturally. That’s why the log collectors having this feature usually also have real-time alert notifications
- Long Retention Period – some regulations such as GDPR require having a clear retention period for any kind of data that can be associated with personal information. Therefore all the log collectors strive to provide flexible solutions. However, some of them (Loggly, Logz.io, Splunk) support log archiving which is not always easy to access.
- Custom Log Parsing – Log parsing can be useful in case you are collecting different types of information in one place. LogSentinel, Loggly, Splunk and Logz.io have developed intuitive log parsing in order to make log collection more readable.
- Rich Data Analysis – such analysis allows organizations to keep an eye on the processes going on. Additionally, rich data can be very useful for the detection of specific trends and behavior. From the listed log collector, only Tierion and GuardTime do not support Rich Data Analysis
- ISO 27001 Compliant – ISO 27001 is the international data protection standard that has lots of similarities with GDPR. One of the similarities is related to evidence and log requirements. In order to be tamper-evident, log collectors involve different technologies. LogSentinel has integrated blockchain-enabled logging in order to protect logs from tampering. This technology is future-proof and it tends to be the most sustainable one in terms of verification
- PCI-DSS Compliant – PCI DSS is the international payment security standard that all organizations need to comply with in case they are dealing with online payment transactions. To ensure their payments’ legitimacy payment providers need to implement secure logging which is tamper-evident. That’s why LogSentinel, Tierion, and GuardTime are most suitable for these purposes
- HIPAA Compliant – HIPAA is a US patient data protection act aiming to reduce sensitive data breaches. When it comes to highly sensitive information, compliance is one thing, but the purpose of that compliance is to actually protect your data. And that’s why cryptographic protection is strongly recommended. Again, the top complying log collectors are LogSentinel, Tierion, and GuardTime: they all ensure data integrity and secure tracking of patient data
- PSD2 – like PCI DSS, PDS2 is a directive concerning payment service providers which also guide organizations to use tamper-evident logs. And when logs are to be used it is recommended to select log collectors that ensure they store logs securely. The three log collectors that comply with this payment service directive are LogSentinel, Tierion, and GuardTime
If you are currently exploring log collectors check out SentinelTrails – LogSentinel’s blockchain-enabled secure audit trail. If you are curious to see how it works, book a DEMO today:
Bozhidar Bozhanov is co-founder and the CEO at LogSentinel. He is a senior software engineer and solution architect with 15 years of experience in the software industry. Bozhidar has been a speaker at numerous conferences and is among the popular bloggers and influencers in the technical field. He’s also a former government advisor on e-government, transparency and information security.