Logging – Missing The Trees For The Forest

When your security perimeter has shifted from usual DMZ and firewalls to a combination of web, mobile devices and cloud infrastructure one cannot continue thinking along the conventional model of security perimeter as defined by network boundaries. With app sprawl IT security has become complex and needs all the help that the apps themselves provide.

While designing applications, organizations must have a secure design mind-set. A secure design of application can take care of some of the fundamental aspects of app security with a pre-emptive stance:

Authentication Mechanism
Authorization Model
Privilege Escalation Architecture
Encryption
Session Management and Session Security
And more ….

One key aspect of App Security design which is either left out or else is dealt with a cursory approach is app logging. Usually a standard logging API or component is picked, and, it is presumed that logging will be taken care of by that component. This presumption is grossly misplaced!

Logging needs to be by design and done exceedingly well, for:

Forensic investigation (a breach, for example)
Debugging (of a running application)
Visibility into user activities (user behaviour)
Monitoring (for Ops and for security)

While designing your application security, pay detailed attention to the logging architecture, and ensure you have usability, coverage and depth in your app logs.

Key Considerations

Here are three checks every Security Architect should look at, while designing logging for an App:

Do I need active run time monitoring of this App?
Would I need a forensic investigation of App activity, if something goes wrong?

Do I need user behaviour monitoring at the app level?

Design that "Logging"

Some of the most critical design considerations are:

Event Architecture
It is super critical to define an event architecture for your application. Without an event architecture, logs would be practically meaningless. A well-defined event architecture, will ensure that you can collect, search and make meaning out of the events in app logs. The event architecture should focus on categorization such as operational events, security events and debug events. Security events should be further classified into authentication events, authorization events and so on.

It is also important to map your event architecture to one of the common and prevalent standards such as CEF. This mapping will help you in correlation at a later point of time.

Collection Ease
If you plan to have your application integrated into monitoring tools, collection of events from logs needs to be simplified. Otherwise, you will be tied up writing a set of plugins for collection technologies of various monitoring technologies/SIEMs.

Event collection from logs becomes simpler, if design considers:

Event data length
Watermarking assistance
Event/log format

Severity Classification
Though almost all tools, which will collect logs from your application, will have a mechanism to allocate a severity to the events. You can make it easier and also a lot more efficient, by clearly designing a severity hierarchy in the data/events, which are logged into apps.

For security specific events (such as authentication) this severity classification needs to be further detailed and should be implemented at fine grained level.

User Event Data
A lot of applications do not have any specific treatment of user related logging in application. They log user specific event, with almost similar treatment, as they would any other operational or process event.

Yet, quality of user event data in logs can dictate the quality of user behaviour analysis in your security tools and apparatus. You should consider atop down approach while designing your user activity specific events in app.

What would you like to see in user behaviour in monitoring tools?
What kind of analytics you would like to have for user activities?

Noise Reduction in Logs
Noise levels in application logs is one of the biggest reason app event logs become meaningless. There is a tendency to shove almost everything to the logs. This leads to massive noise in the logs. It becomes complex for many aggregation tools and tech to extract valuable event data buried within voluminous app logs.

Keep an eye on both log design and developer tendencies to avoid multiple registration of the same event. Also create an abstraction model for similar event categories, so you don’t end up creating too many unnecessary events in your application logs.

App Log Levels
Most applications are not designed for activating and deactivating log levels. Log levels can help you eliminate a lot of unnecessary events from being logged, without compromising anyone’s interest in logged events. A developer should be able to generate and see debug specific events on demand. And, at an operational level disabling/enabling of the specific log categories should be supported.

Security Architects can design their own log levels for specific apps. Here are some standard log levels: