Monitoring

Go back to Tutorial

To monitor or monitoring generally means to be aware of the state of a system, to observe a situation for any changes which may occur over time, using a monitor or measuring device of some sort.

Various types of monitoring are

  • Application performance monitoring, monitoring and management of performance and availability of software applications
  • Business transaction monitoring, managing information technology from a business transaction perspective
  • Network monitoring, systems that constantly monitors a computer network for slow or failing components and that notifies the network administrator
  • System monitoring, a process within a distributed system for collecting and storing state data
  • Website monitoring, the process of testing and verifying that end-users can interact with a website or web application as expected

Event Monitoring

Event monitoring is the process of collecting, analyzing, and signaling event occurrences to subscribers such as operating system processes, active database rules as well as human operators. These event occurrences may stem from arbitrary sources in both software or hardware such as operating systems, database management systems, application software and processors.

Event monitoring makes use of a logical bus to transport event occurrences from sources to subscribers, where event sources signal event occurrences to all event subscribers and event subscribers receive event occurrences. An event bus can be distributed over a set of physical nodes such as standalone computer systems. Typical examples of event buses are found in graphical systems such as X Window System, Microsoft Windows as well as development tools such as SDT.

Event collection is the process of collecting event occurrences in a filtered event log for analysis. A filtered event log is logged event occurrences that can be of meaningful use in the future; this implies that event occurrences can be removed from the filtered event log if they are useless in the future. Event log analysis is the process of analyzing the filtered event log to aggregate event occurrences or to decide whether or not an event occurrence should be signaling. Event signaling is the process of signaling event occurrences over the event bus.

Something that is monitored is denoted the monitored object; for example, an application, an operating system, a database, hardware etc. can be monitored objects. A monitored object must be properly conditioned with event sensors to enable event monitoring, that is, an object must be instrumented with event sensors to be a monitored object. Event sensors are sensors that signal event occurrences whenever an event occurs. Whenever something is monitored, the probe effect must be managed.

Event log analysis is known as event composition in active databases, chronicle recognition in artificial intelligence and as real-time logic evaluation in real-time systems. Essentially, event log analysis is used for pattern matching, filtering of event occurrences, and aggregation of event occurrences into composite event occurrences. Commonly, dynamic programming strategies from algorithms are employed to save results of previous analyses for future use, since, for example, the same pattern may be match with the same event occurrences in several consecutive analysis processing. In contrast to general rule processing (employed to assert new facts from other facts, cf. inference engine) that is usually based on backtracking techniques, event log analysis algorithms are commonly greedy; for example, when a composite is said to have occurred, this fact is never revoked as may be done in a backtracking based algorithm.

Security information and event management – (SIEM) software provides the log management infrastructure encompassing log analysis, log storage and log monitoring tiers. What sets SIEM products apart from traditional log management software is the ability to perform event correlation, alerting, incident management, reporting and forensic investigation based on event analysis. There are many SIEM solutions commercially available today and these solutions provide different set of these features and additional add-ons

Network Monitoring

Network monitoring is the use of a system that constantly monitors a computer network for slow or failing components and that notifies the network administrator (via email, SMS or other alarms) in case of outages. It is usually a part of network management.

While an intrusion detection system monitors a network for threats from the outside, a network monitoring system monitors the network for problems caused by overloaded and/or crashed servers, network connections or other devices.

For example, to determine the status of a webserver, monitoring software may periodically send an HTTP request to fetch a page. For email servers, a test message might be sent through SMTP and retrieved by IMAP or POP3.

Commonly measured metrics are response time, availability and uptime, although both consistency and reliability metrics are starting to gain popularity. The widespread addition of WAN optimization devices is having an adverse effect on most network monitoring tools — especially when it comes to measuring accurate end-to-end response time because they limit round trip visibility.

Status request failures – such as when a connection cannot be established, it times-out, or the document or message cannot be retrieved – usually produce an action from the monitoring system. These actions vary — an alarm may be sent (via SMS, email, etc.) to the resident sysadmin, automatic failover systems may be activated to remove the troubled server from duty until it can be repaired, etc.

Monitoring the performance of a network uplink is also known as network traffic measurement, and more software is listed there.

Logging

Logging is the act of keeping a log. In the simplest case, messages are written to a single logfile. A logfile is a file that records either events that occur in an operating system or other software runs, or messages between different users of a communication software.

Many operating systems, software frameworks, and programs include a logging system. A widely used logging standard is syslog, defined in Internet Engineering Task Force (IETF) RFC 5424). The syslog standard enables a dedicated, standardized subsystem to generate, filter, record, and analyze log messages. This relieves software developers of having to design and code their own ad hoc logging systems.

Types

  • Event logs record events taking place in the execution of a system in order to provide an audit trail that can be used to understand the activity of the system and to diagnose problems. They are essential to understand the activities of complex systems, particularly in the case of applications with little user interaction (such as server applications). It can also be useful to combine log file entries from multiple sources. This approach, in combination with statistical analysis, may yield correlations between seemingly unrelated events on different servers. Other solutions employ network-wide querying and reporting.
  • Transaction logs – Most database systems maintain some kind of transaction log, which are not mainly intended as an audit trail for later analysis, and are not intended to be human-readable. These logs record changes to the stored data to allow the database to recover from crashes or other data errors and maintain the stored data in a consistent state. Thus, database systems usually have both general event logs and transaction logs.
  • Message Logs – Internet Relay Chat (IRC), instant messaging (IM) programs, peer-to-peer file sharing clients with chat functions, and multiplayer games (especially MMORPGs) commonly have the ability to automatically log (i.e. save) textual communication, both public (IRC channel/IM conference/MMO public/party chat messages) and private chat messages between users.

A security log is used to track security-related information on a computer system. Examples include:

  • Windows Security Log
  • Internet Connection Firewall security log

Cyber Security Monitoring

A cyber security monitoring process involves, different components for an organisation to identify, investigate and respond to indicators of compromise effectively. The main components of a typical cyber security monitoring process are

Image 44

Need

The main focus of their cyber security monitoring process was in:

  • Looking for indicators of potential cyber security incidents
  • Adhering to contractual or certification standards (eg. ISO 27001) and good practice (eg. ISF SOGP)
  • Meeting compliance requirements (eg. for PCI DSS or FCA).

Types

Different types of cyber security monitoring can be categorised in a number of ways, such as by looking at a combination of:

  • In-line monitoring (eg. looking at traffic passing through a firewall)
  • Not in line monitoring, which includes:
  • Analysing information about ‘activity’ based intrusion detection
  • Employing heuristic-based rules
  • Taking the ‘Big data’ approach.

Cyber security monitoring process should cover

  • Network perimeter
  • Endpoints (eg. PCs and laptops)
  • Core devices and systems (internal).

Challenges

A range of different monitoring challenges is present, which usually includes

  • Determine where best to spend the cyber security budget
  • Detect anomalous system and network behaviour, distinguishing real anomalous behaviour from ‘noise’
  • Determine that an event is actually a cyber security incident
  • Link threat intelligence and log outputs to provide an overall picture of your threat profile
  • Justify the cost / time to deal with very rare cyber security incidents.

For many organisations, the most challenging part of the monitoring process is accurately detecting and assessing possible cyber security incidents – determining whether an incident has occurred and, if so, the type, extent, and magnitude of the problem.

When monitoring cyber security events, one of the most important actions is to be properly prepared. In the event of a cyber security incident occurring this will help you to recover your systems more quickly, minimise the impact of the attack, instill confidence in your customers and even save you money in the long term.

Process

Image 45

 

Go back to Tutorial

Get industry recognized certification – Contact us

Menu