What is Continuous Monitoring in DevOps

Companies are nowdays adopting DevOps to get faster and better response to constantly changing customer needs. DevOps works on Continuous Delivery and Continuous Deployment model. However Continuous Monitoring is also a very important pillar of our DevOps lifecycle (often neglected), which keeps a check on our Continuous Delivery and Deployment systems within any organisation.

What is Continuous Monitoring in DevOps?

Continuous Monitoring is a process to monitor and identify compliance issues and security risks throughout each phase of DevOps and IT operations lifecycles. Continuous Monitoring and observability can be considered as a final step of the DevOps pipeline. This is one of the most crucial steps in a DevOps lifecycle and will help to achieve true efficiency and scalability.

Continuous monitoring aka ConMon, helps organisations to track, identify, understand key metrics and also resolve application or infrastructure issues in realtime.

The aim of DevOps is to remove silos between Development and Operations team. Once the application is deployed and is available for users, Continuous Monitoring will alert the teams if there are any application issues in that environment. This will provide necessary feedback to the team. And the team can understand the parts of the application that demand additional work.

Different Types of Monitoring

Infrastructure Monitoring: it is necessary to create and maintain an IT infrastructure if your organisation depends on Information Technology to deliver products and services. So what does an IT Infrastructure include? Well, it includes everything IT related – servers, networks, data-centers, computer hardware and software, storage, etc. Managing this infrastructure is called Infrastructure Monitoring in DevOps.The sole purpose behind Infrastructure Monitoring is to collect and analyze data from the IT infrastructure and leverage that data to improve the final results.
So what all does the Infrastructure Monitoring covers ? Well it covers the hardware monitoring, network monitoring,OS monitoring and application monitoring. Some of the popular Infrastructure Monitoring Tools include Nagios, Zabbix, ManageEngine OpManager, Solarwinds,Prometheus etc

Application Monitoring: application monitoring offers runtime metrics of system performance like application uptime, application-performance,application security-monitoring,log-monitoring etc. These metrics are monitored by application monitoring tools(APM) and available to SRE/Devops or Sys-Admins. What do these metrics include? This includes transaction time, transaction volume, API responses, system response, and overall health of back-end applications. It is delivered in the form of graphical figures and statistics. These figures will evaluate application performance and ultimately overall infrastructure of an application. Some of the popular APM tools are Appdynamics, Dynatrace, Datadog, Uptime Robot,Uptrends, Splunk etc

Network Monitoring: network monitoring will monitor and track everything about the network. It is a critical process in any DevOps environment. Network Monitoring will monitor networking components like routers, firewalls, servers, switches, and VMs. In general the Network Monitoring systems(NMS) share five critical data points:

1. Discover
2. Map
3. Monitor
4. Alerting
5. Reporting

A NMS will find faults, measure the performance of these components, and optimise their availability. It is very important for network monitoring to be proactive. A proactive network monitoring will prevent network downtimes and/or failures. Some of the popular NMS tools include Cacti, ntop,nmap, Spiceworks, Wireshark,Traceroute, bandwidth Monitor etc.

Popular DevOps Monitoring Tools in 2020

1. Nagios : Nagios is an open-source DevOps Monitoring Tool. It will help DevOps to find and correct problems with network & infrastructure.
Features:
* Simplifies the process of searching log data.
* Provides complete monitoring of desktop and server operating systems.
* The log management tool is leading in the industry.
* Provides complete monitoring of Java Management Extensions.

2. Monit: Monit is an open-source DevOps Monitoring Tool. Monit is designed especially for monitoring UNIX OS.
Features
* Conducts automatic maintenance and repair.
* Executes meaningful actions in error situations.
* Helps to monitor files, directories, and files on localhost.
* Allows network connections to different servers.

3. Consul: Consul is another open-source monitoring Tool(owned by HashiCorp). Consul is used for discovering and configuring services in any infrastructure.
Features:
* The perfect tool for modern infrastructure.
* It provides a robust API.
* Easy to find services each application needs using DNS or HTTP.
* Supports multiple data centers.

4. Datadog: Datadog is a paid monitoring service, providing monitoring of servers, ,applications,databases, tools, and services through a smart SaaS platform.
Features
* Easily customizable monitoring dashboards.
* 80+ turn-key integrations.
* Complete access to APIs.
* Get visibility in the health and performance of the DevOps tools.
* Visualize data from upstream and downstream systems with metrics, traces, and logs.

5. Prometheus: open-source monitoring solution, built primarily for gathering and analysing time-series data.It uses SNMP to collect performance data from devices across the network.
Features:
* Prometheus actively scrapes data, stores it, and supports queries, graphs, and alerts
* Lowers development environment setup time.
* Works seamlessly with Mac, Windows, and LINUX.
* Very popular for monitoring containerised environments (Docker, Kubernetes)
* Integrates with existing configuration tools like Chef, Puppet, Ansible, Salt.
* Creates a single file for projects to describe which kind of machine and software users want to install.

6. PagerDuty: PagerDuty is a paid Alerting Tool. It helps businesses to enhance the reputation of their brand. PagerDuty allows the DevOps team to deliver high performing apps.
Features:
* Data Collection via a pull model over HTTP
* It supports multiple modes for graphs and dashboards
* It works as autonomous single server nodes with no dependency on distributed storage

7. Snort: Snort is an open-source network intrusion detection system (NIDS) Snort is a powerful tools that help to detect network intruders.
Features:
* Allows real-time traffic analysis and packet logging.
* Performs protocol analysis and content searching.
* Detects buffer overflows, stealth port scans, and OS fingerprinting attempts.

8. Splunk: Splunk is a paid tool. It makes machine data accessible and usable to everyone.It is a software platform to search, analyze and visualize the machine-generated data gathered from across the components of IT ininfrastructure
Features:
* Delivers operational intelligence to DevOps Team.
* Data drive analytics with actionable insights.
* Delivers a single, unified view of various IT services.
* You can extend the Splunk platform with purpose-built solutions for security.