Alerting – DevopsCurry

Best Monitoring and Alerting Practices to follow in 2021

Wed, 10 Feb 2021 17:55:29 +0000

DevOps Monitoring and Alerting Best Practices in 2021

DevOps’ approach has been widely adopted by many businesses to enhance the continuous delivery and integration process. But the main feature is to automate the process within the DevOps pipeline. Automation has changed the overall effectiveness of the business. But the challenge occurs while maintaining and monitoring these automated pipelines. You must keep an eye on what is going on within the business as the network grows. As the business grows, having full visibility of the system is getting complicated, especially when you have fewer resources at your disposal. A proper alerting system will send alerts of any unexpected behavior of the system or an application.

Irrespective of your business’s size, you should consider many aspects of monitoring like what should be monitored, what type of tools should be used, and how you should plan the strategy, to begin with monitoring. You can include various automated monitoring tools that will help you to find potential issues. With software development, the monitoring and alerting process have also evolved. As the DevOps approach is gaining success, you can monitor each DevOps phase from integration, provisioning, and even deployment.

DevOps team can use DevOps strategies to monitor each aspect of the software or the project efficiently and effectively. The below article will discuss the monitoring and alerting strategies that should be implemented for a smooth delivery process.

Be specific- what to monitor?

To ensure better system performance, there are many things that you can monitor. But you have to specify and determine what things within the system you want to watch and to what extent. Some of the categories that need monitoring-

Server health checks

You should check the server’s health to ensure the system performance and uptime concerning the available resources. DevOps team should configure the resources properly to identify the critical endpoints within the application. Also, make sure that your server approves the implemented configurations to work seamlessly.

Application log output

Among all other aspects of DevOps, logging an application’s output is often ignored. You must have a centralized logging system in place to check for errors. Make sure that you place proper tools and strategies to pinpoint and alert the errors in real-time to avoid vulnerabilities. These logs are essential for IT teams and system administrators to work on expected errors to find the root cause and resolve them in time.

Development milestones

After implementing the DevOps approach, we must keep track of how well the strategy works and gain your workflow insights. It helps in checking how well your teams are collaborating to achieve success. You should track the sprint durations, error identification rate, resolve rate, and other features to enhance performance. Maintaining proper checks will help you meet the tasks’ deadlines and how well the DevOps approach is implemented. You can find which job need automation and will save your time.

User activity

There are various types of users, and some of their unusual activity may hinder your software. Every user’s accounts are protected and have role-based access to different applications. You must track what is being accessed by which user to ensure safety.

You can also read our posts on APM and Open Source Monitoring tools:

Monitoring Tools functionalities

Among various monitoring tools, you should consider using those efficient tools to gather performance time-series data from multiple sources, track applications for alerting and collect data in databases. Various tools have a different set of functionalities, but you should choose the right one to solve your purpose of implementing a proper monitoring strategy in place. Some of the functionalities that you might consider are discussed below.

1. Dashboards: The implemented monitoring tool should have an interactive user dashboard to get real-time insights into your applications. It should be more comfortable for every user to work on the tool to analyze what is going on within the system.

2. Diagnosis: The tool should help you to find the critical endpoints for vulnerabilities within the system. It allows you to ensure that all your applications are working as expected.

3. Data collector: It should provide you with open-source agents that support programming language.

4. Data retention: It helps you to gather real-time time-series performance of the data and log data.

5. Notifications: You should consider monitoring tools are well-incorporated with the notification system that alerts you about the tasks and latest updates.

6. Reports: The monitoring tool should allow you to report all the tasks, activities, and insights to look for performance hotspots and planning.

How to optimize your Monitoring and Alerting strategy

Once you adopt a monitoring strategy and required tools, you must know how to optimize your strategy efficiently. As the monitoring process has a standard procedure and manually will consume time manually and repeat the process again and again. It is a better option that we automate the monitoring tasks for regular updates.

We can also set alerts on the notification to generate reports and notifications to be sent across the system. This will optimize the monitoring process by implementing proper alerts on each task to check the system’s performance and speed. It will help you to get a better system insight without intervening in the system.

1. Set your monitoring and alerting priority

First, you need to understand the criticality of the applications, different services, and the tasks involved within your system. After that, you can decide what task is more critical and what notifications you should get in the first place. Suppose you are running deployments, and in the meantime, your server and code get compromised at the same time. You need to prioritize the alerting system to send you a notification so the issue will get resolved in time.

Setting the priority will also help the IT team to work on the priority issue on an urgent basis to support the end-users and the business. It will allow IT to respond quickly to problems, provide better customer service, and save time searching for the potential area of error.

2.Monitor the application stack

The monitoring tool that you will choose should monitor the complete stack with a faster troubleshooting process. The tool should cover the below feature set in the application.

Infrastructure monitoring It should include monitoring your availability of resources like- CPU usage, disk usage, uptime, response time, database, storage capacity, performance, security, network switches, server load, etc.

Performance monitoring It should include monitoring your SLAs, browser speed, end-user transactions, throughput, error rate, page loads, etc.

Network monitoring It should include monitoring your network packet flow, bandwidth, latency, network topology, etc.

Commonly used Monitoring and Alerting Tools

Image Credit: https://pt.slideshare.net/grobie/efficient-monitoring-and-alerting/18

Monitoring Tools

Infrastructure Monitoring Tools: Solarwinds AppOptics, Solarwinds ipMonitor,ManageEngine OpManager, Zabbix,Paessler PRTG
Application Monitoring Tools: New Relic APM,AppDynamics, Stackify Retrace, Dynatrace (Compuware APM),TraceView (SolarWinds),Application Insights (Microsoft)
Network Monitoring Tools: WLC monitoring, Datadog, Site24x7, Nagios,ntop, Obkio,Gif Languard, WhatsUp Gols (WUG)

Alerting Tools

PagerDuty, VictorOps,OpsGenie,AlertOps,Cabot,NgDesk, Bosun, StatsAgg

The post Best Monitoring and Alerting Practices to follow in 2021 appeared first on DevopsCurry.

What is Continuous Monitoring in DevOps

Namrata Joshi — Fri, 25 Sep 2020 12:56:24 +0000

Companies are nowdays adopting DevOps to get faster and better response to constantly changing customer needs. DevOps works on Continuous Delivery and Continuous Deployment model. However Continuous Monitoring is also a very important pillar of our DevOps lifecycle (often neglected), which keeps a check on our Continuous Delivery and Deployment systems within any organisation.

What is Continuous Monitoring in DevOps?

Continuous Monitoring is a process to monitor and identify compliance issues and security risks throughout each phase of DevOps and IT operations lifecycles. Continuous Monitoring and observability can be considered as a final step of the DevOps pipeline. This is one of the most crucial steps in a DevOps lifecycle and will help to achieve true efficiency and scalability.

Continuous monitoring aka ConMon, helps organisations to track, identify, understand key metrics and also resolve application or infrastructure issues in realtime.

The aim of DevOps is to remove silos between Development and Operations team. Once the application is deployed and is available for users, Continuous Monitoring will alert the teams if there are any application issues in that environment. This will provide necessary feedback to the team. And the team can understand the parts of the application that demand additional work.

Different Types of Monitoring

Infrastructure Monitoring: it is necessary to create and maintain an IT infrastructure if your organisation depends on Information Technology to deliver products and services. So what does an IT Infrastructure include? Well, it includes everything IT related – servers, networks, data-centers, computer hardware and software, storage, etc. Managing this infrastructure is called Infrastructure Monitoring in DevOps.The sole purpose behind Infrastructure Monitoring is to collect and analyze data from the IT infrastructure and leverage that data to improve the final results.
So what all does the Infrastructure Monitoring covers ? Well it covers the hardware monitoring, network monitoring,OS monitoring and application monitoring. Some of the popular Infrastructure Monitoring Tools include Nagios, Zabbix, ManageEngine OpManager, Solarwinds,Prometheus etc

Application Monitoring: application monitoring offers runtime metrics of system performance like application uptime, application-performance,application security-monitoring,log-monitoring etc. These metrics are monitored by application monitoring tools(APM) and available to SRE/Devops or Sys-Admins. What do these metrics include? This includes transaction time, transaction volume, API responses, system response, and overall health of back-end applications. It is delivered in the form of graphical figures and statistics. These figures will evaluate application performance and ultimately overall infrastructure of an application. Some of the popular APM tools are Appdynamics, Dynatrace, Datadog, Uptime Robot,Uptrends, Splunk etc

Network Monitoring: network monitoring will monitor and track everything about the network. It is a critical process in any DevOps environment. Network Monitoring will monitor networking components like routers, firewalls, servers, switches, and VMs. In general the Network Monitoring systems(NMS) share five critical data points:

Discover
Map
Monitor
Alerting
Reporting

A NMS will find faults, measure the performance of these components, and optimise their availability. It is very important for network monitoring to be proactive. A proactive network monitoring will prevent network downtimes and/or failures. Some of the popular NMS tools include Cacti, ntop,nmap, Spiceworks, Wireshark,Traceroute, bandwidth Monitor etc.

Popular DevOps Monitoring Tools in 2020

1. Nagios : Nagios is an open-source DevOps Monitoring Tool. It will help DevOps to find and correct problems with network & infrastructure. Features:
* Simplifies the process of searching log data.
* Provides complete monitoring of desktop and server operating systems.
* The log management tool is leading in the industry.
* Provides complete monitoring of Java Management Extensions.

2. Monit: Monit is an open-source DevOps Monitoring Tool. Monit is designed especially for monitoring UNIX OS.
Features
* Conducts automatic maintenance and repair.
* Executes meaningful actions in error situations.
* Helps to monitor files, directories, and files on localhost.
* Allows network connections to different servers.

3. Consul: Consul is another open-source monitoring Tool(owned by HashiCorp). Consul is used for discovering and configuring services in any infrastructure. Features:
* The perfect tool for modern infrastructure.
* It provides a robust API.
* Easy to find services each application needs using DNS or HTTP.
* Supports multiple data centers.

4. Datadog: Datadog is a paid monitoring service, providing monitoring of servers, ,applications,databases, tools, and services through a smart SaaS platform.
Features
* Easily customizable monitoring dashboards.
* 80+ turn-key integrations.
* Complete access to APIs.
* Get visibility in the health and performance of the DevOps tools.
* Visualize data from upstream and downstream systems with metrics, traces, and logs.

5. Prometheus: open-source monitoring solution, built primarily for gathering and analysing time-series data.It uses SNMP to collect performance data from devices across the network.
Features:
* Prometheus actively scrapes data, stores it, and supports queries, graphs, and alerts
* Lowers development environment setup time.
* Works seamlessly with Mac, Windows, and LINUX.
* Very popular for monitoring containerised environments (Docker, Kubernetes)
* Integrates with existing configuration tools like Chef, Puppet, Ansible, Salt.
* Creates a single file for projects to describe which kind of machine and software users want to install.

6. PagerDuty: PagerDuty is a paid Alerting Tool. It helps businesses to enhance the reputation of their brand. PagerDuty allows the DevOps team to deliver high performing apps.
Features:
* Data Collection via a pull model over HTTP
* It supports multiple modes for graphs and dashboards
* It works as autonomous single server nodes with no dependency on distributed storage

7. Snort: Snort is an open-source network intrusion detection system (NIDS) Snort is a powerful tools that help to detect network intruders.
Features:
* Allows real-time traffic analysis and packet logging.
* Performs protocol analysis and content searching.
* Detects buffer overflows, stealth port scans, and OS fingerprinting attempts.

8. Splunk: Splunk is a paid tool. It makes machine data accessible and usable to everyone.It is a software platform to search, analyze and visualize the machine-generated data gathered from across the components of IT ininfrastructure
Features:
* Delivers operational intelligence to DevOps Team.
* Data drive analytics with actionable insights.
* Delivers a single, unified view of various IT services.
* You can extend the Splunk platform with purpose-built solutions for security.

The post What is Continuous Monitoring in DevOps appeared first on DevopsCurry.