Observability: Key to Managing Complex Infrastructures

Network and corporate infrastructures are ever-so more important today as companies deploy new applications and undergo digital transformations. As such, there is a greater need for network visibility, observability, and ultimately the automation of network management functions.

Perhaps the biggest current change in network management is its role in aligning IT and business objectives. Network managers need real-time insights about their operations to ensure the infrastructure supports the needs of the business. To accomplish this, they need awareness and continuous monitoring. Specifically, they must monitor and collect real-time network status information of the underlying systems used to meet the business objective.

That information can be used to dynamically optimize network resources (using real-time status information) to ensure the network delivers the performance and availability needed for the specific business goal.

Currently, many organizations are transitioning from traditional reactive network management approaches to more proactive methods. An example of how the different network management techniques work is how each would ensure an executive video conference goes off without a hitch. In the succession of network management strategies:

• The traditional approach to network management would be to wait for an angry call from executives complaining about the poor quality of their call. Next, an IT manager would use troubleshooting tools to identify the problem. And then make changes (perhaps increase the site’s bandwidth before the next call is made).
• A more proactive approach would spot an increase in dropped or resent packets and other indicators of a poor video conferencing session and take corrective actions in real-time. For example, an IT manager could instruct a router or other edge device to give the video conferencing traffic more bandwidth or assign a lower priority to traffic from other users.
• A more holistic approach would identify the business goal (a high-quality executive video conferencing session at 10 a.m. Monday), translate that into commands that configure the hardware to control bandwidth during the call, monitor the activity of other users and applications that are consuming great amounts of bandwidth, and dynamically adjust bandwidth to the executives while controlling bandwidth used by others. 

See also: 4 Key Trends in Monitoring and Observability

Complexity requires observability and AIOps

Until recently, network infrastructures were relatively static. Physical boundaries separated the corporate network that contained most end-user applications, data, and services within the LAN and WAN. Thus, from a network perspective, if the network devices were up and pushing packets, relatively little added visibility was required. SNMP, ping, traceroute, and Syslog reporting were all that was needed.

The use of cloud-based resources (applications, compute power, infrastructure, and more) makes network management more challenging. Visibility gaps in network monitoring and alerting tools arise with networks now stretching into third-party managed infrastructure-as-a-service (IaaS) clouds and apps/data moving into platform-as-a-service (PaaS) and SaaS environments.

What’s needed is more monitoring and alerting capabilities. However, such capabilities can add to the workload of an already busy network administrator. That is why the industry is undergoing a shift away from separate network, application, and device monitoring tools to a more inclusive approach of artificial intelligence (AI) for IT operations (AIOps).

Modern monitoring and observability platforms offer many of these benefits. AIOps platforms combine traditional monitoring tools with streaming telemetry. They analyze all of the data to spot anomalies, derive insights, and make predictive assessments of the state of the systems. They analyze each data source and correlate multiple anomalies to automate the identification of problems while also providing detailed information about the potential source of and problem. Thus, if a modern monitoring and observability platform, enabled by AI, is properly implemented, it provides more visibility into potential problems and eliminates many manual troubleshooting and remediation tasks.