What It Takes to Make AI Useful in Enterprise Networking

It’s easy to show off AI agents, but making them work well in real-world production is much harder. From our work with internal GPTs, fine-tuning, RAG, and open-source models, we’ve seen that agents only perform well when they are tightly connected to the systems they support. To do that, they need secure, real-time access to the infrastructure, telemetry, code, and workflows that span neoclouds, public cloud platforms, existing data centers, SaaS applications, and enterprise data platforms.

For enterprise teams evaluating AI in networking, that leads to an important question: is the vendor simply adding AI to the interface, or does it have meaningful experience applying AI across its own operations, engineering, and products? In practice, that internal experience often shapes whether AI capabilities translate into secure, practical, and production-ready outcomes for customers.

That matters because the real challenge is not model experimentation on its own. It is turning AI into something operationally useful inside live networking environments.

From Models to Agents. Watch out for Integration Gaps

At first, the industry focused on experimenting with different model sizes, fine-tuning, and prompt strategies. This helped test what Large Language Models (LLMs) could do, but new challenges appeared when the focus moved from chatbots to agents.

For enterprise networking teams, an agent has to do more than generate answers. It becomes part of the operational environment and must have access to the systems, context, and signals that shape the decisions, including:

• Kubernetes clusters for orchestration and service health
• Metrics, alerts, and logs for real-time visibility
• Source control and internal tools to understand configuration intent
• Collaboration platforms to communicate findings and support operator workflows

Without that level of integration, agents can still appear capable while lacking the context needed to produce reliable operational outcomes. Our experience has shown that once AI moves into real environments, integration architecture matters more than prompt design alone. That was a key lesson from our early work applying AI in complex networking environments.

Start with Control, Not Autonomy

As soon as agents begin interacting with real infrastructure, security moves from a design consideration to an operational requirement. To be useful, agents often need credentials such as API tokens, roles, and certificates. But granting broad or unmanaged access too early can introduce unacceptable risk.

A better approach is to place agents behind controlled gateways that enforce clear boundaries around what they can access and what they are allowed to do. In our work, we used Model Context Protocol (MCP) servers as an intermediary layer to define permissions, constrain actions, and separate model reasoning from direct system access.

For buyers evaluating AI-enabled infrastructure, we believe vendors should take a conservative, incremental approach to autonomy:

• Read-only access. Agents can observe, analyze, and report, but not change the system state
• No direct infrastructure modification. Agents should not be able to de-provision, reconfigure, or disrupt critical resources without explicit controls
• No black boxes, we need full observability and auditability. Every output, recommendation, and action should be logged, reviewable, and understandable by human operators

Only after agents have demonstrated safe, reliable behavior in tightly controlled conditions should vendors consider allowing limited corrective actions. Even then, those actions should remain narrowly scoped, policy-driven, and protected by secure protocols. This approach may slow initial deployment, but it is the more responsible path to building trust and reducing production risk.

See also: 6 Proven Day-2 Strategies for Scaling Kubernetes

Economics. Remember Agents Don’t Sleep

Cost becomes a critical consideration once agents move into continuous operation. In networking environments, many agents run around the clock, monitoring systems, correlating signals, and supporting background workflows.

At that point, inference costs can increase quickly, especially if every routine decision depends on an expensive external model call. We recognized early that a useful agent also has to be economically sustainable. That led us to adopt several practical design principles that buyers should also look for in their vendors:

• Separate observation from reasoning: Use lightweight heuristics and deterministic checks for continuous monitoring, and invoke more advanced AI reasoning only when anomalies or higher-order decisions require it
• Use task-specific models: Match the model to the job, using smaller and more efficient models for routine tasks instead of defaulting to a large general-purpose LLM
• Be deliberate about inference: Distinguish between moments that require model-driven reasoning and those that can be handled through standard automation, policy logic, or predefined workflows

This kind of discipline matters because the long-term value of AI is not just determined by what an agent can do, but by whether it can do it efficiently, predictably, and at scale.

Why Inference Architecture Matters

As AI agents scale in networking environments, the architecture behind inference starts to matter just as much as the model itself. Vendors need flexibility in how inference is deployed, whether through external services, local execution, or a hybrid approach, depending on the needs of the task.

What matters most is not a single deployment model, but the ability to balance several operational requirements:

• Predictable performance: Inference should support responsive, reliable behavior for operational workflows
• Cost control: Always-on agents need an execution model that remains economically sustainable over time
• Architectural flexibility: Different networking tasks may call for different models, runtimes, or deployment approaches
• Reduced dependency risk: Vendors should avoid unnecessary reliance on any single external service or execution path
• Strong data inputs: High-quality, structured telemetry and system context often matter more than simply choosing a larger model

In our experience, model performance depends heavily on the quality of the surrounding system, especially the structure of the inputs, the consistency of the data pipeline, and the clarity of the task being performed. For enterprise networking, that is the bigger lesson: effective AI depends as much on architecture and data quality as it does on model selection.

What Buyers Should Demand

When evaluating a networking vendor, buyers should look beyond the user interface and AI branding. The more important question is whether the vendor has meaningful, real-world experience applying AI within its own operations, engineering, and product environment.

Key questions to ask include:

• How are they securing agent access, credentials, and permissions?
• How are they managing inference costs as AI usage scales?
• How are they balancing performance, control, and dependency risk across their inference architecture?
• How are they connecting AI systems to the infrastructure, telemetry, and workflows that shape real operational decisions?

Vendors with hands-on experience applying AI internally are often better positioned to design solutions that are practical, secure, and operationally useful for customers. They understand that effective AI in networking has to be integrated into the underlying environment, not simply added as a surface-level feature.

AI in networking is not about replacing engineers. It is about giving systems the context, access, and safeguards needed to handle routine tasks more effectively, so engineering teams can stay focused on higher-value analysis, architecture, and decision-making. The strongest solutions are typically built by vendors that have already been doing that work inside their own environments.