If you have spent any time looking at modern data center architecture lately, you know that the center of gravity in networking has shifted dramatically. For over a decade, the primary goal of any enterprise network engineer was optimizing north-south traffic—getting data efficiently from the user, through the edge, and into a centralized core or cloud repository. But the explosive growth of generative AI and large language models (LLMs) changed the playbook, supercharging east-west traffic as clusters of GPUs talk to one another in hyper-dense internal fabrics.
Now, major infrastructure players are betting big on the premise that the AI gold rush will be won on the back of massive, high-capacity east-west corridors connecting disparate AI factories. Yet, a more nuanced reality is taking shape on the ground. Even as hyperscalers pour capital into massive data center interconnects, the architectural demands of AI inference are pushing workloads back out to the edge. For network and telecom engineers, this means we are not heading toward a pure east-west future. Instead, we are entering an era of criss-cross communications where balancing east-west AI training fabrics with north-south edge traffic will define infrastructure strategy for the next decade.
The Macro Shift: Fueling the East-West AI Factory Spine
To understand why legacy transport providers are suddenly talking about east-west data flows, you have to look at how modern AI workloads operate. Training a foundational model is not a single-processor job; it requires thousands—sometimes tens of thousands—of specialized accelerators operating in lockstep. Because no single data center can house all the compute power, power, and cooling required for massive clusters, workloads are increasingly distributed across multiple “AI factories.”
This distribution creates an insatiable demand for ultra-low latency, high-bandwidth Data Center Interconnect (DCI). We are moving past the days when a standard 100GbE or even 400GbE wave division multiplexing (WDM) circuit sufficed for enterprise backup and replication. Today, carriers are architecting dedicated, high-capacity optical paths designed explicitly to stitch geographically separate compute clusters into a single, cohesive logical fabric.
For network engineers, this requires a fundamental rethink of transport layers. We are no longer just routing packets; we are extending the backplane of a supercomputer across metropolitan and regional distances. Managing jitter, minimizing propagation delay, and implementing advanced coherent optics are no longer optional best practices—they are the core requirements for keeping expensive GPU clusters from sitting idle waiting for gradient synchronization.
The Edge Counterweight: Why North-South Traffic Won’t Die
While the heavy lifting of model training commands the headlines (and the capital expenditure), inference is where the rubber meets the road for day-to-day enterprise applications. And inference has a gravitational pull of its own. Shipping every user query, IoT telemetry stream, and real-time video analysis back to a centralized AI training hub introduces unacceptable latency and balloons backhaul transit costs.
This economic and physical reality ensures that north-south traffic is far from dead. Enterprises are deploying smaller, specialized inference engines at the network edge—at regional points of presence (PoPs), cell towers, and enterprise premises. These edge nodes take user requests (north-south), process them locally using pre-trained weights, and return the response to the client.
For telecom and network architects, this creates a fascinating balancing act. You cannot simply rip out your edge routing infrastructure and convert every network edge into a dumb pipe. Instead, networks must simultaneously support:
- Massive, deterministic east-west pipes moving petabytes of training data between isolated data center facilities.
- Agile, low-latency north-south paths capable of ingesting high-volume edge queries and distributing localized inference results.
- Dynamic traffic engineering that can dynamically shift resources depending on whether a facility is currently training a model or serving inference requests.
This dual mandate complicates capacity planning. Engineers must provision for peak bursts at the edge while simultaneously underwriting multi-terabit dark fiber and lit services for inter-datacenter AI pipelines.
Engineering the Criss-Cross Network of Tomorrow
As we look at the evolution of carrier and enterprise networks over the coming years, the challenge will be unification. Historically, transport networks and IP routing layers operated in somewhat distinct silos. Today’s AI-driven traffic patterns demand tight integration between optical transport and IP/MPLS or Segment Routing (SRv6) fabrics.
Network automation will play a starring role here. Manual provisioning windows of days or weeks are incompatible with dynamic AI workflows that can spin up massive data-transfer jobs on the fly. Engineers are increasingly turning to SDN controllers that can programmatically spin up high-capacity optical paths on demand, dynamically balancing east-west and north-south resources based on real-time telemetry from the compute clusters.
Security and resilience also take on new dimensions in a criss-cross architecture. When a single east-west transport link carries the equivalent of an entire enterprise’s historical traffic load, a fiber cut doesn’t just degrade performance—it stalls millions of dollars worth of active AI compute processes. Optical protection switching, diverse routing, and automated fast-reroute mechanisms must operate with sub-second precision to protect these massive workloads.
Conclusion
The narrative that AI is purely an east-west phenomenon misses the nuanced reality of how applications are deployed and consumed. While the race to connect massive AI factories is driving unprecedented innovation in DCI and optical transport, the distributed nature of inference guarantees that the edge—and north-south traffic—will remain vital.
For network engineers, this convergence of massive training fabrics and localized inference creates both a challenge and an opportunity. By mastering the art of the criss-cross network—balancing high-throughput internal data center fabrics with agile, low-latency edge connectivity—we can build the resilient infrastructure required to support the next generation of intelligent systems.
For a deeper dive into how major infrastructure providers are navigating this shifting landscape between data centers and edge architectures, check out the original reporting in the RCR Wireless article on Lumen’s AI networking strategy.