Chaos Computer Club - recent events feed
Ethernet Won Inside the AI Data Center. Can It Cross an Operator Boundary? (denog18)
· 29 min
Jakson kuvaus
Inference is quietly overtaking training as the dominant AI workload, and unlike training — concentrated in a handful of hyperscale halls — inference wants to sit close to users, pulling AI capacity into distributed sites across Europe.
The industry has spent the last two years re-engineering Ethernet (via the Ultra Ethernet Consortium and platforms like NVIDIA Spectrum-X) to match InfiniBand's performance for exactly this kind of workload, and now even ships Ethernet fabrics purpose-built to stitch multiple physical data centers into one logical AI factory.
That's a solved problem when one company owns every site. This talk asks what happens when it isn't — when distributed inference has to span operators, jurisdictions, and ownership boundaries — and what a federated, policy-aware Ethernet fabric for that world would need to look like.
Ethernet is rapidly becoming the preferred back-end network for AI infrastructure. Technologies such as RoCE, Ultra Ethernet, NVIDIA Spectrum-X, and emerging “scale-across” architectures promise predictable, high-performance connectivity within and between AI data centers.
These architectures generally make one important assumption: a single organization controls the servers, switches, telemetry, congestion policies, and inter-site connectivity. Under that model, several physical data centers can be engineered to behave as one logical AI environment.
Distributed European AI infrastructure may not have that luxury. Inference capacity will increasingly be deployed across hyperscalers, regional clouds, colocation facilities, metro-edge sites, and specialist GPU providers. The networks connecting those sites may also belong to different fiber and connectivity operators. Each participant controls only part of the infrastructure, exposes limited operational information, and may operate under different commercial and jurisdictional requirements.
This creates a question that faster Ethernet alone cannot answer: what happens when an AI fabric reaches an operator boundary?
The talk begins by separating two ideas that are frequently conflated. Distributed AI services can steer requests between independent sites without extending a tightly coupled GPU fabric between them. Some workloads, however, including model distribution, distributed inference, resilience between GPU sites, and certain forms of model parallelism, place much stronger demands on inter-site connectivity. We will examine which workloads genuinely benefit from network-level federation and which should remain distributed at the application layer.
We will then look at what changes when traffic crosses a network-to-network interface. Can congestion-control assumptions survive between independently operated domains? What latency, loss, and bandwidth guarantees are technically credible? How can operators exchange useful telemetry without exposing internal topology? Who selects the end-to-end path, who responds to failures, and how can customers verify that…
Sivusto: media.ccc.de
