In the high-stakes world of Content Delivery Networks (CDNs), the shortest path to a user isn’t a straight line—it’s a calculation. During the AIORI-2 Hackathon, our team—FunnyCast from the Heritage Institute of Technology, Kolkata—investigated a phenomenon called Anycast Flipping. By combining BGP routing logic with real-world latency modeling, we built a simulation to measure how unstable routing “flips” degrade the very performance CDNs are meant to provide.
1. The Problem: When Anycast “Flips”
Anycast allows multiple servers to share the same IP address, routing users to the “closest” node. However, routing instability (BGP churn) can cause a user to suddenly “flip” from one CDN node to another mid-session. This isn’t just a technical hiccup; it adds significant Round-Trip Time (RTT) and delays the First Contentful Paint (FCP)—the moment a user actually sees a website loading.
2. Implementation: From Emulation to Reality
Our project evolved from simple latency shells to a robust, BGP-enabled container environment. We used Docker and FRR (Free Range Routing) to simulate a multi-Point of Presence (POP) architecture.
- Approach 1 (Mahimahi): We used Mahimahi to emulate 30ms vs. 80ms delay shells, proving that a flip can cost a user 50ms or more in pure latency.
- Approach 2 (BGP + Connmark): We implemented actual BGP peering between nodes. To replicate real-world data, we used CONNMARK-based policy routing to probabilistically flip ~3.2% of flows—matching the prevalence observed in global Internet studies.
3. Technical Benchmarks
Using Puppeteer for automated browser testing, we captured the performance cost of these routing shifts across HTTP/1.1 and HTTP/2.
| Metric | Baseline (Stable) | Flipped (Unstable) | Impact |
|---|---|---|---|
| RTT (Latency) | 30 ms | 80 ms | +166% increase |
| BGP Convergence | < 3 seconds | N/A | High session stability |
| Flip Probability | 0% | 3.2% (Modeled) | Matches IETF MAPRG data |
| FCP (User Exp) | Fast | Delayed | Visible rendering lag |
4. Challenges: The SSL & Namespace Trap
During our sprints, we hit a major wall: Nested Mahimahi namespaces.
The isolated shells blocked external connectivity, causing SSL certificate errors when testing against live sites like Google or YouTube.
- The Pivot: We moved to a hybrid approach using connmark-based policy routing within the Docker stack. This allowed us to simulate flips at the packet level while maintaining valid BGP semantics and internet connectivity.
5. Lessons from the Sprints
We learned that Anycast is a balancing act. While it improves resilience, the lack of “state” in BGP means that the network doesn’t know it’s hurting the user’s web experience.
“Building this project made me appreciate the subtle interplay between routing protocols and user-perceived performance. Working with FRR and Docker taught me the value of controlled experimentation in network research.” — Aayushmaan, Team Lead
6. Future Work: Protocol Resilience
Our next goal is to share our dataset with the IETF MAPRG mailing list. We want to test how modern protocols like QUIC (RFC 9000) handle these flips compared to standard TCP, specifically looking for ways to “pin” routes and prevent mid-session degradation.
