Network Architecture Overview
We have chosen the equipment and design of Xandr's network architecture specifically to support production-level applications.
The core of each datacenter is a pair of Juniper EX9200 L3 switches. In front of the switches, a pair of Juniper MX480/960 routers manage all traffic into and out of the datacenter, connecting to redundant IP transit providers (currently Cogent, Level 3, NTT and Hurricane Electric in NYM2). A pair of BIP-IP F5 5200s in each datacenter provide line-speed load balancing and support global load balancing (GSLB) between datacenters.
We have remote and out-of-band access to all network equipment using Opengear IM4200 remote console servers.
Each rack has:
- Two redundant Juniper EX4200 switches, with redundant connections back to the EX9200's and to each server in the rack
- Two redundant 10 Gigabit Ethernet uplinks to the EX9200's
- Redundant power from the datacenter
- An out-of-band switch, which connects to the DRAC card in each server for console and troubleshooting in case of hardware failure
- Switchable PDUs to provide power on/off capabilities when needed
One cabinet in each datacenter is dedicated to Nexenta clustered storage. Each Nexenta node is connected directly to the EX9200 switches for maximum performance.
Each component in Xandr's system is designed for high availability:
- Two - four different IP transit providers in each datacenter
- Pair of Juniper EX9200 L3 Switches in each datacenter
- Pair of Juniper MX480/960 routers in each datacenter
- Pair of F5s in each datacenter
- Pair of Juniper EX4200 switches in each rack
Every customer is allocated a VLAN upon account creation. The size of the VLAN depends on customer need, ranging from 8 to thousands of IPs. The first 8 addresses of the IP range are reserved so that Xandr's networking gear (switches and load balancers) has access to the VLAN. All remaining addresses are reserved for customer virtual machines. Each instance, when started, receives a public IP. A customer may choose to allocate multiple IP addresses to a single instance or even float a single address across multiple instances.
Public addresses are being used for a few reasons, the main ones being:
- NAT provides a false sense of security.
- The use of public IPs provides maximum flexibility, manageability, and scalability. Having a single IP address to reference for each instance greatly reduces complexity and increases flexibility.
- The load balancers are directly connected to each customer VLAN and as such, require an IP from the customer's pool.
Here is a list of the IP Addresses Xandr uses for customers and internal use.