Welcome to the first in a new series looking at some of the solutions we have implemented. As time goes by, please select the Case Studies category for other similar posts.
In today’s post, we look at network edges. A network edge is where your network ends and you connect to the Internet. For an ISP, it’s how you connect your customers to the outside world; for a content provider, it’s how your customers access your content and services. If your business provides an Internet based service and you host (or want to host) it from your own equipment, then you need a network edge.
We should clarify that we’re not talking about office networks here which connect to the Internet via DSL or wireless products (for these, consider talking to some of our customers such as Questzones who have a product that provides resilient Internet access over DSL falling back to 3G; or imag!ne who have recently launched a WiMAX product along side their DSL and wireless offerings).
What we are talking about is companies who have higher bandwidth connections than typical DSL (minimum would be 10Mbps full duplex) who need a resilient office connection and/or want to deliver Internet services on their own platforms. Typical customers are those who are making the move to dedicated rack(s) in a data centre or building a dedicated data room who want their internet connectivity to just work all of the time but don’t have or want the in house resources to build and manage it themselves.
A Typical Network Edge
Our typical network edge has two edge routers and two distribution switches. We pick at least two independent wholesale access providers and connect these to either a router each or a distribution switch each for a full mesh BGP configuration (both routers connected to both providers).
These routers present gateway(s) to internal routers and servers using a high availability protocol (such as VRRP or HSRP). As such, if either router fails, external access is maintained via one or both (if full mesh) Internet access providers and all internal servers still have an active gateway.
Connecting to the distribution switches is then done in a fault tolerant manner using link aggregation or STP for example. Other considerations of course include power which should be provided via a UPS system with independent A+B feeds where possible (reducing to a least separate circuits for each router/switch pair).
So, as you can see, at each level of the network edge through to your own network core, there is a fault tolerant configuration in place.
Example #1
The first example of a network edge we built on a budget for a customer is discussed on our main site. In this example, we used two Dell PowerEdge servers as the edge routers running stable versions of Quagga. These connected to manageable (by which of course we mean CLI access) Netgear switches which included essential features such as .1q.
The servers were each connected to both switches using Linux’s bonding capabilities in fail over mode (i.e. only one 1Gb port active at a time). The switches themselves were connected with a 2 x 1Gb port channel.
Three upstream providers provide 1Gbps links (with various CDRs) with full BGP routing tables for both IPv4 and IPv6.
This installation was completed in a Dublin based data centre over eighteen months ago and has never had an outage to date.
Example #2
For another ISP customer of ours who had their own dedicated data centre, we built a network edge using two Cisco 7301s and two Foundry FESX424s (now Brocade).
Internet connectivity was provided via:
- 2 x 1Gbps metro-Ethernet service with full IPv4 BGP tables layered on top by eunetworks who dug in their own fibre;
- 2 x 1Gbps metro-Ethernet fibre links from Broighter Networks to their own PoP in DEG where we established a mini PoP to bring in connections from INEX and other transit providers.
This edge was established in 2005 and, while it has under gone many upgrades and changes, has never had a service affecting network outage due to its resilient design.