International Peering

At the end of every Internet Peering Workshop we end up at the same discussion: How should we think about our international peering strategy? This chapter presents a simple framework that has proven helpful for creating an international peering strategy.

Simplifying the International Peering Strategy

International peering strategies rest upon a complex web of strategic intentions, the foreign business contexts, the market for Internet services, and a variety of underlying assumptions. Network operators expand their network for a variety of reasons, but primarily among these are servicing current and future customer needs. Building into a new Internet Region involves dealing with the local operations environment and understanding and adhering to the regulatory regime. Add in a few more variables associated with peering politics and the international peering discussions quickly become overly cloudy and complex.

To reduce this complexity we have found it helpful to start discussions with a simple international peering model based on explicit assumptions that can be refined or removed in later stage discussions. These discussions often lead to the broad strokes of an international peering strategy.

An International Peering Model

We will build our model by making some simplifying assumptions.

First, we will continue our naming convention and call our network operator “ISP A.” We assume that ISP A is interested in interconnecting in three Internet Regions, with one of these regions being a “gateway” Internet Region where Internet Transit is inexpensive. The other Internet Regions will be called “Internet Region X” and “Internet Region Y.” Let’s assume that ISP A has a single router (shown as ‘A’ in the figure) in each region providing interconnections for that region. These routers are interconnected across transport circuits ‘t’ as shown in Figure 15-1.

Interconnections in Each Internet Region

In each Internet Region we assume that ISP A establishes a Point-of-Presence (POP) to interconnect with others in

1. free and reciprocal peering relationships (shown as ‘P’ in the figure), or in
2. transit relationships (shown as ‘T’ with a ‘$’ highlighting which side gets paid).

Transit customers are numbered and the peers are lettered so we can discuss traffic flows and corresponding revenue flows. We will refer to this diagram throughout the rest of the chapter.

Simplifying Assumptions

In the context of our discussions, we will hold five variables constant.

1. Customers. All customers are assumed to be homogeneous in the sense that they all offer the same traffic load, and they are all entirely content-heavy or access-heavy. Therefore, all traffic travels in the dominant direction (content-to-customer or customer-to-content).
2. Transit Price. ISP A sells Internet Transit for $T/Mbps in each Internet Region. This includes the transport from the customer to ISP A.
3. Global Internet Transit Price. ISP A pays $GIT/Mbps for transit from its upstream provider in the Gateway Internet Region.
4. Transport Price. Transport between Internet Regions costs $t/Mbps.
5. Peering Cost. Free peering is assumed in each Internet Region; the peering port cost is treated as fixed and is not included in the per-Mbps math.

With these assumptions in place, we can examine the traffic flows.

Traffic Flows

There are only three categories of traffic to be considered when looking at ISP A’s interconnect strategy:

1. Customer-to-customer traffic (also called “customer routes”), and
2. Customer-to-peer’s-customer-traffic (also called “peer routes”),
3. Customer-to-the-rest-of-the-Internet-traffic (also called “default route”).

Let’s consider each category of traffic flow.

1) Customer-to-Customer traffic

Traffic between ISP A customers and the other ISP A customers will be exchanged within the Internet Region or between Internet Regions. For example, customer #9 may send content to customer #8, or customer #6 may send traffic to customer #7 as shown in Figure 15-2.

Revenue. For this customer-to-customer traffic, ISP A receives revenue from both customers, or 2 * $T/Mbps. If traffic has to go between Internet Regions, there may be a cost of $t/Mbps. As a result, for this type of traffic we see that ISP A obtains revenue from customer-to-customer traffic flow of

C-C Revenue = 2 * $T/Mbps − $t/Mbps

If both customers are within the same Internet Region or if the network operator owns the physical transport, the value of $t is zero.

2) Customer-to-Peer’s-Customer traffic

The second type of traffic is customer traffic that is destined to or coming from a peer’s customer. In this case, as shown in Figure 15-3, traffic between ISP A’s customers and its peers’ customers will either be satisfied within the country (such as Peer J sending content to customer #7) or satisfied by sending traffic between Internet Regions (such as Peer B sending traffic to customer #6).

Revenue. ISP A collects revenue of $T/Mbps from its customers. Customers #6 and #7 pay ISP A for the traffic that is ultimately going to or coming from ISP A’s peers. If traffic has to go between Internet Regions, ISP A may bear an additional transport cost of $t/Mbps.

As a result, for this type of traffic we see that ISP A obtains revenue from customer-to-peer traffic flow of

C-P Revenue = $T/Mbps − $t/Mbps

If the customer and peer’s customer are within the same Internet Region or if the network operator owns the physical transport, the value of $t is zero.

3) Customer-to-Internet traffic

Traffic that can’t be satisfied between ISP A customers, and can’t be satisfied with ISP A’s peers, is ultimately delivered via ISP A’s upstream transit provider in the Gateway Internet Region as shown in Figure 15-4.

Revenue. For this type of traffic, ISP A receives revenue of $T/Mbps but pays transit fees of $GIT/Mbps. If traffic has to go between Internet Regions, there may be a cost of $t/Mbps.

As a result, for this type of traffic we see that ISP A obtains revenue from customer-to-Internet traffic flow of

C-I Revenue = $T/Mbps − $GIT/Mbps − $t/Mbps

If the customer happens to interconnect with ISP A in the Gateway Region or if the network operator owns the physical transport, the value of $t is zero.

Traffic Flow and Revenue Summary

The key point is, with all three types of traffic, customer traffic generates revenue for ISP A. As long as transit revenue exceeds traffic exchange costs, ISP A will derive incremental revenue from customer traffic as shown in Table 15-1.

Now that we have discussed the three traffic types and corresponding revenues and costs, let’s take a look at a few interconnection strategies.

International Interconnection Strategies

The interconnection strategies usually consist of two parts: an Internet Transit strategy and an Internet Peering strategy.

Internet Transit Strategies

Transit strategies tend to be based on the company’s strategic intent and the market context.

Compete in the Wholesale Transit Market. If ISP A wants to be the leader in the wholesale Internet Transit across Internet Regions, it might deploy a node and a small sales office in each region. In this case, ISP A might not care so much about the Internet Region itself, nor about the Internet Peering Ecosystem dynamics in this Internet Region since it only wants to sell wholesale access for the Internet routes it picks up in the Gateway Region for example.

Lead the Internet Transit Market. On the other hand, if the strategic intent is to “own” or “lead” the market for certain Internet Regions, then they may want to accelerate the growth of this particular Internet Peering Ecosystem and take a more active role in the Internet Region. They might establish a larger local team and focus on sales and growing their own local hybrid peering and transit with an aggressive peering and customer attainment strategy. They may create an informal alliance with the local Internet Exchange Point, try to pull together the Tier 2 ISP population for peering, and perhaps establish a strategic alliance with an ISP that owns some local fiber.

Perform Transit Arbitrage. Another common strategy is to rapidly build a customer base in several Internet Regions by arbitraging the transit markets between the inexpensive gateway country and the expensive home market. For example, ISP A will drive the price for Internet Transit down within each region until the price of transit approaches the price of transit in the Gateway Internet Region.

The degree of competitive aggressiveness is a strategic decision.

Next we turn our attention to Internet Peering in emerging Internet Peering Ecosystems, and how ISP A determines its peering inclination, its articulated peering policy, and how ISP A may evolve as these Internet Peering Ecosystems evolve.

Peering in Emerging Internet Peering Ecosystems

To discuss peering strategies, we first must understand the context of the Internet Region. For example, we need to understand:

1. How well developed is the Internet Peering Ecosystem?
2. Who are the Tier 1 ISPs in the Internet Peering Ecosystem?
3. Who are the Tier 2 ISPs and are they peering?
4. Where are the Internet Exchange Points (IXPs), who runs them, who is there, and what are the costs for participating (local loop, colocation, peering ports, membership fees, etc.)
5. What is the market price for Internet Transit?
6. What is the cost for transport capacity into the exchange?
7. What are the dominant traffic patterns?
8. How much traffic stays within the Internet Region? What are the popular destinations? Who are the leading access providers in the region?

In the Peering Workshops we go through this data collection in each region under consideration. We then create the business case for peering in each region.

While this in depth analysis of the Internet Region is important, it is beyond the scope of this book. To give you a flavor of the discussion however, we will walk through a sample peering strategy case study based the model and discussion so far.

Case Study — African ISP A Expands into London

Strategic Intent. Let’s assume that ISP A is an African ISP that is interested in expanding its network into nearby countries and more importantly, into London to purchase Internet Transit there for $2/Mbps instead of the $300/Mbps that is available in their home market. As for strategic intent, ISP A wants to provide better Internet Transit service for the home market while also being the price leader.

Let’s enumerate some relevant context generalized for the continent of Africa.

1. 98% of all African traffic is coming from the US and Europe. A trivially small percentage of African traffic is locally produced and consumed content.
2. With such a small percentage of traffic satisfied locally, it is not surprising that there is not much peering traffic exchanged within the Internet Regions across Africa.

Let’s enumerate some additional assumptions for the case study.

1. The Internet Gateway location of choice is London
2. The cost of transport up to London is $275/Mbps.
3. Relevant content providers are in London and willing to peer openly.

ISP A’s Peering Policy

Since 98% of all African traffic comes from the U.S. and Europe, ISP A should peer openly in London. Given the original assumptions that the peers are all in the ISP A colocation center in London, that Traffic will either come from the Global Internet Transit interconnect or via the free peer. Free peering is better than paid transit for this readily available traffic.

One could refine the analysis by performing a Peering vs. Transit calculation or limit peering to those that meet certain criteria, or not bother with peering at all given the low price of transit. However, by peering in the same building, ISP A will generally improve performance while simultaneously reducing costs. Peering in London will also provide a good sales and marketing tool for ISP A since it can then promote itself as having direct peering with the most popular content sites in the world up in London. Peering openly in London is aligned with the explicit strategic intent.

ISP A should therefore identify (or hire) a peering coordinator to build these peering relationships. This will require some travel budget, staff time, and a learning curve, but ultimately peering in Europe will be great preparation for peering in the African Internet Regions.

Assertion: ISP A should take a community leadership role in each Internet Region

Many Internet Regions across Africa are just now starting to grow. There is little value derived from peering in these regions today however, so an argument could be made to not bother with peering there. But these markets will not always be small. As the Internet Peering Ecosystem grows, ISP A’s market for international transit grows. It is therefore in ISP A’s best interest to promote and help grow these ecosystems. Over time,

1. Broadband access (perhaps mobile) will become more widely and inexpensively available, and
2. Lower transit prices will lead to more local Internet attachments, and
3. More Tier 2 ISPs will therefore launch services for their local markets, and
4. The local Internet Peering Ecosystem will grow enough that local eyeballs will seek local content, and
5. The business case for peering will make sense.

If ISP A takes the leadership role now, promotes local peering and the local exchange points, it will not only help establish goodwill, but it will also grow a broader base of customer prospects that may ultimately prefer to purchase transit from a richly peered ISP A.

The point is that this community leadership is a very powerful tool, and helped position South Africa as a Internet Regional focal point. ISP A should do the same for each Internet Region of interest.

Assertion: ISP A should adopt an Open Peering in each Internet Region of interest across Africa.

At this point during the peering workshops people usually ask, “Why will customer prospects buy from ISP A when they could just peer with ISP A for free?”

The answer is that Internet Transit and Internet Peering are not perfect substitutes.

ISP A’s peers only get access to ISP A’s downstream Internet Transit customers, which, as we have already established, is a very small percentage of the ecosystems Internet traffic (less than 5% of African traffic is local sourced and sunk). So initially there is very little value to peering with ISP A. This peering discussion however can lead to this next sales discussion with prospects:

Internet Transit on the other hand provides access to all routes:

1. Routes learned from other ISP A customers, and
2. Routes from ISP A’s peers’ customers, and
3. Routes from ISP A’s upstream Internet Transit provider in London.

Ultimately, the transit service provided in each Internet Region should evaluate and compare favorably to the value of peering.

The Value of an Internet Transit Network

When one of the local ISPs purchases Internet Transit from ISP A today, it is primarily buying access to the global Internet delivered out of London. Most of the value derived by purchasing Internet Transit from ISP A today is the quality of its network capacity up to London and the quality of the global routes it picks up there. The primary question customers should ask today is “How well does ISP A deliver traffic to me from the U.S. and Europe?”

Today, the secondary question is, “How well does ISP A meet my long-term local traffic needs?” Let’s discuss the long-term effect of this open peering policy in the emerging local markets has on the value of the transit network (shown in Figure 15-6).

Adding Peers makes ISP A a more Attractive Transit Provider

The ISP A network is a more attractive network to its transit customers when it peers its traffic locally. Why? Peering provides ISP A’s customers with direct access to ISP A’s peers’ customers. This means that customers get better performance when their traffic doesn’t have to go all the way up to London to be served! As the ecosystem develops, more local ISPs and more local content comes on-line, so the value of ISP A’s Internet Transit network increases since it peers this traffic better locally as shown in Figure 15-6. From a marketing perspective, the performance benefits become an important differentiator, and from a public relations perspective, peering positions ISP A as promoting nationalistic interests. This will lead to more customers that want not only access to the global Internet but also want great performance to this emerging local content.

More Customers make ISP A a more Attractive Transit Provider

The ISP A network is a more attractive network to its transit customers as it obtains more customers. Why? First, the performance is even better when all customer traffic can be satisfied entirely on the same network. Second, these routes may be unique; if the customers are singly homed to ISP A, the next best alternative to reach these customers might require sending traffic through London. Therefore, a strong argument can be made that purchasing transit from ISP A will provide the best performing path to these local destinations.

A Strong Local Network of Customers and Peers will Fend Off Foreign Competitors

Over time, and as the Internet Peering Ecosystems across Africa develop, ISP A will be able to peer more of its traffic locally and therefore be able to demonstrate that much of this local traffic can be satisfied without leaving the country. These peering sessions will help it differentiate itself and help it compete aggressively against the foreign transit providers that build into this now more established Internet Peering Ecosystem. These foreign Internet Transit providers can initially only offer global routes from London, not the more desirable local routes that ISP A has cultivated in this emerging ecosystem. The competitor’s traffic will have to traverse London to get to ISP A’s customer base, and it may have to traverse London to get to ISP A’s peers’ customers as well. ISP A is in a more defensible market position from this strong local peering and customer acquisition strategy.

Over time, by driving down the price of Internet Transit, and perhaps by migrating to a selective peering policy, foreign competitors will see these markets as less lucrative arbitrage opportunities.

Open Peers Emerge as Tier 1 ISPs in Emerging Markets

At some point, a “Tier 1 ISP” club will probably emerge in these ecosystems similar to what we see in the rest of the world. These “Tier 1 ISPs” will peer only with each other and be able to reach all in-country routes through these free and reciprocal peering arrangements. They will adopt a restrictive peering policy in the country and de-peer all of those networks that are customers of the other Tier 1 ISPs. They will deny requests for peering with anybody else so competitors may have to send traffic through London to reach their customers. Every other Internet Region has gone through this transition and ISP A may, with this strategy, be well positioned to emerge as a Tier 1 ISP in various regions. The uniqueness of the routes and the greater volume of traffic may even allow ISP A to negotiate peering from a stronger power position in London.

International Peering for Content Providers

We have been talking about ISP A and its interconnection strategy as a provider of Internet Transit, but we have not yet discussed how the model applies to content providers. Every developing Internet Region sees cost and performance issues when content is not available locally. As a result, the large-scale network savvy content providers are constantly invited to extend their presences into other Internet Regions.

Content providers generally consider three things when evaluating these requests:

1. “How do they currently reach customers in this region, and what is the current price and performance for this traffic?” and,
2. “How might they better reach customers in this region and what would be the price and performance improvement?” and finally,
3. “What are the implications of expanding into the Internet Region?”

For content providers and content-heavy ISPs and CDNs, it is all about the end-user experience. They might even pay more for a substantial improvement in performance for an important user base. For example, companies like LinkedIn have a substantial, important, and growing user base in India and parts of South America. They might consider expanding into these markets directly rather than delivering their content from Japan or from the U.S.

So when Content Providers apply the international peering model, they don’t look at the ability to sell transit. They look at the current latency, packet loss, perhaps jitter and other metrics between customers and their servers via a transit provider from an adjacent but centrally located Internet Region. The next best alternative might be to expand into a new Internet Region and serve up the traffic locally. If they build in, how much of their traffic can be peered away for free, how much might Internet Transit cost there, and how much will it cost to build in? They will apply the business case for peering but instead of looking at the peering break-even point, these network savvy content providers primarily consider the number of milliseconds from the round trip time to display their web pages.

Generally content providers and content-heavy ISPs and CDNs will adopt an open peering policy globally. In some cases, the content provider or CDN may even require their hosting company in the region to pay for their bandwidth and hosting expenses.

Content Providers and CDNs may require that the hosting company’s competitors be allowed to interconnect with the content company or CDN directly, something the hosting company may be disinclined to allow otherwise. No “walled gardens are allowed” has been Google’s approach as they deploy caches into emerging Internet Regions to help a local Internet Peering Ecosystem grow.

Another set of issues that the content providers look at are around the business context in the foreign Internet Region. If they build into Europe for example, they may have to honor the strict European privacy standards. If they build into China they may need to honor the governmental censorship orders and deal with a strong-armed and invasive government controlled Internet Region. Some countries have large import duties and some require majority local ownership before a company is granted a license to offer telecommunications services there.

These are the sorts of discussions that come up during the peering workshops, and the framework serves as the skeleton for dissecting and discussing some of these types of topics.

Summary

This chapter provided a simple model for exploring international peering strategies that have come up in almost every Internet Peering Workshop. The group discussions surrounding the model exercise the definitions and helps us sketch out broad stroke strategies about where we want the “ISP A” in question to be in 5-10 years as the ecosystems develop. The simplifying assumptions have proven necessary to avoid getting caught up in the complexity and unique characteristics of each market.

This resulting model is a good first pass for exploring an international peering deployment. Refinements are often made to consider different price points, different peering policies, and different market penetration strategies for each region.

In this analysis we used for an example deploying across Internet Regions in Africa but the same logic applies to many international deployments. In the next few years much of the African Internet will emerge as a set of first-class independent Internet Regions. It may take some time, and there is risk here, but in a world where so much of the ecosystem is hardened, Africa looks to be a fertile ground for the next generation Internet consisting of a billion users consuming new local Internet content.