According to the RIPE RIS service, the links between MWEB (AS10474) to Telkom South Africa (AS5713) were disconnected on the November 2nd – Telkom being the original transit provider that MWEB used.

MWEB have detected that congestion reduces, therefore service levels increase when traffic bypasses the incumbent and is delivered directly to other ISPs in their region via peering links.  If a network refuses to peer, MWEB simply deliver the traffic to local providers via their international links – possibly just as congested, but available at a fraction of a cost.  If traffic is then delivered to the incumbents via links they themselves pay for, the incumbents also have a financial incentive to peer.

Peering is the best way to encourage enormous capacities between ISPs and other networks, because a direct one-to-one connection can be monitored and well managed in order to guarantee availability for internet traffic.  Peering therefore increases available bandwidth and reduces bandwidth costs.  This will enable high the sort of services that require high-bandwidth availability, like streaming media and high definition video conferencing.

Interestingly, thirty minutes after the adjacency with Telkom was severed, it appeared that MWEB picked up a new transit customer – Yebo, AS12258, with Yebo’s prefixes being advertised to Interoute (again, according to RIPE RIS).  The commercial nature of this downstream relationship is, however, not revealed by the routing table.

The incumbent is perfectly entitled to – and well placed to – sell excellent transit links into the local market, but their strategy to do that, as I explained in my last article, must be to make the transit product in their key regions excellent – this means to peer with the key other local providers (not all providers) in the market, and to ensure that capacities across their backbone and to customers are well managed and available for traffic.

It demonstrates clearly that deregulated markets offer enormous advantages over controlled ones, and should serve well as a reminder to operators and policy makers that simply getting out of the way could be the best way to further their aims for industry in any given region.  This is mainly because:

• Allowing networks to interconnect freely (calculated as number of active ASNs in a region), and the size of the market (calculated from the pool of announced ip addresses in a region) are strongly correlated (slide 8).  My guess is that more organisations get online, because competition leads to price falling, whilst the versatility and relevance of services offered increases.
• When there are a larger number of networks in a region, the global carriers have a greater incentive (more customers!) to run diverse connectivity into the region.  This leads to a huge advantage to firms in a region, their connectivity carries on despite local major fibre breaks. (slide 25, 36)
• Content moves out of the US/Western Europe and into the local market place, creating opportunity (and jobs) for local players, and improving the performance of services for local users.

Incumbent networks in this region have a huge opportunity to grow revenues, as the market expands, as long as they are willing to interconnect widely in this region.  As the number of providers in a region expands, customers will be able to (and, according to this research, actually do) pick between innovative and disruptive new providers with excellent regional (via peering), and international (via transit) capacities.  Peering also makes capacity cheap, because traffic can stay local to the ISP.  An incumbent provider that refuses to peer in order to retain market share will not be able to compete in quality terms with the new providers.  Defending a 100% market share is impossible in a competitive market, so the strategy must change, the aim must become enjoying the fruits of a booming market instead of monopoly.  As the Renesys slides say, there is no dominent IXP in this region yet, with many networks dragging traffic to London, Amsterdam or Frankfurt to exchange, but this will change as the density of providers in the Middle East reaches a critical mass.

Setting up an Internet Exchange point is simple from a technology point of view, but requires significant planning, and community support for the plans.  Read the slides to find out more about what must be planned.

Download:  [Slides + Notes (recommended)] ~ [Slides alone]

View directly from Slideshare (requires flash):

Solving this bandwidth starvation is the role of fibre optics and next-generation broadband.  A relatively simple way to roll out fibre backed technology is to use VDSL – service providers run high capacity fibre optic networks to distribution boxes in streets (FTTC – Fibre to the Cabinet), and utilise the existing copper infrastructure between street and house or business carries high speed internet.  The shortness of the copper run enables higher speeds. This is increasingly available from companies such as Digital Region Broadband, who offer 40Mbit broadband at consumer prices, but is obviously only available in specific neighbourhoods where the streetboxes have been rolled out.

Removing the copper element will enable much higher speeds and new products like premise-to-premise connectivity.  FTTP – Fibre to the Premises opens up a world where connectivity between service provider and your office can run at 100Mbit or Gigabit speeds.  Office-to-home or Office-to-office connectivity that runs at Gigabit or even 10Gigabit would be possible too. This would make remote-working via high definition video conferencing, ultra high speed access to company resources and files, and also better quality and more interactive entertainment services a normal thing for everyone.

However, a national – even urban wide – fibre rollout project is expensive because of the construction (civils) costs, legal costs, and impact on neighbourhoods.

Earlier this month, Ofcom released a statement on wholesale access products, explaining that they were planning to require BT to make access to their existing ducts, intending to make fibre rollout cheaper.  The two key mechanisms are:

• Virtual Unbundled Local Access – BT offer other service providers access to existing fibre.
• Physical Infrastructure Access – BT offer service providers space in their fibre ducts, allowing service providers to run their own fibre.

I welcome this development, but hope that the regulation framework mandated by Ofcom does not remove the incentive BT to roll out new ducts and fibres.  The regulation will be a success if it enables more regional FTTC broadband schemes like the one cited in South Yorkshire, and also if it makes new FTTP the ‘norm’ for all new housing developments and telecoms upgrades.  Further, another huge disincentive from rolling out fibre based services — the UK fibre tax — must also be repealed in order to achieve the typical 30Mbit/sec broadband that the EU wish to see for all citizens by 2020.

When firms avoid fibre, it hurts us all.  When fibre is cheap, firms can use it to roll out super-fast broadband to their users, using the sort of technology that facilitates connections tens or hundreds of times faster than a typical UK home enjoys.  It also allows service providers to increase the capacity of their network edge, and to improve the robustness of their network – for instance by building more links between their network points.  Improved robustness also means better business continuity planning options, improving the availability of their services.  This tax kills faster access, and better services.

The fibre tax also worsens the conditions for international networks looking to build into the UK, for instance in order to bring their content and services to the UK market .  This is not a hypothetical risk, it is a game-changer that has destroyed the business cases of several projects that we have been contributed to last year.

This morning, George Osborne was on BBC TV explaining that he saw an advancement in next-gen broadband (based on fibre optic cabling) as a priority. If this is the case, then he must commit to repealing the 21st Century Window Tax. To date, they have only considered repealing the 50p per month tax on telephone lines that has been suggested by the hugely flawed Digital Britain study.

We are just not competitive with this tax.

On Nanog, Owen DeLong and Larry Sheldon were discussing the relative size of the IPv6 address space:

>> 64 bits is enough networks that if each network was an almond M&M,
>> you would be able to fill all of the great lakes with M&Ms before you
>> ran out of /64s.
> Did somebody once say something like that about Class C addresses?

Well, this seemed like a challenge for Maths, and the answer is:

No. There are only 2,097,152 Class C networks.

Assuming all M&Ms are spheroids of uniform oblate nature, radius major axis=6mm, minor axis=3mm. Volume is (4/3)Pi (Major²) Minor

They will be poured into a great lake of your choice, and we will assume random close packing (agitation mechanisms are probably best discussed off-list) and a (generous, but this Wikipedia article insists) void fraction of 32%.

Volume of m&m = 0.452cm³, occupies 0.665cm³.

Lake Erie is 484km³ – See: http://www.epa.gov/glnpo/factsheet.html

1 km³ = 1,000,000,000,000,000 cm³

484,000,000,000,000,000 * 0.665 = 321,860,000,000,000,000 m&ms needed to
fill this lake.

There are 4,294,967,296 /64s in my own /32 allocation. If we only ever use 2000::/3 on the internet, I make that 2,305,843,009,213,693,952 /64s. This is enough to fill over seven Lake Eries. The total amount
of ipv6 address space is exponentially larger still – I have just looked at 2000::/3 in these maths.

THE IPv6 ADDRESS SPACE IS VERY, VERY, VERY BIG.

Can we please now just go ahead and roll out some ipv6 services?

Since January 1st, a few key milestones have passed, indicating how urgent the IPv4 rundown problem has become. Firms that rely on internet connectivity must take urgent action in light of the events:

• The allocation last week of two further /8s (blocks of IPv4 addresses with the same number before the first dot) to APNIC mean that for the first time, less than just ten percent of the IPv4 unallocated pool is available to be assigned.  At current utilisation rates, this pool will be exhausted in only 600 days.  Of course, the internet could stop growing, but all signs point away from this…
• The allocation of 1.0.0.0/8 is the assignment of the first really ‘dirty’ block of addresses, signalling that we really are in the run-down period.  Bad network design decisions in the past have meant that networks have ‘borrowed’ the use of addresses starting 1. for ‘internal use only’ or special applications on their network.  This means that organisations assigned address space starting ’1′ may well have partial connectivity even though they are rightfully assigned the space.  Examples are the braindead hotspot operators who take addresses like 1.1.1.1 to trigger hotspot logout, but a handful of examples appear across this address range.
• RIPE NCC, the organisation who assign addresses to networks in and around Europe have this month implemented their ‘run down’ policy which will mean that organisations requesting space will only be able to cater for their growth requirements for a very short amount of time.  This is to evenly spread the inevitable misery across the ISP community.

RIPE members should thoroughly audit their address space so that they can ensure that their records are accurate, because RIPE are more likely to ensure that address space is assigned to your end users in line with the community’s policies.  ISPs and services providers who need help can contact me for further information or specific assistance.

Organisations who rely on internet connectivity for their products should ensure their providers have an IPv6 migration plan in place.  Otherwise end-to-end connectivity for your home or office is unlikely to be something you can enjoy looking beyond the runout period.  Companies hosting network services, for example a website, should enquire what their host’s IPv6 plans are, and start to enable their services via v6.

There is real traction to ensure v6 support appears in both the hardware and services you need to connect to the internet.  It is easier today than before to find help making your services available via v6.  The alternatives – patchy connectivity via nested stacks of ipv4 islands, or no more end-to-end connectivity (so that your internet service is a walled garden), have much worse consequencies than learning to roll v6.

Engineers know the facts by now and have no excuse.  For more information, see the RIPE NCC’s information site, ipv6actnow.

The slides show that the new route-servers perform splendidly well compared with traditional Quagga based MLPs, also that route-servers are now free of ‘first generation code’ bugs, and also that they handle your prefixes transparently – as you would expect.

Interestingly, BIRD and OpenBGPd behave identically ‘on the wire’ so IXPs are encouraged to use multi-vendor MLP on their platform for increased reliability and stability.  The new breed of route-server code is dependable and tested, so networks that would like to connect should draw confidence from this testing, and IXPs wishing to roll out MLP services should feel confident in the software tested.

Happy peering!

Would domain name users who are concerned about the accuracy of data served pay extra for the ability to sign their DNS zone ?  A handful of people in the room raised their hand in agreement, but the overwhelming majority of operators did not.

His argument was that this compared well with SSL certification authorities who sell certificates that suggest that visitors to a website are interacting with a validated entity, and the technology guarantees privacy between the visitor and the website.  It’s this technology which makes buying and selling online safe.

However, I think that DNSSEC has different aims altogether – simply to guarantee that DNS data is not changed en-route between the authoratative server, through the caches, all the way to users.  Therefore there are significant attack mitigation reasons to deploy DNSSEC, so I hope that operators will begin trials (we are doing so), and that the pace of trials will quicken as the root zone will be signed this year.

If DNSSEC is deployed as designed, then temporary and brief mistakes will not be imported into DNS caches, users will not fall foul to tampered data in caches, and we all receive an authenticated/secure channel for distributing DNS data inside an organisation.

The argument that Dr. Schweiger used is that DNSSEC adds an operational and technical burden to registries (extra communication with registrars, more complex software, additional CPU and bandwidth requirements).

I hope that my colleagues in other organisations agree that there are significant infrastructure advantages to freely allowing DNSSEC to grow, and that Moore’s Law, automation, and the fact that DNS registries normally find it simple to peer widely with ISP networks will offset the needs to consider the commercial signing model.

trials:/usr/src/openssh-5.1p1# ssh hextreeme -l netadmin

Keyboard-interactive authentication

Enter password for netadmin:

channel 0: open failed: resource shortage: Channel open failed

The bug is described in Debian bug 495917, and it also prevents connection to some NetScreen firewalls.  I have this bug with Debian 4 (openssh-client 4.3p2-9etch3) and Debian 5 (openssh-client 1:5.1p1-5).

If anyone else is experiencing the same bug and needs a quick fix, then you can download my Debian packages which replace openssh-client.  You of course need to hold the packages if you don’t want them overwriting by a security fix.

• Debian 4 openssh-client_4.3p2-9etch3_i386.deb
• Debian 5 openssh-client_5.1p1-5_i386.deb

By using this software you agree to hassle both the debian-ssh team and extreme to sort their stuff out!