my web 0.2 website

If VoIP kills phreaking, who are tomorrow’s engineers?

“Ma Bell is a system I want to explore. It’s a beautiful system, you know, but Ma Bell screwed up. It’s terrible because Ma Bell is such a beautiful system, but she screwed up. I learned how she screwed up from a couple of blind kids who wanted me to build a device. A certain device. They said it could make free calls.”

That’s a paragraph from an article linked to from Steve Wozniak’s website, which Steve describes as “The Article that changed history“. He is one of the most important engineers of our time, and like thousands more, he was driven to learn more and more about how computer systems interact, after snooping around telephone networks. The telephone system has always been a prime target for attack for two reasons – vulnerabilities have historically been well published, and telephony was so expensive that it was worth working out the ways to subvert the system and talk for free.

But what happens when talking across the world is so cheap that its not worth stealing any more? You may think this is an irrelevant point, calls from BT users to France are still 18.5p per minute, to New Zealand are still 31p per minute. But what if these calls to France were a penny a minute? Calls to New Zealand 1.4p a minute?

Well, they are now that price if you are a Localphone user. Does this mean no more Steve Wozniaks, young men driven to explore big networks so that they can use their skills to build something even bigger and better?

The first ‘Phreaks’ – the collective name for people who like to exploit vulnerabilities in the phone system found their skills by accident. A blind eight year old called Josef Carl Engressia discovered that he could stop a phone accounting for a call he was making by whistling a particular note in a long distance call. He’d accidently discovered the 2600Hz tone which signals to long-distance telephony kit that a user had hung the phone up.

The later Phreaks like Steve Wozniak were more methodical, they took this learning and approached the exercise as engineers – phreaking was a learning experience – as Steve puts it, “The blue box year was 1972. Apple started in 1975. The biggest connection was some design tricks and techniques that I honed on the blue box.” Fooling around with the telephone drove innovation and learning for the early Apples.

The telephone system acted as a central point of interest for those interested in information security, and gave the movement focus. Whilst the 2600Hz trick no longer works, the number features in the name of the world’s most popular security journal, 2600 The Hacker Quarterly, which specialises in distributing information to IT personnel about improving their systems by demonstrating weaknesses in flawed systems. Again, without Phreakers would such openness and publicity for information security exist?
I admit that phreaks are not only motivated by the prospect of free telecoms, they are fascinated with the huge telephone network. I only ask if calls were as cheap as they are through services like Localphone, would so many engineers have found value exploring telephone systems, learning techniques to use in their later masterpieces.

I hope that tomorrow’s engineers will still explore telecoms. In fact, its easier today than before – downloading a free PBX like Asterisk means you nolonger need to be a criminal in order to explore how a telephone network interacts. VoIP networks have existed as islands within a corporation, or groups of interested people (e.g. the closed FWD system permitted free calls between friends on their network, no matter where they were in the world, but did not allow calls to route to other telephone networks, such as the one your mobile is connected to). Cheaper telecoms was our drive to build Localphone, so it can still act as a motivator for engineers to create something, its just that today you can have more fun doing this legally!

Disclaimer: the author is an engineer at Localphone.

Making the right ipv6 noises

I’ve been allowing the webcast of RIPE55 to mutter away in my ears all week and have let myself get distracted from time to time when the topics turned relevant to networks I operate or the chatter got interesting.  A bit like the end of today’s ipv6-wg session.
Six months ago I was quite sure that v6 was not likely to be viable as a way to guarantee the continuity of fresh (and useful) addressing resources when the IANA ipv4 pool is expired (around two years from today).  I thought that our best chances remained with working with our remaining v4 options; reclassifying 240/4 as normal unicast, pressure or perhaps even (shudder) regulation imposed on the holders of unused legacy class-A space, tighter policy control on remaining v4, and eventually a market model for new address distribution.

This is operator speak for burying one’s head in the sand.  240/4 is not going to useful for end-to-end internet use any time soon, thanks to the amount of equipment which is configured to not permit assignment of a class-e address to an interface, which unless you are using a very old or patched operating system will include the computer you are sitting in front of.  The best we can hope for is that it will be useful for private internetworking at some point in the future.  The class-A holders wont give their legacy allocation back because they think its worth lots of money, but the holders (and their future customers who ‘buy’ rights to a slice of addresses) are going to be disappointed when their announcements are ignored – anyone who thinks that HP can deaggregate 15.0.0.0/8 into 65,000 /24 networks and sell it off (HP not subject to the cost of every single other router in the world carrying the routes for free) is mistaken.

I have changed my mind and started testing v6 out.  I care about the end to end internet, and I care that addressing resources should be available to anyone who can justify need and that this is good for innovation. I am therefore pleased to hear the ipv6-wg discussing text for a new RIPE recommendation document that boils down to:

• New v4 addresses wont be available in two to four years, and this means your future network plans STOP HERE if they only involve v4.
• RIPE urges the deployment of v6

I am however concerned about one thing.  There are a number of networks which use provider independent addresses so that they can multihome (connect to more than one ISP for redundancy and performance.)  This is not catered for in general circumstances with RIPE v6 policies, except in very special circumstances.  This is due to a desire to keep the v6 routing table small so that we avoid the problems of an unwieldy routing table like today’s (by today’s equipment standards) in the future.

I don’t think the RIPE community can justify publishing such a document until 2006-01 reaches concensus, and in fact that this resolution is that IPv6 PI is available, it is a /48, and it’s possible to get some if you are multihoming.  You can’t tell operators that they must move to v6 without giving them a migration path.

The argument revolves around routing table size and control, but the community must allow networks that rely on connectivity to multihome, it’s an accepted technique for many operations now.  By all means require the organisation to continuously multihome or give the addresses back, but we must let the concept of v6 PI exist.

Otherwise needs based address resource allocation dies in around two years.

Making Round Robin DNS usable.

I have been fairly consistently telling people a lie for the last ten years – and that is that Round Robin DNS can not be used for high availability. Its a view I have held pretty strongly, but two people have shown me techniques in the last week that have made me change my mind.

First, a note on what Round Robin DNS is. Hopefully you know what dns is! When a resource record is answered in a round robin fashion, a single resource record has several answers. When high-availability is build into a protocol (e.g. DNS itself – retries) RRDNS works pretty well. dig . @a.root-servers.net if you want a reference service which uses this principal.

RRDNS is also used to share load between nodes in a cluster where the protocol is not as forgiving, with unpredictable results. If I serve my website via two nodes, so the dns looks like:

www IN A 10.1.1.2
www IN A 10.1.1.3

then my website is not at all highly available. Pretty far from it, in fact – if either the nodes 10.1.1.2 or 10.1.1.3 fail, then I have suboptimal results, when in a highly-available configuration, I only want to suffer a performance degredation or outage if all of my nodes fail.Two colleagues showed me two different systems earlier this week that solve this fundamental issue with traditional RRDNS implementations – and that is handling an outage. Both of them use the premise that you never use the box’s own, unique (e.g. management) IP address in the RRDNS Resource Record, instead you use ‘virtual’ or aliases IP addresses that are free to roam around your server cluster. Consider my zone with this change:

www1 IN A 10.1.1.2
www2 IN A 10.1.1.3
www IN A 10.1.1.201
www IN A 10.1.1.202

Now I can address both servers in this small cluster with their own address, and also point customers to an address which can be served by any of those machines. If both web servers are up, they each have one of the additional addresses aliased. If 10.1.1.2 goes down, the computer 10.1.1.3 can adopt both 10.1.1.201 and 10.1.1.202 as aliased interfaces.

I can’t take the credit for finding either of the technologies I mention below, that can manage the automated failover of aliased interfaces, but I have seen both work this week.

Firstly, Wackamole which is brilliant if you are already using ‘Spread’. Nodes on a Spread ring send health messages to each other. If a node goes silent, or sends a communication that it is failing (deliberately stopping), another server in the cluster adopts that server’s aliased address. You can also configure a mechanism that notifies other devices that their arp cache entry for the vip is staled.

The second is good-old VRRP, which I had only ever considered as a redundant first-hop protocol, but actually can be run on a Linux server in order to provide these Virtual-interface floating features. This system does not rely on the Spread messaging bus, but in order to float several VIPs in a cluster, one vrrpd instance, and therefore group, must be configured for each vip. For my simple cluster of two servers above, I would need to run vrrpd twice on each server, to balance both vips as such :

/usr/local/sbin/vrrpd -Ni eth0 -v 48 -p 120 -g 3 -t 10.1.1.201 -S /var/run/vrrpd_48.state
/usr/local/sbin/vrrpd -Ni eth0 -v 49 -p 100 -g 3 -t 10.1.1.202 -S /var/run/vrrpd_49.state

Careful planning needs to be done in order to use RRDNS to share load. I have deliberately refrained from describing this technique as Load Balancing, because at best it is load sharing – you can’t balance traffic, only politely suggest to visitors that that might like to share traffic across your cluster. Additionally, you need to do some capacity thinking – if you have two, or even three nodes in your cluster, if one of the servers fails, one of the servers which is left is dealing with a doubling of the traffic that it was doing before. There is really no elegant way to share three vips across two servers.

Its cheap though. And as I have learned, you can force it to work with a bit of ingenuity.