// archives

Sys Admin

This category contains 33 posts

Mastercard Securecode Rant.

I ranted on the Ecommerce Experts mailing list earlier in the week after canceling an order on a cabling website, after it prompted me to enroll in Mastercard Securecode, with no way out.

My gripes are that

  • The general public should NOT be encouraged to enter their secret personal data at a checkout, in random popups.  The commerce community should be sending the opposite signal; that filling in forms requesting private data on a random website is precisely how you get your identity stolen and used fraudulently !
  • The form looks like a XSS attack, not something genuine, so I have no way to work out whether it is genuine, or whether I am being phished.
  • The card may well be a company card, and not attributable to personal details.

I complained to the retailer and explained that I was not willing to order from them whilst they used and enforced securecode, and the retailer lied to me, explaining that they had no option but to use it, but that I could telephone through an order.  I think they miss the point of e-commerce.

Please do not deploy Securecode or VBV on client sites.  Please abort the transaction if you are prompted to enter your details on an untrustworthy third-party form during checkout online.

Scalability – a talk at LugRadio Live 2007

I’m at LRL (currently watching Ted Haeger’s talk), and I am publishing my notes for my talk tomorrow, “Scaling Up for Champions”.

There are quite a lot of config examples in the notes which aren’t on the slides to support the talk.

Overview:

  • Definition of Scaling & Problems of scaling
  • You have to monitor things
  • Scaling individual machines – Disk IO, Processors, Memory, Connectivity
  • Multiple Servers – L7 proxying, L4 switching, CDNs
  • Memcache
  • Simple tuning

Its a 30 minute talk so its an overview of all of those things.

Download the slides here. 

Was PGP Signing the first social network?

I spent some time last night making some cards for the LugRadio Key Signing event. I’ve used pgp for a while (since 2001 – I am now on my second key) and have not worked on building up the number of signatures I had on my first key.

I understand that PGP works best with a ‘Web of Trust’, and it suddenly hit me – I think PGP key signing is the first online social network. It has many of the other features of Web 2.0 social networks :

  • A common interest (in this case, in security – just like music on last.fm)
  • A list of real people advertising a relationship (nomatter how tenuous – “I have verified your ID”)
  • It is published (keyserver network)

Wikipedia thinks that Classmates.com was the first social network, released in 1995. PGP was released in 1991. I’d be really interested if someone who knows whether the web-of-trust features were there in 1991 – perhaps please comment if I am wrong.

In any case, if you know me, and would like to sign my key – these are the details

andy@pringle:~ $ gpg –fingerprint CCBCBE9A
pub 1024D/CCBCBE9A 2007-02-06
Key fingerprint = 2B62 D54D CF4A 8093 5189 804E 8991 FF62 CCBC BE9A
uid Andy Davidson
sub 2048g/9002F1A8 2007-02-06

If I know you, I would be delighted to sign your key.

I use subkeys.pgp.net as my keyserver.

Trend Monitoring Suites

I hate cacti. Sorry guys, there are lots of things that are good about it, and those things are that if you want to monitor just switch/router interface stats, via snmp, and that’s *it*, its very easy. When you want to plot technical data that you source through something other than snmp, working through the cacti template system is like wading through tar.

ganglia.pngStep in some newer projects. Ganglia was really interesting, and a colleague found it thanks to some presentation that Flickr demo’d. I really liked how easy it was to configured. Set the agent up on a bunch of PCs, run the web interface on one, and bang, graphs. Its that easy. We installed the agent on a couple of trial PCs and we had graphs. We then wrote some scripts to measure other metrics from custom applications. If we could write a script that produced a number, then we could graph that metric in ganglia, just by passing the number to the bundled ‘gmetric’ application. Brilliant. But what about if we can’t run an agent on the device that we want to monitor, such as a switch ? There has been talk on the ganglia developers list since 2004 about incorporating snmp support, but no real evidence of traction. So it wont work for me.

So let me offer a golden rule of performance monitoring. If you are going to write a performance monitoring suite, make sure it supports SNMP on day one. If you are writing a monitoring layer for your application, make sure it uses SNMP.

In steps Zabbix. The best of both worlds. Here, there’s an agent again, so if you want to monitor the health over time of a server, you configure the agent and send back figures to a monitoring box. Figures appear. There’s also an snmp interface, so you point it at a router, tell it the community, and more figures appear.

No graphs yet, but thats because you configure them yourself, but its really easy. Want to aggregate all of the exit ports on your router – make a graph using those metrics ! If you can imagine it, you can graph it with Zabbix. Some of it is quite clunky, i.e. configuring the snmp community for each device is a bit slow, but the back end if just MySQL, so you can change the community for a device with an “update items set snmp_community =’xx’ where hostid=’yy’;” instead of using the clunky interface. Also, to measure interface stats, you must change the ifInOctets and ifOutOctets delta to ‘speed over time’ not just accept the counter value, otherwise your graphs will show nothing more than the port counting more data as time goes by.

I strongly recommend Zabbix to anyone who finds cacti arduous to configure.

Common Event Expression.

I am getting quite excited about some of the material I have been reading on Common Event Expression (pdf).  CEE is a desire to standardise the way that events are described.  I can see this being of significant advantage to sysadmins who need to produce large scale monitoring systems.

We already all use syslog-ng or rsyslogd or similar to aggregate our logs centrally, but it would be great to be able to aggregate logs inside our monitoring systems in such a way that when we add servers to our networks, any issues that they raise, in the application layer, or in hardware, are described to monitoring systems in a common and expected way.

If the taxonomy of error handling was equivalent on, say, routing kit as well as desktop systems, this allows sysadmins to deploy complex monitoring systems with less effort.  Understand how to handle a mistake with system-X and every single system you deploy from then on benefits from tried and tested monitoring and management.

Its early days for CEE, but I am optimistic about the benefits we could all realise if there was a desire to standardise logging.  Looking forward to what happens next.

Debian Package for Adaptive Readahead

I am testing out Adaptive Readahead in Linux quite a lot at the minute. ARA offers particular performance improvements to file-reading logic, and should give significant performance wins on database servers.

I now have a stable enough kernel package for Debian users who may want to try it out. The kernel package uses the same defaults as etch, but there’s a version bump to 2.6.21, and of course I have turned on the ARA features!

Download the Debian Adaptive Readahead package and install with dpkg -i

Please post a comment if you try it out and have anything to report.

Friendly SIP URIs in Asterisk

I have typed this info into several irc privmsgs in the last month, so I’ll write up how to setup ‘friendly’ sip uris with Asterisk.

Firstly let’s look at DNS. Say my email address is abc@example.com. The A record for example.com probably points to example.com’s webserver, so that people who skip the www can still see your website. Therefore, if you don’t run Asterisk on the web-server, how do you redirect packets to your voip server ?

The answer is the service (SRV) record. If example.com’s voip server was ‘voip-in.example.com’, your SRV record would look something like this :

_sip._udp IN SRV 1 0 5060 voip-in.example.com.

You then need to configure Asterisk to handle the sip packets properly. The default domain on this asterisk box is probably ‘voip-in.example.com’. Fortunately, multi-domain support is pretty easy, you just have multiple ‘domain’ lines, thus :

domain=example.com,visitors

…which would send requests for abc@example.com to the ‘abc’ extension in the [visitors] context in your dialplan.

The easy thing to do here is to just force requests to for the email-form sip uris to ‘goto’ the right section of the dialplan in your ‘normal’ phone dialplan, e.g.

[visitors]
exten => abc,1,Goto(default,1001,1)

This avoids lots of duplication. Ensure that your default/unauthenticated sip contexts do not allow access to your pstn gateway!

Solaris 10′s in.telnetd exploit.

My telnet mantra has always been ‘Telnet is only dangerous if you use it’. Leaving it enabled as an emergency way-in during ssh upgrades, for example, is a good idea. It’s a better idea to leave it turned on and visable to just your home or office, but not use it unless you only pass through your own switched network between your desktop and server running telnet.

All this changed at the weekend with the disclosure of a flaw in the authentication model for Solaris’ in.telnetd. To try it out, first enable telnet

# inetadm -e telnet

Then connect from another machine in this way:

factory:~ andy$ telnet -l “-fbin” grumps.nosignal.org
Trying 213.232.80.121…
Connected to grumps.nosignal.org.
Escape character is ‘^]’.
couldn’t set locale correctly
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
————————————————————

Welcome to grumps.
$

Then (quickly) disable telnetd with an

# inetadm -d telnet

This attack raises some questions, the most interesting ones in my mind include :

  • Does the publication of this flaw support or destroy the open source model?

The flaw was only discovered because the source to in.telnetd had been released as part of the disclosure of source to the OpenSolaris project. If the code had not been released this flaw would have not been noticed (yet), and therefore might have been picked up by the vendor and handled in a sensitive manner. On the other hand, releasing the code to the community has picked up this flaw quickly, so does the open source model lead to inherently more secure code?

  • Is a mail to full-disclosure the best way to handle vulnerabilities?

It is certainly much more polite to contact the vendor so that a structured damage-limitatione exercise can begin! I don’t care that someone picks up a vulnerability in the software I use as long as one of the first people to find out is the vendor. With open source software, there is possibly no central ‘vendor’ to inform, but for the examples of the commercial companies, and huge projects like the Mozilla Foundation. The person reporting it to full-disclosure may have tried to tell Sun first, but there is no evidence to suggest this.

Well, Sun have released a patch in any case, but now that my remote sun boxes have Lights out Management cards I think it’s now time for a new mantra: Let Telnet Die. I still maintain that it’s no inherantly less secure than ssh (if used with ssl, of course), and perhaps more secure than ssh because the port-forwarding features in ssh aren’t present, but in the interests of exposing as little as possible to the outside world, I’m leaving it turned off.