The wider lesson from a college admissions glitch

February 1st, 2012

My alma mater, Vassar College, has been in the news lately over an error in it’s admissions web site, which led 76 early decision applicants to believe they had been admitted when they had not. As the Times tells it:

On Friday, around 4 p.m., 122 students who had applied for binding early admission to Vassar saw what the school later called a “test letter” congratulating them on their acceptance. Hours later, the students received a message saying the letter had been posted in error. Once the correct decisions were displayed, only 46 of the students were told they had been accepted.

Vassar has since sent apologies to the students, to all of us alumnae/i, and other interested parties.  They are refunding the application fees for the rejected students. There has been some debate in the blogosphere and elsewhere about whether or not these applicants should have been admitted anyway.

For technologists, this speaks to the cost of the error, but the interesting part is the underlying cause.  This looks to me like yet another example of the bad things that happen when testing is done in the production environment. In this case 76 teenagers involved in what was already a very stressful process were forced to ride an emotional roller coaster of disappointment and embarrassment.  In other cases there can be large financial and even regulatory consequences when test transactions were accidentally put onto production systems and sent to counter-parties. Most senior technologists know a horror story or three on this topic, although they don’t like sharing them.

While there is increasing recognition of the need to better segregate production and test environments, it is both expensive and inconvenient to do so. In times of budgetary restraint, projects to fix this problem can get postponed until the next accident happens. I think companies would be wiser to work on this gradually and incrementally.  Trying to do nothing and trying to do everything all at once are equally unrealistic.

Security Thought Leadership

February 27th, 2011

Hat tip to Brian Krebs for this one.  Also congratulations to Brian for winning a security blogger award at the RSA Conference.  One of his fellow winners, Chris Eng, won an award for the best single blog post for this wonderful piece. (Thanks to xtranormal.com for providing the embedding code on their website.  The original post may be found at http://www.xtranormal.com/watch/7897173.)

Attachment or Link?

July 27th, 2009

Steve Bellovin has an interesting analysis of the question “which is a better way to send a file, as an attachment or as a link?” The discussion centers around the process of opening a file in either a mail user agent or a web browser, and how vulnerabilities in each get tickled. There is no clear hands down winner, although since sending links is also more courteous he gives that a slight nod.

Password Security

July 24th, 2009

From our friends at XKCD:

No Free Lunch

July 24th, 2009

A hat tip to Spaf for this one. Hillary Clinton held a town hall for State Department employees recently, where a recent transfer from one of the intelligence agencies asked why they couldn’t have Firefox, which was approved by NSA for use in the intel community.  Secretary Clinton turned to one of her aides, Patrick Kennedy, who replied that they had to look into the budgetary issues.  This drew cries of “but it’s free” from the crowd, which then got the “nothing is really free” explanation.

This explanation drew snarky hoots of derision from The Register and Gizmodo, both of whom ridicule the notion that a piece of free software could cost anything to manage.

Clearly, you don’t have to actually know anything about managing IT to write about it for these publications.  There is no such thing a “free” software, if by that you mean that the total cost of ownership is zero. Here’s what it takes to deploy Firefox to tens of thousands of desktops:

  • Decide what lockdowns you need in your environment and build a local build of Firefox that implements.
  • If you care about plugins, include in the lockdowns a restriction that plugins come from a local repository of approved ones.
  • Package it.
  • Distribute it.
  • Support it.
  • Rinse and repeat for each patch release.

What you can’t do in an environment where the user desktop is a managed resource is have users download and self-maintain a complex security-sensitive piece of software.  I’ve worked in organizations that decided that the costs of doing the above was worth spending.  But there was no illusion that supporting Firefox was free.  Even a “best effort” support model requires people to execute it.

One encouraging note was that lots of IT professionals gave these articles the comments they deserved.

Cybersecurity Bill of 2009

April 27th, 2009

Steve Bellovin has an excellent analysis of the CyberSecurity Bill of 2009 on his blog. It is a long and thoughtful piece that you should read for yourself rather than having me try to summarize it.

Devolution

April 22nd, 2009

A hat tip to Bruce Schneir for spotting this one.  The BBC is reporting that the NHS Central Lancashire backed up health data on 6,360 prisoners and ex-prisoners by copying it on to a USB stick. They even encrypted the data.

Then they lost the stick.

With a yellow sticky attached to it with the password.

We really can’t make this stuff up.

Captcha? Gotcha

April 6th, 2009

Beware of security questions that you didn’t create!  From XKCD (http://xkcd.com/565/)

Not out of the Conficker woods yet

April 5th, 2009

Dan Kaminsky has a very good call for continued vigilance against Conficker. Entitled “A Marathon, Not a Sprint” he writes:

Of course, you may be thinking:  The world didn’t come to an end.  Clearly, this whole thing was just a Y2K hypefest.  I’m sorry the bad guys aren’t quite the eschatologists some people would like them to be, but somebody’s been investing extraordinary amounts of resources making a worm very difficult to kill.  It’s not like there was a contingent of rogue coders, sitting around figuring out where they could put two-character date fields after January 1st, 2001.  There’s a bad guy out there, and while we shouldn’t panic, we shouldn’t quite ignore the situation either.

I agree with his advice.  Don’t panic, but don’t drop your guard either.  Now that network based scanners can spot this, the owners of enterprise networks should be scanning and cleaning up.

Conficker “doomsday” passes without incident

April 2nd, 2009

Security Wire Daily (among many other sources) reports that the April 1st detonation date for the Conficker / Downadup worm passed without incident. For those who have not been paying attention, Conficker is a worm that exploits a vulnerability in the Microsoft Windows RPC code to install itself.  The payload is aimed at forming a large (between 9 and 15 million hosts so far) botnet.  The command and control channel is an outbound connection to a host selected from a pool of domain names generated by algorithm.  Largely due to the efforts of the Conficker Working Group the domain generation algorithm was cracked and registrars cooperated to prevent (or revoke) registration of those domains.  Thanks to that effort, very few of those millions of computers were actually able to reach an update server, which has kept these machines from getting instructions to do anything nastier than spreading.

The Conficker Working Group is a (rare) collaboration among a broad spectrum of technology companies and organizations including Microsoft, Afilias, ICANN, Neustar, Verisign, CNNIC, Public Internet Registry, Global Domains International, Inc., M1D Global, AOL, Symantec, F-Secure, ISC, researchers from Georgia Tech, The Shadowserver Foundation, Arbor Networks and Support Intelligence. Their work was aimed at both detection (hence the AV companies) and keeping the worm from phoning home (the registrars and ICANN).

Reports are that some small percentage of infected machines did manage to connect to an update server, but did not immediately change their behavior.  This has led some to speculate that the April 1 date was a blind and the worst may be yet to come.  Certainly, it is no time for complacency, and organizations should remain vigilant in detecting and cleaning up Conficker-infected machines.

The most interesting work actually came over the weekend on the detection front.  Dan Kaminsky reports on work he did jointly with Tillman Werner and Felix Leder of The Honeynet Project to detect infected machines from the network rather than the host. The authors of Conficker want to protect their botnet from poaching, so after they own a host they patch the buffer overflow that let them in.  Werner and Leder exploited the differences between the Conficker patch and the official Microsoft patch to develop a malformed RPC request that will elicit different responses from healthy and infected machines, allowing detection.  This is vital to remediating infections because the worm disables Windows update and any security software it finds on the machine to avoid detection.

In an amazing piece of coordination, the group figured this out last Friday and working code was in the major vulnerability scanners by Monday.  I don’t impress easily, but I’m impressed.