Aching Brain Just another WordPress weblog

Uniqueness

Something we've been ruminating on at work recently is uniqueness.  We'd quite like everything in our system to have a unique identifier, usually referred to as a GUID or Globally Unique IDentifier.  This way you can access any entity from any place, and the display method of said entity is determined by the place of access.  By way of example, if you were to access an email, you'd see the email, but if you were looking at a folder full of emails, you'd see the email in the context of the folder that contains it.

Back to uniqueness though.  We don't really need our GUIDs to be truly globally unique, only unique within our system.  Ideally we'd also like them to be short so if you email someone a url their client won't wrap the line in the middle of our GUID.

The simplest way to ensure this would be with a number.  This is easy to maintain as they can be sequential, generated by a database and don't need any special rules.  The only problem is the namespace is not very big.  If you allow five characters for your GUID in the familiar base 10 number system, you have 10,000 potential values, or 10⁵.

10,000 is a good start but what if you've got entities in different tables?  Entity#1 in table A is not the same as entity#1 in table B.  Perhaps we can do better.

The canonical GUID implementation is the Universally Unique Identifier or UUID.  UUID has been standardised as RFC4122 by people a lot cleverer than me so let's assume it's pretty good.  A UUID takes the following form:

550e8400-e29b-41d4-a716-446655440000

Right away we can see that it's going to result in a much larger namespace than using numbers as it incorporates letters as well.  This expands the value each character can be from 10 to (26 + 10), giving us a base 36 namespace.  A five character GUID in base 36 has 60,466,176 possible values, or 36⁵.

Version 5 UUIDs are calculated by taking an existing source of uniqueness (a URL is recommended) and hashing it using the SHA-1 algorithm.  This is not fantastic for us as we're trying to create a GUID which will be used in the creation of a URL, so we're going to have to look elsewhere.

The simple thing to do would be to generate them randomly.  This is not as bad as it sounds and appears to be a standard way of doing it.  We've just introduced a subtle problem though.  As we are now randomly generating the GUID we become vulnerable to the birthday paradox.  From this rather nifty calculator, we can deduce that we'll reach a 99.9% probability of a collision (eg. two items with the same GUID) with a five character GUID in base 36 after 29,903 items.  Although this means we are fairly likely to have collisions with randomly generated GUIDs, it's still much better than using integers which reach 99.9% after just 372 items.

Clearly the way to increase the namespace is to increase the base of the number system we are using.  There's easy thing we can do here - include capital and lower case letters.  YouTube do this to great effect, even including punctuation:

http://www.youtube.com/watch?v=ZOU8GIRUd_g

So lower case (26), upper case (26), integers (10) and underscores (1) gives us base 56 - 550,731,776 potential values at five characters, or 56⁵. No need to stop at underscores though - RFC2396 tells us that the only restricted characters for URLs are ; / ? : @ & = + $ ,.  That leaves !@£%^*()[]{}"'\|><. etc etc.

At 11 base 56 characters, YouTube can accommodate nearly 17,000,000,000,000,000,000 videos (or 1.7 x 10^-8 hella videos) before running out of GUIDs.  When they do, they can just add another character on to literally increase the namespace exponentially or simply allow additional punctuation marks.  Taking the birthday paradox into account, they can have 15,325,262,787 videos before a 99.9% probability of collision occurs.  If they currently have 120,000,000 they are still a long way off needing to extend their namespace.

So there you have it.  Increase your number base to reduce the chance of GUID collision and make your life easy when trying to make everything in your database have a unique, email friendly identifier.

Popularity: 1% [?]

The Register’s guide to Linux Distributions

Tomorrow, we'll tell you how to get them, burn them and set them up to dual-boot with Windows and on Wednesday there will be a guide to tweaking your new setup and getting it ready for use.

Three days to set up and have a Linux desktop ready to use.  Yep, that about tallies with my memory of it.

http://www.theregister.co.uk/2010/06/21/reg_linux_guide_1

Popularity: 1% [?]

Updates

I'd forgotten what a time consuming, involving and generally ill thought through process updating a copy of Windows is.  On the Mac you use a desktop application, on Windows a flaky looking web app.  On the Mac you select things like "QuickTime Update 7.6.2" which kind of makes sense, on Windows you select "Critical Update for Windows XP (KB9832082)" which might as well be in Esperanto.

Operating system updates on the Mac are all rolled together, so if you are on 10.5.3, updating to 10.5.7 is one step (and a big download).  On Windows you need to repeatedly visit Windows Update to ensure that no new updates have been unlocked by installing the last set.

After selecting your desired updates, on the Mac any further interaction is done right at the start of the process - EULA clicking, etc.  On Windows the process is constantly interrupted with wizard after wizard after wizard, meaning that you can't just click "go", put the kettle on and come back half an hour later.  No, you have to constantly hand hold the process like some it's some sort of attention seeking toddler.

It also offered me the option of installing something called Microsoft Genuine Windows Notifications which promised, if my copy of windows was not "Genuine", to constantly bombard me with nagging reminders to "solve" this problem.

Who in their right mind would actually choose to install such a piece of software?  If your copy is legit you don't need it and if it's not, you don't want it.

It then prompted me to follow a link to a list of benefits of using "Genuine" Windows software.  I didn't follow the link, but presumably it's just a picture of Steve Balmer in a bath of money.

Sorry that this post is so unashamedly pro Mac.  I'd have mentioned the Linux update process as these days it's pretty painless.  The update process itself is painless, I mean.  The pain comes after the update when you find that half of your shit doesn't work any more.

Popularity: 9% [?]

Running FMS 3 on Gentoo Linux

I've been trying to install Macromedia Adobe Flash Media Server 3 on my local development box and have run into some problems.

To help out anyone else trying to do the same thing, here's how I did it.

First you need to install nspr

Add a user/group for it to run under:

Then download and extract the media server as per usual. Before installing it, you need to fiddle the install script so that it will install on your "unsupported" platform. Open installFMS in your favourite text editor and find the line that reads

Change it to something appropriate like

Then start the installation script as normal. Tell it that you don't want it to run as a daemon (as it requires the Red Hat only chkconfig command) and that you don't want it to start the server when done.

After you've installed the server, you'll need to make the bundled libasneu.so.1 library available for use:

For some reason the install messed up the configuration files, adding things like the administrative user name twice. Check conf/fms.ini to make sure this hasn't happened to you.

The installer creates init scripts at /etc/init.d but in order to use them you first need to create a file called .autostart in the installation directory:

You should then be able to start and stop the server normally:

And use rc-update to have the server start on boot:

Good luck.

Popularity: 98% [?]

Prototype String.toQueryParams() weirdness

I just came across this while using Prototype:

To save you the trouble of running it yourself, I'll tell you what happens - the unescaped percent symbol causes a URI malformed error to be thrown.

Very odd. The Prototype documentation for toQueryParams says:

Parses a URI-like query string and returns an object composed of parameter/value pairs.

Since it's a URI-like query string and not an actual query string (by which I mean one that's been through the browser address bar and as such subject to whatever text munging the browser does), my example above should be a legal use case.

After a little digging around in prototype.js, it seems that the problem is caused by the JavaScript function decodeURIComponent() not enjoying being given a string that contains an unescaped percent symbol. Usually percent symbols are used in conjunction with a number to represent a non-alphanumeric character in a URL - %20 for space, etc, but in the above it's a literal percent symbol.

My first attempt to fix it changed lines 98 and 100 of string.js in the Prototype source to use the built in escape function:

This worked for my use case but caused Prototype to fail a different unit test. Changing them to the following worked out okay:

Granted the unescape(...(escape(...)) is a little clumsy, but it seems to get the job done.

Adding the following unit test to string.html allows to test for the above:

I've created some test pages which demonstrate the problem.

There is also a patch file available. Apply it to /src/string.js in your Prototype source tree.

Update

I've also filed a bug.

Popularity: 100% [?]

Code

I haven't posted here for a while, mainly due to the time sink that is Facebook, but I have been writing code in my spare time, honest.  I thought I'd have a little skim through what's on my machine and share a few choice nuggets.

Recently I've been trying to get into programming applications for Mac OS X - I use it on a day to day basis so thought it would be best to get involved.  I've put some new things on the Stuff page for your perusal.  They are:

Backgrounderer

A little app to download desktop wallpapers from veer.com. If you wish to build from source, you'll need to install RegexKit first.

Mail RSS Exporter

An application that exports your RSS feeds from Apple Mail and stores them in either OPML format or Safari Bookmarks.

TouchMe

A program to run AppleScripts on a computer on your local network from your iPhone or iPod Touch - I use this to control a Mac mini attached to a projector.  The projector is not always switched on so Front Row was insufficient.  It also allows you to force quit applications that have hung.

SimpleHTTPd

A Cocoa native web server packaged as a framework so you can use it in your own projects.  Based on Jurgen Schwiezer's SimpleHTTPServer.

All of the above require OS X 10.5 Leopard as coming from a Java/PHP background I have a strange fetish for garbage collectors and consequently find manual memory management tedious.

They are all BSD licensed and source is included with each program so go crazy.

I'll get round to posting about each one in more depth in the near future.

Popularity: 92% [?]

Button, button, who’s got the button?

From an article on the Wall Street Journal about Steve Job's button phobia.

When the company introduced the iPod in late 2001, the most common calls to Apple's technical support lines for a time were about how to turn the device, which lacked a clearly defined power button, off and on, says a former Apple executive.

Wow, I'd forgotten all about that. Years ago Will left his iPod at our house for a week or two and I had such trouble trying to figure out how to turn the damn thing on.

Popularity: 93% [?]

Apache2 and OS X

This is more a note to myself than anyone else, but to compile Apache2 on OS X, use the following configure:

When Apache starts whinging about "dlname not found, assuming libtool archive" and shit, and refuses to produce .so modules instead kicking out a bunch of .a and .la files in their place, it means you've fucked up somewhere along the line, probably because this isn't the first time you've tried to run the configure script.

Delete the entire Apache source directory and start again. If you are lucky you'll even get a nice copy of libexpat.0.dylib at the end of it all.

Popularity: 93% [?]

Orwellian

The BBC ... has always taken a platform agnostic approach to its internet services.

And then a little later:

It is not possible to put an exact timeframe on when BBC iPlayer will be available for Mac users.

--bad, indeed.

The corporation's governing body asked the BBC to ensure that the iPlayer could run on different systems - such as Apple Macs - within "a reasonable time frame", initially twenty-four months.

Twenty four months for porting to Mac OS X is reasonable?! I dread to think about the poor old Linux users out there.

The BBC has previously said it cannot commit to a two-year time frame as many decisions would have to be made by third parties.

i.e. Either Microsoft decide to port their DRM software or someone else will have to decide to write some.

A statement from the BBC read: "Our ability to deliver this open approach will be influenced by the availability of alternative DRM systems on the market.

i.e. Microsoft have no plans to port their DRM software to any other operating system so the BBC will have to find some other system to use and so far haven't so we'll just have to wait.

What a load of rubbish. I've always been a strong supporter of the BBC but this is utterly unacceptable.

http://news.bbc.co.uk/1/hi/technology/6236612.stm

Popularity: 91% [?]

Glastonbury tickets

Alex is currently wondering if the Glastonbury ticket website is run off of a beige box in the corner of someone's flat with a cat chewing on the network cable.

Honestly, what with all the pre-registering bollocks it's not like they were unaware of the demand. Why can't they design a server infrastructure that can handle enough requests?

Popularity: 93% [?]

svn:externals and different usernames

How to use a different username with svn:externals.

Genius. Well, almost. Close enough.

Popularity: 92% [?]

Pink Zune

Me: Oh look, they've released a pink Zune. Would you like one for Christmas?

Her: I'd rather stick my head on a pike.

Although on closer reading of the article, it seems that the pink Zunes are a very rare limited edition and they are not marked as such. Consequently the guy who bought it thought he was getting a white one, didn't like the pink, saw an opportunity and promptly put it on eBay.

Honestly, that's a stroke of genius only matched by releasing a brown Zune in the first place.

You can just see little Jimmy's parents in some electronics shop going:

Parent: Oh look, there's one of those Zune things Jimmy wants. Excuse me, Mr Shop Assistant, we'd like to buy a Zune.

Shop Assistant: Wonderful, here you go.

Parent: Excellent, he's going to be so pleased.

Of course, the parents left it to the last minute and the black/white Zunes sold out a long time ago - something the shop assistant neglects to mention lest it should damage his chances of making a commission and/or missing his sales target.

Little Jimmy then spends the next six months being bullied at school for having a poo coloured Zune before it's finally stolen and destroyed by muggers on the bus who can't bear to suffer it's existence any longer. Little Jimmy is secretly thankful.

Popularity: 50% [?]

Vista

I downloaded the release version of Vista today via my work MSDN account and my first impressions were:

Hmm. I can't quite focus on the text on this screen. Either my hangover is really, really bad or I need to get my eyes tested.

Oh, wait, it's just ClearType.

Aside from that, I'm amazed at how complicated they've made everything. For example, when downloading files through IE7, you now have a "Downloads" folder in your home directory. Can you get to this via the start menu? No. Is it a sub-folder of your Documents folder? No. What's wrong with just putting them on the Desktop? Isn't it what it's there for - files you are currently working on and/or might not want to keep?

The Windows Explorer seems to have become a maze of semi-coherent short cuts and file system abstractions - see the address bar for what I mean. Even the Control Panel has succumbed to this mess. It seems they've sat around a table and gone "What are the most popular things people do with the Control Panel" and made shortcuts to them all, rather than organising the different areas in a logical manner in the first place.

And then there's the whole off button thing.

Never mind the hundreds of pop ups "This program wants to do this" "Do you know this program?" "Do you want to let this program do this?" "Are you sure you want to let this program do this?". And the really obnoxious one that greys out your entire screen and puts the dialogue box right in the middle. "LOOK AT ME", it screams. "FUCK OFF", I think.

And what happened to the File/Edit/etc menus? It took me a good twenty minutes to work out how to show hidden files and folders. In the end I had to ask Google for the answer.

Ugh. Maybe I'm just getting old, but does it really have to be this way?.

Popularity: 49% [?]

Magic bullet

Biometric passports cloned. It seems all it takes is 174 quids worth of hardware and 48 hours of a programmer's time to write the software.

Given the government's track record for IT, is anyone really surprised by this?

Popularity: 34% [?]

Beta software

John Gruber has a go at shonky software hiding behind the "Beta" label (so hot right now).

Too bloody right.

Popularity: 31% [?]

eBay infrastructure

There's an article on eWeek.com about the storage requirements of eBay.com. It's mostly large number willy waving, but this quote from the third page caught my eye:

eBay's application servers, according to Strong:

* use monolithic two-tier architecture
* boast 3.3 million line C++ ISAPI DLL (150MB binary)
* have hundreds of developers, all working on the same code
* are hitting compiler limits on number of methods per class.

Now, I could be wrong, but the last three points don't really strike me as much to be proud of, not to mention possibly inaccurate in the case of the first two.

In other news, this morning I did something that some people might find a little odd. I cleaned my bike.

Truly, I am entering into the realm of the unwell.

Popularity: 22% [?]

ClassLoader

I have a sort of love hate relationship with OO support in PHP. On one hand it looks like a dirty hack due to a combination of the duck typing, lack of function overloading, no proper namespaces and the way everyone talks about object methods when the code itself calls them functions. On the other hand it just feels right (although I'm told this kind of wooly thinking is my degree showing) and if you are putting anything together where scripts run to more than a hundred lines or so it's really the only way to do it and keep it all manageable. One of the major problems I have with the whole thing is that there is no standardised way to store your class definitions and consequently no way to import them in a neat and tidy way. I personally favour having each class and interface in it's own file, named thus:

DatabaseConnection.class.php
SQLQuery.class.php
JSONableItem.interface.php
etc..

This way you can identify what is in each file just by reading the name and you can order them in directories like so:

ajax
JSONableItem.interface.php
database
DatabaseConnection.class.php
SQLQuery.class.php
etc..

Even giving you a sort of memetic namespace of

ajax.JSONableItem
database.DatabaseConnection
database.SQLQuery
etc..

Which encourages code re-use by making it very easy to transfer classes between projects. This brings me on to the next problem in that there is no standardised way of importing these classes into your script files. I normally have one file that imports all the class definitions into the project, somewhere high up on the bootstrap page but this means that every single class in the project is being loaded for each request to the server that is answered by PHP - not exactly ideal. I found a little class on the Pure PHP website called Include_once which is based on an interesting little idea - keep a list of imported files and only import a requested file if it is not in the list. Inspired by this I've written a little utility class rather unimaginatively named ClassLoader which does the above but via a sort of namespace that allows for nice structuring of class definition files in the file system and also the importing of whole folders of class files (packages). An example of it's use is as follows:

or even

There are also a couple of other fun methods

tells you how many classes have been loaded and

tells you how many times ClassLoader has been asked to load class definition files. The reason that these may be different is because you can use ClassLoader to include class definition files at the top of other class definition files much in the same way you'd use the import statement in Java. So for a trite example, with the following file structure:

classes
  pretend
    InterfaceA.interface.php
    ClassA.abstract.php
    ClassB.class.php
    ClassC.class.php
  util
    ClassLoader.class.php
index.php

InterfaceA.interface.php

ClassA.abstract.php

ClassB.class.php

ClassC.class.php

index.php

Neat. Of course, you don't have to follow my file naming convention, feel free to do whatever you want, as long as the file has the name of the contained class before the first '.' and the last four characters are '.php' Configuration of the class is pretty simple - just change the static member variable ClassLoader::$classPath to suit your set up. Just make sure it's readable by PHP. So, for example:

or

or similar. Given the above, there's a really tiny part of me that says I should start using proper stuff like J2EE or something. I try not to listen to it.

Popularity: 26% [?]

Colourblind

I've just been reading this presentation paper on colourblindness while trying to be a back seat designer as the actual designer in the office chooses colours for buttons.

An amusing statistic from the paper is that 8% of caucasians have difficulty distinguishing between red and green. From the perspective of a web designer, it's more worthwhile designing for their needs than the entire macintosh user base (even after accounting for the 8% of red/green colourblind mac users).

Popularity: 19% [?]

Web 2.0 phones home

Just picked up a Sony Ericsson K800i. It's a bona fide Web 2.0 (tm) telephone complete with rounded corners, drop shadows, an RSS reader and photo blogging tools.

I briefly had this phone on a contract with 3, but their network is so restrictive that I had to send it back - for example, you can't access any websites that haven't been "specially formatted for 3 Mobiles" (e.g. vetted to make sure they aren't selling any mobile content) and you can't use the built in email client unless you a) pay through the nose and b) have some huge pre-specified smart phone. No thanks.

Then I noticed that T-Mobile are currently being foolish enough to offer unlimited (okay, 2GB) mobile Internet access for a mere seven pounds fifty a month, even with 3G phones. Bwahaha. I will show them the error of their ways.

Now all I need to do is actually call someone with it.

Popularity: 19% [?]

Time machine

Is it just me, or does anyone else think that the Time Machine UI is just a tad excessive?

It's a neat idea though, sort of like having a single user Subversion repository for your whole computer.

Popularity: 20% [?]