Joe's Jottings

Or…yet another reason to watch who you befriend….

Facebook attempts to be what’s known in the online world as a ‘Closed Garden’ – interactions with the rest of the Internet are restricted somewhat to make the user experience better…or to keep you in the loving arms of Facebook, depending on how cynical you are.  One of the tools in this process is the Facebook API – a set of programming tools that Facebook produce to make it possible for programmers to write software that works within the Facebook framework.  Indeed, Facebook get very peeved if you try automating any aspect of the site’s behaviour without using the API.

One thing that the API enforces is the privacy controls; and one thing that you cannot get through the API is an email address.  Which is cool – it prevents less scrupulous people who’ve written games and such from harvesting email addresses from their users to use for other purposes.  It also ensures that all mass communications are done through Facebook.

Of course, if you’re determined enough you could go to every Friend’s profile page and copy the email address from there…or there are scripts that people have written to do the task by simply automating a browser.  The former is tedious, the latter is likely to get you thrown off of Facebook.

However, a method documented hereshows how this can be done through the auspices of a Yahoo mail account.   It is apparently a legitimate application available within Yahoo Mail for the benefit of subscribers.  How long Facebook will allow this loophole to be exploited is anyone’s idea, but given that I have a number of Facebook friends I felt it worthwhile warning folks.

The problem is not you, my trusted and good and wonderful reader, who would only use the tool for what it’s intended for – added convenience in contact management.  The problem lies with people who are a bit free and easy about who they make friends with.  If you do end up befriending a less than trustworthy individual, they could quite happily get your email address through this method, and soon enough you’ll be receiving all those wonderful offers for life enhancing medication and get rich quick schemes.

So…watch who you befriend.  Today might be a good day to prune out those folks that you’re not one hundred percent sure about!

A short while ago I wrote a couple of posts about the issues around Real time Search (How important is Real Time Search and Google and the Dead Past) – that is, Internet based searches that include Internet content that has been generated in the few minutes (or even less!) prior to the search.  Those of us who’ve been around the Internet for long enough will remember the days when you could wait days or weeks for stuff to show up in a Google search; nowadays Tweets can turn up in search results almost immediately.

There are many reasons – most expressed in the two posts above – that I have for feeling rather uneasy about the whole idea of real time search, particularly around personal privacy.  I think the main mistake I made when I wrote those two posts last year was to underestimate the speed with which things would move.  Recent developments in geolocation based systems – that record the location from which a post is made – such as FourSquare and the geocoding side of Twitter have made it easy for Tweets and similar online posts to locate people in the real world.  A particularly fine example of this phenomena is the suitably named ‘Please Rob Me’ - this site uses some clever coding to detect when people Tweet that they’re away from home. 

The publication of ‘exploits’ for web browsers and other software could also become a hot topic.  At the moment, a hacker may determine how to ‘poison’ a website with a specially manufactured piece of code that can infect an unprotected PC with a virus or Trojan Horse program.  The hacker can then publicise the fact via various means, hoping that others will get the chance to use it before the manufacturer of the browser relaeses a ‘patch’ for the bug that the code exploits.  Real time search could very much help hackers – by releasing details of an exploit, then linking to it from a few sites, then tweeting it, it’s quite possible that details of such exploits could be showing up in search results within minutes or hours of the exploit being identified.  Unless the search results are sanitised in some way to prevent this happening – highly unlikely – then this will surely lead to decreasing online safety.

A related problem might be in the creation of online Pop-up Shops’ for ‘warez’ or other illegal content.  For those who’ve never come across a ‘Pop-up Shop’ these are shops that take out a very short lease on a retail property – typically a month or so around Christmas or some other busy event that will guarantee good local footfall.  They then sell cheap goods, Christmas cards, etc. and then shut up shop and disappear – whilst these shops are totally legit business, the Internet equivalents are frequently not.  Given real time search, a suitably optimised ‘instant site’ with an arbitrary URL could be put on a server, show up in search engine indexes / Tweet indexes within the hour , make material available and be gone before the authorities even know it was there.

Real time search is here – faster and probably more effective than I feared.  And it’s not going to be pretty.

I would say that I’m something of an ‘online person’ I ran a Bulletin Board ‘the hard way’ in the late 1980s / early 1990s using a phoneline, a modem and a PC at home, and have been on the Internet in one way or another for over 20 years, and was involved with Prestel back in 1982/83.  However, this article from the BBC made me do a serious reality check.   Nearly four out of five people in a survey done of 27,000 folks around the world considered that Internet access should be regarded as a ‘fundamental right’.

Now, this sort of thing crops up every now and again, and it always elicits the same response from me.  At this point in the history of our planet, nonsense.  Yes, information is increasingly important – even, or perhaps especially – in developing countries and economies.  But a ‘fundamental right’? No.  Let’s not forget that the Internet is a communications technology first and foremost – similar to the phone system, road and railway network, etc.  And let’s face it, there are many people in the world without access to a reasonable road and railway system, let alone  a phone system and the Internet.

Let me give you the run-down on precisely why I think that there are many rivers to cross before we get to the luxurious position of the Internet being a fundamental right.

The Internet can’t carry food…

Or people, or goods, or equipment.  An information superhighway is great in an information economy, but of limited use when you have a subsistence, agricultural or manufacturing based economy.  And let’s face it, whilst information is essential in developing new skills and supporting economies, it can be delivered in lots of old fashioned ways – like books, pamphlets, radio, TV.

The Internet needs power…

To deliver a reliable Internet service in to a country requires that that country have a viable and effective power supply.  Even now, many developing countries do not have reliable power.  Is it realistic to prioritise the right to the Internet over the right to a reliable and cheap energy source that can provide power for light, heating, entertainment, energy for industry? 

What’s the point of an Internet without machines…

Even with projects like OLPC and other ideas to get computers in to developing nations, there will still be the problem of providing equipment and software in to developing nations in a sustainable and long term manner.  A laptop computer – or a mobile phone, for that matter – is a complex piece of kit and is unlikely to be easily manufactured or maintainable locally. 

The Internet doesn’t educate or heal

Whilst the information on the Internet may be helpful in education, just how much of it is relevant without literacy?  And which is a more effective means of delivering basic and even advanced education in a developing nation?  $1000 spent on a computer that might help 1 person, or the same amount spent on books and similar resources for a class?  the Internet does not provide basic health care – it may provide useful information but cannot vaccinate.

The bottom line is that we live in a world of limited resources in which we have to prioritise those resources.  To claim the Internet is a fundamental right is to forget that the real fundamental rights – a home, food, safe water and no local Gestapo kicking the door in because you disagree with your Government – are yet to be achieved over much of the planet.  In a technologically advanced society their might be an excuse for this sort of comment, but in parts of the world where the next drink of water could kill you, it’s a luxury that cannot be realistically afforded.

Regular readers of my ‘jottings’ might recall a recent post of mine in which I debated the value of Tweeted Wisdom.  Always one to consider returning to the scene of past musings, I was today motivated back in to Twitter criticism territory after I read a Tweet that suggested that:

 ”100 is the new 140 for massive retweetlove”.

Now, I have enough problems with 140 characters, but then again I’m using Twitter to communicate ideas and concepts as well as gossip, funnies and bon-mot to the good folks following me.  Whether I get re-tweeted or not is not the first thing in my mind when I put a Tweet together – what matters to me is whether I can marshall the idea effectively in to the 140 character limit.

Starting to apply lower character limits to Tweets based purely on the possibility of re-tweeting does seem rather ‘arse about face’ to me – it IS putting the process of communication ahead of the content – i.e. putting the medium before the message.

Some years ago, the Ford Motor Company were in pretty dire straits – losing money and market.  There was a serious concern amongst the higher echelons at Deerborn that Ford might actually go under.  Various policies were implemented throughout the organisation, including cuts to the design and manufacturing base of the company.  The story goes that at one Board Meeting, some of the directors were commenting that they had managed to get the books looking better by reducing costs, and that most of the cost reductions had come from savings made by closing down manufacturing facilities.  A grizzled old veteran who DID know the difference between a carburetor and a Carbonara pithily pointed out that, based on that thesis, the best way to save the company was to close ALL the company’s manufacturing facilities and stop making cars altogether….

And this is how this sort of emphasis on the mechanism of Twitter strikes me; people get way too wound up with the phenomena and culture and technology of Twitter rather than the function – and the function of Twitter is to allow rapid, succinct communication and conversation between people.  Or even between people and other computer programs!  But the emphasis is on communication and conversation – and when we start emphasising the possibility of a re-tweet over the quality of content, we are in danger of making Twitter more ‘gimmicky’ – something that is not good.

So, for what it’s worth – use that character allowance for the purpose it was originally given to us – to communicate.  Giving 30% of available space up for posisble re-tweets seems pointless.  What matters is what you say; not necessarily how many times it gets re-tweeted.  The ultimate re-tweetable message accoridng to some folks would be a single word – don’t let the usefulness of Twitter be compromised by ego.

When I started in IT, I encountered a program called ‘The Last One’.  It was a menu-driven application generator that allowed a non-programmer to specify the sort of system they wanted (within a limited range) and generate a BASIC program that would do the job.  When it was first announced – and before any of us got to take a look at it – there was a little nervousness amongst the ranks of programmers, based on the advertising strapline for the program, that suggested the software was called ‘The Last One’ because it was the last program you would ever need to buy…

Which was, of course, utter rot.

I was reminded of it today after coming across this piece in which the bods at Google are predicting the  end of the desktop computer.  And the reason I was reminded was that the ‘The Last One’ story just went to show how bad IT pundits – and those in the industry – are at predicting the future.   You see, the problem with predicting the future is that you have to make certain assumptions and extrapolations from today in to the future, and then work out consequences based on those assumptions.  And if you get your assumptions of teh future wrong – or the assumptions of how the world works now – then it can all go horribly wrong.  And that’s what’s happened to Google.

The demise of the desktop computer – to be replaced by iPads, Smartphones and similar mobile devices.  Note that Google aren’t even suggesting that laptops and netbooks and their ilk will be delivering the goods – it’s all going to be a mobile wonderland.  Now, short of some sort of high tech ‘Rapture’ occuring in December 2012 that whisks away all the computers we use in our homes and offices whilst leaving only mobile computing devices behind, I very much doubt that this is going to happen.

Google have mixed up predicting the future with what they (with their interest in mobile operating systems and desire to compete with Apple) want the future to be.  A dangerous thing for a technology company to do.  Whilst in Google’s idea world of media and search consumers everyone would be able to do what they need to do on some sort of mobile gizmo, those of us who work with computers for serious amounts of time each day will NOT be able to function with  poxy little touchscreen keyboards or Blackberry QWERTY pads.  Sorry guys, we need real sized keyboards which will be realistically associated with a decent sized screen and so will be at the very least a reasonably sized laptop – which we’ll sit on a desk and run from the mains.

Quite a few of us also like the idea of storing data locally – not in ‘The Cloud’ or on Google’s application servers – something that isn’t easy on many mobile devices right now.

Google – you’re wrong.  Stop looking at the dreams of your own and other researchers, and start looking at how real people use computers – especially in their work.  And make that the basis of any more crystal ball gazing.

The second point of interest is why the sudden flurry of attempts to compromise Twitter accounts. It’s been suggested that one reason is that the compromised accounts will be used to promote sites in to search engines, based on the recent development of search relationships between Yahoo and Microsoft’s ‘Bing’.  Getting hold of the Twitter accounts would have been the first stage of the operation; the idea would be to automate those accounts to ’spam’ other users with  other links over the next few weeks to attempt to increase the search engine standing of those links.

But the thing that’s surprised me most is how often people have actually gone along with the phishing request – to enter your Twitter user name and password into an anonymous web page, with no indication as to what the page is!  To be honest, it stuns me.  And it isn’t just Internet neophytes – according to this BBC story an invitation to improve one’s sex life was followed through on by banks, cabinet ministers and media types.  Quite funny, in a way, but also quite disturbing – after all, these are people who’re likely to have fairly hefty lists of contacts on their PCs, and whilst an attack like the one detailed in this article is quite amusing, a stealthier attack launched by a foreign intelligence service against a cabinet minister’s account would be of much greater potential concern.

There are no doubt technical solutions that twitter can apply to their system to reduce the risk of the propagation of these Phsihing attacks.  For example, looking at the content of DMs sent from an account and flagging up a warning if a large number of DMs are sent containing the same text.  Twitter have also been forcing password changes on compromised accounts – again, this has to be a good move.  It might also be worth their while pruning accounts that have been unused for a length of time – or at least forcing a password change on them. 

A further part of the problem is with the use of Link Shortening services like Bit.ly to reduce the length of URLs in Tweets.  This means that you can’t even take a guess at the safety or otherwise of a shortened link;  a link that is goobledegook could lead to the BBC Website to read the story I mentioned above, or to a site that loads a worm on to a Windows PC – or prompts you for your Twitter credentials.  perhaps a further move for Twitter would be to remove the characters in URLs from the 140 character limit.  That way, full URLs could be entered without shortening.

But ultimately a lot of the responsibility for Twitter phishing attacks lies with us users.  We need to bear the following in mind:

1. If you get a DM or Reply from ANYONE that says ‘Is this you’ or ‘Read this’ form a friend, then to be honest, check with the person concerned to see whether they have sent them.  If you get such a message from anyone who’s not well known to you, then just ignore the message.
2. DO NOT enter your Twitter username and password in to any website that a link takes you to.  If you do do this, change your password as soon as possible, and don’t use the Twitter password on ANY other system.
3. Keep an eye on your Followers – if there is someone you don’t like the look of, just block them.  It may seem extreme but it stops possible miscreants ‘hiding in plain sight’.
4. Ensure your anti-virus and anti-malware software is up to date – this is your last line of defence designed to stop malware that YOU have allowed on to your machine by falling for phishing scams.

So…play your part in reducing the impact of Twitter Phishing attacks by not clicking those links!

One of my professional interests is in Artificial Intelligence – AI.  I think I’ve had an interest in the simulation of human personality by software for as long as I’ve been interested in programming, and have also heard most of the jokes around the subject – particularly those to do with ’making friends’.   In fiction, most artificial intelligences that are portrayed have something of an attitude problem; we’ve had HAL in 2001 – insane.  The Terminator designed to be homicidal.  The Cylons in the new version of Battlestar Galactica and the ‘prequel’ series, Caprica – originally designed as mechanical soldiers and then evolving in to something more human with an initial contempt for their creators.  The moral of the story – and it goes all the way back to Frankenstein – is that there are indeed certain areas of computer science and technology where man is not meant to meddle. 

Of course, we’re a long way away form creating truly artificial intelligences; those capable of original thought that transcends their programming.  I recently joked that we might be on our way to having a true AI when the program tells us a joke that it has made up that is genuinely funny!  I think the best we’ll manage is to come up with a clever software conjuring trick; something that by deft programming and a slight suspension of disbelief of people interacting with the software will give the appearance of an intelligence.  This in itself will be quite something, and will probably serve many of the functions that we might want from an artificial intelligence – it’s certainly something I find of interest in my involvement in the field.

But the problem with technology is that there is always the possibility of something coming at us unexpectedly that catches us out; it’s often been said that the human race’s technical ability to innovate outstrips our ethical ability to come up with the moral and philosophical tools we need to help our culture deal with the technical innovations by anywhere from a decade to 50 years; in other words, we’re constantly trying to play catch up with the social, legal and ethical implications of our technological advances.

One area where I hope we can at least do a little forward thinking on the ethical front is in the field of AI; would a truly ‘intelligent’ artificial mind be granted the same rights and privileges as a human being or at the very least an animal?  How would we know when we have achieved such a system, when we can’t even agree on definitions of intelligence or whether animals themselves are intelligent? 

Some years ago I remember hearing a BT ‘futurist’ suggesting that it might not be more than a decade or so before it would be possible to transfer the memory of a human being in to a computer memory, and have that memory available for access.  This isn’t the same as transferring the consciousness; as we have no idea what ‘conciousness’ is, it’s hard to contemplate a tool that would do such a thing.   But I would accept that transferring of memories in to storage might be possible and might even have some advantages, even if there are ethical and the ultimate in privacy implications to deal with.  Well, it’s certainly more than a decade ago that I heard this suggestion, and I don’t believe we’re much closer to developing such a technology, so maybe it’s harder than was thought.

But what if….

In the TV series ‘Caprica’, the artificial intelligence that controls the Cylons is provided by an online personality created by a teenage girl for use as an avatar in cyberspace that is downloaded in to a robot body.  In Alexander Jablokov’s short story ‘Living Will’   a computer scientist works with a computer to develop a ‘personality’ in the computer to be a mirror image of his own, but that won’t suffer from the dementia that is starting to affect him.  In each case a sentient program emerges that in all visible respects  is identical to the personality of the original creator.  The  ’sentient’ program thus created is a copy of the original.  In both Caprica and ‘Living Will’ the software outlives it’s creator.

But what if it were possible to transfer the consciousness of a living human mind over to such a sentient program?  Imagine the possibilities of creating and ‘educating’ such a piece of software to the point at which your consciousness could wear it like a glove.  From being in a situation where the original mind looks on his or her copy and appreciates the difference, will it ever be possible for that conscious mind to be moved in to that copy, endowing the sentient software with the self awareness of the original mind, so that the mind is aware of it’s existence as a human mind when it is in the software?

Such electronic immortality is (I hope) likely to be science fiction for a very long time.  The ethical, eschatological and moral questions of shifting consciousnesses around are legion.  Multiple copies of minds?  Would such a mind be aware of any loss between human brain and computer software? What happens to the soul?

It’s an interesting view of a possible future  for mankind, to live forever in an electronic computer at the cost of becoming less than human?  And for those of us with spiritual beliefs, it might be the last temptation of mankind, to live forever and turn one’s back on God and one’s soul.

You write one article about Appliance Computing and the following morning this BBC story pops up – Laptop Launched to aid computer novices’.  The ‘Alex’, a Linux based laptop, is aimed at people who’re occasional computer users and comes with an Office suite, mail, browser, broadband connection and a monthly fee.  In other words, a PAYG laptop.  There’s nothing new about this; a number of Mobile Phone Companies offer mobile broadband access packages that include a Windows laptop, and in the recent past there have been a few occasions when companies have attempted to launch similar schemes, sometimes backed with advertising.

I say attempted, because they’ve tended not to work, and I’m not at all convinced that this one will be any more succesful.  The company’s website describes the package available here,and to be honest it does seem rather over-priced for what is a modified and stripped down Ubuntu distro.  And one that seems to only work when your broadband connection is running.  It’s a good business model provided that you can get people to buy in to it.  There’s a review of the package to be read here.

Now, first question – who is the market place?  The Broadband company who’ve developed this package claim that almost 25% of people in the UK with computers don’t know how to use it.  really?  That I find difficult to believe.  Most folks I know – across the board, non-techies, techies, old, young, whatever are quite au fait with using their computer to do what they want to do.  There may be aspects of computing that they don’t get, in the same way that I don’t ‘get’ iTunes, for example, or the intricacies of computer or video gaming, but I know no-one who’s bought a computer who doesn’t make some use of it.  Perhaps that 25% didn’t really want a computer, or have ended up with one totally unsuitable for them?

If the market sector is this 25%, then what proportion are willing to buy a £400 computer and a £10 access fee?  Apparently a ’sofwtare only’ option that can be installed on older computers and that will simply cost you the monthly fee is out in the next couple of months, which might allow people with older computers to make use of them.  the package comes with 10Gb online storage; does this mean that local storage is not available?  If so, what happens to your data if you don’t pay your monthly fee or cancel your subscription?  To be honest, that sounds like something of a lock-in akin to Google Docs.  According to this review, on stopping the subscription, the PC effectively ‘expires’ – along with the access to your data.

I’m afraid that from what I can see I’m not impressed with either the environment or the limitations offered.  One of teh things that you learn after a while in putting together user interfaces is that people who come in knowing nothing soon gather skills and in some cases start finding the ’simple interface’ that originally attracted them to be a limitation.  With a standrad PC, you just start using more advanced programs and facilities; with something like the Alex you’re stuck with what you’re given.  And whilst you could just buy a PC, and ask someone to set it up ’simple’ for you (to be honest, it isn’t THAT difficult with a Windows PC, Mac or Linux machine if you ask about) and use a more ‘mainstream’ machine, you’re still stuck with your data being locked in to the Alex environment.

The solution to this problem is perhaps to look at front ends that sit on existing platforms, rather than work to further facilitate the move towards a computer appliance future split between a large number of manufacturers who lock us in to proprietary data stores.

Well, the fuss over the launch of the iPad has died down somewhat – it wasn’t the Second Coming or the Rapture, the world didn’t suddenly turn Rainbow coloured (not for me. anyway) and the Apple Fans have gone quiet.  So, perhaps it’s time to take a few minutes to think about what the iPad might mean in the future.   This is an interesting viewpoint - that the iPad could be the first step on the road to the computer as a true ‘appliance’. 

In some ways, this might not be a bad thing – after all, it’s the way that all technology has tended to go over the years.  Take for example radio – the first radio receivers required the operators to be reasonably knowledgeable about the equipment, and in some cases be able to build and maintain their own equipment.  Radios required large outside aerials, and I clearly remember a ‘Home Maintenance’ book that my mum had that dated from the 1920s that had great amounts of information about how to service your wireless were it to go wrong.  By the 1930s they were more self contained ‘black boxes’ – OK, self contained walnut wood boxes – and by the time we hit the 1970s little radios were being given away as children’s toys.  We’re moving along that path with computers; when home computers first became available then you were expected to want to write some of your own programs or even build the machine, then published software came along, then we have the time we’re in now when very few people write their own software at all. 

But the thing is with contemporary computers is that you can still write your own software if you wish to; you can go out, buy a copy of VB.NET, download Python or PHP or Java and with some application write your own software.  And if your computer doesn’t support media you want to view or listen to, you can just get a piece of software installed that will do the trick.  And if you want it to do something totally new, you can again find an application somewhere, or write your own, or commission someone else to write it for you – all without fear or favour.

If computers follow the logical progression, then we could expect to see them move on to a stage of development where they are pretty much ‘closed units’ – the old joke of ‘no user serviceable parts’ will be very applicable.  Think of the computer of tomorrow as being a little like your smartphone or a digital TV with Satellite TV and a DVD recorder built in; there’s content for you to view, you can save it, there may be services to buy, but you’re not going to be able to add functionality to it by producing your own code or content to run on it.

In other words, surprisingly like an iPad.  And some analysts have noted that the apparent lack of expandability of the iPad might not be a design omission, but might actually be a deliberate design policy.

Producing computers that are simply glorified media players has a number of advantages for many parts of the hardware and content industries.  To start with, if you can totally control the hardware and software environment then you can restrict your support calls; many software houses that produce applications for Windows have to have reasonable support functions in their companies because whilst their software runs on Windows, each PC running Windows is to a great degree unique, and therefore offers a near unique environment on which the application runs. 

A further point is that once you stop people from being able to put their own software on these machines, then you also prevent a lot of the issues of illicit copying.  By controlling the platform you can control the way in which the platform handles content that might be protected by some sort of Digital Rights Management software.  Indeed, it’s not too difficult to imagine a situation in which the functionality available on the unit can be remotely enabled and disabled  based on the payment of licenses or rental fees – similar to the way in which satellite TV receivers can be activated or de-activated remotely.

The Appliance Computer has a lot to offer manufacturers and content providers; it locks users in; it protects content; it makes the equipment more reliable.  But it also eats away at the very foundations of what has made so many software applications possible – the ability for anyone to write their own software.

Don’t let Appliance Computing remove the freedom to compute.

Back in January the UK Government opened a web site up that was described as “a one-stop shop for developers hoping to find inventive new ways of using government data”.   The site, http://data.gov.uk/, aims to pull together government generated data sets in a form that application developers can use to create ‘mashups’ of data from different sources of public and private data, create map based information from the data, etc.  In other words, the idea if to open up public data for private use.

I was pretty excited; professionally I’ve used some public data in the past and acquiring it is usually quite hard going.  Even if you know where to find the data, it’s not easy to just grab and download, and then it comes in various formats that need pre-processing to make useful.  So, I was pretty excited when I heard about this project.  I wouldn’t go so far as to say that my nipples were pinging with excitement, but there was definite anticipation.

So….my thoughts.  Bottom line for me at the moment is ‘Sorry chaps, sort of getting there but there’s a long trail a-winding before you reach your goal’.  Now, this may sound rather churlish of me, but allow me to explain….

Nature of data

First of all, a lot of the data on the site has been available in other places before now – however it is at least now under one roof, so to say.  The data is also available in disparate formats, like CSV files, etc.   The data is also pre-processed / sanitised – depending upon how you want to view it.  In some cases the data is in the form of Spreadsheets that are great for humans but dire for automated processing in to mashups.  The datasets are not always as up to date as one might expect; for example, on digging through to the Scottish Government data, I found nothing more recent than 2007.

Use of SPARQL and RDF

Although the SPARQL query language has been implemented to allow machine based searching of the site to be done, the data available via this interface seems to be pretty thin on the ground AND, to be honest, I’m not sure that the format is the best for the job.  SPARQL is a means of querying data that is represented in the RDF format to search what’s called the ‘Semantic Net’ – a way of representing data on teh Internet that is more easily made meaningful to search tools. But for a lot of statistical data, this isn’t necessarily the best way to search for data,  and the SPARQL language is not widely used or understood by developers.

No API

There’s no API available such as a Web Service to get at the data.  The site acknowledges this and states :

“The W3C guidance on opening up government datasuggests that data should be published in its original raw format so that it’s available for re-use as soon as possible. Over time, we will covert datasets to use Linked Data standards, including access through a SPARQL end-point; this will provide an API for easy re-use.”

I think this is a rather facile argument.  Apart from the data not being that up to date, one can surely publish the contentof the data raw – i.e. no numerical alterations – whilst still making it available via a SOAP, JSON or other similar API that more developers might have experience of and access to.   As it stands it just seems that some of the time spent on this project could have been spent in getting the data in to a format that could be served up in a consistent format to a wider range of developers.

This current interface – wait for the heresy, people – may be wonderful for the Semantic Web geeks amongst us BUT for people wishing to make widescale, real use of the data it’s NOT the best format to allow the majority of non bleeding-edge developers to start making use of the data available.

Summary

This is an early stage operation – it is labelled ‘Beta’ in the top right of the screen, and as such I guess we can wait for improvements.  But right now it just seems to be geared too much towards providing a sop for the ‘Open Data’ people rather than providing a widely usable and up to date resource.