Joe's Jottings

I recently commented on whether Web 2.0 had ‘jumped the shark’ in terms of strange applications, and also remarked on whether the biggest threat to online privacy was ourselves.  However, I don’t think I was prepared for Blippy - a web site that allows you to share details of products and services that you’ve bought via different routes- Amazon, iTunes, Mastercard, Visa, etc.  Now this I find very weird and, dare I say it, slightly compelling viewing.  The system was on an invite only basis until late last year, but now seems to be open to all comers.

It’s sort of like the online version of being in the check out at the local Morrisons Supermarket and peering in to the basket of the person next in the queue.  I took a look on the site and randomly selected a user.  From 5 minutes looking at their recent transactions I was able to work out that they either lived in San Francisco and had recently traveled to New York (or vice versa), that they had a baby / toddler, that they’d done some DiY recently and various other aspects of their lives based on the purchasing records that they were willing to share.

Now, there’s nothing here that falls in to the ‘blackmail’ category, and I’m quite sure that people using Blippy would keep their ‘special’ purchases off of the system, but to be honest I do find it a rather strange thing for someone to want to do.  Maybe I’m just old.  It wasn’t long ago that people were protesting about the use of RFID tags in goods to track our shopping behaviour in shopping malls; now we seem to be falling over ourselves to give the information away for free, along with the amounts spent!

The Blippy owners said last December that they weren’t yet sure how to monetise the project.  Well, I think they were being rather disingenuous because it appears that Blippy have joined forces with the people who bought you the (now scrubbed) Facebook Beacon project.  And then there’s the very direct link between the data that Blippy collects and what has been called the ‘database of intentions’ - data that allows the prediction of buying activity based on past behaviours.  You have a large collection of data on buying habits; you have an individual with a recent history of purchases; it’s a relatively trivial software process to take the individual’s list and use the collection of data to predict what other items might be of interest.  You can then contact businesses in those market sectors with what is at least a warm prospect for a sale.

Blippy is again an interesting example of how people are willing to put lots of information in to this ‘database of intentions’.  Their lack of concern about their own privacy impacts upon us all by making it easier to predict our behaviour even if we only ‘leak’ small amounts of data. 

Are we heading for a ‘speculative bubble’ effect in the portions of the media and IT economy that are tied up with Social Media and Social networking?  Regular readers will know that I’m something of  a cynic about the importance of Social Media and Social Networking; whilst it’s clearly an important aspect of marketing for the future, I am rather concerned about the importance that the ‘industry’, if we can call it that, applies to itself.

Take the following article, from a Canadian newspaper, for example.  Real world businesses are still doubting the importance and relevance of Social networking and Media to their ongoing business activity.  Unsurprisingly, the practitioners are effectively saying ‘Ignore us and you’re doomed, doomed I tell you! Doomed!’  Now, some of us who were out of school in the late 1990s can probably remember the comments made by a number of folks with possible vested interests that anyone without a web presence would be out of business within 5 years.  What actually happened was that within 5 years a lot of web companies were out of business, and many businesses with no web presence or strategy whatsoever were going along quite happily.

Just because you find something sexy and interesting doesn’t mean it’s important; passion is a wonderful thing to have but one also needs to be pragmatic along with it.  In a recession, surely any business is likely to be most interested in keeping existing customers and is likely to be playing a ‘safe hand’ with it’s resources.  It’s unlikely to want to adopt techniques that it’s customers may not actually be aware of or care about.  There is absolutely no point in extensively using social media and social networking technologies if your customers are not aware of them!  It’s rather like advertising in French when you have no one in France reading the ads!

The arrogance of Social Media zealots in assuming that real businesses are lagging behind is astonishing; surely Social Media / Networking is a support function for most companies, part of marketing and advertising.  It’s not as disruptive a technology as the web itself is, and shouldn’t be treated like it is.  Take a look at this definition of a bubble - the phrases that immediately struck me were “emerging social norms”, “positive feedback mechanisms”,”they create excess demand and production”.  I think it’s fair to say that we’re seeing all these effects.

In addition, it’s difficult to value the Social networking / Media market place and individual services and companies within it.  And then we have the other issues often associated with bubbles:

Moral Hazard- how much of the market place is supported by ‘other people’s money’ – if supported mainly by VC capital then companies may take risks that they wouldn’t take with their own money.

Herding- the more folks who say it’s good, the more the markets are likely to follow.

All in all….I think a ‘correction’ to the emergent Social Networking and Media sector is likely.  And then we can get back to realistic use of this technology as part of an integrated marketing strategy for businesses.

The Greeks had the Oracle at Delphi; we have consultants. A recent comment on Twitter suggested that the Apocalypse would be heralded by everyone on Twitter being a ‘Social Media Expert’ – sometimes this is how Twitter feels, with everyone who starts following me appearing to be the online equivalent of those guys who clean your car windows when you stop at junctions…

It set me thinking – is the whole Social Media field (that part of the media / Internet that deals with interactive and group based applications and developments, like Facebook, Twitter, Blogging, etc.) too young to have real experts?

Years ago I worked with a guy who hated the word ‘expert’.  His take was that an ‘ex’ was a has been and a ‘spurt’ was a drip under pressure.  Which sort of summed it up… A more widely heard belief in IT is that an expert is someone who’s read 3 pages further in the manual than you have….

Whilst I wouldn’t go that far, I think that at this stage in the social media game it’s too soon to tell what is true expertise and what isn’t.  It’s similar to the many people who thought they were successful property developers during the UK housing boom; the market added value; they did nothing, and when the market slipped the dilettantes got whacked.

At this stage in the game I believe the best policy to be to encourage the client to adopt the generally stated ‘best practice’.  This may be a conservative approach, but it allows the client to develop their social media expertise organically and as part of their normal marketing strategy.  Having said that, a recent discussion with a practitioner in the field suggested that we may not yet even have the maturity needed for ‘best practice’ to have evolved, so that approach may not yet be of value.

So, what is the answer?  Perhaps it’s time to stop going on about Social media as a separate discipline and start looking at the technologies and techniques it encapsulates as being just different aspects of existing business practices.  For example, a company may use Facebook to establish brand awareness and communicate with customers.  OK – that’s a new approach for both Marketing and Customer Care to learn.  Someone else may be using a blog; that Public Relations / the Press Office.  Twittering to announce special offers?  Sales, anyone?

The technology is new, and there will be a steep learning curve, but the business processes being supported are the same as we have seen in businesses for the last 60 years.  Any technology or technique applied to a business must surely have one objective; to ultimately increases the value of the company or organisation to it’s stakeholders.  We’re just using new methods, which means that we’re going to have to learn them.  Most of these technologies are so cheap to implement (and are usually pretty straight forward to set up in the first instance) that perhaps we just need to try a few different approaches out and take note of what works for us, and then implement what works, rather than expect ‘expert’ guidance to solve all our problems.

In the classic comedy series, The Hitch Hiker’s Guide to the Galaxy’ the only guidance offered to cultures that hadn’t yet mastered the new technology of fire was ‘Keep banging the rocks together’.

Good advice.

I’m old enough to have used an address book and still have a Rolodex on the phone table.  When I actually sit down and think about the people with whom I have reasonably regular ‘quality’ contact in a 3 month period, either electronically or face to face, it probably amounts to no more than a hundred or so.  I guess it’s safe to say that in the world of networking I’m a ‘quality over quantity’ sort of fellow.  I’ve never been a great collector of large numbers of business cards or people details – collections are fine for stamps, coins and locomotive numbers but are kind of creepy for people.

Back in the late 1990s / early 2000s I used a networking site called Ecademy – I stopped after a while because it seemed that people were making contact with you purely from a sales oriented viewpoint.  Allow me to explain – if I’m interested in AI, and someone brings something to my attention that’s even vaguely related to the field – that’s cracking!  That’s exactly what I’m there for – and hopefully I’ll be able to reciprocate.  On the other hand, if someone steams in with a ‘Hi, I’m Fred, I’m in marketing, blah, blah, blah’ I get the feeling I’m receiving a boilerplate message which is likely to end up as a boiler room selling attempt.  The site seemed to encourage numbers of contacts over quality – and that’s one of the reasons why I eventually jacked it in.

I’ve noticed in recent days that I’m being followed by people who are following thousands of others.  And the odd thing is most of them appear to be selling something that is as relevant to me as a comb to Sir Patrick Stewart.  The ‘Bio’ of one such follower (soon to be ex-follower in my daily purge) – “A Business Dedicated to providing free online MLM training videos, articles, books and webinars”.  If I received an email like this I’d call it spam – pure and simple.  I know that Twitter has policies around spam, but my point is that most folks following 20,000 people seem to be in the MLM, ‘sales and marketing’, ‘social media consultancy’ sort of areas.  They’re cold calling – they sure ain’t networking.

Bottom line – there is NO WAY, realistically, that the content generated by the 20,000 people these bods follow is ever registering in any meaningful manner with these people – I assume it’s simply being harvested electronically and searched for keywords that might suggest a sales lead. 

Joe’s categorisation of Twitter users…

1. Vast number of followers, smallish number of followed – publisher / celeb.
2. Vast number of followers, vast number of followed – probably sales / mass marketing
3. Smallish followers, large number of followed – probably spammer
4. Smallish followers / smallish followed – personal / business networking

OK – it’s not a brilliant classification but it works for me.  Just watch out if you’re in category 2 or 3 ‘cos I’m binning you!

 Whilst I was drafting this yesterday, I came across this piece on the same topic:  http://juliorvarela.wordpress.com/2009/12/28/when-twitter-numbers-are-meaningless/

Don’t get too hung up on your numbers on Twitter.  If you’re following lots of people, just check WHY.  Do they add value to your day?  Amuse / entertain you?  Educate you?  Guide or enlighten you?  If not, ditch ‘em.  And those following you – just take a look at their numbers and think about what I’ve said.

And I hope you don’t chuck me off your lists.

I appreciate that this is likely to be one of those posts that will annoy some folks, but, here we go. A couple of days ago I was invited by two or three separate people to join Facebook Groups and sign petitions against the UK Digital Economy Bill. Now, I believe that we need a Digital Economy Bill like I need a hole in the head; what we actually need is less red tape and a more hands off approach from Central Government to let entrepreneurs get on with the job without requiring a chit from a bureaucrat to go to the toilet. However, I didn’t sign any petitions or join any groups; why? Because the total pre-occupation of everyone was whether it’s right to have a legal structure in which it’s possible to remove or restrict someone’s Internet access if they’re guilty of or accused of sharing copyright materials. 

There are some truly stinky parts of that bill, like there are with most pieces of New Labour legislation – but I want to look today at the filesharing issue in a wider sense.  

Let’s start with the ’The Internet is an essential part of modern life and it shouldn’t be possible to be cut off from it.’  argument.  Bollocks. Water is essential. A roof over your head is essential. Electricity and Power are pretty important. And yet you can lose all of these by simply not paying your mortgage and utility bills, ultimately resulting in being thrown out of your house and living in a cardboard box. If that’s possible, why on Earth does anyone living in the real world and not in Second Life think that your Internet connection should have some sort of God-given right of protection? And as for essential – quite a few people manage quite happily without an Internet connection, thank you very much.

In the context of this argument, what the Internet HAS done for some people is to allow them to access, free of charge, a large tranche of media that they would have had to borrow off of their friends 20 years ago.  The actually physical act of borrowing and copying probably restricted copying in that few people had the brass neck (or stamina) to borrow 20 CDs from a friend and copy them in one sitting, for example.  The Internet is, no doubt, an essential tool for people in ripping off media.   I’ve heard most of the arguments, and there are some good points on both sides of the debate. Rather than re-hash the usual debates, here are a few observations to provide food for thought for anyone approaching this argument with an open mind.

We have a number of open licenses like Creative Commons, Open Source, etc. Why shouldn’t it be possible, therefore, for a creator of software, film or music put their material out through a proprietary license that requires payment to use the material? After all, I am restricted with what I can do within the GNU licence, for example; I have to allow other people to copy the material – it’s part of the GNU licence and I am more than happy to play along with that. If you want to have a recording of a film, then you should surely, by the same logic, abide by the license of that film. Typically pay money and don’t copy it. No one is forcing you to adopt that license – you have the choice not to buy the DVD. Similarly with Open Source; if I don’t want to pass on my source code to other people, I choose to write my own code and not use the GNU license. I can see no difference.

I’d be interested to know how many people who regularly engage in file sharing of copyrighted materials have ever created something non-trivial and original and have tried to sell it. I may be wrong but a quick poll of folks I know (outside the professional digerati) would indicate that the answer is ‘not many’. Perhaps to have experienced running a small business that deals with created works like software, or publishing a book, etc.  might change people’s attitudes a little.  Same argument as above; if I wish to place material I create in the public domain or under GNU or CC, then it’s my right to do so, and your right as consumers of media to take advantage of my action. If, on the other hand, I choose to publish under what I might start calling the Dr Johnson License (‘Only a fool writes and doesn’t get paid for it!’) then you have a right to not purchase what I produce, and I have the risk of not seeing my works go out in to the world. If you make an illegal copy of my work, then I have a right to pursue you to make you abide by my Dr Johnson License – just as the creators of software licensed under the GNU license can pursue someone who breaches that license under the law.

I’ve heard the argument of ‘try before buy’ and it’s a good one.  Which is why I have a Spotify account – legal music downloading, free of charge, some advertising, no actual physical ownership of the music outside the service even if I pay a monthly fee (which removes the ads).  I have to say that I find the latter a pain in the rear – there are some advantages to having physical ownership of the files – but then again, it’s THEIR license and I choose to go with them or not.  It’s an imperfect system.  If you download stuff on a ‘try before buying’ basis, then perhaps the case could be made to allow you to download any piece of media with a built in duration, that renders the media unplayable after, say, 30 days unless you buy it – the act of purchase then generates an unlock code.  The argument has been expressed that downloaders tend to be purchasers of music – again, I’m yet to be convinced.  Anecdotal evidence from people I know would suggest that whilst they may purchase material, the value of that that they download illegally vastly exceeds that which they purchase.

To all the freetards, can you explain why wrong for me to put the material I’ve created out in to the world, want to be paid for it, and take action when I’m not?  I am exercising my personal, creative freedom to want to be paid for my work.  If my view of my own value is wrong, don’t pay me – but don’t copy either. 

I was reminded earlier today, whilst reading a book called ‘Life 101′,  of a useful piece of advice from one of the more under-rated personal development gurus of the mid 20th Century – Sergeant Ernest Bilko of the United States Army.  Let’s listen to what he has to say on the topic of a three letter word…

You said, “but.” I’ve put my finger on the whole trouble. You’re a “but” man. Don’t say, “but.” That little word “but” is the difference between success and failure. Henry Ford said, “I’m going to invent the automobile,” and Arthur T. Flanken said, “But . . .”

And so it was, according to Bilko, that Ford remains in history whilst Flanken doesn’t even make the footnotes.

‘But’ is indeed one of the words in the English language that fills me with trepidation.  During my years in consulting, hearing someone agree with what you were proposing, and then adding the word ‘…but’ (complete with pause) to the end of a sentence was the equivalent of telling me that I was as likely to get cooperation as I was to win the Nobel Prize for Physics and Literature in the same year.

There some occasions when it’s valuable to pull someone up short before they thunder off and implement some plan or other that at best can be described as ‘unwise’.  And there are times when the use of but can provide a useful reminder for folks that their master plan requires a few tweaks before it will work properly.  But often ‘but’ is used as a prelude to a road-block.

Rather than ‘but’ I now try and use ‘and’ or ‘or’ instead of ‘but’ – then rephrase the part of teh sentence after the old ‘but’ to look towards solutions.  For example:

I’d like to buy a new computer, but it costs too much.

becomes

I’d like to buy a new computer, and in order to give me time to save the extra money, I’ll put the purchase off for a month and see if I can do some overtime in the meantime to help raise the extra cash.

The first sentence becomes, in the but-less second sentence, an intention with a timescale and a partial solution to the problem of money.  As the guys at Honda say, ‘and’ is a great little word – it opens up opportunities for solutions, rather than closing things down.

Don’t be a but-nik!

One of my interests is in GIS systems – Geographical Information Systems – and other aspects of computerised and online mapping.  Thanks to Googlemaps, it’s been possible for developers to create map-driven applications for nothing – Google allows access to their mapping infrastructure free for many applications, and it’s brilliant.  To anyone who hasn’t taken a look or had a play, have a look at Google Maps and for you programming types out there, take a look at the Google Maps API.

Now, what really peeves me as a UK citizen is that our own Ordnance Survey – the folks who make maps – haven’t got any facility for getting hold of mapping data free of charge.  I am aware of a rather scrappy ‘trial set’ of data that is available for use with GIS systems, but honestly – the OS was traditionally funded by the UK Government and it is only in recent years that it has been spun off.  It should not be beyond the capabilities of the current Government – who’ve always whined about innovation and creativity being a driving force of British business – and the OS to make available a system similar to the Google Maps one using UK Centric OS data, at negligible cost to software developers and end users, to actually make it easier for the development of geographically based applications on the Web, on the Mobile Internet and on our desktops.

But it hasn’t happened yet.  And this morning I find out about the ‘Geovation’ project - a project to attempt to generate innovative ideas based on the use of geographical data and concepts.  Hey, it’s supported by the OS!  I can see nothing on the site that suggests that there’s any OS data available to play with – indeed I think the only data set mentioned is Google Maps!

To be honest, this is shaping up to be an astonishing lost opportunity for the Ordnance Survey – they could have leveraged this project by making data or even some sort of API available at a reasonable cost for small businesses  or zero cost for non-commercial development and research.  It doesn’t look like it’s going to happen – I get the impression they’re going to lurk around picking up good ideas from people and then take them back and see what money they can make from them.

I may be wrong on all counts – I genuinely and sincerely hope I am, and that there is a nice, cheap, API and full UK dataset out there waiting to support companies and individuals looking at the Geovation Challenge.  Why do I think there isn’t, though?

Hmmm…this is becoming WA corner recently - take a look at my previous piece here.  I was less than impressed with the technology and considered it either over-hyped or released too early in to the world.  However, I did hope that as time progressed there might be improvements in the results set returned and, more importantly from a developer point of view, an API published that would allow developers to build new applications to stretch and maybe improve WA.

So, this week an API was announced for Wolfram Alpha on the company’s Blog and I was pretty excited about the prospect of trying out a few things.  Despite my grumbles about the results returned, I was hopeful that with a suitable API encouraging third party developments, the underlying technology and data sets at WA might see an improvement.  My hopes survived for as long as it took me to start reading the small print – in particular this little document, the price list.  Now, I’m aware that WA has cost money to develop but to charge for developers to make use of teh API seems to be one of the dumbest and most counter-productive things they could do.  There are some ‘pioneer grants’ available for the developers, but I get the impression that these are still likely to involve shelling out money.

Google do not charge developers for use of the API until you start using the API in ‘closed’ systems and with a large number of calls.  They certainly don’t charge you during the development cycle – they have more sense.

Now, let’s assume I wanted to develop an API based application for WA – what we in the trade call a ‘proof of concept’ model – i.e. something that proves whether or not the bright idea that we sketched out on the back of a beer-mat in the pub will actually work.  How many requests might I get through to develop such an application?  Well, the other day I wrote some code to retrieve data from a Postcode / Geocode system’s API.  Now, this was a VERY simple application – send a Postcode, retrieve a list of addresses, send a code number, retrieve a full street address with map reference.  Let’s say 2 calls to the remote API for something very straight forward.  During code development and ‘in house’ testing I made about 30 or 40 API calls.  Now, during more formal testing on the client site that’s going to increase somewhat – probably in to the low-hundreds.  And this is for a problem with a well defined structure, with a finite returnable answer set – i.e lists of addresses, a single address or nothing at all, all in a set, predictable format. 

By the very nature of the sort of problem that WA has been set up to deal with, the problems passed up via an API are unlikley to be as well defined and the results set returned is also unlikely to be as simple to deal with as my addresses.  When I did some API work with Google for a client I found I was generating hundreds of API calls and responses during development, let alone testing.  For WA, I’m looking at $60 for 1000 API requests, and $0.08 for each additional request beyond the thousand I initially pay for.  Obviously, I can buy a bigger bundle, but the inference is clear – it ain’t gonnna be cheap developing for the WA API. 

API developments typically involve a learning curve for the API syntax and methods of use.  This is par for the course and to be expected.  However, when the API is interfacing to a curated data set like WA, we have an additional problem of whether the data set will actually contain the sort of data that we’re wanting to get back.  And whether it will be available in the sort of format we’re interested in.  And whether the curated data is timely compared to the data that is being made available through non-curated data sets like those available via Google – or other APIs, for that matter.  Clearly, if your problem space IS covered by WA and the data set WA has available contains what you want in the format in which you want it, then perhaps the API fee is worthwhile.  But for those developers wanting to try something new out, they’re most likely to look to free APIs to test their ideas, and spend time and energy working the wrinkles out in an environment that isn’t costing them pennies for the simplest query.

I’m afraid WA have dropped the ball big time here; by charging for ALL development use of the API they’ve alienated a large source of free development and critical expertise.  Look at how Google has benefited from the sheer number of developers doing ‘stuff’ with their various APIs.  Can you imagine that happening had they charged all the way?  Hardly likely. 

If WA were to make a limited  ’sandbox’ set of data available for developers via a free of charge API, that would at least allow the developers to get the wrinkles out of their code.  The company could then charge for use of the ‘live’ WA datasets, and would have the additional advantage of the code being run against the live system being reasonably bug free.  By charging from the first line of code written, they’re restricting the development of their own product and driving people in to the arms of Google, Amazon, Bing and the like.  WA doesn’t appear to be offering a lot that is truly revolutionary; so-so natural language query interface against a curated data set.  I doubt it will be long before third party developers start producing the same from Google.

In case you’re saying, “Wolfram what?”, here’s a little reading:

http://www.wolframalpha.com/

http://www.bbc.co.uk/blogs/technology/2009/05/does_wolfram_work.html

http://news.bbc.co.uk/1/hi/technology/8052798.stm

http://www.guardian.co.uk/news/blog/2009/may/18/wolfram-review-test-google-search

http://www.theregister.co.uk/2009/05/19/dziuba_wolfram/

http://www.theregister.co.uk/2009/03/17/wolfram_alpha/

http://www.theregister.co.uk/2009/05/18/wolfram_alpha/

OK – I’ll start by announcing a vested interest here.  I occasionally write software that attempts to make sense out of straight English questions and phrases, and then by cunning trickery makes the response from the program appear ‘sensible’ as well.  So I know something about how to make software appear smarter than it actually is.  And I’m afraid that at first glance I regard Wolfram Alpha as over-hyped, under-delivering and pretty much unsure of it’s position in the world.

But, the folks at Wolfram Research score highly for getting the coverage they’ve managed!

WA is described as a Computational Knowledge Engine, rather than a search engine.  However, it’s raison d’etre is to answer questions, and nowadays any piece of software on the internet that does that is always going to be regarded by users as some sort of search engine, and the ‘Gold Standard’ against which all search engines tend to be judged is Google.  So, first question…

Is it fair to compare WA and Google?

Not really, and Wolfram himself acknowledges this.  WA is regarded by the company as a means of getting information out of teh raw data to be found on the Web, and it does this by having what’s called ‘curated’ data – that is, Wolfram’s team manage sources used for the data and also the rpesentation of the data.  This makes it very good at returning solid factual and mathematically oriented data in a human readable form. 

Whereas Google will return you a list of pages that may be useful, WA will return data structured in to a useful looking page of facts – no links, just the facts.  And a list of sources used to derive the infromation. The results displayed are said to be ‘computed’ by Wolfram Research, rather than just listed as is the case of a search engine.

Is it a dead end?

WA relies on curated data – that is, a massaging and manipulation process to get the existing web data in to a format that is searchable by the WA algorithms and that is then also presentable in a suitable fomat for review.  This is likely to be a relatively labour intensive process.  Let’s see why…

In a perfect world, all web data would be tagged with ‘semantic tagging’ – basically additional information that allows the meaning of a web page to be more explicitly obvious.  Google, for all it’s cleverness, doesn’t have any idea about the meaning of web page content – just how well or poorly it’s connected to other web pages and what words and phrases appear withjin the page.  They do apply a bit of ‘secret sauce’ to attempt to get teh results o your search closer to what you really want, assuming you want roughly the same as others who’ve searched the Google search space for the same thing.  Semantic tagging would allow a suitably written search engine to start building relationships between web pages based on real meaning.  Now, you might just see the start of a problem here…..

If a machine can’t derive meaning from a web page, then the Semantic tagging is going to have to be human driven.  So for such a tool to be useful we need to have some way of ensuring as much web data as possible would be tagged.  Or, start from tomorrow and say that every new page should be tagged, and write off the previous decade of web content.  You see the problem.

What the WA team have done is taken a set of data from the web, and massaged and standardised it in to a format that their software can handle, then front-ended this system with a piece of software that makes a good stab at natural langauge processing to get the meaning of your question out of your phrase.  For example, typing in ‘Compare the weather in the UK and USA’ might cause the system to assume that you want comparative weather statistics for those two countries.  (BTW – it doesn’t, more on this later)

The bottom line here is that the data set has had to be manually created – something that is clearly not posisble on a regular basis.  And a similar process would ahve to be carried out to get things semantically tagged.  And if we COULD come up with a piece of sofwtare that could do the semantic analysis of any piece of text on the web, then neither of tehse approaches would be needed anyway.

In a way, WA is a clever sleight of hand; but ultimately it’s a dead end that could potentially swallow up a lot of valuable effort.

Is it any good?

The million dollar question.  Back to my ‘Compare the weather in the UK and US’ question.  the reason I picked this was that WA is supposed to have a front end capable of some understanding of the question, and weather data is amongst the curated data set.  I got a Wolfram|Alpha isn’t sure what to do with your input. response.  So, I simplified and gave WA : “Compare rainfall london washington” – same response.  I then went to Google and entered the same search.  And at the bottom of Page 1 found a link : http://www.skyscrapercity.com/showthread.php?t=349393 that had the figures of interest.  Now, and before anyone starts on me, I appreciate that the data that would have been provided by WA would have been checked and so would be accurate.  But I deliberately put a question to WA that I expected it should be able to answer if it was living up to the hype.

I then gave WA ‘rainfall london’ as a search and got some general information (not a lot) about London.  Giving ‘rainfall london’ to Google and found links to little graphs coming out of my ears.  A similar search on rainfall washington to Google gave me similar links to data on Washington rainfall.

WA failed the test, I’m afraid. 

Will it get better?

The smartness of any search tool depends upon the data and the algorithms.  As we’re relying on curated data here, then improvements might come through modifications to data, but that might require considerable effort.  If the algorithms are ‘adaptive’ – i.e. they can learn whether answers they gave were good or bad – then there might be hope.  This would rely on a feedback mechanism from searchers to the sofwtare, basically saying ‘Yes’ or ‘No’.  If the algorithms have to be hand crafted – improvement is likely BUT there is the risk of over-fitting the algorithms to suit the questions that people have asked – not the general searching of what MAY be asked.

And time passes…

As it turned out, this post never moved from ‘Draft’ to ‘Published’ because of that thing called ‘Life’.  So, a month or two have passed, and I’ve decided to return to Wolfram Alpha and see what’s changed….

Given the current interest in the band Boyzone, I did a quick search.  WA pointed me to a Wiki entry – good – but nothing else.  Google pointed me to stacks of stuff.  Average rainfall in London got me some useful information about rainfall in the last week.  OK….back to one of my original questions ‘Compare rainfall London Washington’ – this time I got the London data with the Washington equivalent on it as well – sort of what I wanted.  Google was less helpful this time than back when I wrote this piece.

So…am I more impressed?  Maybe a little.  Do I feel it’s a dead end?  Probably, yes, except in very specific areas taht might already be served by things like Google and Wiki anyway.

Do I have an alternative solution for the problem?

If I did, do you think I’d blog it here and expose myself to all that criticism?

And you get monkeys.

I assume most of us have heard this phrase. It’s become almost a mantra with me in my professional life because the last 6 months have exposed me to an interesting aspect of the freelance world that I’ve not been aware of until now; the fact that there are a Hell of a lot of people out there expecting a lot of work for next to nothing!

Allow me to elaborate…I get most of my work through ‘word of mouth’ – this has always been the way and after 20 odd years in IT it seems to have worked well. But I still like to chase the odd new client – after all, nothing wilts faster than laurels that have been sat on, as they say. In many ways, the availability of Internet web sites that allow people wishing work to be done to advertise their requireents for people like me to pick up the jobs should have ade things easier, but it hasn’t.

In fact, I’m beginning to regard such sites as one of the worst things that has happened to ‘professional’ freelancers and contractors, because they have totally distorted the market. Don’t get me wrong; I’m a firm believer in market forces but these sites are actually pushing the markets for freelance development work to the brink of extinction. And this isn’t going to be a rant about out-sourcing…

My concern is that people are posting requests for work like the following:

“Develop a highly interactive and very aesthetic media review website. A good example is Yahoo! TV. The site is going to cater for commercial considerations i.e web ads. Want a site that would load fast as well.
Hence, beautiful but efficient. Must do the job. “

This is a real advert, tweaked for punctuation and spelling in two places.  Now – this isn’t a hobby site, it’s not a charity.  The poster is open in that there will be advertising and will be catering for ‘commercial considerations’.  That’s the full ‘job brief’ against which people are expected to bid, by the way.  Now, let’s assume that we can put something together like the Yahoo TV site – here and ignore the content and imagery side of things for now.  It’s got forums, photo galleries, all sorts of cute stuff.  I wouldn’t even want to try tackling it – a wise man knows his limitations, after all.  But I can guess the sort of development time – you’re looking at the minimum of 2-3 man-months here, I’d estimate.  

And the suggested budget?  £250.  Yes, Two Hundred and Fifty Pounds.  No missing zeroes.

I cannot imagine the most desperate out sourcer being willing to work for that sort of money, let alone a programmer in the UK, US or Europe.

Oddly enough I came across this today:

http://technology.timesonline.co.uk/tol/news/tech_and_web/the_web/article5483244.ece?token=null&offset=0&page=1

An article in the Times dealing with Amazon’s Turk’ project which harnesses the available time of people to do online jobs of various sorts.  Where you might be expected to work for a couple of pence an hour, if that.

Digital exploitation?  You betcha.  There are projects that rely on the good nature of people to get things done – projects where the bottom line is a better, publically and freely available service, rather than profits to corporations who can already dictate terms to much of the online world.

Some years ago I was involved in film making and there was a very rich culture of ‘No-budget’ filming, where productions were put together with no budget except for the essentials of film stock or tape – everything else was borrowed, begged or blagged.  But part of the contract was that anyone involved would get a copy of the material for their own portfolio and an on-screen credit – ‘Credit and VHS’ – as well as being fed and watered on set.  This model could, of course, be exploited but rarely was, because the world of film making was relatively insular and someone pulling a fast one would immediately find it difficult to crew-up next time around.

Perhaps we need to start being similarly watchful in the information marketplace?