The One Thing Needful

The weather has been very nice here in Guri today. I’ve just read two articles predicting that this summer will be good for barbecues in Britain.

The Met Office apparently predicted the same thing last year and got it completely wrong. My guess is that they will be wrong again this year. My thinking is not really very scientific, just a willful and immature contrariness coupled with a lifetime of disappointing summers in England.

The sun is at the bottom of its sunspot cycle as well:

Noticing correlations between seemingly unrelated data has always been a rich source of new knowledge. I understand that correlation is not causation, but the reverse does appear to be true. If the expected outcome of a hypothesis does not coincide with the recorded data, then the hypothesis should not be trusted or a least questioned thoroughly.

I watched a lecture last night on Wolfram|Alpha

I imagine that when this goes live, there will be a lot of bloggers who use the data to show relationships that are very questionable. The old line about statistics being used like a drunkard uses a lamp post, for support rather than illumination, will no doubt apply. I am interested to see what all the amateur climatologists who have sprung up since climate change has obsessed the world will come up with. Like everything in life, more than 99% of it will be banal and worthless. It’s the rest that’s intesting. Luckily, the internet provides incredible filters for sorting through enormous amounts of information for finding the most interesting things to think about.

One of the problems that I can see with W|A is that it is closed and proprietary, although users will be able to access the data for free. The company may be able to run profitably. The search engines have done well being run this way. As far as I know, this is a new sort of service that has not been tried before. I hope that the likes of Google, Yahoo! and Microsoft try to build rivals to this quickly.

I hope that the diverse open source communities of the world try to come up with something to compete with it. At this stage, it is clear that a lot of human intervention is needed to get the data into the system. Wikipedia has shown that this is something that people are willing to give up their free time to do. Providing the vast computing resources for an open source version of this project is also a hurdle. I would certainly consider giving up some of my computer’s spare cycles for a distributed and open source version of this came along, and I am sure that I would not be alone. However, the popularity of Google compared to, for example, Yacy shows how difficult it is for these sorts of things to be fully open.

Another development that I hope that W|A brings about is to force universities and other publicly funded research institutions to do more to make all of their experimental data available in machine readable formats. A single open source project that can absorb all of the scientific, engineering, technical, sociological, economic and financial data in the world might not come about, but lots of smaller projects that each try to solve part of the problem might. No doubt, such projects would take pains to cooperate with the other projects.

What developments occur in the next few years in this field are the subjects of anyone’s guesses. As Dickens pointed out in Hard Times,

facts alone do not make a person educated or complete. However, I imagine that everyone being able to ask lots of little questions involving data and relationship between them will have a similar impact to that of Google and Wikipedia. In the past, if we wondered to ourselves “What is the name of the Aztec sun god?” we might not have bothered to go to a public library or even take an encyclopedia of the shelf to find out. Now, we are much more likely to find out about

because it takes such a short amount of time with our modern tools.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s