Goodbye to certainty
Phil Dawes has an excellent post on problems scaling classic Semantic Web approaches to a moderately sized, messy, real world environment. I have watched him develop his solutions to these problems, the evolution of tag triples and JAM*VAT and it is very impressive. One of the lessons I take away from Phil’s work is that for things to scale in the real world they are necessarily fuzzy. If your system needs everything to work out up-front and will only handle 99.999% of cases then scaling to real world sizes quickly generates far too many problems to cope. As brittle systems (those that require certainty) grow they have to increase their complexity enormously. Building a system that can hold ambiguity without worrying about it until the information is needed means that special cases can be ignored until they have to be dealt with. Which is essentially what Phil has done by leaving the disambiguation to the user of the data rather than the source. What this does mean is that its hard to be certain about the meaning of your data. But that’s just the real world for you.