There has been a fair bit of noise about Google owing an explanation about the recent announcement
that they were considering nofollow from now on. The SEO community has called for further explanations and for information about PageRank and how Google functions. There is a feeling that people feel entitled to this information, possibly because many of us earn a living through SEO, through rankings and links in Google, from traffic from Google because of it’s huge market share. This is however largely wrong, because Google owe the SEO community nothing. Nothing at all.
David Harry wrote an excellent post called “SEOs are not criminals” where he goes on to say that they are the enemy. In research circles I am sometimes called NME as a bit of a joke, because I share the SEO information with them quite openly, but then I’m happy to expose flaws in their solutions with my tampering. In fact I also work on blocking the holes SEO can take advantage of too. I enjoy it and I’ll justify it by saying that what I’d like to see are nicely presented and information rich websites doing really well. It’s good for the user, it’s good for the engines. Meta-tags and everything in the Google guidelines support that also.
The issue with SEO is that most of the websites on the web are not optimised, which means that they will often be ranked after the optimised ones even if they are more relevant. This is because the selection is done automatically by a machine. The machine is built to do this. It is hard to tell it to weigh up the optimisation over the relevance and be fair. There have been changes though because in the past if you searched for “latent semantic indexing” you would get a list of SEO sites and not the educational sites and the official LSI site.
There is plenty of work being done in contextual analysis and in natural language understanding which is my specialist area. This alone shows that PageRank is being or in the process of being supported by more sophisticated, language centered methods. You will be looking at those and should be already and you will be excited and impressed no doubt, as well as enjoy working out how to work with that.
Oh but “Google release papers and there are patents that give incomplete information about their methods, which isn’t fair”. These papers are released for peer review because that’s all part of the scientific method. They are discussed by the scientific community, not because anyone wants to game the system or because they want to copy their work (that’s why there is a patent) but because it is every scientist’s duty to investigate novel methods. They are not published for SEOs primarily.
PageRank is discussed at length both in the SEO community and in the scientific community. The scientific community has produced a wealth of papers about how it falls short and have also proposed a lot of improvements to the method, which you can pretty much accept that Google have too. SEOs should be reading those papers, otherwise basically they are using old blueprints. I’ve given a list of papers about PageRank that you should have read and be aware of if you are looking at PageRank right now.
Google does not owe anyone outside of their organisation any explanation of what they do. If they wanted they could just drop PageRank altogether and use an astronomy technique or something. Matt Cutts is a wonderful diplomat and deals with the SEO community admirably. He is fair and provides explanations that Google really are not obligated to give. They are smart like other search engines are and opt to try and work with the SEO community because I guess you may as well limit the damage and make your peace.
Matt is heavily involved with AirWeb and you”ll notice that the topic of discussion is rather pertinent to SEO:
The Potential for Research and Development in Adversarial Information Retrieval
Selective Sampling in Active Learning for Splog Detection
There’s actually a whole workshop dedicated yearly to “Adversarial information retrieval”, which is the more elegant name for SEO 
Don’t go expecting and feeling you are due an answer to your questions. None of us are. The work of an SEO is partly to work with that, educate himself/herself and do their best to provide a clean and well optimised site. Gaming the system is not part of the job imho, but then I’m biased to an extent.
Remember: There is no silver spoon…you’re being handed one of these:
Couldn’t agree more, actually. There is an obvious, constant, tension between helping clients and creating a more information-rich web. And, the bottom line is, if SEO was easy, then everyone would do it.
I have no issue with Matt. In fact, I think he’s rather bold for always sticking his chin out in front a (mostly) rabid SEO crowd. This was a great post, and I hope that more people read it and remove that large chip on their shoulder. Google doesn’t owe you a damn thing. Get better at your job.
Hi Anthony, thanks I’m glad you liked you post. Once in a while I have a rant. I think Matt is pretty good too, and yes the crowd is rabid!
These are great comments! I started following your blog more earlier this year. I think you should be heard more often by the SEM community
You are giving out lots of great advice over here!
Thanks!
Please email me when you have a chance I have a quick question if you have a moment.