Science for SEO

Peer review is a very important process in any profession. It ensures that certain standards are met, and that the quality of work being done in the profession is of a high standard. When that standard is not met, reviewers or peers are entitled to reject the work outright, or to ask for further research or work to be done. After that the work can be reviewed once again. This process means that nothing average or passable makes it through. If such work is accepted and becomes common practice, standards slip and the profession as a whole suffers.

The point of this post is to look at peer review in the seo/online marketing world and trying to gain some insight into how it works in that field and also if it’s done in a relaible way right now.

Definition:

“referee: evaluate professionally a colleague’s work” – WordNet

“Peer review (also known as refereeing) is the process of subjecting an author’s scholarly work, research or ideas to the scrutiny of others who are experts in the same fiel”d (wikipedia)

“The review of an article or other publication by a group of experts on the topic. Used by scholarly publications as a way of determining whether an article should be accepted for publication”. (USG)

“The procedure by which academic journal articles are reviewed by other researchers before being accepted for publication.”(Cornell)

“A system using reviewers who are professional equals; a process used for checking the work performed by one’s equals (peers) to ensure it meets specific criteria.” (NYU)

Basically, we need to show our work for approval to others who are also professionals of the same standard in the field. This standard must be be the highest standard possible.

Who is your peer?

In academia or in research your peers are those who wrote your textbooks, those who are active in the field and consistently publish or make available work that is regarded as cutting edge, novel, thorough…At the end of my PhD years, all I have to do is look at the references on my papers. Those names are those of my peers.

In the SEO world, it’s not so clean-cut is it? Who would you choose to be your peers? Would you be comfortable presenting a site you had worked on to a group of say 12 highly regarded professionals in a public forum?

If it were me, I would not be expecting to be patted on the back and told it was all wonderful. A good result is when there is discussion over something you have done or found out. This is about finding out what you could have done better, and also what you have done plain wrong.

You’re supposed to come away with a good idea of how you can improve your work. If it gets past those 12 reviewers, then it gets presented to everyone else in the profession. This ensures that th standard of information put out there is high.

There are always several reviewers because it avoids the result being subjective

Our profession:

The journals in the seo profession are actually posts on blogs. Some are well regarded, and the news that comes from them is considered to be of some authority. The problem with this divided model is that hobbyists, beginners, amateurs and so on are taking part in that peer review. This does not say much about the standard of the work being published.

It can be difficult to judge which research claims, techniques, and work should be taken seriously. When you are a reviewer, you can’t just say that you liked it or didn’t, you have to explain why you have a particular opinion, and back it up too.

There are places like SEO Scoop, SeoMoz, Search Engine Journal and others that carefully review who they have posting on their sites. I would however say that William Slawski is someone to be reckoned with, and David Harry and Jill Whalen for example are as well. Would you say that these people would be some of the most informed and active in the business?

I think so, but each also have their own specific areas of expertise as well, which is fortunate! There are many specialisms in our profession from link building techniques to understanding search engine algorithms.

In research:

“Research is a competitive endeavor. Researchers are accustomed to constant assessment: any work submitted even, sometimes, invited is peer reviewed; rejection is frequent, even for senior scientists. Once published, a researcher’s work will be regularly assessed against that of others. Researchers themselves referee papers for publication, participate in promotion committees, evaluate proposals for funding agencies, answer institutions’ requests for evaluation letters. The research management edifice relies on assessment of researchers by researchers.” (The ACM)

Maybe this is why we don’t have “rockstars” in science. Anyone can be rejected, and it’s true that if you’re doing something new, you may be judged to not have quite enough to contribute something useful. Yet. That’s the whole point, go back, make it better, and try again. Einstein and Edison were often rejected at review.

An art and a science:

SEO is sometimes described as and Art and a Science. For something to be a science it has to follow the scientific method for starters. If SEO really does want to be considered a serious science, then I see no reason why it should be spared the scrutiny and high standards set in other scientific professions. We can’t call it a science and then bumble around producing whatever we like to define as research. You might as well call it philosophy or art history.

How do you become a reviewer (peer):

- You’ve contributed pertinent and useful research/work

- You’ve passed peer review consistently

- You’ve got the appropriate expertise

- Have passed peer-part (assessment for reviewers)

- You have also probably also been rejected at peer review countless times in your career!

Peer review can be:

- single blind (reviers don’t know the author)

- Double blind (reviewers and authors don’t know each other)

- Triple blind (editors, reviewers and authors don’t know each other)

Acceptance rate:

The acceptance rate in computer science journals is low:

- SIGIR 17%

- KDD 12%

- IAT 19%

- IJCAI 24.7%

- SIGKDD 17.9%

- ECCV 4.4%

(This will also depend on what field you work in. Some conferences will accept more papers.)

There is no shame in being rejected, it will happen often if you are active in research. I have been through this myself, and so has every other scientist I know.  In fact Bob Carpenter who is a well known research scientist wrote an entire post about how he failed peer review and what he learnt:

It’s called “Lacks a Convincing Comparison with Simple Non-Bayesian Approaches” and that’s what peer review concluded. He makes available the feedback, and it’s an inspiring post.

The problem:

In SEO everything is published, 100%. All you have to do is set yourself up with a blog and start writing stuff. SEO’s though don’t always write for the love of it, or for the contribution they can give the professional community. They write to attract customers by sounding knowledgeable. Customers are not experts for the most part and get drawn in. Also the nonsense that is publish accross a wide number of blogs is detrimental to the profession.

New seo’s are faced with the horrible task of having to read lots and lots of rubbish as well as the good stuff. This is a decider. If they’re talented, they’ll quickly learn what is rubbish and what is gold. Clients concern me less in this post at least. What I’m concerned about is the standard of information available for education in SEO.

I’m not saying that peer review as it is modeled in science should be directly applied to the SEO profession, but having some kind of peer review process will help those starting out in the profession and those who want to learn from others and contribute.

Related Posts:

• Masters Degree in Social Media
• Can gibberish rank? Why Certainly.
• The end of SEO: an interview
• Using the Science of Social Proof in SEO
• What Google owes you: nothing