Are you looking after your robot customers?

Most people reach websites via search enquiries, using search engines such as Google, Yahoo and Bing. But how do these search engines choose which pages to suggest to you out of the more than 13.5 billion webpages contained in their databases, and on what basis do they choose to add one page or ignore thousands of very similar pages out of the billions of new pages added to the web yearly? Understanding search engines is a keystone to building an effective digital strategy.

In the simplest sense, search engines deploy small programmes called spiders or bots which trawl the internet. These robotic explorers crawl websites, and report what they find back to the search engine. Using proprietary, secret algorithms the search engines automatically analyse this data and index and rank web pages according to perceived relevance. When a search enquiry is made, the search engine returns pages of suggestions (known in the trade as SERPs) that relate to the keywords entered into the search bar, with top-ranked pages showing first.

However, this does not mean that if your keyword search was ‘books’ that a website called www.books.com would automatically be listed near the top of search results. It is also not simply a matter of repetition of keywords in a page’s content (for example “Book your books at the book-lover’s most bookish bookery”) – search engines regularly penalise websites for blatant keyword stuffing. Writing good online content that accommodates both your human and bot audiences is a fascinating art called digital copywriting, but unfortunately beyond the extent of this article.

Apart from site content, there are many other factors that affect ranking – Google reportedly has over 200 ranking factors in its algorithm – including popularity of a page (not only the number of visitors, but also the time they spend on that page), the number of external pages that link to it (not only the number of these referral links is considered, but also the reputation of the referrer site) and site download speed, amongst many others.

The search engine also considers factors such as what device you are using at the time, your geographic location; and may even list results that your friends have recommended! So, if you are driving in Soweto and entered the keyword ‘books’ into Google Mobile Search on your smart phone, the search results could include bookshops located in Soweto and give preference to websites that are mobile-friendly.

Unlike earlier search engine results pages, modern blended search results not only include relevant website pages but also relevant images, sound and video files, social media pages and even documents, all of which compete for eyeball time and clicks in the limited results visible per page. This should tell you that to compete for traffic, you should pay attention to optimising all of your online assets, not only your web pages for search.

An exciting new development in the power of search engines is semantic search, which seeks to deliver results based on context and user intent. For example, in the search phrase ‘book Cape Town’, is the user looking for a book about Cape Town, wanting to buy a book in Cape Town, or book a hotel in Cape Town? Addressing ambiguity and lemmas will become increasingly important in helping our slightly dumber bot readers make sense of site content.

Remember that rankings are not static, they shift all the time. If a suggested page in search results begins to garner more clicks, more quality referral links and social shares, it can suddenly improve its ranking. Similarly, a well-optimised new competitor page could easily knock your page from its long-standing top ranking. For these reasons alone it is worth keeping a close eye on competitor page rankings and taking appropriate action.

The way your website is structured and coded is also very important, and can impact heavily on the ability of search engine bots to ‘make sense’ of your site – another reason to choose your website developer carefully. Error-free coding, clean navigation structures, properly made robots.txt files, well-labelled page headers and considered, relevant metadata can prove just as important as well-written, useful and persuasive content.

Be aware of and avoid using so-called ‘Black Hat’ practices – these are measures which attempt to ‘trick’ the search engines into ranking pages higher than what they should be ranked organically. Employing these (even unwittingly) can cause your website to be removed from search ranking altogether, rendering it almost invisible. Keyword stuffing, content scraping (basically, stealing another sites content), certain URL redirects and masked text which is not visible to human visitors are among these nefarious tricks.
Understanding how search engines rank pages and trying to tweak a website’s organic search ranking is a technical digital marketing field called Search Engine Optimisation or SEO. Hopefully this article has highlighted the importance of catering to your invisible, but valuable ‘robot’ customers in your digital strategy.

Alan Vesty is a Quirk-accredited digital marketer. Alan manages a growing suite of digital assets for the African Branch of Cambridge University Press, including a corporate website, several news blogs, email newsletters and social media channels.
Visit http://booksellers.cup.co.za