Preferring the robot over the human…
Having to type www before most websites you visit isn’t very friendly is it? While I am well aware of the signficance of the WWW in DNS / network topology terms, it dates back to the days when the internet was largely used by nerds & geeks (takes one to know one!) – the demographic of the average internet user today really couldn’t be much different.
That, coupled with the fact that most websites start with www, just means it’s needless complication to my mind. Most big companies realise this, and also find it snappier to advertise just their domain. They will sort it out for you, try it – type microsoft.com, or dell.com, or bbc.co.uk into your browser…. you get the www. version, don’t you?
The www. has been dropped, just like the http:// has been too.
Unless, that is, you’re trying to get at some of the biggest UK government websites. Take for example Her Majesty’s Revenue & Customs – possibly the one agency every citizen has no choice to use. hmrc.gov.uk fails to resolve. www.hmrc.gov.uk works fine though. Same with the Air Accident Investigation Branch, Parliament, Prison Service and counltess others.
I should say, that some do work – and importantly the gateway to government websites, direct.gov does work.
Why would they do this?
Well, it could just be they forget to configure it – either in the DNS for the domain, or the webserver itself. I’d say that’s lazy or sloppy – every domain I register and host is configured to allow both URLS to be used.
The more likely reason, I think, is duplicate content. Search engines, especially Google, penalise duplicate content at different addresses – and this will hurt where your site is positioned in the search results.
How can it be duplicate if it is the same content on the same site? Well, simple – it’s at two different addresses; and so is indexed twice. It’s duplicate. For example, these two URLs are different, but the content the same:
The way this should be handled is with URL re-writing or a 301 redirect. That is the webserver should make sure only 1 version ever appears on the web by changing it and redirecting the user (and thus also Google). If you go to the second address above, you will find you actually end up at the first one! The DVLA is using a redirect to ensure only one version is available.
This is really simple to set up on both Apache and IIS webservers; and it means your users have a much nicer experience, and who is more important, the bots or the user?….maybe it should be a standard for government departments!