Ever wondered what the purpose of User Agent settings were for? Well it has changed over the years, and was used to determine what type of browser or service was accessing your website. Mainly it is used by people to determine what browser their main subscription base is using in order to tailor their website to that viewing. One thing that I have seen is that some people use a spoofed user agent setting are trying to see things that they only want a website crawler such as Google’s GoogleBot to view. This may be taking advantage of if they are showing the crawler special files that they may try to SEO optimize in order to get better ranking.
If you are looking to see whether or not the GoogleBot that has visited your site is legit, visit this list to check and see. Also, if you are looking to tweak your user agent settings (that I mentioned in my “Top Firefox Pluggins (Addons)”) through your Mozilla Firefox, download it here.
Make sure that you use the correctly formatted user agent information, which you can get a huge list here. For more information, please feel free to contact me with questions or tips.
]]>