Avoiding the SEO pitfalls of URL parameters
URL parameters are the parts of the address of a web page often visible after a question mark (?), ampersand (&), equals (=) or other attribute. Parameters might also be referred to as “query strings” and are commonly used on e-commerce websites.
Numerous websites make use of query string parameters as a core part of their site architecture for tracking session IDs, languages, powering search queries and more. However, query string parameters can often result in multiple variations of the same URL all serving the same content.
For example, the following URLs would, in theory, all point to the same content: a collection of bikes. The only difference here is that some of these pages might be organized or filtered slightly differently:
- http://www.example.com/products/bikes.aspx
- http://www.example.com/products/bikes.aspx?category=mountain&color=blue
- http://www.example.com/products/bikes.aspx?category=mountain&type=womens&color=blue
URL parameters like those listed above, are common on e-commerce websites that use faceted navigation systems. Faceted navigation systems or site filter options allow users to browse products using a selection of pre-defined filters. This ultimately helps the user to find what they want, but it can lead to problems.
Example of a bike website’s faceted navigation or product filtering options. Each filter option adds a parameter to the URL string.
Using URL parameters to serve content based on filters applied via faceted navigation menus commonly results in URLs similar to those above. While this can be helpful to users trying to drill down to specific product groups within e-commerce sites, they can give search engines a headache if not managed correctly.
Common issues with URL parameters duplicate content
The generation of URL parameters based on website filters can create serious on-page issues that can impact the rankings of e-commerce category pages. We have already discussed how filters can be used to sort or narrow the content of a page, thus creating additional URLs that add no real value. For example, if your website enables users to sort content by price or feature (color, size etc.), and these options aren’t actually changing the content of the page but just narrowing the results; then this could hamper your website’s performance.
One way to check whether this is the case for your site is to test some of the filters available on one of your product category pages to assess whether the content changes significantly after the products have been filtered. For example, let’s imagine that the content of an original category page on a cycling website included a paragraph or two of copy to promote a specific type of bike. Then, when filters were applied using the faceted navigation to select female bikes, the URL of the page changes to include a query string (example.com/bikes?type=female) – if the majority of the page content remained the same, these pages could be classed as duplicate content by Google IF their relationship isn’t made clear by the website owner.
Keyword cannibalization
Keyword cannibalization occurs when multiple pages on a website target the same or similar keywords. This often causes search engine problems when deciding which page is the most appropriate page to rank for a particular search query, which in turn can lead to the “wrong” or “undesired” page ranking for that term.
A prime example of this can be seen on the bike company Halfords’ website www.halfords.com. As is commonplace on e-commerce sites, Halfords.com uses URL parameters to dynamically serve product and search filter results for types of bikes, brands, sizes, styles or kids or adult audiences. The problem for Halfords is that their site doesn’t deal with the numerous dynamically generated URL parameters page which ultimately, creates competing category pages for Google to index.
To better illustrate how parameter URLs can harm organic performance, let’s take a look at the following results for “women’s bikes”. For this search, Halfords’ main bike category appears in the search results rather than their women’s bikes category page.
The Halfords site doesn’t rank as well as it could in Google due to URL parameter issues on their website.
The organic results for Decathlon and Evans Cycles are their women’s bike pages, which makes sense as these pages best match the search query “women’s bikes”. This is what I would expect from a properly optimized e-commerce site, but unfortunately for Halfords, their failure to address the numerous duplicate parameter pages for the keyword phrases has resulted in Google not ranking their women’s bikes page.
To understand the extent of Halfords’ problems, I ran a site command search on the Halfords domain (site:halfords.com women’s bikes) to identify competing pages for the term women’s bikes. The search results for this query revealed why the right page isn’t ranking.
I conducted a site command search on the Halfords site for to identify competing womens bikes pages (site:halfords.com “womens bikes”). The results show the duplicate parameter URLs that are holding back Halfords organic performance for this term.
I conducted a site command search on the Halfords site for to identify competing women’s bikes pages (site:halfords.com “womens bikes”). The results show the duplicate parameter URLs that are holding back Halfords’ organic performance for this term.
There are numerous duplicate parameter URLs for women’s bikes and the screenshot above includes two results with product_storeId_10001_catalogId_14552_productId_ in the URL.
The first result from halfords.com is http://www.halfords.com/webapp/wcs/stores/servlet/product_storeId_10001_catalogId_14552_productId_1135992_langId_-1_categoryId_null&cm_re=Cycling+SideNav-_-bikes-_-womens
One of the many womens bikes pages on the Halfords site. An SEO own goal!
One of the many women’s bikes pages on the Halfords site. A prime example of the problem that can be caused by parameter URLs.
The second result URL is a similar URL, but a different Product ID parameter is appended to the same URL as the first result. This is essentially a duplicate page. http://www.halfords.com/webapp/wcs/stores/servlet/product_storeId_10001_catalogId_14552_productId_1135992_langId_-1_categoryId_null&cm_re=Cycling+SideNav-_-bikes-_-womens
Halfords Womens Bikes 2nd page in Google’s results is similar to the first but with a different Product ID appended. This is duplicate content and explains why Halfords women’s bikes page does not rank.
Halfords Womens Bikes 2nd page in Google’s results is similar to the first but with a different Product ID appended. This is duplicate content and explains why Halfords women’s bikes page does not rank.
Halfords also have their mobile site in Google’s desktop index which isn’t good, as it provides yet another duplicate parameter URL.
Mobile parameter URLs are in Google’s index for the Halfords site
Mobile parameter URLs are in Google’s index for the Halfords site. This doesn’t provide a good experience for desktop users.
If we visit the Halfords site, it’s possible to find even more URLs available for women’s bikes queries. The link within the main navigation menu takes you through to a women’s bike page powered by parameters.
The product category filter options and additional search filter options are creating further pages that are competing for the same term. Halfords are missing out on organic traffic because they are not using technical SEO techniques to avoid parameter issues and help Google to understand which pages to index. Ideally, your pages should use a search engine friendly URL structure, but you can still use parameters as long as you take the necessary steps to avoid duplication and indexing issues.
The fact that Halfords has multiple pages for women’s bikes with variations of women’s bikes in their page titles and headings, performance is being held back. This is a classic example of keyword cannibalisation, where a website is simply confusing search engines by having two or more pages covering the same topic, meaning search engines will be unsure of which page to rank.
When we originally wrote this post in 2015, we cited Booking.com as a prime example of an accommodation website with parameter issues. Booking.com used URL parameters to display unique hotel results related to specific locations across the globe. It looks like they have solved the parameter issues with their site, but to illustrate the problems that they had and that many sites continue to have, I have referenced our original observations below. These results show the results for the query ‘Booking Hotels in Dubai’.
For this search, Booking.com appeared twice in the search results:
hotels in dubai
The first result from booking.com that appeared for that search query was this page: http://www.booking.com/city/ae/dubai.en-gb.html
dubai
This page uses a search engine friendly URL structure, and serves a customised landing page that acts as a search portal for all hotels they offer in Dubai.
However, Google also indexed a second page from Booking.com’s website for that term: http://www.booking.com/searchresults.en-gb.html?city=-782831
dubai2
This page was substantially different to the first page we looked at; and had been developed so that the heading is generated through the name of the city being searched for, and the number of properties being returned for that search. The city attribute had been used to populate the page’s meta title to ensure it’s not seen as a duplicate of the first page.
Now, while these pages won’t be classed as exact duplicates of one another, this doesn’t mean that they won’t cause problems. The fact that Booking.com had two pages that feature the phrase ‘Hotels in Dubai’ in their page titles, headings, and copy and held back the performance of both pages.
Booking.com resolved their duplicate parameter URL content issues, and now we’re going to take a look at a few of the techniques for doing this below.
How to avoid pitfalls associated with URL parameters
There are a number of solutions to ensure URL parameters don’t cause SEO issues for your website. But before you rush in and implement any of the fixes below, you need to check whether URL parameters could be causing your website problems by asking the following questions:
- When using a search filter on your website (see faceted navigation), does the URL change and the copy remain the same as the copy on the original URL?
- When using a search filter on your website, does the URL change and the page title and meta description remain the same or contain the same target keyword?
If you answered yes to one or both of these, URL parameters could be holding back the performance of your site in organic search, and it might be time to take action.
Canonical tags
Canonical tags are used to indicate to search engines that certain pages should be treated as copies of a certain URL, and that any rankings should actually be credited toward the canonical URL.
Web developers can set the canonical version of a piece of content to the category page URL before a filter is applied. This is a simple solution to help direct search engine robots toward the content you really want to be crawled, while keeping filters on the site to help users find products closely related to their requirements. For more information on implementing rel=”canonical” tags, see our guide on how to use canonical tags properly. We have also written a guide on how to implement canonical tags on Magento e-commerce sites.
URL parameter tools
Use the URL Parameters tool within Google Search Console to give Google information about how to handle URLs containing specific parameters. Please do tread carefully when making changes as you can easily exclude the wrong pages from Google’s index which may result in a significant loss of traffic.
Bing Webmaster Central also provides a tool for ignoring URL parameters. You can find a guide on how to use this tool here.
Robots.txt – disallowing query strings
The robots.txt file can help you remedy a duplicate content situation, by blocking search query parameters from being crawled by search engines. However, before you go ahead and block all query strings, I’d suggest making sure everything you’re disallowing is something you definitely don’t want to be indexed. In the majority of cases, you can specify search engines to ignore any parameter based pages simply by adding the following line to your website’s robots.txt file:
Disallow: /*?*
This will disallow any URLs that feature a question mark. Obviously, this is useful for ensuring any URL parameters are blocked from being crawled by search engines, but you first need to ensure that there aren’t any other areas of your site using parameters within their URL structure.
To do this, I’d recommend carrying out a crawl of your entire website using a tool like Screaming Frog’s SEO spider, exporting the list of your website’s URLs into a spreadsheet and carrying out a search within the spreadsheet for any URLs containing question marks (?).
Common things to look out for here would be the use of URL parameters to serve different language variants of a page, which in itself is a bad idea. If this is the case, you don’t want to block search engines from crawling those variants via robots.txt. You’ll need to instead look into implementing a viable URL structure to target multiple countries.
If you’ve worked through the list of URLs and confirmed that the only pages using URL parameters are those causing duplicate content issues, I’d suggest adding the above command to your website’s robots.txt file.
Conclusion
Using a faceted navigation can prove extremely useful for consumers looking for specific products within your website, but you need to ensure that any URLs generated as a result of filters being applied don’t hold back the performance of your original category pages in organic search results.
While I have detailed three of the most common fixes for URL parameters, every website platform is a little different, and you should, therefore, take the time to assess each situation on a case by case basis before jumping in and implementing any of the solutions I’ve described.
If you need help with your Search Engine Optimization (SEO) don’t hesitate to contact us.
PTC Computer Solutions works extensively on understanding the Needs of Our Clients and what they are looking for so that understanding of the Internet and business technologies for their needs and objectives is fully explained. Don’t fail to meet your objectives with your customer experience and other technologies. Stay on top of things. Train your employees or hire an agency that can handle it for you such as PTC Computer Solutions.
David WB Parker is a principal of Parker Associates of Jacksonville, Florida, marketing consultants to the real estate industry; President of PTC Computer Solutions, IT Specialist, and an active real estate sales professional with Barclay’s Real Estate Group based in Jacksonville, FL. He can be reached at 904-607-8763 or via email davidp@ptccomputersolutions.com.