Magento URL reindex (core_url_rewrite) duplicates patch

There is one annoying problem in Magento – the url rewrite indexer creates tons of unnecessary custom rewrites for products with duplicate url_keys. This happens during every indexer run. As a result, the size of the core_url_rewrite table grows to enormous sizes. Please note that the problem is typical for Magento 1.7 – 1.9 and its above versions. Below, I will explain how to fix this bug with the help of our patch. But first of all let’s start with one real world example.

Recently I was working on ecommerce store based on Magento Community Edition 1.9.1. The store already had around 20k products and 4 store views. After cleaning core_url_rewrite table and reindex, I’ve noticed a significant increase in a number of records. The default quantity was 170k records. After every Catalog URL reindex, additional 1500 records appeared in a core_url_rewrite. Note that it happened without modifying or adding new products or categories. Thus, the bug in out-of-the-box Magento produced around 50k new trash records every month that was pretty annoying.  But why did this happen?

Since Community Edition doesn’t offer any restrictions on the product URL key duplicates, multiple products can have the same URL key. At the same time, the product URL path generator should be able to handle it.

Thus, the generator checks the duplicate’s URL key and founds that the respective URL path is occupied. Moreover, the same is about the URL key with the product’s ID in its end. That’s why the product URL path generator creates a new URL key and a custom redirect to the new URL. This results in three major problems:

• As mentioned before, you get enormously big core_url_rewrite which is growing after every reindex.
• Every new reindex requires more time.
• The process provides negative influence on SEO since a duplicate’s URL path is always changing.

Now when you know the reason of the problem and see how it affects your Magento website, it’s time to tell you about the healing.

Atwix has described one reliable solution. Unfortunately, it is too complex to be used by everyone. That’s why I decided to create a simple patch, which fixes the Magento URL reindex (core_url_rewrite) duplicates bug. Hit the below link to download the patch.  Install the patch, and you will fix the problem.

Download the Magento URL reindex (core_url_rewrite) duplicates patch