How to Run Product Import on Daily Basis in Magento 2

Below, we shed light on how to run Magento 2 import processes on a daily basis. You can do that either in your admin interface or via CLI. While Magento 2 doesn’t let you automate data transfer processes and there is no way to use CLI to control imports and exports, you can enable the missing functionality with the help of third-party solutions. In the article below, we explain how to run Magento 2 import processes daily and use CLI to control them with the help of Improved Import & Export.

Also, you can look for an alternative point of view on Magento StackExchange. Take a look at the question by Massimo Camoni and the answer by Khoa TruongDinh. Since we leverage the advanced import functionality of the Improved Import & Export extension to address the issues mentioned in the discussion, you will also find a section that describes other possibilities the module provides.

So, how to run product import in Magento 2 on daily basis? And how to use CLI in this process? The question has two answers, since everything depends on the platform version you use. As we all know, the difference between Community and Enterprise editions is huge, therefore, the same goals on both platforms can be achieved in a slightly different manner or after CE is enhanced with the help of third-party solutions. We’ve created a guide teaching how to cover the gap between Magento 2 Community Edition and Magento 2 Enterprise Edition, but let’s return to our muttons.

Enterprise Edition

If you have an Enterprise Edition-based website, running product import on daily basis is quite simple, since it is a feature of this platform version. Go to System > Data Transfer > Scheduled Imports/Exports.

Here, you can create import/export processes that will be launched daily, weekly, or monthly managed by the Cron job scheduler. Furthermore, the default capabilities allow you to import files from both local Magento servers as well as remote FTP servers.

Check this step by step guide for better understanding of the process:

• Go to the Admin sidebar and tap System. Find  the Data Transfer section and select Scheduled Imports/Exports.
• There are appropriate buttons for creating new scheduled import/export jobs, tap the appropriate one and follow the instructions. You can learn more about scheduling import and export here: Add Scheduled Export; Add Scheduled Import. Save the new job and it will appear in the Scheduled Import/Export grid.
• Note that a copy of each import/export file is stored in the var/log/import_export directory (your Magento local server), but details of each operation are not available in the log. At the same time, if an error occurs, you get a notification that describes it. For further information, check this official documentation.

But what to do, if you have a Community-based webstore?

Community Edition

By default, Magento 2 Community Edition doesn’t provide the ability to run product import/export on daily, weekly, or monthly basis automatically, but it is always possible to solve this problem. As mentioned above, you can easily get most of the EE features on the Community storefront with the help of third-party extensions. Of course, you can build your own module, but we’ve already solved the problem. In Firebear’s portfolio, you can find a robust tool called Improved Import. The extension is designed to simplify both import and export processes as well as add missing features to the Community Edition platform. We’d like to introduce you to the core functionality of this Magento 2 module and then shed light on how it solves the problem discussed in this post.

The first thing that makes our product even better than the default import of Magento 2 EE is wider selection of sources. In addition to local or remote FTP server, Improved Import provides the ability to grab CSV from Dropbox or via a direct URL. Besides, the Magento 2 extension allows you to import product as well as dedicated simple categories (note that the default Magento 2 platform allows to import categories only with assigned products).

Another great feature of our product is the ability to create new product attributes and their values on the fly during the import process.

The next feature is the most interesting as part of this discussion. The Improved Import extension fully supports con job import/export for products, category, customer, and stock data as well as attributes. The flexible and extremely powerful Magento 2 Enterprise Edition feature is available for Community Edition merchants. Below, we describe it in more details.

Configurable products are fully supported and can be effortlessly imported into the system. And to make import more precise, our extension supports field mapping. Thus, it is possible to map Magento 2 product attributes to any custom column in a CSV file!

Import Jobs Feature

With the help of the Import Jobs feature, you can easily leverage the cron job functionality for creating schedules. It supports any kind of import processes and can be launched via CLI. Thus, running product import on a daily basis in Magento 2 CE via CLI becomes possible. The feature is available under Magento admin -> System -> Data Transfer -> Import Jobs.

Here, you can see a grid that contains all import cron jobs. The grid consists of the following columns:

• ID – each job gets an ID number;
• Checkbox – you can select jobs to apply mass action;
• Title – each job gets a title;
• Status – enabled or disabled;
• Cron – the frequency of jobs in a cron format;
• Frequency – how often import is run: daily, hourly, weekly, etc;
• Entity Type – data type to be imported: products, customers, categories, etc;
• Import Source – local file, Dropbox, FTP, or URL;
• Actions – delete or edit.

To create a new job that will run product import on a daily basis, click the “Add New Job” button. You will be redirected to the following screen:

Now, let’s figure out how to configure import parameters.

Title of Import Job allows you to name your import job in a self explanatory manner to easily find it in future. Next, you should select an import preset frequency. Note that it is possible to run import every minute or hour, day at specified time, every specified day of the week at certain time, every specified day of the month at selected time, or the module allows you to create a custom schedule:

• Every minute (*/1 * * * *)
• Every hour (* */1 * * *)
• Every day at 3:00am (0 3 * * *)
• Every Monday at 3:00am (0 3 * * 1)
• Every 1st day of month at 3:00am (0 3 1 * *)
• Custom

As for the aforementioned custom format for your Cron job-scheduled import, it can be easily configured. You only need to understand the standard Cron syntax. Here it is:

Next, you should select import behaviour. Our Magento 2 Improved Import module allows to add new entities, as well as update, replace, or delete the existing ones with new items.

For further information about the standard Cron configuration in Magento 2, check our guide: Magento 2 Cron Configuration. Besides, there is the official documentation here: Configure CLI Subcommands Cron.

Now, let’s tell a few words about the usage of CLI in the import procedure. Our extension allows to enable, disable, or start product import via CLI. Go to the Magento 2 root folder. Open the command line and run the following command:

Next, you can use CLI, to disable, enable, and run cron jobs by adding the appropriate command to “import:job:”. These are three available commands:

• import:job:disable
• import:job:enable
• import:job:run

We hope this was a comprehensive answer, so now you know how to run product import on daily basis in CE and EE or run import via CLI in CE. As for the Improved Import extension, you can purchase it here for $238:

Download / Buy Firebear Improved Import Magento 2 Extension

The Magento 1 version costs $119 and is available here:

Download / Buy Firebear Improved Import Magento Module

More tips from Magento 2 Cookbook.

Magento 2 Import & Export FAQ

Below, we share several other nuances of importing and exporting data with Improved Import & Export.