How to Run Product Import on Daily Basis in Magento 2

- E-Commerce, Magento 2

Magento 2 Development

Below, we shed light on how to run Magento 2 import on daily basis and describe how to do this via CLI. We’ve discovered both the question and the answer on Magento StackExchange, and they inspired us to share our own opinion on this topic. Question by Massimo Camoni; answer by Khoa TruongDinh.

So, how to run product import in Magento 2 on daily basis? And how to use CLI in this process? The question has two answers, since everything depends on the platform version you use. As we all know, the difference between Community and Enterprise editions is huge, therefore, the same goals on both platforms can be achieved in a slightly different manner or after CE is enhanced with the help of third-party solutions. We’ve created a guide teaching how to cover the gap between Magento 2 Community Edition and Magento 2 Enterprise Edition, but let’s return to our muttons.

Enterprise Edition

If you have an Enterprise Edition-based website, running product import on daily basis is quite simple, since it is a feature of this platform version. Go to System > Data Transfer > Scheduled Imports/Exports.

Here, you can create import/export processes that will be launched daily, weekly, or monthly managed by the Cron job scheduler. Furthermore, the default capabilities allow you to import files from both local Magento servers as well as remote FTP servers.

Check this step by step guide for better understanding of the process:

  • Go to the Admin sidebar and tap System. Find  the Data Transfer section and select Scheduled Imports/Exports.
  • There are appropriate buttons for creating new scheduled import/export jobs, tap the appropriate one and follow the instructions. You can learn more about scheduling import and export here: Add Scheduled Export; Add Scheduled Import. Save the new job and it will appear in the Scheduled Import/Export grid.
  • Note that a copy of each import/export file is stored in the var/log/import_export directory (your Magento local server), but details of each operation are not available in the log. At the same time, if an error occurs, you get a notification that describes it. For further information, check this official documentation.

But what to do, if you have a Community-based webstore?

Community Edition

By default, Magento 2 Community Edition doesn’t provide the ability to run product import/export on daily, weekly, or monthly basis automatically, but it is always possible to solve this problem. As mentioned above, you can easily get most of the EE features on the Community storefront with the help of third-party extensions. Of course, you can build your own module, but we’ve already solved the problem. In Firebear’s portfolio, you can find a robust tool called Improved Import. The extension is designed to simplify both import and export processes as well as add missing features to the Community Edition platform. We’d like to introduce you to the core functionality of this Magento 2 module and then shed light on how it solves the problem discussed in this post.

Cron import Magento 2 , categories import Magento 2

The first thing that makes our product even better than the default import of Magento 2 EE is wider selection of sources. In addition to local or remote FTP server, Improved Import provides the ability to grab CSV from Dropbox or via a direct URL. Besides, the Magento 2 extension allows you to import product as well as dedicated simple categories (note that the default Magento 2 platform allows to import categories only with assigned products).

Another great feature of our product is the ability to create new product attributes and their values on the fly during the import process.

The next feature is the most interesting as part of this discussion. The Improved Import extension fully supports con job import/export for products, category, customer, and stock data as well as attributes. The flexible and extremely powerful Magento 2 Enterprise Edition feature is available for Community Edition merchants. Below, we describe it in more details.

Configurable products are fully supported and can be effortlessly imported into the system. And to make import more precise, our extension supports field mapping. Thus, it is possible to map Magento 2 product attributes to any custom column in a CSV file!

Import Jobs Feature

With the help of the Import Jobs feature, you can easily leverage the cron job functionality for creating schedules. It supports any kind of import processes and can be launched via CLI. Thus, running product import on a daily basis in Magento 2 CE via CLI becomes possible. The feature is available under Magento admin -> System -> Data Transfer -> Import Jobs.

Import Jobs

Here, you can see a grid that contains all import cron jobs. The grid consists of the following columns:

  • ID – each job gets an ID number;
  • Checkbox – you can select jobs to apply mass action;
  • Title – each job gets a title;
  • Status – enabled or disabled;
  • Cron – the frequency of jobs in a cron format;
  • Frequency – how often import is run: daily, hourly, weekly, etc;
  • Entity Type – data type to be imported: products, customers, categories, etc;
  • Import Source – local file, Dropbox, FTP, or URL;
  • Actions – delete or edit.

all Import Cron jobs

To create a new job that will run product import on a daily basis, click the “Add New Job” button. You will be redirected to the following screen:

Click Add New Job to add a new process

Now, let’s figure out how to configure import parameters.

Title of Import Job allows you to name your import job in a self explanatory manner to easily find it in future. Next, you should select an import preset frequency. Note that it is possible to run import every minute or hour, day at specified time, every specified day of the week at certain time, every specified day of the month at selected time, or the module allows you to create a custom schedule:

  • Every minute (*/1 * * * *)
  • Every hour (* */1 * * *)
  • Every day at 3:00am (0 3 * * *)
  • Every Monday at 3:00am (0 3 * * 1)
  • Every 1st day of month at 3:00am (0 3 1 * *)
  • Custom

As for the aforementioned custom format for your Cron job-scheduled import, it can be easily configured. You only need to understand the standard Cron syntax. Here it is:

Cron syntax

Next, you should select import behaviour. Our Magento 2 Improved Import module allows to add new entities, as well as update, replace, or delete the existing ones with new items.

For further information about the standard Cron configuration in Magento 2, check our guide: Magento 2 Cron Configuration. Besides, there is the official documentation here: Configure CLI Subcommands Cron.

Now, let’s tell a few words about the usage of CLI in the import procedure. Our extension allows to enable, disable, or start product import via CLI. Go to the Magento 2 root folder. Open the command line and run the following command:

Next, you can use CLI, to disable, enable, and run cron jobs by adding the appropriate command to “import:job:”. These are three available commands:

  • import:job:disable
  • import:job:enable
  • import:job:run

Magento CLI Import Commands

We hope this was a comprehensive answer, so now you know how to run product import on daily basis in CE and EE or run import via CLI in CE. As for the Improved Import extension, you can purchase it here for $238:

Download / Buy Firebear Improved Import Magento 2 Extension

The Magento 1 version costs $119 and is available here:

Download / Buy Firebear Improved Import Magento Module

More tips from Magento 2 Cookbook.