How to Run Product Import on Daily Basis in Magento 2

- E-Commerce, Magento 2

Magento 2 Development

Below, we shed light on how to run Magento 2 import processes on a daily basis. You can do that either in your admin interface or via CLI. While Magento 2 doesn’t let you automate data transfer processes and there is no way to use CLI to control imports and exports, you can enable the missing functionality with the help of third-party solutions. In the article below, we explain how to run Magento 2 import processes daily and use CLI to control them with the help of Improved Import & Export.

Also, you can look for an alternative point of view on Magento StackExchange. Take a look at the question by Massimo Camoni and the answer by Khoa TruongDinh. Since we leverage the advanced import functionality of the Improved Import & Export extension to address the issues mentioned in the discussion, you will also find a section that describes other possibilities the module provides.

'

So, how to run product import in Magento 2 on daily basis? And how to use CLI in this process? The question has two answers, since everything depends on the platform version you use. As we all know, the difference between Community and Enterprise editions is huge, therefore, the same goals on both platforms can be achieved in a slightly different manner or after CE is enhanced with the help of third-party solutions. We’ve created a guide teaching how to cover the gap between Magento 2 Community Edition and Magento 2 Enterprise Edition, but let’s return to our muttons.

Enterprise Edition

If you have an Enterprise Edition-based website, running product import on daily basis is quite simple, since it is a feature of this platform version. Go to System > Data Transfer > Scheduled Imports/Exports.

Here, you can create import/export processes that will be launched daily, weekly, or monthly managed by the Cron job scheduler. Furthermore, the default capabilities allow you to import files from both local Magento servers as well as remote FTP servers.

Check this step by step guide for better understanding of the process:

  • Go to the Admin sidebar and tap System. Find  the Data Transfer section and select Scheduled Imports/Exports.
  • There are appropriate buttons for creating new scheduled import/export jobs, tap the appropriate one and follow the instructions. You can learn more about scheduling import and export here: Add Scheduled Export; Add Scheduled Import. Save the new job and it will appear in the Scheduled Import/Export grid.
  • Note that a copy of each import/export file is stored in the var/log/import_export directory (your Magento local server), but details of each operation are not available in the log. At the same time, if an error occurs, you get a notification that describes it. For further information, check this official documentation.

But what to do, if you have a Community-based webstore?

Community Edition

By default, Magento 2 Community Edition doesn’t provide the ability to run product import/export on daily, weekly, or monthly basis automatically, but it is always possible to solve this problem. As mentioned above, you can easily get most of the EE features on the Community storefront with the help of third-party extensions. Of course, you can build your own module, but we’ve already solved the problem. In Firebear’s portfolio, you can find a robust tool called Improved Import. The extension is designed to simplify both import and export processes as well as add missing features to the Community Edition platform. We’d like to introduce you to the core functionality of this Magento 2 module and then shed light on how it solves the problem discussed in this post.

Cron import Magento 2 , categories import Magento 2

The first thing that makes our product even better than the default import of Magento 2 EE is wider selection of sources. In addition to local or remote FTP server, Improved Import provides the ability to grab CSV from Dropbox or via a direct URL. Besides, the Magento 2 extension allows you to import product as well as dedicated simple categories (note that the default Magento 2 platform allows to import categories only with assigned products).

Another great feature of our product is the ability to create new product attributes and their values on the fly during the import process.

The next feature is the most interesting as part of this discussion. The Improved Import extension fully supports con job import/export for products, category, customer, and stock data as well as attributes. The flexible and extremely powerful Magento 2 Enterprise Edition feature is available for Community Edition merchants. Below, we describe it in more details.

Configurable products are fully supported and can be effortlessly imported into the system. And to make import more precise, our extension supports field mapping. Thus, it is possible to map Magento 2 product attributes to any custom column in a CSV file!

Import Jobs Feature

With the help of the Import Jobs feature, you can easily leverage the cron job functionality for creating schedules. It supports any kind of import processes and can be launched via CLI. Thus, running product import on a daily basis in Magento 2 CE via CLI becomes possible. The feature is available under Magento admin -> System -> Data Transfer -> Import Jobs.

Import Jobs

Here, you can see a grid that contains all import cron jobs. The grid consists of the following columns:

  • ID – each job gets an ID number;
  • Checkbox – you can select jobs to apply mass action;
  • Title – each job gets a title;
  • Status – enabled or disabled;
  • Cron – the frequency of jobs in a cron format;
  • Frequency – how often import is run: daily, hourly, weekly, etc;
  • Entity Type – data type to be imported: products, customers, categories, etc;
  • Import Source – local file, Dropbox, FTP, or URL;
  • Actions – delete or edit.

all Import Cron jobs

To create a new job that will run product import on a daily basis, click the “Add New Job” button. You will be redirected to the following screen:

Click Add New Job to add a new process

Now, let’s figure out how to configure import parameters.

Title of Import Job allows you to name your import job in a self explanatory manner to easily find it in future. Next, you should select an import preset frequency. Note that it is possible to run import every minute or hour, day at specified time, every specified day of the week at certain time, every specified day of the month at selected time, or the module allows you to create a custom schedule:

  • Every minute (*/1 * * * *)
  • Every hour (* */1 * * *)
  • Every day at 3:00am (0 3 * * *)
  • Every Monday at 3:00am (0 3 * * 1)
  • Every 1st day of month at 3:00am (0 3 1 * *)
  • Custom

As for the aforementioned custom format for your Cron job-scheduled import, it can be easily configured. You only need to understand the standard Cron syntax. Here it is:

Cron syntax

Next, you should select import behaviour. Our Magento 2 Improved Import module allows to add new entities, as well as update, replace, or delete the existing ones with new items.

For further information about the standard Cron configuration in Magento 2, check our guide: Magento 2 Cron Configuration. Besides, there is the official documentation here: Configure CLI Subcommands Cron.

Now, let’s tell a few words about the usage of CLI in the import procedure. Our extension allows to enable, disable, or start product import via CLI. Go to the Magento 2 root folder. Open the command line and run the following command:

Next, you can use CLI, to disable, enable, and run cron jobs by adding the appropriate command to “import:job:”. These are three available commands:

  • import:job:disable
  • import:job:enable
  • import:job:run

Magento CLI Import Commands

We hope this was a comprehensive answer, so now you know how to run product import on daily basis in CE and EE or run import via CLI in CE. As for the Improved Import extension, you can purchase it here for $238:

Download / Buy Firebear Improved Import Magento 2 Extension

The Magento 1 version costs $119 and is available here:

Download / Buy Firebear Improved Import Magento Module

More tips from Magento 2 Cookbook.

Magento 2 Import & Export FAQ

Below, we share several other nuances of importing and exporting data with Improved Import & Export.

How to import products in Magento from CSV?

Magento 2 lets you import products via a CSV file by default. However, you need to use the standard attributes and follow other requirements for Magento 2 product import. As for the Improved Import & Export extension, it provides the ability to import any CSV file. And there is no need to edit it. Instead, you match third-party attributes to ones used in Magento 2, teaching the system how to replace the unsupported data.

What file format Magento accepts in import?

Magento 2 only lets you import CSV files. However, you can dramatically extend the platform’s default functionality with the help of the Improved Import & Export extension. The module provides the ability to import CSV, XML, XLSX, JSON, and ODS files.

How to import Amazon listings to Magento?

You can import Amazon listings to Magento 2 as follows. Firstly, it is necessary to export data from Amazon. Secondly, you need to edit the output according to the Magento 2 requirements. Thirdly, import your Amazon listings to your e-commerce website. Alternatively, it is possible to use the Improved Import & Export Magento 2 module to establish a direct connection between the two systems. The plugin lets you create an API connection for moving product data and other entities between the two systems.

How to bulk import images in Magento 2

You need to upload product images to your server under the pub/media/import folder and then specify their addresses in a product data file that you are going to import. Enter each image file name to be imported on the correct row, by SKU, and in the correct column according to image type.

How to import up-sell and cross-sell products to Magento?

To perform the import of up-sells, cross-sells, and related products, create a CSV file with the corresponding data. Specify related products, up-sells, and cross-sells via the following three columns in the table.
– related_skus – for related products
– crosssell_skus – for cross-sells
– upsell_skus – for up-sells
Note that all columns should have the same formatting. You can assign a product as related by specifying its SKU. To assign multiple related products, use commas to separate numerous SKUs.

How to import and export customers in Magento 2 on a daily basis?

To move customers between Magento 2 and other external systems and e-commerce websites, use Improved Import & Export to create import and export jobs. Next, choose Customers as your import or export entity in the corresponding settings section. Note that you can transfer all customer data.

How to import and export products in Magento 2 on a daily basis?

To move products between Magento 2 and other external systems and e-commerce websites, use Improved Import & Export to create import and export jobs. Next, choose Products as your import or export entity in the corresponding settings section. Our module works with all product types: simple, configurable, grouped, bundle, virtual, downloadable, and even gift cards!

How to import and export orders in Magento 2 on a daily basis?

To move orders between Magento 2 and other external systems and e-commerce websites, use Improved Import & Export to create import and export jobs. Next, choose Orders as your import or export entity in the corresponding settings section. You can transfer orders along with all the related documents.

How to import and export categories in Magento 2 on a daily basis?

To move categories between Magento 2 and other external systems and e-commerce websites, use Improved Import & Export to create import and export jobs. Next, choose Categories as your import or export entity in the corresponding settings section. Our module lets you map imported categories to ones that are already represented in your category tree.

How to import and export prices in Magento 2 on a daily basis?

To move advanced pricing between Magento 2 and other external systems and e-commerce websites, use Improved Import & Export to create import and export jobs. Next, choose Products as your import or export entity in the corresponding settings section. Note that it is necessary to edit the product data table in a specific manner to enable the price update.

How to import and export product attributes in Magento 2 on a daily basis?

To move product attributes between Magento 2 and other external systems and e-commerce websites, use Improved Import & Export to create import and export jobs. Next, choose Product Attributes as your import or export entity in the corresponding settings section. Note that the module provides the ability to move product attributes, attribute sets, and attribute groups as dedicated entities.

How to import and export newsletter subscribers in Magento 2 on a daily basis?

To move newsletter subscribers between Magento 2 and other external systems and e-commerce websites, use Improved Import & Export to create import and export jobs. Next, choose Newsletter Subscribers as your import or export entity in the corresponding settings section. Thus, you can subscribe the existing customers to your new newsletter as well as add new subscribers.

How to import and export data between Magento 2 and other websites automatically?

The Improved Import & Export module lets you automate your data synchronization between Magento 2 and external systems. The plugin uses a cron scheduler, providing the ability to choose or create intervals for automated data transfers. Specify a period of updates, and the module will do everything for you.

How to import data with third-party attributes to Magento 2 on a daily basis?

As we’ve already mentioned, it is impossible to import third-party data to Magento 2. However, you can use Improved Import & Export to carry this burden. The extension offers a mapping section where you match the unsupported attributes to ones used in Magento 2, teaching the system what designations to use to replace the unsupported data.

'