How to import data from Amazon DynamoDB to Magento 2

In the following post, we compare two different ways of importing data from a remote Amazon DynamoDB table to Magento 2. First of all, you will find out how to do that via a default algorithm suitable for all databases. Next, we will draw your attention to a more user-friendly approach. Thus, if you need to import data from Amazon DynamoDB to Magento 2, you’ve come to the right place. Let’s start the exploration.

The Common Way of Moving Amazon DynamoDB Bases into Magento 2

The common way of moving Amazon DynamoDB bases to Magento 2 consists of three steps. Firstly, you should create a CSV output. Only this format is supported by Magento 2, so you don’t have other options. Next, the CSV file should be edited. You need to make its attributes suitable for the internal Magento 2 requirements. Otherwise, no data will be transferred. When the file is ready, you can finally update your database. Let’s take a look at each step separately.

CSV Export from Amazon DynamoDB Database

There are several ways of exporting data from Amazon DynamoDB to CSV. If your items have the same attributes (id and name both are strings), then you can run the following command:

As a result, you will receive a tab-separated output. Next, it is necessary to redirect it to file using > output.txt. Next, it is necessary to convert tabs into commas to get a CSV file.

Another option is the DynamoDBtoCSV project available at GitHub. The application provided there will export the content of a DynamoDB table into a CSV file without additional steps. Update config.json with your AWS credentials and region to achieve this goal. Note that the application produces a comma-separated output where each field is enclosed by double quotes (“). For further information, check the project on GitHub.

Map Third-Party Attributes to Magento 2 Attributes

Now, when you have the exported CSV file, it is possible to proceed to the second step of importing a remote Amazon DynamoDB base – you should edit its content.

If there are no headers – add them. Next, make attributes suitable for the Magento 2 requirements.

If you import data from an external system, match its attributes to the ones used in Magento 2. Otherwise, no export will occur. In our case, the problem should be fixed manually. It is necessary to open the CSV file generated in the previous step and review each attribute. The procedure requires additional patience and may consume lots of time especially if you need to edit not only attributes but also their values. Luckily, there is a solution that dramatically improves both routine processes. We describe it a little bit later. Now, let’s take a look at the third step of importing data from Amazon DynamoDB to Magento 2.

Import Amazon DynamoDB Table into Magento 2

Go to the import interface situated in your Magento 2 admin and transfer the file from the previous steps. The system will import it updating your database. Note that you will need to repeat this procedure every time a new update file is provided. Furthermore, the CSV export and attributes editing cannot be automated.  

Read this article for further information: The Complete Guide to Magento 2 Product Import / Export. Here, you will discover what attributes to use and how to adopt an external output to the needs of Magento 2 making the import procedure more efficient.

Pros & Cons

Let’s take a look at the pros and cons of using the default algorithm of transferring data from DynamoDB to Magento 2.

Pros

The only positive aspect here is the ability to implement the data migration. Nothing else can be mentioned here. Although you may argue that it is our initial goal, there is a better way to achieve it. But let’s focus on drawbacks first.

Cons

The first major inconvenience is the necessity to create a CSV output. Next, you should edit it which is also a drawback. Besides, it is necessary to import the data file manually and repeat all the actions mentioned above every time an update is necessary. There are enough reasons to look for a better option, and we describe one below.

The Easy Way of Moving Amazon DynamoDB Bases into Magento 2

The improved approach of importing Amazon DynamoDB bases into Magento 2 is based on a third-party module, so meet the Improved Import & Export extension – your most reliable time-saver for data transfers.

Amazon DynamoDB Database Connection

The first competitive advantage over the common approach to the import is the ability to avoid files from data transfers. The Improved Import & Export module connects Magento 2 to Amazon DynamoDB directly. As a result, the first two steps described above are eliminated. So how does the import procedure looks?

Magento 2 Amazon DynamoDB Data Import

You only need to configure and enable a new import profile. Next, the Improved Import & Export Magento 2 extension will do everything for you. All further transfers will be fully automated which is another great advantage over the default approach.

But the direct connection doesn’t solve the problem of different attribute standards. Luckily, the Improved Import & Export extension does. It offers mapping presets that modify external attributes to a suitable form within a few clicks. At the same time, you can do everything manually in the appropriate section of the profile. These and other features are described further in the article. Now, let’s take a look at the improved algorithm of importing data from Amazon DynamoDB into Magento 2.

How to Import Data From Amazon DynamoDB to Magento 2

You need to pass the following three steps:

1. Go to your Magento admin, open Improved Import & Export, find its import section, and create a new profile there.
2. Now, configure the profile specifying where your database is situated, what attributes should be mapped, what schedule should be applied, etc;
3. Save and launch the profile.

That’s it! You’ve just connected your e-commerce store to a remote Amazon DynamoDB database. You must admit that it is a way easier than the default approach.

Pros & Cons

Let’s see what pros and cons the Improved Import & Export offers then it comes to the Magento 2 Amazon DynamoDB integration.

Pros

1. You transfer data from DynamoDB to Magento 2;
2. You store is connected to the database directly;
3. There is no need to create a CSV output;
4. You don’t have to edit data files;
5. All updates are automated entirely.

Cons

As for negative aspects, you only have to install a third-party extension. If any difficulties occur, contact our support and we will help you. Furthermore, you can ask us about various nuances of the extension configuration. Thus, the necessity to install the Improved Import & Export Magento 2 extension is hard to call a drawback. It is inevitability especially if you compare the user-friendly algorithm with a community-proven one. Let’s take a look at some features of Amazon DynamoDB and then proceed to a more in-depth exploration of the Improved Import & Export extension.

Get Improved Import & Export Magento 2 Extension

Amazon DynamoDB Facts

Amazon DynamoDB is often compared with MongoDB, Microsoft Azure Cosmos DB, and Cassandra. It is a fast and flexible NoSQL database for all applications that require consistent, single-digit millisecond latency at any scale. Furthermore, DynamoDB is a fully managed cloud database that supports both document and key-value store models. The flexible data model and reliable performance make it suitable for mobile, web, gaming, ad tech, IoT, and many other areas.

As for other features, they include:

• Integration with AWS Lambda to provide Triggers which enables you to architect applications that automatically react to data changes;
• AWS Identity and Access Management (IAM) integration for fine-grained access control;
• Advanced flexibility caused by the use of both document and key-value data structures;
• The initial release of DynamoDB is dated back to 2012;
• It operates under a commercial license;
• And you can fully leverage its cloud-based offer;
• The server operating system is hosted;
• The data scheme is schema-free;
• Typing and secondary indexes are supported, but there is no SQL;
• APIs and other access methods are represented by RESTful HTTP API;
• Supported programming languages include .Net, ColdFusion, Erlang, Java, JavaScript, PHP, Python, Ruby, etc;
• Server-side scripts are not represented;
• Triggers are available;
• Sharding is the partitioning method;
• Replication methods are supported
• No MapReduce is available;
• Consistency concepts are represented by Eventual and Immediate Consistency;
• Foreign keys and transaction concepts are available;
• Concurrency and durability features are supported.

Improved Import & Export Magento 2 Extension

Now, when you are familiar with all the features of Amazon DynamoDB and know what approach to importing data from the database to your Magento 2 website is better and why, we can pay some more attention to the functionality of the Improved Import & Export Magento 2 extension. Let’s take a look at the automation features.

Automated Import & Export Processes

Or module offers two ways of making every data transfer automated. First of all, you can create a schedule of updates. Besides, it is possible to set triggers that will launch import/export processes. Let’s see how both features work.

Schedules

The Improved Import & Export Magento 2 extension supports cron. Consequently, you can create any custom schedule of updates. It is only necessary to use the default cron syntax. Alternatively, you can select one of the predefined intervals.

Also, note that the Improved Import & Export Magento 2 extension allows creating import and export jobs that have no schedule. You can use them for asynchronous or one-time updates. And don’t forget that our module provides the ability to launched every scheduled profile manually whenever you want.

Below, you can see how to configure a schedule:

Events

As for triggers, they are based on Magento 2 events and observers. You can select either a system event or a custom process that will be considered a trigger for the corresponding update action. Furthermore, the Improved Import & Export Magento 2 extension lets you create conditions that make the update action more precise. For further information, follow this link: How to Run Magento 2 Import or Export After Specific System Event or Process.

Advanced Mapping Features

Now, we’d like to draw your attention to the mapping functionality of Improved Import & Export as well as features that improve your work with attributes.

Mapping Presets

As mentioned above, you can freely use mapping presets to solve the problem of different attribute standards. It is only necessary to select and apply a scheme suitable for your integration. Next, the Improved Import & Export Magento 2 extension will analyze third-party attributes and map them to ones used internally. Your role is reduced to a few clicks -0 a dramatic improvement of the attributes editing process described at the beginning of this post.

Below, you can see how to match Magento 1 attributes to ones used in Magento 2 with the help of mapping presets:

Matching Interface

If a preset for your particular integration is not available or you want to keep the full control over the transfer, the Improved Import & Export extension provides you with a convenient interface where you can do everything manually. Select an external attribute and specify the internal one in front of it. Besides, you can leverage a column for hardcoded values. Here, a default attribute value can be set. Next, the plugin will add it to all items which include this attribute.

The process of manual matching is displayed below:

Attribute Values Mapping

Additionally to attributes, you can map their values. The Improved Import & Export Magento 2 extension provides conditions similar to the procedure illustrated above. The process is very straightforward. Select an external attribute, specify its values, and match them to ones used internally.

For further information, read this article: Attribute Values Mapping.

Attribute Values Editing

And you can edit multiple attribute values in bulk. The Improved Import & Export Magento 2 extension allows you to:

• Add a prefix to several attribute values simultaneously;
• Add a suffix to multiple attribute values in bulk;
• Merge data related to the same value;
• Split data related to multiple values.

Read this article for further information: How to Modify Attribute Values During Import and Export in Magento 2.

Category Mapping

Another feature related to mapping allows matching external product categories to ones used internally. The process is similar to attributes matching: you select an external category and specify the corresponding one used internally in front of it like shown below:

You can even create new categories during import with our extension. Both features are described here: Category Mapping.

Attributes On The Fly

And if any attributes are not available in the imported data, you can easily close that gap by specifying parameters necessary to generate the missing designations. Use the following general form to create attributes on the fly:

Attribute|attribute_property_name:attribute_property_value|…

The feature is described here: Product attributes import.

Extended Connectivity Options

Although we’ve been talking about the direct connection between Magento 2 and Amazon DynamoDB, it is necessary to say a few words about file transfers available with our module and alternative ways of data migration.

Multiple File Standards

While Magento works with CSV files only, you need to convert an external output to this format. That’s why we recommend exporting DynamoDB tables into this format if you use the default approach. As for the Improved Import & Export Magento 2 extension, it works with multiple more standards. With our module, you can freely import and export data via XML, JSON, ODS, and Excel.

Multiple File Sources

And if you perform a data import/export process with files, it is necessary to specify a source that will become a point of data transfers between your e-commerce store and connected systems. Our plugin offers three options:

• FTP/SFTP – use a local or remote server to exchange data files;
• Dropbox – use your Dropbox account for the same purpose;
• URL – specify a file address to import it directly.

Also, note that our extension can easily import compressed data. It unzips/untars all archives automatically.

Alternative Ways of Import & Export

And if you want to avoid data files from your integrations, there are two different options to achieve this goal. First of all, you can use intermediary services such as Google Sheets, Office 365 Excel, or Zoho Sheet. The Google Sheets integration is illustrated below:

The second way is based on API connections. The Improved Import & Export Magento 2 extension lets you transfer data between your e-commerce store and an external system via  REST, SOAP, and GraphQL.

Note that all mapping features, schedules, and other enhancements work equally well with data transfers and connections established directly or with the help of the intermediary services.

Watch this video to see the extension in action:

Final Words

As you can see, the Improved Import & Export Magento 2 extension is a powerful tool that covers all nuances of data transfers. And the Magento 2 Amazon DynamoDB integration is not an exception. You can freely use our module to save tons of time and effort that could be consumed by the default algorithm. In case of any questions regarding the Improved Import & Export extension, follow the link below and contact our support – we will provide you with answers immediately.

Get Improved Import & Export Magento 2 Extension