Magento 2 has been released, so we are trying to dig more useful materials related to the new version of the popular ecommerce platform. This time, our attention has been drawn to Alan Kent’s article related to Magento 2 testing, so we’ve also decided to cover this topic on the Firebear blog. For those developers willing to improve the quality of their Magento 2 projects, testing is mandatory; therefore, we propose you to examine the following Magento 2 guide, which is based on the aforementioned article, the official documentation, as well as our own experience related to both the Magento 2 platform and testing.
UPD (11.02.16): Magento 2 code katas and Getting your hands dirty testing Magento 2
According to Alan Kent, one of the key Magento 2 goals is better test automation, since it can dramatically improve the code quality by finding bugs at early stages of development which leads to more frequent releases. Thus, we should start with discussing different types of tests available in Magento 2:
Unit Tests (dev/tests/unit). With the aid of unit tests developers check isolated individual classes or functions in order to find logic bugs. Such tests are often run along with integration and functional tests. Highly focused unit tests make it easy to identify the root cause of a problem, but at the same time turn writing highly coupled code into a headache.
Integration Tests (dev/tests/integration). As for integration tests, they are helpful when you try to prevent problems in interactions between components, layers, and the environment. This group of tests is designed to examine multiple real classes in combination. Use them in addition to unit tests to improve the quality of your MAgento 2 testing.
Performance Tests (dev/tests/performance). Tests for tracking and maintaining certain performance level per build. Don’t confuse such tests with full Load and Performance tests.
Static Code Analysis Tests (dev/tests/static). Such tests analyze code instead of running it. Their major goal is to check whether the code conforms to coding standards or not.
Static integrity tests (dev/tests/static/testsuite/Magento/Test/Integrity). They examine how the application is linked, as you might have guessed, without running the code.
Integrity Tests (dev/tests/integration/testsuite/Magento/Test/Integrity). As for integrity tests, they also check how the app is linked, but deal with more advanced scenarios, so the application have to be run.
Legacy Tests. These tests are extremely helpful for porting modules to Magento 2, since they examine the usage of methods/classes/constants and show their substitutes.
Functional Tests (dev/tests/functional). With the help of functional tests, developers find out if product features work correctly, since such tests imitate user’s interaction with the system.
You can check all these Magento 2 tests running publically here. Before introducing you to the Magento Testing Framework, we’d like to draw your attention to some popular testing solutions:
JMetrix brings an aspect oriented approach into the measurement of application performance. With this solution, developers can easily add specific measurement features into the existing projects.
Selenium automates web applications for testing purposes. It supports major browsers, browser automation tools, APIs, and frameworks. Selenium is a perfect solution for creating robust regression automation suites and tests.
PHPUnit is a testing framework for PHP and at the same time an xUnit instance for unit testing frameworks.
Magento 2 Testing Framework
The Magento Testing Framework (MTF) provides the ability to create and run functional tests, simplifying acceptance, smoke, regression, or other testing. At the same time you should remember that the framework itself does not contain tests; all functional tests are available in the Magento code.
Besides being open source, MTF is a cross-platform solution that enables the development and usage of functional tests. The platform is capable for running multiple tests simultaneously, but you can still perform a single test.
Below, you will find a list of official posts that describe the Magento Testing Framework and a way you should use it. Then, there is a section related to core actions necessary for running Magento 2 tests.
INSTALLATION – this post sheds light on the installation of MTF entities. Start your journey into the world of Magento 2 testing here.
Go to magento2/dev/tests/functional/etc, find config.xml, and open it in editor.
Set credentials in your <application> element: change content of <backendLogin> and <backendPassword> to your actual credentials.
Choose browser. Mozilla Firefox is the default solution, but you can easily specify another browser in config.xml. You have to copy your <server> element from magento2/dev/tests/functional/vendor/magento/mtf/etc/config.xml>and paste it in magento2/dev/tests/functional/config.xmlafter the <install> element. Now, set actual data in the following attributes: <browserName>and <browser>. The Google Chrome example looks as follows:
3. The following step requires editing the credentials.xmlfile, since some Magento modules require credentials:
Go to magento2/dev/tests/functional, find credentials.xml, and open it in editor. You can create credentials.xmlfrom credentials.xml.dist if you cannot find it.
To set credentials for a Magento 2 module you are going to test, in credentials.xml find the block of elements related to this module and enter actual data in the valueattribute of the corresponding fieldelement.
Preparations for test run
The Selenium Server
The Selenium Server utilizes your browser to execute tests. You can download it here. Please note that:
To run tests on a non default browser (chances are the Selenium Server will not work directly with it) utilize the corresponding WebDriver.
Set a browser that MTF will use for testing Magento 2 in the config.xmlfile. Check the Google Chrome example above. Now, you can run the Selenium Server with an additional argument. Again, we provide an example for Google Chrome:
This tool generates fixtures, repositories, as well as page objects. Thus, After initializing MTF, all classes will be pre-generated. As a result, you facilitate creating and running tests for your Magento 2 project. To run the generator, enter the following command in terminal:
You can find all tests classified by categories in magento2/dev/tests/functional/tests/app/Magento. As for the testing tool, the framework (MTF) utilizes PHPUnit from magento2/dev/tests/functional/vendor/bin.
Please note that before running any tests, you should be confident that
To run all tests from magento2/dev/tests/functional/tests/app/Magento/, enter the below command in terminal:
How to run a particular test in Magento 2
To run a particular test in Magento 2, enter the following command in terminal
phpunit--filter<name of test>
where <name of test> is the name of PHP file with the test.
How to get logs for failed tests
All failed tests are logged in magento2/dev/tests/functional/var/log, while all Magento 2 errors – in magento2/var/log.
We expect that extensions in the new version of Magento Connect will have a complete test coverage. Check our blog posts related to Magento testing and TDD for further information.
Our team also recommends you to check the Entities of the Magento Testing Framework post on the official devdoc. The guide provides a complete data set overview and structure as well as describes how to merge data set. Check the full tutorial here: Entities of the Magento Testing Framework.
A simple and productive article about Custom Module Unit Tests in Magento 2 by Creare is also among our favourite materials related to Magento 2 testing. Don’t waste your chance to get a deep insight into this complex procedure without any headaches. For further information, follow this link: Beginners Guide to Magento 2 Custom Module Unit Tests.
Magento 2 code katas
Vinai Kopp creates a lot of amazing Screencasts of TDD practice sessions on the appropriate YouTube channel here: Mage2Katas. You get a great opportunity for learning TDD in Magento 2 via code katas – basic coding patterns practiced until you can use them automatically. You can read more about the term here: What is a Code Kata?