### Setup Environment Variables (Bash) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/mftf Updates configuration parameter values in the `.env` file or creates it if it doesn't exist. Parameters are passed as options with values, e.g., `--MAGENTO_BASE_URL=http://magento.local/`. If a parameter doesn't exist, an error is returned. Example parameters are sourced from `etc/config/.env.example`. ```bash vendor/bin/mftf setup:env --MAGENTO_BASE_URL=http://magento.local/ --BROWSER=firefox vendor/bin/mftf setup:env ``` -------------------------------- ### Install Magento 2 Dependencies with Composer Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/getting-started This section covers navigating to the Magento 2 directory, checking out a specific development branch, and installing the application's dependencies using Composer. Ensure Composer is installed and accessible in your PATH. ```bash cd magento2/ git checkout 2.4-develop composer install ``` -------------------------------- ### Define Starting Test in XML Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/merge-points/extend-tests This XML snippet defines a base test case for creating a simple product via the admin interface. It includes annotations, setup steps (before), cleanup steps (after), and a sequence of action groups to execute the test. ```xml <description value="Admin should be able to create a Simple Product"/> <severity value="CRITICAL"/> <testCaseId value="MAGETWO-23414"/> <group value="product"/> </annotations> <before> <createData entity="_defaultCategory" stepKey="createPreReqCategory"/> </before> <after> <amOnPage url="admin/admin/auth/logout/" stepKey="amOnLogoutPage"/> <deleteData createDataKey="createPreReqCategory" stepKey="deletePreReqCategory"/> </after> <actionGroup ref="AdminLoginActionGroup" stepKey="AdminLoginActionGroup1"/> <actionGroup ref="AdminFillSimpleProductFormActionGroup" stepKey="fillProductFieldsInAdmin"> <argument name="category" value="$$createPreReqCategory$$"/> <argument name="simpleProduct" value="_defaultProduct"/> </actionGroup> <actionGroup ref="AssertProductInStorefrontCategoryPage" stepKey="assertProductInStorefront1"> <argument name="category" value="$$createPreReqCategory$$"/> <argument name="product" value="_defaultProduct"/> </actionGroup> <actionGroup ref="AssertProductInStorefrontProductPage" stepKey="assertProductInStorefront2"> <argument name="product" value="_defaultProduct"/> </actionGroup> </test> ``` -------------------------------- ### Build Project with MFTF CLI Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/mftf Builds the Codeception project by cloning example configuration files. The `--upgrade` option can be used to upgrade installed tests after a framework upgrade. ```bash vendor/bin/mftf build:project vendor/bin/mftf build:project --upgrade ``` -------------------------------- ### MFTF Test Generation Example (testNames.txt) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/mftf An example format for a `testNames.txt` file used with the `generate:tests -p` command. It lists test names and suite names for targeted test generation. ```text testName1 testName2 testNameN suiteName:testInSuite ``` -------------------------------- ### Run Codeception Tests Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/codeception Executes the default test suites defined in the Codeception configuration. This is the most basic command to start your test execution. ```bash vendor/bin/codecept run ``` -------------------------------- ### Define Starting Test for Merging - XML Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/merge-points/merge-tests This XML code defines a starting test case for merging. It includes annotations, setup steps (before), teardown steps (after), and action groups for logging in, filling a product form, and asserting product details. This serves as the base test to which new functionalities will be added. ```xml <test name="AdminCreateSimpleProductTest"> <annotations> <features value="Catalog"/> <stories value="Create a Simple Product via Admin"/> <title value="Admin should be able to create a Simple Product"/> <description value="Admin should be able to create a Simple Product"/> <severity value="CRITICAL"/> <testCaseId value="MAGETWO-23414"/> <group value="product"/> </annotations> <before> <createData entity="_defaultCategory" stepKey="createPreReqCategory"/> </before> <after> <amOnPage url="admin/admin/auth/logout/" stepKey="amOnLogoutPage"/> <deleteData createDataKey="createPreReqCategory" stepKey="deletePreReqCategory"/> </after> <actionGroup ref="AdminLoginActionGroup" stepKey="adminLoginActionGroup1"/> <actionGroup ref="AdminFillSimpleProductFormActionGroup" stepKey="fillProductFieldsInAdmin"> <argument name="category" value="$$createPreReqCategory$$"/> <argument name="simpleProduct" value="_defaultProduct"/> </actionGroup> <actionGroup ref="AssertProductInStorefrontCategoryPage" stepKey="assertProductInStorefront1"> <argument name="category" value="$$createPreReqCategory$$"/> <argument name="product" value="_defaultProduct"/> </actionGroup> <actionGroup ref="AssertProductInStorefrontProductPage" stepKey="assertProductInStorefront2"> <argument name="product" value="_defaultProduct"/> </actionGroup> </test> ``` -------------------------------- ### Download and Run Selenium Server for Functional Testing Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/getting-started Download the Selenium server standalone JAR file and run it to facilitate browser automation for functional tests. This example uses version 3.14.0 and specifies the ChromeDriver path. ```bash curl -O http://selenium-release.storage.googleapis.com/3.14/selenium-server-standalone-3.14.0.jar ``` ```bash java -Dwebdriver.chrome.driver=chromedriver -jar selenium-server-standalone-3.14.0.jar ``` -------------------------------- ### Basic Assertion Example with Variable Substitution Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test/assertions This example demonstrates a common assertion pattern where a value is first grabbed from a page and then asserted to match an expected string. It showcases the use of `stepKey` for referencing previous actions and embedding variables within assertion strings using `{$stepKey}`. ```xml <grabTextFrom selector="#elementId" stepKey="stepKeyOfGrab"/> <assertEquals message="This is an optional human readable hint that will be shown in the logs if this assert fails." stepKey="assertEquals1"> <expectedResult type="string">Some String</expectedResult> <actualResult type="string">A long assert string {$stepKeyOfGrab} with an embedded variable reference.</actualResult> </assertEquals> ``` -------------------------------- ### Inline Alert Slot Example Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/best-practices An example demonstrating the use of slots within an inline alert component, suggesting a pattern for content injection or customization. ```html <div data-slots="text"> <!-- Text content will be placed here --> </div> ``` -------------------------------- ### REST API Header Example (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/metadata Provides an example of how to define a REST API header parameter using the '<header>' tag. The 'param' attribute specifies the header name, and the content is the header value. ```xml <header param="status">available</header> ``` -------------------------------- ### Build Efficient Selectors by Starting with Parent Context Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/tips-tricks Illustrates the recommended approach for building selectors for form elements by starting with the parent context and then specifying the element's `name` attribute. This pattern `{{section_selector}} {{input_selector}}` or `{{section_selector}} {{button_selector}}` ensures accurate targeting and improves lookup efficiency. ```xml <element name="productName" type="input" selector="*[data-index='product-details'] input[name='product[name]']"/> ``` ```xml <element name="productName" type="input" selector=".admin__field[data-index=name] input"/> ``` -------------------------------- ### Build Project with Options Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/mftf Builds the project and allows setting environment configuration parameters. The `--upgrade` option can be used to upgrade installed tests. ```bash vendor/bin/mftf build:project --MAGENTO_BASE_URL=http://magento.local/ --MAGENTO_BACKEND_NAME=admin214365 vendor/bin/mftf build:project --upgrade ``` -------------------------------- ### Example Suite Configuration (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/using-suites An example of an XML suite definition for the Magento Functional Testing Framework. This suite, named 'WYSIWYGDisabledSuite', disables WYSIWYG before running tests from the 'Catalog' module and re-enables it afterward, excluding a specific incompatible test. ```xml <suites xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:mftf:Suite/etc/suiteSchema.xsd"> <suite name="WYSIWYGDisabledSuite"> <before> <magentoCLI stepKey="disableWYSIWYG" command="config:set cms/wysiwyg/enabled disabled" /> </before> <after> <magentoCLI stepKey="enableWYSIWYG" command="config:set cms/wysiwyg/enabled enabled" /> </after> <include> <module name="Catalog"/> </include> <exclude> <test name="WYSIWYGIncompatibleTest"/> </exclude> </suite> </suites> ``` -------------------------------- ### Use Descriptive stepKey Names in Tests (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/tips-tricks Illustrates the importance of using descriptive `stepKey` names for better readability and clarity in test scripts. The 'Good' example shows the recommended practice of using meaningful names, while the 'Bad' example shows the less maintainable approach of using generic numbers. ```xml <click selector="{{StorefrontNavigationSection.topCategory(SimpleSubCategory.name)}}" stepKey="clickSimpleSubCategoryLink" /> <waitForPageLoad stepKey="waitForSimpleSubCategoryPageLoad" /> <click selector="{{StorefrontCategoryMainSection.productLinkByHref(SimpleProduct.urlKey)}}" stepKey="clickSimpleProductLink" /> <waitForPageLoad stepKey="waitForSimpleProductPageLoad" /> <!-- Perform some actions / Assert product page --> <click selector="{{StorefrontNavigationSection.topCategory(CustomCategory.name)}}" stepKey="clickCustomCategoryLink" /> <waitForPageLoad stepKey="waitForCustomCategoryPageLoad" /> <click selector="{{StorefrontCategoryMainSection.productLinkByHref(CustomSimpleProduct.urlKey)}}" stepKey="clickCustomSimpleProductLink" /> <waitForPageLoad stepKey="waitForCustomSimpleProductPageLoad" /> ``` -------------------------------- ### Configure .credentials file for Functional Testing Framework Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/credentials This snippet demonstrates how to set up and use the .credentials file for storing sensitive data in the Functional Testing Framework. It includes steps for copying the example file, adding it to .gitignore, and defining key-value pairs for credentials. ```bash cd dev/tests/acceptance/ cp .credentials.example .credentials ``` ```bash git check-ignore .credentials ``` -------------------------------- ### XML Example: Nested Assertion Syntax - Functional Testing Framework Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/backward-incompatible-changes Demonstrates the nested assertion syntax required by the updated XSD schema. This example shows how to use `<assertEquals>` with explicit actual and expected result types. ```xml <assertEquals stepKey="assertAddressOrderPage"> <actualResult type="const">$billingAddressOrderPage</actualResult> <expectedResult type="const">$shippingAddressOrderPage</expectedResult> </assertEquals> ``` -------------------------------- ### Use Isolation for Selectors (CSS Example) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/selectors Explains the concept of 'isolation' in selector writing by targeting a parent element first, then a child. This example shows how to select a 'Sign In' button within a specific login form using a combined selector. ```css #login-form .sign-in-button ``` -------------------------------- ### Define Starting Action Group - XML Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/merge-points/extend-action-groups Defines a starting action group named 'AdminFillSimpleProductFormActionGroup' for filling out a simple product form. It includes arguments for category and product details, navigates to the product index page, fills various product fields, selects a category, opens the SEO section, fills the URL key, saves the product, and asserts the success message and field values. ```xml <actionGroup name="AdminFillSimpleProductFormActionGroup"> <arguments> <argument name="category"/> <argument name="simpleProduct"/> </arguments> <amOnPage url="{{AdminProductIndexPage.url}}" stepKey="navigateToProductIndex"/> <click selector="{{AdminProductGridActionSection.addProductToggle}}" stepKey="clickAddProductDropdown"/> <click selector="{{AdminProductGridActionSection.addSimpleProduct}}" stepKey="clickAddSimpleProduct"/> <fillField userInput="{{simpleProduct.name}}" selector="{{AdminProductFormSection.productName}}" stepKey="fillName"/> <fillField userInput="{{simpleProduct.sku}}" selector="{{AdminProductFormSection.productSku}}" stepKey="fillSKU"/> <fillField userInput="{{simpleProduct.price}}" selector="{{AdminProductFormSection.productPrice}}" stepKey="fillPrice"/> <fillField userInput="{{simpleProduct.quantity}}" selector="{{AdminProductFormSection.productQuantity}}" stepKey="fillQuantity"/> <searchAndMultiSelectOption selector="{{AdminProductFormSection.categoriesDropdown}}" parameterArray="[{{category.name}} প্রদর্শন]" stepKey="searchAndSelectCategory"/> <click selector="{{AdminProductSEOSection.sectionHeader}}" stepKey="openSeoSection"/> <fillField userInput="{{simpleProduct.urlKey}}" selector="{{AdminProductSEOSection.urlKeyInput}}" stepKey="fillUrlKey"/> <click selector="{{AdminProductFormActionSection.saveButton}}" stepKey="saveProduct"/> <seeElement selector="{{AdminProductMessagesSection.successMessage}}" stepKey="assertSaveMessageSuccess"/> <seeInField userInput="{{simpleProduct.name}}" selector="{{AdminProductFormSection.productName}}" stepKey="assertFieldName"/> <seeInField userInput="{{simpleProduct.sku}}" selector="{{AdminProductFormSection.productSku}}" stepKey="assertFieldSku"/> <seeInField userInput="{{simpleProduct.price}}" selector="{{AdminProductFormSection.productPrice}}" stepKey="assertFieldPrice"/> <click selector="{{AdminProductSEOSection.sectionHeader}}" stepKey="openSeoSectionAssert"/> <seeInField userInput="{{simpleProduct.urlKey}}" selector="{{AdminProductSEOSection.urlKeyInput}}" stepKey="assertFieldUrlKey"/> </actionGroup> ``` -------------------------------- ### Array with Primitive Items Example (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/metadata Shows how to populate an array with primitive data type items. This XML structure represents the actual data assigned to an array, like a list of IDs. ```xml <array key="tax_rate_ids"> <item>1</item> <item>2</item> </array> ``` -------------------------------- ### Define Starting Page Structure (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/merge-points/merge-pages This XML snippet defines the initial structure of a page, listing various sections that are part of its default configuration. It serves as the base for merging additional sections. ```xml <page name="AdminCategoryPage" url="catalog/category/" area="admin" module="Magento_Catalog"> <section name="AdminCategorySidebarActionSection"/> <section name="AdminCategoryMainActionsSection"/> <section name="AdminCategorySidebarTreeSection"/> <section name="AdminCategoryBasicFieldSection"/> <section name="AdminCategorySEOSection"/> <section name="AdminCategoryProductsSection"/> <section name="AdminCategoryProductsGridSection"/> <section name="AdminCategoryModalSection"/> <section name="AdminCategoryMessagesSection"/> <section name="AdminCategoryContentSection"/> </page> ``` -------------------------------- ### MFTF Action Group Declaration Example (Login) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test/action-groups This example demonstrates a complete action group declaration for logging into the Admin area in MFTF. It includes annotations for description, arguments with default values, and a sequence of actions like navigating to the page, filling fields, clicking login, and closing notifications. The `userInput` variables reference data values for test execution. ```xml <?xml version="1.0" encoding="UTF-8"?> <actionGroups xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:mftf:Test/etc/actionGroupSchema.xsd"> <actionGroup name="LoginAsAdmin"> <annotations> <description>Login to Backend Admin using provided User Data. PLEASE NOTE: This Action Group does NOT validate that you are Logged In.</description> </annotations> <arguments> <argument name="adminUser" type="entity" defaultValue="DefaultAdminUser"/> </arguments> <amOnPage url="{{AdminLoginPage.url}}" stepKey="navigateToAdmin"/> <fillField selector="{{AdminLoginFormSection.username}}" userInput="{{adminUser.username}}" stepKey="fillUsername"/> <fillField selector="{{AdminLoginFormSection.password}}" userInput="{{adminUser.password}}" stepKey="fillPassword"/> <click selector="{{AdminLoginFormSection.signIn}}" stepKey="clickLogin"/> <closeAdminNotification stepKey="closeAdminNotification"/> </actionGroup> </actionGroups> ``` -------------------------------- ### Example composer.json for Functional Test Module Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/mftf-tests-packaging This JSON file demonstrates the required format for a functional test module's composer.json. It specifies the 'magento2-functional-test-module' type and lists module dependencies using the 'suggest' block with specific type and name formats. ```json { "name": "magento/module-configurable-product-catalog-search-functional-test", "description": "test module for Magento_ConfigurableProduct and Magento_CatalogSearch", "type": "magento2-functional-test-module", "config": { "sort-packages": true }, "require": { "magento/magento2-functional-testing-framework": ">=2.5" }, "suggest": { "magento/module-configurable-product": "type: magento2-module, name: Magento_ConfigurableProduct, version: *", "magento/module-catalog-search": "type: magento2-module, name: Magento_CatalogSearch, version: *" }, "license": [ "OSL-3.0", "AFL-3.0" ] } ``` -------------------------------- ### Use Numbers in stepKey for Order Importance (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/tips-tricks Provides an exception to the rule of using descriptive `stepKey` names. This example shows when it is appropriate to use numbers within `stepKeys`, specifically when the order of operations is critical, such as in testing sort order. ```xml <createData entity="BasicMsiStock1" stepKey="createCustomStock1"/> <createData entity="BasicMsiStock2" stepKey="createCustomStock2"/> <createData entity="BasicMsiStock3" stepKey="createCustomStock3"/> <createData entity="BasicMsiStock4" stepKey="createCustomStock4"/> ``` -------------------------------- ### Automate customer sign-in steps (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test/actions This example demonstrates a sequence of actions to automate the customer sign-in process. It includes navigating to the sign-in page, filling in the email and password fields, and clicking the sign-in button. Each action uses specific selectors and input data, with `stepKey` attributes for referencing. ```xml <amOnPage url="{{StorefrontCustomerSignInPage.url}}" stepKey="amOnSignInPage"/> <fillField userInput="$$customer.email$$" selector="{{StorefrontCustomerSignInFormSection.emailField}}" stepKey="fillEmail"/> <fillField userInput="$$customer.password$$" selector="{{StorefrontCustomerSignInFormSection.passwordField}}" stepKey="fillPassword"/> <click selector="{{StorefrontCustomerSignInFormSection.signInAccountButton}}" stepKey="clickSignInAccountButton"/> ``` -------------------------------- ### MFTF Category Operations Example (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/metadata Illustrates how to define different operations (create, update, delete, get) for a 'category' data entity within an MFTF metadata file. This allows for comprehensive management of category data in tests. ```xml <operations> <operation type="create" dataType="category"> ... </operation> <operation type="update" dataType="category"> ... </operation> <operation type="delete" dataType="category"> ... </operation> <operation type="get" dataType="category"> ... </operation> </operations> ``` -------------------------------- ### Serve Allure Report (Bash) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/reporting Serves the generated Allure HTML report in a temporary folder and automatically opens it in the default web browser. This provides immediate access to test results for review. ```bash allure serve dev/tests/acceptance/tests/_output/allure-results/ ``` -------------------------------- ### Enable CLI Commands for Functional Testing Framework Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/getting-started Copy the sample .htaccess file to enable the Functional Testing Framework to send CLI commands to your Adobe Commerce or Magento Open Source instance. This step is performed in the project root directory. ```bash cp dev/tests/acceptance/.htaccess.sample dev/tests/acceptance/.htaccess ``` -------------------------------- ### Install hoa/console for Interactive Pause Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/interactive-pause This command installs the 'hoa/console' library, which is a prerequisite for the interactive pause feature to function correctly within the MFTF. Ensure you run this in your project's root directory. ```bash composer require hoa/console ``` -------------------------------- ### MFTF Section Example: Admin Category Sidebar Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/section An example of an MFTF section definition in XML for the 'AdminCategorySidebarActionSection'. It demonstrates how to declare button elements with their types and selectors. ```xml <?xml version="1.0" encoding="utf-8"?> <sections xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:mftf:Page/etc/SectionObject.xsd"> <section name="AdminCategorySidebarActionSection"> <element name="AddRootCategoryButton" type="button" selector="#add_root_category_button" timeout="30"/> <element name="AddSubcategoryButton" type="button" selector="#add_subcategory_button" timeout="30"/> </section> </sections> ``` -------------------------------- ### Build Tests Using Action Groups (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/tips-tricks Demonstrates how to build tests using action groups, even for single actions. This approach enhances maintainability for extension developers by allowing updates to multiple tests through a single action group modification. The 'Good' example shows the recommended structure. ```xml <test name="NavigateClamberWatchEntityTest"> <annotations> <!--some annotations--> </annotations> <actionGroup ref="StorefrontOpenProductPageActionGroup" stepKey="openProductPage"> <argument name="productUrl" value="{{ClamberWatch.url_key}}" /> </actionGroup> <actionGroup ref="StorefrontAssertProductNameOnProductPageActionGroup" stepKey="assertProductName"> <argument name="productName" value="{{ClamberWatch.name}}" /> </actionGroup> <actionGroup ref="StorefrontAssertProductSkuOnProductPageActionGroup" stepKey="assertProductSku"> <argument name="productSku" value="{{ClamberWatch.sku}}" /> </actionGroup> <actionGroup ref="StorefrontAssertProductPriceOnProductPageActionGroup" stepKey="assertProductPrice"> <argument name="productPrice" value="{{ClamberWatch.price}}" /> </actionGroup> <actionGroup ref="StorefrontAssertProductImagesOnProductPageActionGroup" stepKey="assertProductImage"> <argument name="productImage" value="{{ClamberWatch.image}}" /> </actionGroup> </test> ``` -------------------------------- ### Array Containing Data Field Example (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/metadata Demonstrates declaring an array that holds specific fields from other data types. This example shows an array of 'id' fields from the 'tax_rate' data type. ```xml <array key="tax_rate_ids"> <value>tax_rate.id</value> </array> ``` -------------------------------- ### Inline Alert Warning Example Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/best-practices A simple example of an inline alert with a warning variant, likely used for UI feedback or notifications within the framework's context. ```html <div data-variant="warning"> <!-- Alert content --> </div> ``` -------------------------------- ### Serve Allure Test Reports Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/getting-started Generate and serve visual representations of test reports using the Allure Framework. This command serves the artifacts located in the specified allure-results directory, making them accessible via a GUI. ```bash allure serve dev/tests/acceptance/tests/_output/allure-results/ ``` -------------------------------- ### Run Codeception Tests with Functional Testing Framework Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/mftf A wrapper command that executes `vendor/bin/codecept run` for the functional suite. Tests must be generated prior to execution. Supports running all tests, specific suites, or individual test files with various options. ```bash vendor/bin/mftf codecept:run [<suite|test>] --[<option(s)>] ``` ```bash # Run all tests in functional suite vendor/bin/mftf codecept:run functional ``` ```bash # Run all tests in functional suite with options vendor/bin/mftf codecept:run functional --verbose --steps --debug ``` ```bash # Run one test vendor/bin/mftf codecept:run functional Magento/_generated/default/AdminCreateCmsPageTestCest.php --debug ``` -------------------------------- ### Example of Bad CSS Selector Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/selectors This CSS selector is an example of what not to do. It relies on excessive hierarchical details, making it brittle and difficult to understand. If the DOM structure changes, this selector is likely to break. ```css #html-body > section > div > div > div > div ``` -------------------------------- ### Run MFTF Static Checks Help Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/mftf Displays help information for the MFTF static-checks command, outlining available subcommands and options. This is useful for understanding the full capabilities of the tool. ```bash vendor/bin/mftf static-checks --help ``` -------------------------------- ### Clone Adobe Commerce Repository Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/cicd Clones the Adobe Commerce or Magento Open Source repository from GitHub. This is the first step to setting up an instance for testing. ```bash git clone https://github.com/magento/magento2 ``` -------------------------------- ### XPath Parent Selector Examples Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/selectors The parent selector ('..') allows navigation up the DOM tree to select parent elements. These examples demonstrate how to select a parent or grandparent element based on text content or other attributes. ```xpath //*[text()='Unique Value']/../.. ``` ```xpath //div[text()='Unique Value']/../..//a ``` -------------------------------- ### Run Codeception Tests with Filters (Bash) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/codeception These bash commands demonstrate how to execute Codeception tests using the 'vendor/bin/codecept' CLI. They show how to run all tests, skip specific groups, or include specific groups while skipping others. The '-c' flag specifies the configuration file. ```bash vendor/bin/codecept run functional -c dev/tests/acceptance/codeception.yml ``` ```bash vendor/bin/codecept run functional --skip-group skip -c dev/tests/acceptance/codeception.yml ``` ```bash vendor/bin/codecept run functional --group example --skip-group skip -c dev/tests/acceptance/codeception.yml ``` -------------------------------- ### Explicit Page Declaration Example (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/page An example of an explicit page object declaration in XML format. This defines the 'AdminCategoryPage' with its URL, module, area, and associated sections. This page can be referenced directly in tests using its declared URL. ```xml <?xml version="1.0" encoding="UTF-8"?> <pages xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:mftf:Page/etc/PageObject.xsd"> <page name="AdminCategoryPage" url="catalog/category/" module="Magento_Catalog" area="admin"> <section name="AdminCategorySidebarActionSection"/> <section name="AdminCategorySidebarTreeSection"/> <section name="AdminCategoryBasicFieldSection"/> <section name="AdminCategorySEOSection"/> </page> </pages> ``` -------------------------------- ### Avoid Overly Specific Selectors (XPath Example) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/selectors Shows an overly specific XPath selector mirroring the CSS example above. Like its CSS counterpart, this XPath is fragile and prone to breaking due to minor structural modifications in the DOM. ```xpath //*[@id='container']/*[@class='dashboard-advanced-reports']/*[@class='dashboard-advanced-reports-description']/*[@class='dashboard-advanced-reports-title'] ``` -------------------------------- ### Codeception `codecept run` Command Reference Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/codeception Provides a comprehensive list of arguments and options for the `codecept run` command. This includes specifying suites, tests, overriding configurations, enabling extensions, and generating various report formats. ```bash Full reference: Arguments: suite suite to be tested test test to be run Options: -o, --override=OVERRIDE Override config values (multiple values allowed) -e, --ext=EXT Run with extension enabled (multiple values allowed) --report Show output in compact style --html[=HTML] Generate html with results [default: "report.html"] --xml[=XML] Generate JUnit XML Log [default: "report.xml"] --phpunit-xml[=PHPUNIT-XML] Generate PhpUnit XML Log [default: "phpunit-report.xml"] --tap[=TAP] Generate Tap Log [default: "report.tap.log"] --json[=JSON] Generate Json Log [default: "report.json"] --colors Use colors in output --no-colors Force no colors in output (useful to override config file) --silent Only outputs suite names and final results --steps Show steps in output -d, --debug Show debug and scenario output --bootstrap[=BOOTSTRAP] Execute custom PHP script before running tests. Path can be absolute or relative to current working directory [default: false] --no-redirect Do not redirect to Composer-installed version in vendor/codeception --coverage[=COVERAGE] Run with code coverage --coverage-html[=COVERAGE-HTML] Generate CodeCoverage HTML report in path --coverage-xml[=COVERAGE-XML] Generate CodeCoverage XML report in file --coverage-text[=COVERAGE-TEXT] Generate CodeCoverage text report in file --coverage-crap4j[=COVERAGE-CRAP4J] Generate CodeCoverage report in Crap4J XML format --coverage-phpunit[=COVERAGE-PHPUNIT] Generate CodeCoverage PHPUnit report in path --no-exit Don't finish with exit code -g, --group=GROUP Groups of tests to be executed (multiple values allowed) -s, --skip=SKIP Skip selected suites (multiple values allowed) -x, --skip-group=SKIP-GROUP Skip selected groups (multiple values allowed) --env=ENV Run tests in selected environments. (multiple values allowed) -f, --fail-fast Stop after first failure --no-rebuild Do not rebuild actor classes on start --seed=SEED Define random seed for shuffle setting --no-artifacts Don't report about artifacts -h, --help Display this help message -q, --quiet Do not output any message -V, --version Display this application version --ansi Force ANSI output --no-ansi Disable ANSI output -n, --no-interaction Do not ask any interactive question -c, --config[=CONFIG] Use custom path for config -v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug ``` -------------------------------- ### Run Tests from Manifest File (Bash) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/commands/mftf Executes all tests specified in a `testManifest.txt` file. This command requires a pre-existing manifest file and does not generate tests. It accepts a path to the manifest file as an argument. ```bash vendor/bin/mftf run:manifest path/to/your/testManifest.txt ``` -------------------------------- ### Example of Bad XPath Selector Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/selectors This XPath selector is also an example of poor practice due to its reliance on deep DOM hierarchy. Such selectors are prone to breaking with minor HTML changes and lack clarity regarding the intended target element. ```xpath //*[@id='html-body']/section/div/div/div/div ``` -------------------------------- ### Create Adobe Commerce Project with Composer Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/cicd Creates a new Adobe Commerce or Magento Open Source project using Composer. This is an alternative method for setting up a testing instance. ```bash composer create-project --repository=https://repo.magento.com/ magento/project-community-edition magento2ce ``` -------------------------------- ### Extending LoginAsAdmin with Two-Factor Authentication Field Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test/action-group-best-practices This XML snippet shows another example of extending the `LoginAsAdmin` action group, this time to fill a Two-Factor Authentication input field. Similar to the CAPTCHA example, it uses `before="signIn"` to insert the action at the appropriate point in the sequence. ```xml <actionGroup name="LoginAsAdmin"> <fillField selector="{{TwoFactorSection.twoFactorInput}}" before="signIn" .../> </actionGroup> ``` -------------------------------- ### Configure Adobe Commerce Instance Settings Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/cicd Sets essential configuration values for an Adobe Commerce or Magento Open Source instance. These commands adjust locale, admin sharing, form key usage, and WYSIWYG editor enablement. ```bash bin/magento config:set general/locale/timezone America/Los_Angeles bin/magento config:set admin/security/admin_account_sharing 1 bin/magento config:set admin/security/use_form_key 0 bin/magento config:set cms/wysiwyg/enabled disabled ``` -------------------------------- ### Run All Functional Tests with Codeception Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/getting-started Execute all generated functional tests using the Codeception runner. This command references the codeception.yml configuration file located in the acceptance directory. ```bash vendor/bin/codecept run functional -c dev/tests/acceptance/codeception.yml ``` -------------------------------- ### Absolute vs. Relative XPath Selectors Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/selectors Absolute XPath selectors ('/') start from the root of the document and specify a direct path, making them brittle. Relative XPath selectors ('//') search for elements anywhere in the DOM, starting from a specified element or the entire document if none is specified, offering more flexibility. ```xpath /html/body/div[2]/div/div[2]/div[1]/div[2]/form/div/input ``` ```xpath //div[@class='form-group']//input[@id='user-message'] ``` -------------------------------- ### Open Allure Reports Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/reporting Opens the generated Allure report in the default web browser. This command assumes a report has already been generated in the current directory's `allure-report/` folder. ```bash allure open ``` -------------------------------- ### XML: Example of Parameterized Selectors in Page Objects Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/best-practices Demonstrates the correct way to handle parameterized selectors in page object definitions within the Functional Testing Framework. It shows how to define individual elements instead of using parameterized selectors for static elements. ```xml <element name="relatedProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='related']"/> <element name="upSellProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='upsell']"/> <element name="crossSellProductSectionText" type="text" selector=".fieldset-wrapper.admin__fieldset-section[data-index='crosssell']"/> ``` -------------------------------- ### Manual Test Steps in XML Format Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/test-prep This XML snippet outlines the manual steps for creating and verifying a product in Adobe Commerce. It includes navigation, form filling, saving, and storefront verification. These steps serve as the basis for converting into an automated functional test. ```xml <!-- Navigate to Catalog -> Products page (or just open by link) --> <!-- Fill field "Name" with "Simple Product %unique_value%" --> <!-- Fill field "SKU" with "simple_product_%unique_value%" --> <!-- Fill field "Price" with "500.00" --> <!-- Fill field "Quantity" with "100" --> <!-- Fill field "Weight" with "100" --> <!-- Click "Save" button --> <!-- See success save message "You saved the product." --> <!-- Navigate to Catalog -> Products page (or just open by link) --> <!-- See created product is in grid --> <!-- See "Name" in grid is valid --> <!-- See "SKU" in grid is valid --> <!-- See "Price" in grid is valid --> <!-- See "Quantity" in grid is valid --> <!-- Open Storefront Product Page and verify "Name", "SKU", "Price" --> ``` -------------------------------- ### Clone Magento 2 Repository using Git Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/getting-started This snippet demonstrates how to clone the Magento 2 source code repository using either the HTTPS or SSH protocol. Ensure Git is installed and configured on your development environment. ```bash git clone https://github.com/magento/magento2.git ``` ```bash git clone git@github.com:magento/magento2.git ``` -------------------------------- ### Open Allure Report (Bash) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/reporting Opens a previously generated Allure HTML report located in the specified directory within the default web browser. This command is useful for revisiting existing reports. ```bash allure open dev/tests/acceptance/tests/_output/allure-report ``` -------------------------------- ### Define and Use Parameterized Selectors in XML Tests Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/section/parameterized-selectors This snippet demonstrates the step-by-step construction of an XML test case using parameterized selectors. It shows how to initialize a test, add a click action, and progressively define a parameterized selector by referencing sections, elements, and various parameter types like default category data, string literals, persisted data, and variables. ```xml <test name="SampleTest"> </test> ``` ```xml <test name="SampleTest"> <click selector="" stepKey="click1"/> </test> ``` ```xml <test name="SampleTest"> <click selector="{{}}" stepKey="click1"/> </test> ``` ```xml <test name="SampleTest"> <click selector="{{SampleSection}}" stepKey="click1"/> </test> ``` ```xml <test name="SampleTest"> <click selector="{{SampleSection.threeParamElement}}" stepKey="click1"/> </test> ``` ```xml <test name="SampleTest"> <click selector="{{SampleSection.threeParamElement()}}" stepKey="click1"/> </test> ``` ```xml <test name="SampleTest"> <click selector="{{SampleSection.threeParamElement(_defaultCategory.is_active)}}" stepKey="click1"/> </test> ``` ```xml <test name="SampleTest"> <click selector="{{SampleSection.threeParamElement(_defaultCategory.is_active,'StringLiteral',$createDataKey.id$)}}" stepKey="click1"/> </test> ``` -------------------------------- ### createData Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test/actions Creates an entity (for example, a category or product). To create an entity, the Functional Testing Framework makes a `POST` request to the Adobe Commerce or Magento Open Source API. ```APIDOC ## createData ### Description Creates an entity (for example, a category or product). To create an entity, the Functional Testing Framework makes a `POST` request to the Adobe Commerce or Magento Open Source API according to the [data](/commerce/testing/functional-testing-framework/data) and [metadata](/commerce/testing/functional-testing-framework/metadata) of the entity to be created. ### Method Not applicable (this is a custom action within the framework, not a direct HTTP method). ### Endpoint Not applicable. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body This action is configured using XML attributes. It can optionally contain one or more `requiredEntity` child elements. - **`entity`** (string) - Required - Type of entity to be created. - **`storeCode`** (string) - Optional - ID of the store within which the data is created. - **`stepKey`** (string) - Required - A unique identifier of the action. - **`before`** (string) - Optional - `stepKey` of action that must be executed next. - **`after`** (string) - Optional - `stepKey` of preceding action. ### Request Example ```xml <!-- Create an entity with the "SampleProduct" name. --> <createData entity="SampleProduct" stepKey="createSampleProduct"/> ``` ### Response #### Success Response This action does not return a specific response body. Its success is determined by the creation of the entity in the system. #### Response Example N/A ``` -------------------------------- ### Array with Primitive Value Example (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/metadata Demonstrates how to declare an array of primitive data types, such as integers, using the '<array>' and '<value>' tags. This structure is used for defining arrays in metadata. ```xml <array key="tax_rate_ids"> <value>integer</value> </array> ``` -------------------------------- ### XML Description Annotation Example Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test/annotations This XML snippet shows the usage of the `<description>` annotation, which is an implementation of the Allure tag for extended test descriptions. It requires a 'value' attribute of type string. ```xml <description value="Add Catalog via Admin"/> ``` -------------------------------- ### Prioritize Critical Actions in <after> Block (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/tips-tricks Illustrates the best practice of performing non-browser driving actions, such as `magentoCLI` commands and data deletion, before UI-heavy actions within the `<after>` block. This ensures critical setup or cleanup tasks are executed even if UI steps fail. ```xml <after> <magentoCLI command="indexer:set-mode" arguments="schedule" stepKey="setIndexerMode"/> <magentoCLI command="config:set catalog/frontend/flat_catalog_category 0" stepKey="setFlatCatalogCategory"/> <deleteData createDataKey="category" stepKey="deleteCategory"/> <deleteData createDataKey="createSimpleProduct" stepKey="deleteSimpleProduct"/> <actionGroup ref="AdminDeleteStoreViewActionGroup" stepKey="deleteStoreViewEn"> <argument name="customStore" value="customStoreEN"/> </actionGroup> <actionGroup ref="AdminDeleteStoreViewActionGroup" stepKey="deleteStoreViewFr"> <argument name="customStore" value="customStoreFR"/> </actionGroup> <actionGroup ref="logout" stepKey="logout"/> </after> ``` ```xml <after> <actionGroup ref="AdminDeleteStoreViewActionGroup" stepKey="deleteStoreViewEn"> <argument name="customStore" value="customStoreEN"/> </actionGroup> <actionGroup ref="AdminDeleteStoreViewActionGroup" stepKey="deleteStoreViewFr"> <argument name="customStore" value="customStoreFR"/> </actionGroup> <deleteData createDataKey="category" stepKey="deleteCategory"/> <deleteData createDataKey="createSimpleProduct" stepKey="deleteSimpleProduct"/> <magentoCLI command="config:set catalog/frontend/flat_catalog_category 0" stepKey="setFlatCatalogCategory"/> <magentoCLI command="indexer:set-mode" arguments="schedule" stepKey="setIndexerMode"/> <actionGroup ref="logout" stepKey="logout"/> </after> ``` -------------------------------- ### Array Containing Data Entities Example (XML) Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/metadata Illustrates how to define an array that contains complex data entities. The '<value>' tag specifies the data type of the entities within the array, such as 'product_option'. ```xml <array key="product_options"> <value>product_option</value> </array> ``` -------------------------------- ### Company ID JSON Payload Example Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/metadata This JSON object represents the payload structure for creating a company ID, as defined by the 'CreateCompanyId' metadata. It contains a single key 'company_id' with an integer value. ```json { "company_id": 1 } ``` -------------------------------- ### URL Construction and Authentication Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/metadata Explains how URLs are constructed by concatenating environment base URLs with specific paths and outlines the available authentication methods for API requests. ```APIDOC ## URL Construction and Authentication ### Description This section explains how URLs are constructed by concatenating environment base URLs with specific paths and outlines the available authentication methods for API requests. ### URL Construction - `*url` - full URL is a concatenation of `*ENV.baseUrl` + `/rest/` + `*URL`. To reuse data of a required entity or returned response use a field key wrapped in curly braces such as `{sku}`. When the data to reuse is of a different type, declare also the type of data such as `{product.sku}`. Example: `"/V1/products/{product.sku}/media/{id}"`. ### Authentication - `**auth` - available values: - `adminOath` is used for REST API persistence in the Admin area with [OAuth-based authentication](https://developer.adobe.com/commerce/webapi/get-started/authentication/gs-authentication-oauth/). - `adminFormKey` is used for HTML form persistence in the Admin area. - `customerFormKey` is used for HTML form persistence in the Customer area. - `anonymous` is used for REST API persistence without authorization. ``` -------------------------------- ### Company Relation JSON Payload Example Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/metadata This JSON object illustrates the payload for creating a company relation. It includes a 'relations' array, where each element is an object containing the 'company_id' of the child company to be associated with the parent. ```json { "relations": [ { "company_id": 1 } ] } ``` -------------------------------- ### CSS Attribute Selectors Source: https://developer.adobe.com/commerce/testing/functional-testing-framework/test-writing/selectors Provides examples of CSS attribute selectors, which allow targeting elements based on their attributes and attribute values. This is useful for more specific element selection. ```CSS [attribute] [attribute=value] [attribute~=value] [attribute|=value] [attribute^=value] [attribute$=value] [attribute*=value] ```