Parallel Test Execution With Appium, Serenity ScreenPlay, and Android Studio Emulators

In the realm of Android app development, thorough testing is paramount to ensure the quality and reliability of applications across various devices and scenarios. However, with the myriad of devices and configurations in the Android ecosystem, testing can become a time-consuming and challenging endeavor. Fortunately, by leveraging the power of Serenity, Android Studio, and Appium, developers can streamline and accelerate the testing process through parallel execution.

Testing Android applications comprehensively involves evaluating their functionality, performance, and user experience across different devices and operating system versions. Traditionally, testing on multiple devices sequentially can be laborious and inefficient, leading to prolonged testing cycles and delayed releases. Parallel execution offers a solution by allowing tests to run concurrently on multiple devices, significantly reducing testing time and accelerating feedback loops.

Step 1: Setting up Parallel Execution Environment

To set up parallel test execution for Android applications, we’ll utilize Serenity, a powerful testing framework that integrates seamlessly with popular testing tools like Cucumber and JUnit. Additionally, we’ll leverage Android Studio for emulator management and Appium for automating interactions with Android applications. We need to create two emulators (emulator-5554 and emulator-5556) using Android Studio and make those running. We also need to create two (port: 4723 and 4724) Appium server using Appium GUI. Depending on the requirements more emulators and servers can be created which can communicates with each other for parallel test execution.

Step 2: Configure Test Runner

In our test project, we configure the test runner using Cucumber with Serenity. Here’s a snippet of the TestSuite.java class:

@RunWith(CucumberWithSerenity.class)
@CucumberOptions(
plugin = {
„pretty“,
„html:target/serenity-reports/serenity-html-report“,
„json:target/serenity-reports/cucumber_report.json“
},
tags = („@emulator1 or @emulator2“),
features = {
„src/test/resources/features/“
}
)
public class TestSuite {
}

Test Runner Configuration:

• @RunWith(CucumberWithSerenity.class): Specifies that the test suite will be executed using the CucumberWithSerenity test runner, which integrates Serenity reporting with Cucumber.
• @CucumberOptions(…): Provides options to configure the behaviour of the Cucumber test execution. Here’s a breakdown of the options specified:
  • plugin: Configures the plugins used for reporting. In this case, it specifies that test results should be outputted in a „pretty“ format and also generate HTML and JSON reports in the specified directories.
  • tags: Specifies the tags to include for test execution. Tests tagged with either „@emulator1“ or „@emulator2“ will be executed. For our case, feature files with ‘@emulator1’ tag will be executed on ‘emulator-5554’ and feature files with ‘@emulator2’ tag will be executed on ‘emulator-5556’.
  • features: Specifies the directory where feature files containing Cucumber scenarios are located. Feature files are typically stored in the „src/test/resources/features/“ directory.

Step 3: Configure pom.xml

Next, in our Maven project’s pom.xml file, we define profiles for each emulator device. In our case, we have two profile that represents two individual emulators (emulator-5554 and emulator-5556). These profiles contain configurations for running tests targeting specific emulators in parallel. Here’s a simplified snippet:

<profile>
<id>emulator-5554</id>
<build>
<directory>src/test/target/target-5554</directory>
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<!–<includes>
<include>**/TestSuite.java</include>
</includes>–>
<argLine>-Dudid=emulator-5554</argLine>
<argLine>-Dhub=http://localhost:4723/wd/hub</argLine>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>net.serenity-bdd.maven.plugins</groupId>
<artifactId>serenity-maven-plugin</artifactId>
<version>${serenity.version}</version>
<executions>
<execution>
<phase>post-integration-test</phase>
<goals>
<goal>aggregate</goal>
</goals>
<configuration>
<outputDirectory>src/test/reports/report-5554/emulator-5554-reports-${buildNumber}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>

Key-Indicators:

• <profile>: This line indicates the start of a Maven profile definition.
• <id>emulator-5554</id>: This line specifies the identifier for the profile, which is set to „emulator-5554“. Profiles are used in Maven to customize the build process for specific environments or scenarios.
• <build>: This line indicates the start of the build configuration for the profile.
• <directory>src/test/target/target-5554</directory>: This line specifies the output directory for the build process. Here, it is set to „src/test/target/target-5554“. We have to define individual build directory for each profile. Because in parallel run if those are not separated, there will be compilation problem.
• <goal>integration-test</goal>: This line specifies the „integration-test“ goal of the Maven Failsafe Plugin, which is responsible for running integration tests.
• <goal>verify</goal>: This line specifies the „verify“ goal of the Maven Failsafe Plugin, which is responsible for verifying the results of the integration tests.
• <argLine>-Dudid=emulator-5554</argLine>: This line specifies a command-line argument to be passed to the Maven Failsafe Plugin. Here, it sets the „udid“ system property to „emulator-5554“.
• <argLine>-Dhub=http://localhost:4723/wd/hub</argLine>: This line specifies another command-line argument to be passed to the Maven Failsafe Plugin. Here, it sets the „hub“ system property to the URL of the Appium server.
• <phase>post-integration-test</phase>: This line specifies the Maven build phase in which the plugin execution should occur. Here, it is set to „post-integration-test“.
• <goal>aggregate</goal>: This line specifies the „aggregate“ goal of the Serenity Maven Plugin, which aggregates test results for reporting.
• <outputDirectory>src/test/reports/report-5554/emulator-5554-reports-${buildNumber}</outputDirectory>: This line specifies the output directory for the aggregated test reports generated by the Serenity Maven Plugin. It includes a placeholder „${buildNumber}“ for dynamic naming of the reports based on the Maven build number.

This configuration defines how Maven should handle the build process for the „emulator-5554“ profile, including running integration tests with the Maven Failsafe Plugin and generating aggregated test reports with the Serenity Maven Plugin. For ‘emulator-5556’ there is another profile in pom.xml

Step 4: Our Feature files

Example of our feature files:

1. Outbox.feature

2. Inbox.feature

4.1) Functionality: Edit outbox

As an insured person, I would like to be able to process my outgoing mail

@emulator1
Scenario: View and delete a document from your Outbox
Assuming Peter Outbox has logged into the mobile portal
If he wants to view a document in his outbox
The document “Resources” should then be in your outbox

4.2) Functionality: Edit inbox

As an insured person, I would like to be able to process my inbox

@emulator2
Scenario: View the inbox
Assuming Adi has imported the test data for Show Inbox
Assuming Klaus Inbox has logged into the portal again from Mobile
When he looks at his inbox
Then he should be able to see all his messages in his inbox
And all new messages should be marked as unread

Step 5: Configure Serenity.Conf file

Now we need to configure our serenity.conf file in a how that our the parameters from pom.xml file can interact with it and make a successful run for each individual emulator to make a parallel execution.

appium {
hub = „${hub}“
automationName = „uiautomator2“
platformName = „Android“
platformVersion = „11.0“
udid = „${udid}“
newCommandTimeout = „1000“
browserName = „chrome“
}

hub = „${hub}“: This line sets the hub URL for connecting to the Appium server. The value „${hub}“ suggests that the URL is retrieved from a system property named „hub“. This allows for dynamic configuration of the Appium hub URL.

udid = „${udid}“: This line specifies the unique device identifier (UDID) of the device on which the tests will run. The value „${udid}“ suggests that the UDID is retrieved from a system property named „udid“. This allows for dynamic configuration of the device ID.

Step 6: Run Maven Command

Here we are going to execute our parallel test using maven command in the terminal as follows:

mvn clean verify -P emulator-5554 -Dudid=emulator-5554 Dtags=”@emulator1” -Dhub=https://localhost:4723/wd/hub

mvn clean verify -P emulator-5556 -Dudid=emulator-5556 Dtags=”@emulator2” -Dhub=https://localhost:4724/wd/hub

Let’s break down the commands:

• mvn clean verify: This part of the command instructs Maven to clean the project (remove any existing build files) and then verify it (compile, test, and package the code).
• P emulator-5554: This specifies the Maven profile to activate during the build process. In this case, it activates the profile named „emulator-5554“.
• Dudid=emulator-5554: This sets a system property named „udid“ to the value „emulator-5554“. This property specifies the unique device identifier (UDID) of the emulator or device to be used for testing that helps the configuration file to determine the appropriate emulator.
• Dtags=”@emulator1”: This sets a system property named „tags“ to the value „@emulator1“. This property is used to specify the tags or categories of tests to run. In this case, it indicates that tests tagged with „@emulator1“ should be executed.
• Dhub=https://localhost:4723/wd/hub: This sets a system property named „hub“ to the value „https://localhost:4723/wd/hub“. This property specifies the URL of the Appium server hub where the tests will be executed. Therefore, configuration understand which hub to be chosen for which emulator to run.

Below is how the setup looks like while two emulators are running different feature files in parallel:

After successful or failure run, it should generate two separate report for both emulators (one for emulator-5554 and other for emulator-5556 including the date and time of test execution) into the pre-defined report output directory which we have previously configured in our pom.xml file.

Best of luck.