Create dynamic test suite at runtime
A dynamic test suite is a test suite with multiple test cases added via a search query. You can dynamically add additional test cases while running the test suite. To learn more about dynamic test suites in Katalon Studio, you can refer to this document: Dynamic test suite in Katalon Studio.
From Katalon Studio version 7.8.2 onwards, you can create a dynamic test suite at runtime with Katalon Runtime Engine (KRE). This article guides you on how to do so.
- Katalon Studio Enterprise version 7.8.2 onwards.
- Katalon Runtime Engine version 7.8.2 onwards.
- An active Katalon Runtime Engine license.
You can download the sample project from our Github repository: Dynamic test suite sample.
Enable the Query Provider
The Query Provider is to provide the query syntax to search test artifacts within the dynamic test suite. When you implement the dynamic test suite for the first time, the Query Provider is set to No query provider available by default.
To enable the query provider for the dynamic test suite, install one of the following plugins from the Katalon Store:
Return to Katalon Studio and activate your plugin. To do so, click on the Profile icon, then click Reload Plugin.
- To use plugins in console mode, while generating commands, use the API Key of the users who have the plugin installed. Both API key command-line options work, either
- From version 7.7.0 onwards, if you belong to more than one Organization subscribing to Runtime Engine licenses, you can choose which Organization validates your license usage with the following command line:
After you successfully activate the plugin, the Query Provider automatically applies the query syntax of the installed plugin. For example, after activating the Test Case Management with Tags plugin, the Query Provider is set to Advanced Tag Search. You can now add test cases for dynamic test suite execution.
Create dynamic test suite at runtime with KRE
Create a new dynamic test suite
Open Katalon Studio and create a new dynamic test suite. In the Test Explorer panel, right-click at the Test Suites folder > New > Dynamic Test Suite.
The New dialog opens. Name the dynamic test suite. Here, we name the test suite DTS_Verify Successful Login and Appointment.
Generate commands with the command builder
You can use the command builder in Katalon Studio (KS) to generate commands quickly and precisely.
Follow these steps:
Click on the Build CMD button in the main toolbar. The Generate Command for Console Mode dialog appears.
Configure your execution as follows:
2.1. Test Suite: select the dynamic test suite you want to execute. Here, we want to execute the DTS_Verify Successful Login and Appointment dynamic test suite.
2.2. Executive Platform: Click Edit in each field to choose the environment and execution profile you want to execute with. Here, we select Chrome and the default execution profile.
- Katalon API key: the API key is auto-generated.
- Katalon Organization: Select the Organization that validates your license usage.
For detailed information on the command builder, see Command Builder.
Click Generate Command. The Generated Command dialog appears as below.
For example, the sample command is:
./katalonc -noSplash -runMode=console -projectPath="/Users/HTK/Downloads/dynamic-test-suite-sample-main/test.prj" -retry=0 -testSuitePath="Test Suites/DTS_Verify Successful Login and Appointment" -browserType="Chrome" -executionProfile="default" -apiKey="<api-key>" --config -proxy.auth.option=NO_PROXY -proxy.system.option=NO_PROXY -proxy.system.applyToDesiredCapabilities=true
Click Copy to Clipboard. Open the KRE folder in the command prompt or terminal, then paste the generated command to your cmd/terminal.
Pass the search query to the CLI for dynamic test suite execution
To pass the search query to the CLI for the dynamic test suite execution, add the
-testSuiteQuery parameter to the generated command. You can learn more about the
-testSuiteQuery parameter in this document: General commands.
For example, we want to execute the TC1_Verify Successful Login and TC2_Verify Successful Appointment test cases in the dynamic test suite, we add the
-testSuiteQuery parameter as follows:
./katalonc -noSplash -runMode=console -projectPath="/Users/HTK/Downloads/dynamic-test-suite-sample-main/test.prj" -retry=0 -testSuitePath="Test Suites/DTS_Verify Successful Login and Appointment" -browserType="Chrome" -executionProfile="default" -apiKey="<api-key>" --config -proxy.auth.option=NO_PROXY -proxy.system.option=NO_PROXY -proxy.system.applyToDesiredCapabilities=true -testSuiteQuery="ids=(Test Cases/TC1_Verify Successful Login,Test Cases/TC2_Verify Successful Appointment)"
- Suppose the associated dynamic test suite has an existing search query. In that case, the
-testSuiteQueryparameter overrides the current search query and executes the dynamic test suite with the stated search query in the CLI instead.
- You can add one or more test cases via the search query by test case IDs and separate them by commas.
- You can only search for one keyword at a time when searching by tag, description, or comment.
Execute the dynamic test suite in KRE
After finalizing parameters in your command, hit Enter to execute the dynamic test suite.
- Make sure to update the browser by clicking Tools > Update WebDrivers > Choose browser. You can learn more about updating webdrivers here: Update or Downgrade WebDrivers.
After the test suite execution, to view your test reports, go to the Reports folder in the Test Explorer panel.
Alternatively, you can also view your reports and details in
<your-project-folder>/Reports. Katalon Studio supports exporting test reports into different formats, such as HTML, CSV, PDF, and JUnit. You can learn more about exporting reports here: Generate reports.
- For real-time monitoring and better reporting capabilities, consider integrating your project with Katalon TestOps. Learn more about test result reports here: Upload Test Results to Katalon TestOps from Katalon Studio.