Edit

Share via

AzureTestPlan@0 - Azure Test Plan v0 task

Run manual and automated tests in test plan in Java, JavaScript and Python language.

Note

This task is in Public Preview.

Syntax

# Azure Test Plan v0
# Run manual and automated tests points of test plan for different testing frameworks like Maven and Gradle for Java, PyTest for Python and Jest for JavaScript.
- task: AzureTestPlan@0
  inputs:
    testSelector: # 'manualTests' | 'automatedTests'. Required. Test cases to be executed. 
    testPlanOrRunSelector: 'testPlan' # 'testPlan' | 'testRun'. Required. Select tests using. Default: testPlan.
    #testRunId: '$(test.RunId)' # string. Required when testPlanOrRunSelector = testRun. Test Run. Default: $(test.RunId).
    testPlan: # string. Required when testPlanOrRunSelector = testPlan. Test plan. 
    testSuite: # string. Required when testPlanOrRunSelector = testPlan. Test suite. 
    testConfiguration: # string. Required. Test configuration. 
    #testLanguageInput: # 'JavaMaven' | 'JavaGradle' | 'Python' | 'JavaScriptJest'. Select Test framework language. 
    #pomFilePath: # string. Optional. Use when testLanguageInput = JavaMaven. Pom file path. 
    #gradleFilePath: # string. Optional. Use when testLanguageInput = JavaGradle. Gradle file path. 
    #failTaskOnFailedTests: true # boolean. Fail if there are test failures. Default: true.
    #failTaskOnFailureToPublishResults: false # boolean. Fail if there is failure in publishing test results. Default: false.
    #failTaskOnMissingResultsFile: false # boolean. Fail if no result files are found. Default: false.
  # advanced
    #publishRunAttachments: true # boolean. Upload test results files. Default: true.

Inputs

testSelector - Test cases to be executed
string. Required. Allowed values: manualTests (Manual tests), automatedTests (Automated tests).

  • Manual tests: Use this option to trigger manual tests from your test plan.
  • Automated tests: Use this option to run tests from your test plan that have automated test method associated with it.
    • .

testPlanOrRunSelector - Select tests using
string. Required. Allowed values: testPlan (Test plan), testRun (Test run). Default value: testPlan.

  • Test assembly: Use this option to specify one or more test assemblies that contain your tests. You can optionally specify a filter criteria to select only specific tests.
  • Test plan: Use this option to run tests from your test plan that have an automated test method associated with it.
  • Test run: Use this option when you are setting up an environment to run tests from the Test hub. This option should not be used when running tests in a continuous integration / continuous deployment (CI/CD) pipeline.
    • .

testRunId - Test Run
string. Required when testPlanOrRunSelector = testRun. Default value: $(test.RunId).

Test run based selection is used when triggering automated test runs from the test hub, value for this should be kept as it is.


testPlan - Test plan
string. Required when testPlanOrRunSelector = testPlan.

Type or paste the test plan ID containing test suites with test cases.


testSuite - Test suite
string. Required when testPlanOrRunSelector = testPlan.

Select one or more test suites containing test cases.


testConfiguration - Test configuration
string. Required.

Select Test Configuration.


testLanguageInput - Select Test framework language
string. Allowed values: JavaMaven (Java - Maven), JavaGradle (Java - Gradle), Python (Python - PyTest), JavaScriptJest (JavaScript - Jest).

Test Framework Language of automated tests in test plan.


pomFilePath - Pom file path
string. Optional. Use when testLanguageInput = JavaMaven.

Relative path from the repository root to the Maven POM file.


gradleFilePath - Gradle file path
string. Optional. Use when testLanguageInput = JavaGradle.

Relative path from the repository root to the build.gradle file.


publishRunAttachments - Upload test results files
boolean. Default value: true.

Upload logs and other files containing diagnostic information collected when the tests were run.


failTaskOnFailedTests - Fail if there are test failures
boolean. Default value: true.

Fail the task if there are any test failures. Check this option to fail the task if test failures are detected in the result files.


failTaskOnFailureToPublishResults - Fail if there is failure in publishing test results
boolean. Default value: false.

Fail if there is failure in publishing test results. Check this option to fail the task if publishing test results is failed partially.


failTaskOnMissingResultsFile - Fail if no result files are found
boolean. Default value: false.

Fail the task if no result files are found.


Task control options

All tasks have control options in addition to their task inputs. For more information, see Control options and common task properties.

Output variables

None.

Remarks

Integrate your automated tests with a test plan

Follow the steps below to integrate your automated tests with manual test cases and execute them from Test Plans or via the Azure Test Plan task in your build pipelines:

  1. Ensure your project is in Azure DevOps Repos (or GitHub Repos).

  2. Create a pipeline that runs the tests using tasks such as Gradle or Maven. If your tests are already executed as part of an existing pipeline, you can skip this step. Running the tests is essential—only after they have been executed at least once will they be available for association with manual test cases.

     trigger: none

 pool:
   vmImage: ubuntu-latest

 steps:
 - task: Gradle@3
   inputs:
     gradleWrapperFile: 'gradlew'
     workingDirectory: '$(Build.SourcesDirectory)'
     tasks: 'build'
     publishJUnitResults: true
     testResultsFiles: '**/TEST-*.xml'
     testRunTitle: 'gridinitialexecution'
     javaHomeOption: 'JDKVersion'
     sonarQubeRunAnalysis: false
     spotBugsAnalysis: false

  3. The next step is to link the test cases from the Test tab of the pipeline run summary to a manual test case work item. To do this, create a new test case or use an existing one (note the Automation Status column for TC4):

    Screenshot of a test case without automation.

  4. Once you have a test case, return to the pipeline run summary page and associate your automated test with the test case.

    Screenshot of the association experience in Azure Pipelines

  5. Here’s how you can tell if a manual test case has an associated automated test:

    Screenshot of a test case with automation.

    Screenshot of a test case with an associated automated test.

  6. Now that the test case is associated with a manual test case work item, you can execute it as part of a pipeline by passing the test plan as an argument in the Azure Test Plan task.

    trigger: none
pool:
  vmImage: ubuntu-latest

steps:
- task: AzureTestPlan@0
  inputs:
    testSelector: 'automatedTests'
    testPlanOrRunSelector: 'testPlan'
    testPlan: '21294'
    testSuite: '229461'
    testConfiguration: '82'
    testLanguageInput: 'JavaGradle'

Save and run the pipeline.

Here is the pipeline result summary, which shows the outcome for TC4 along with other test cases:

Screenshot of the associated automation tab.

Requirements

Requirement Description
Pipeline types YAML, Classic build, Classic release
Runs on Agent, DeploymentGroup
Demands None
Capabilities This task does not satisfy any demands for subsequent tasks in the job.
Command restrictions Any
Settable variables Any
Agent version 2.144.0 or greater
Task category Test