Test Runs

This chapter explains how to create, search, edit, execute, and generate reports for Test Runs in Kiwi TCMS. A Test Run in Kiwi TCMS contains the execution results of selected test cases against particular product builds and environment.

To view Test Runs you have created or are assigned to you click Main menu::DASHBOARD or alternatively click Personal menu::My Test Runs.

Searching for Test Runs

To search for Test Runs:

  1. From the Main menu click SEARCH::Search Test Runs.

    The Testing menu 2

  2. In the Search Test Run screen, enter the required search details.

    The Search Test Run screen

  3. Click Search button and the results will appear.

    Test Run search results

Creating a Test Run

Test Runs are created for a specific Test Plan. Only Test Cases with a status of CONFIRMED can be added to the Test Run. A Test Run can be assigned to any user in Kiwi TCMS. To create a Test Run:

  1. Open a Test Plan and click the Cases tab

  2. From the Run sub-menu click on Write new run

    The New Run button

  3. In the Create New Test Run screen, perform the following actions:

    • Edit the Summary.
    • Select the Product.
    • Select the Product Version.
    • Select the Build.
    • Edit the Run Manager.
    • Edit the Default Tester.
    • If applicable, select the Set Status Automatically checkbox.
    • Enter the Estimated time.
    • Enter any Notes.
    • Select Environment property values.

    The Create New Test Run screen

  4. Test Cases will be shown under the above screen.

  5. Click Remove action on any Test Cases that are not required for this Test Run.

  6. Click Save button.

Note

Kiwi TCMS notifies the default tester by email that they have been assigned a new Test Run!

Add Test Cases to an existing Test Run

To add a Test Case to an existing Test Run:

  1. Open the Test Plan containing this Test Case.

  2. Select Test Cases you want to add.

  3. From Run sub-menu click Add into Run item.

    The Add cases to run button

  4. Select the Test Run to which Test Cases will be added.

  5. Click Update button.

    The Update button

Note

Test Cases can be added via the Test Run view as well. While the Test Run is opened youmay use the Cases sub-menu to add/remove other Test Cases to this Test Run.

Cloning a Test Run

Test Runs can be cloned for easier creation of testing tasks between team members. To clone a Test Run:

  1. Open the Test Run.

  2. Select which Test Case executions (aka test case-runs) to be cloned. Use a filter, if required, to help restrict the number of visible runs.

  3. Click Clone button at the top of the page.

    The Clone button 2

  4. Enter the details for the cloned Test Run. Details are auto-populated from the original.

  5. Click Save button.

Editing a Test Run

The Edit function modifies fields in a Test Run.

  1. Open the Test Run to be edited, and then click Edit button.
  2. Edit the fields as required:
    • Summary
    • Product
    • Product version
    • Manager
    • Default Tester
    • Estimated Time
    • Environment Property values
    • Notes
    • Finished
  3. Click Save button.

Changing the status of a Test Run

A Test Run’s status can be changed from Running to Finished even if all Test Cases have not been completed.

If the check box Set Status Automatically is selected in the test run, when all the test cases in the run have a passed, failed or blocked result the test run’s status will be changed to Finished.

To change the status of a Test Run:

  1. Open the Test Run.

  2. Click Set to Finished.

    The Set to finished button

  3. To re-activate a Test Run, click Set to Running.

    The Set to running button

Note

It is also possible to change the status of a Test Run from the Edit Test Run menu.

Deleting a Test Run

To delete a Test Run:

  1. Open the Test Run to be deleted.

  2. Click Delete button.

  3. Click Ok to delete or Cancel to return.

    The Delete confirmation screen.

Executing a Test Run

Test Runs can be executed at any time. The user can execute any of the Test Cases within a run, regardless of the order they appear. Use the Comment field to make notes about a Test Case. All comments will be displayed when a report is generated for a Test Run.

To execute a Test Run:

  1. From the Dashboard or a Test Runs list, click the Test Run to execute. The Test Run summary is displayed. You are able to change Test Case statuses from this page.

    The Test Run summary

  2. After executing a Test Case expand its widget, enter a Comment and select the appropriate Status icon.

    A Test Case

Icon Meaning
image78 Idle - Default value. The Test Case has not been examined.
image79 Running - Test Case is in progress.
image80 Paused - This status is used to denote a problem with the test case itself that prevents the test from being completed.
image81 Passed - Test Case met all the expected results.
image82 Failed - Test Case did not meet all the expected results, or produced an unhandled exception.
image83 Blocked - Test Case has a dependency that has failed.
image84 Error - Test environment has problems that prevent Test Case     execution.
image85 Waived - Test Case is not suitable for this run or blocked by other cases.

Bulk update of Test Cases

Bulk operations include change case-runs status, add/remove bug by entering bug ID, add comment to case-run.

  1. Select the Test Cases to be updated.

  2. Click on the sub-menu for the required operation:

    Test Case-run bulk menu

Generating a Test Run report

Kiwi TCMS generates reports for Test Runs, regardless of their state. A report provides the following information:

  • Plan details:
    • Product
    • Product version
    • Plan
    • Plan version
    • Platform
    • Operating system
    • Run summary
    • Run notes
    • Start date
    • Stop date.
  • Test Case details:
    • Closed at
    • ID
    • Summary
    • Case ID
    • Tested by
    • Group
    • Status
  • Summary statistics:
    • Total number of Test Cases Run.
    • Total number of Pending Test Cases.
    • Test Run completed (%).
  • Bug List:
    • Individual bugs
    • View all bugs (if bug tracker allows it)

To generate a report for a Test Run:

  1. Open the Test Run.

  2. From the Case Status widget, click Report.

    The Report button

  3. A printer friendly version displays.