Automation Frameworks plugins

Kiwi TCMS can be used with test automation frameworks. Test names and results are fetched automatically from the test automation framework using a plugin! This is an additional software package that you have to install in your test environment and configure it to talk back to Kiwi TCMS.

Available plugins

• tap-plugin: for reading Test Anything Protocol (TAP) files. Written in Python

• junit.xml-plugin: for reading junit.xml formatted files. Written in Python

• Native JUnit 5 plugin written in Java

• PHPUnit plugin written in PHP

• Robot Framework plugin

• Django test runner plugin

• Native py.test plugin written in Python

Plugins still in progress

N/A

Watch their repositories or subscribe to our newsletter to be notified when they are officially released.

Proposed plugins

• Native Test NG plugin to be written in Java

• GitHub status plugin

Vote with a :+1: reaction on GitHub to give them priority!

Plugin configuration

Plugins will perform 2 high-level actions:

1. Parse the test results from the test runner/automation framework

2. Send these results to Kiwi TCMS via the API

The second is controlled via environment variables and behavior is described here. Important variables are:

• TCMS_PREFIX - a text prefix for new TestPlan and TestRun records. Plugins provide their own default prefix if not specified

• TCMS_DEFAULT_TESTER_ID - user ID to use for new records where needed. Defaults to the user making the API request

• TCMS_PLAN_ID - if defined will create test runs under this TestPlan

• TCMS_PARENT_PLAN - if defined newly created TestPlan will be a child of this one

• TCMS_RUN_ID - if defined will report results to this TestRun

• TCMS_PRODUCT or TRAVIS_REPO_SLUG or JOB_NAME - defines the product under test if we have to create new objects in the database

• TCMS_PRODUCT_VERSION or TRAVIS_COMMIT or TRAVIS_PULL_REQUEST_SHA or GIT_COMMIT - defines the version under test

• TCMS_BUILD or TRAVIS_BUILD_NUMBER or BUILD_NUMBER - defines the build which we are testing

You are free to adjust these variables and how they get assigned different values in your CI workflow. This will change how/where results are reported. For example this is how the environment for kiwitcms-tap-plugin looks like:

#!/bin/bash

if [ "$TRAVIS_EVENT_TYPE" == "push" ]; then
    # same as $TRAVIS_TAG when building tags
    export TCMS_PRODUCT_VERSION=$TRAVIS_BRANCH
fi

if [ "$TRAVIS_EVENT_TYPE" == "pull_request" ]; then
    export TCMS_PRODUCT_VERSION="PR-$TRAVIS_PULL_REQUEST"
fi

export TCMS_BUILD="$TRAVIS_BUILD_NUMBER-$(echo $TRAVIS_COMMIT | cut -c1-7)"

The above configuration creates a separate TestPlan for each branch, a separate TestPlan for each pull request (recording possible multiple test runs) and separate TestPlan and TestRun for each tag on GitHub!