Automation Frameworks plugins

Kiwi TCMS can be used with test automation frameworks. Test names and results are fetched automatically from the test automation framework using a plugin! This is an additional software package that you have to install in your test environment and configure it to talk back to Kiwi TCMS.

Available plugins

Plugins still in progress

Watch their repositories or subscribe to our newsletter to be notified when they are officially released.

Proposed plugins

Vote with a :+1: reaction on GitHub to give them priority!

Plugin configuration

Plugins will perform 2 high-level actions:

  1. Parse the test results from the test runner/automation framework
  2. Send these results to Kiwi TCMS via the API

The second is controlled via environment variables and behavior is described here. Important variables are:

  • TCMS_RUN_ID - if defined will report results to this TestRun
  • TCMS_PRODUCT or TRAVIS_REPO_SLUG or JOB_NAME - defines the product under test if we have to create new objects in the database
  • TCMS_PRODUCT_VERSION or TRAVIS_COMMIT or TRAVIS_PULL_REQUEST_SHA or GIT_COMMIT - defines the version under test
  • TCMS_BUILD or TRAVIS_BUILD_NUMBER or BUILD_NUMBER - defines the build which we are testing

You are free to adjust these variables and how they get assigned different values in your CI workflow. This will change how/where results are reported. For example this is how the environment for kiwitcms-tap-plugin looks like:

#!/bin/bash

if [ "$TRAVIS_EVENT_TYPE" == "push" ]; then
    # same as $TRAVIS_TAG when building tags
    export TCMS_PRODUCT_VERSION=$TRAVIS_BRANCH
fi

if [ "$TRAVIS_EVENT_TYPE" == "pull_request" ]; then
    export TCMS_PRODUCT_VERSION="PR-$TRAVIS_PULL_REQUEST"
fi

export TCMS_BUILD="$TRAVIS_BUILD_NUMBER-$(echo $TRAVIS_COMMIT | cut -c1-7)"

The above configuration creates a separate TestPlan for each branch, a separate TestPlan for each pull request (recording possible multiple test runs) and separate TestPlan and TestRun for each tag on GitHub!