Automation Frameworks plugins¶
Kiwi TCMS can be used with test automation frameworks. Test names and results are fetched automatically from the test automation framework using a plugin! This is an additional software package that you have to install in your test environment and configure it to talk back to Kiwi TCMS.
Available plugins¶
tap-plugin: for reading Test Anything Protocol (TAP) files. Written in Python
junit.xml-plugin: for reading junit.xml formatted files. Written in Python
Native JUnit 5 plugin written in Java
Native py.test plugin written in Python
Plugins still in progress¶
Native PHPUnit plugin written in PHP
Watch their repositories or subscribe to our newsletter to be notified when they are officially released.
Proposed plugins¶
Native Test NG plugin to be written in Java
Vote with a :+1:
reaction on GitHub to give them priority!
Plugin configuration¶
Plugins will perform 2 high-level actions:
Parse the test results from the test runner/automation framework
Send these results to Kiwi TCMS via the API
The second is controlled via environment variables and behavior is described here. Important variables are:
TCMS_PREFIX
- a text prefix for new TestPlan and TestRun records. Plugins provide their own default prefix if not specifiedTCMS_DEFAULT_TESTER_ID
- user ID to use for new records where needed. Defaults to the user making the API requestTCMS_PLAN_ID
- if defined will create test runs under this TestPlanTCMS_PARENT_PLAN
- if defined newly created TestPlan will be a child of this oneTCMS_RUN_ID
- if defined will report results to this TestRunTCMS_PRODUCT
orTRAVIS_REPO_SLUG
orJOB_NAME
- defines the product under test if we have to create new objects in the databaseTCMS_PRODUCT_VERSION
orTRAVIS_COMMIT
orTRAVIS_PULL_REQUEST_SHA
orGIT_COMMIT
- defines the version under testTCMS_BUILD
orTRAVIS_BUILD_NUMBER
orBUILD_NUMBER
- defines the build which we are testing
You are free to adjust these variables and how they get assigned different values in your CI workflow. This will change how/where results are reported. For example this is how the environment for kiwitcms-tap-plugin looks like:
#!/bin/bash
if [ "$TRAVIS_EVENT_TYPE" == "push" ]; then
# same as $TRAVIS_TAG when building tags
export TCMS_PRODUCT_VERSION=$TRAVIS_BRANCH
fi
if [ "$TRAVIS_EVENT_TYPE" == "pull_request" ]; then
export TCMS_PRODUCT_VERSION="PR-$TRAVIS_PULL_REQUEST"
fi
export TCMS_BUILD="$TRAVIS_BUILD_NUMBER-$(echo $TRAVIS_COMMIT | cut -c1-7)"
The above configuration creates a separate TestPlan for each branch, a separate TestPlan for each pull request (recording possible multiple test runs) and separate TestPlan and TestRun for each tag on GitHub!