Automation Frameworks plugins¶
Kiwi TCMS can be used with test automation frameworks. Test names and results are fetched automatically from the test automation framework using a plugin! This is an additional software package that you have to install in your test environment and configure it to talk back to Kiwi TCMS.
- tap-plugin: for reading Test Anything Protocol (TAP) files. Written in Python
- junit.xml-plugin: for reading junit.xml formatted files. Written in Python
- Native JUnit 5 plugin written in Java
- Robot Framework plugin
- Django test runner plugin
- Native py.test plugin written in Python
Plugins still in progress¶
- Native PHPUnit plugin written in PHP
Watch their repositories or subscribe to our newsletter to be notified when they are officially released.
- Native Test NG plugin to be written in Java
- GitHub status plugin
Vote with a
:+1: reaction on GitHub to give them priority!
Plugins will perform 2 high-level actions:
- Parse the test results from the test runner/automation framework
- Send these results to Kiwi TCMS via the API
The second is controlled via environment variables and behavior is described here. Important variables are:
TCMS_PREFIX- a text prefix for new TestPlan and TestRun records. Plugins provide their own default prefix if not specified
TCMS_DEFAULT_TESTER_ID- user ID to use for new records where needed. Defaults to the user making the API request
TCMS_PLAN_ID- if defined will create test runs under this TestPlan
TCMS_PARENT_PLAN- if defined newly created TestPlan will be a child of this one
TCMS_RUN_ID- if defined will report results to this TestRun
JOB_NAME- defines the product under test if we have to create new objects in the database
GIT_COMMIT- defines the version under test
BUILD_NUMBER- defines the build which we are testing
You are free to adjust these variables and how they get assigned different values in your CI workflow. This will change how/where results are reported. For example this is how the environment for kiwitcms-tap-plugin looks like:
#!/bin/bash if [ "$TRAVIS_EVENT_TYPE" == "push" ]; then # same as $TRAVIS_TAG when building tags export TCMS_PRODUCT_VERSION=$TRAVIS_BRANCH fi if [ "$TRAVIS_EVENT_TYPE" == "pull_request" ]; then export TCMS_PRODUCT_VERSION="PR-$TRAVIS_PULL_REQUEST" fi export TCMS_BUILD="$TRAVIS_BUILD_NUMBER-$(echo $TRAVIS_COMMIT | cut -c1-7)"
The above configuration creates a separate TestPlan for each branch, a separate TestPlan for each pull request (recording possible multiple test runs) and separate TestPlan and TestRun for each tag on GitHub!