Automation Frameworks plugins¶
Kiwi TCMS can be used with test automation frameworks. Test names and results are fetched automatically from the test automation framework using a plugin! This is an additional software package that you have to install in your test environment and configure it to talk back to Kiwi TCMS.
Plugins still in progress¶
Watch their repositories or subscribe to our newsletter to be notified when they are officially released.
- Django test runner plugin
- Native Test NG plugin to be written in Java
- GitHub status plugin
- Robot Framework plugin
Vote with a
:+1: reaction on GitHub to give them priority!
Plugins will perform 2 high-level actions:
- Parse the test results from the test runner/automation framework
- Send these results to Kiwi TCMS via the API
The second is controlled via environment variables and behavior is described here. Important variables are:
TCMS_PLAN_ID- if defined will create test runs under this TestPlan
TCMS_RUN_ID- if defined will report results to this TestRun
JOB_NAME- defines the product under test if we have to create new objects in the database
GIT_COMMIT- defines the version under test
BUILD_NUMBER- defines the build which we are testing
You are free to adjust these variables and how they get assigned different values in your CI workflow. This will change how/where results are reported. For example this is how the environment for kiwitcms-tap-plugin looks like:
#!/bin/bash if [ "$TRAVIS_EVENT_TYPE" == "push" ]; then # same as $TRAVIS_TAG when building tags export TCMS_PRODUCT_VERSION=$TRAVIS_BRANCH fi if [ "$TRAVIS_EVENT_TYPE" == "pull_request" ]; then export TCMS_PRODUCT_VERSION="PR-$TRAVIS_PULL_REQUEST" fi export TCMS_BUILD="$TRAVIS_BUILD_NUMBER-$(echo $TRAVIS_COMMIT | cut -c1-7)"
The above configuration creates a separate TestPlan for each branch, a separate TestPlan for each pull request (recording possible multiple test runs) and separate TestPlan and TestRun for each tag on GitHub!