Getting Started

Getting started with CTM

On this page you can find some examples, which should give you a quick start into CTM.
If you want to deepen your CTM knowledge, please see our CTM Guidebook on more configuration options.

Example conditions

In all given examples below we use GitHub issues to define our product requirements:
See our CTM Guidebook which other tools are supported by CTM
❗ All CTM command line calls are given here as ctm <arguments>.
In case you used go get github.com/SAP/quality-continuous-traceability-monitor to obtain your CTM instance, please use quality-continuous-traceability-monitor <arguments> to execute CTM.
Mac and Linux: You could also create a symbolic link as ctm to quality-continuous-traceability-monitor to call CTM by it's abbreviation (ln -s $GOPATH/bin/quality-continuous-traceability-monitor $GOPATH/bin/ctm).

Requirment mapping in the source code

The product's source code is written in Java (see the CTM Guidebook for more supported languages).
Each automated test, that safeguards a product requirement, gets a specific comment which assigns the test to the corresponding product requirement.
```
...
  /** 
   * Overwhelming complex test, which covers my
   * product requirement #1
   */
  // Trace(GitHub:doergn/sourcecodeRepo#1)
  public void testApp() {
      assertTrue(true);
  }
...
```
Note the // Trace(GitHub:doergn/sourcecodeRepo#1) comment which assign this test to product requirement #1

CTM requires a configuration file to run. So we create a text file called myCTMconfig.json with the following content:

{
  "github": {
    "base_url": "https://github.com"
  },
  "workDir": "/tmp/CTM/workdir",
  "outputDir": "/tmp/CTM/reports",
  "log": {
    "level": "INFO"
  },
  "sourcecode": [{
    "local": "/home/doergn/sourcecodeRepo",
    "language": "java"
  }],
  "testReport": [{
    "type": "xunit-xml",
    "local": "/home/doergn/sourcecodeRepo/target/surefire-reports"
  }]
}

Note the sourcecode element which points to the local source code folder. The element testReport points to the local test result report.
Details on the other configuration elements can be found in the CTM Guidebook.

Execute CTM with the just created config file: ctm -c myCTMconfig.json which should generate an output like:

I0817 10:20:56.924698    7759 performance.go:13] Parse java sourcecode (/home/doergn/sourcecodeRepo) took 1.029272ms
I0817 10:20:56.924872    7759 xunit.go:127] Parsing /home/doergn/sourcecodeRepo/target/surefire-reports/TEST-com.mycorp.AppTest.xml
I0817 10:20:56.925226    7759 performance.go:13] Scan xunit XML test reports took 372.21µs
I0817 10:20:56.925253    7759 performance.go:13] Create Traces took 1.251µs
I0817 10:20:56.925312    7759 performance.go:13] Create HTML and JSON reports took 51.785µs

According to the output, CTM started parsing the java source code in /home/doergn/sourcecodeRepo for traceability annotated tests which took it 1.029272ms
Second line shows it started parsing the xunit-xml test report /home/doergn/sourcecodeRepo/target/surefire- reports/TEST-com.mycorp.AppTest.xml as it was obviously the only xunit-xml test result it found in the configured path /home/doergn/sourcecodeRepo/target/surefire-reports (see testReport configuration above)

With the collected data, CTM generated the overall traceability report (json and html) and wrote it into the given outputDir:

/tmp/CTM/reports/ctm_report_all.json

{
  "doergn/sourcecodeRepo#1": {
    "link": "https://github.com/doergn/sourcecodeRepo/issues/1",
    "test_cases": [
    {
        "test_fullname": "com.mycorp.AppTest.testApp",
        "test_name": "testApp",
        "test_class": "com.mycorp.AppTest",
        "passed": true,
        "skipped": false
     }]
  }
}

/tmp/CTM/reports/ctm_report_all.html

You could further improve the generated overall traceability report, by adding the GitHub repository to the sourcecode element:

{
  "github": {
    "base_url": "https://github.com"
  },
  "workDir": "/tmp/CTM/workdir",
  "outputDir": "/tmp/CTM/reports",
  "log": {
    "level": "INFO"
  },
  "sourcecode": [{
    "local": "/home/doergn/sourcecodeRepo",
    "git": {
      "organization": "doergn",
      "repository": "sourcecodeRepo",
      "branch": "master"
    },
    "language": "java"
  }],
  "testReport": [{
    "type": "xunit-xml",
    "local": "/home/doergn/sourcecodeRepo/target/surefire-reports"
  }]
}

Note the new git element under sourcecode

The generated overall traceability report will now also contain a link to the corresponding source code in the GitHub repository:

/tmp/CTM/reports/ctm_report_all.json

{
  "doergn/sourcecodeRepo#1": {
    "link": "https://github.com/doergn/sourcecodeRepo/issues/1",
    "test_cases": [
      {
        "test_fullname": "com.mycorp.AppTest.testApp",
        "test_name": "testApp",
        "test_class": "com.mycorp.AppTest",
        "test_source": "https://github.com/doergn/sourcecodeRepo/blob/master/src/test/java/com/mycorp/AppTest.java",
        "passed": true,
        "skipped": false
      }]
    }
}

Note the new test_source element

/tmp/CTM/reports/ctm_report_all.html Note the new link at the source code

In case your source code is spread across multiple GitHub repositories and you want to have a combined traceability view on your requirements, you can enter multiple source code repositories in the sourcecode element of your configuration file like this:

{
  "github": {
    "base_url": "https://github.com"
  },
  "workDir": "/tmp/CTM/workdir",
  "outputDir": "/tmp/CTM/reports",
  "log": {
    "level": "INFO"
  },
  "sourcecode": [{
    "local": "/home/doergn/sourcecodeRepo",
    "git": {
      "organization": "doergn",
      "repository": "sourcecodeRepo",
      "branch": "master"
    },
    "language": "java"
  },
  {
    "local": "/home/doergn/sourcecodeRepo2",
    "git": {
      "organization": "doergn",
      "repository": "sourcecodeRepo2",
      "branch": "master"
    },
    "language": "java"
  }],
  "testReport": [{
    "type": "xunit-xml",
    "local": "/home/doergn/sourcecodeRepo/target/surefire-reports"
  }]
}

Requirement mapping via one dedicated file

What if the product's source code is not written in any of the supported languages? Also some teams don't want/can't add comments to their automated tests. In this case you can pass a requirement mapping file to CTM. The requirement mapping file is a json file which maintains the relations between product requirement and automated test.

Let's create a requirement mapping file and save it as r_mapping.json. The content of the file would be:
```
[
  {
    "source_reference": "com.mycorp.AppTest.testApp()",
    "github_keys": [
      "doergn/sourcecodeRepo#1"
    ]
  }
]
```
Note the brackets () after com.mycorp.AppTest.testApp which indicates testApp is a method/function.

The corresponding CTM configuration file (which we save as myCTMconfig.json) will have the following content:

{
  "github": {
      "base_url": "https://github.com"
  },
  "workDir": "/tmp/CTM/workdir",
  "outputDir": "/tmp/CTM/reports",
  "log": {
      "level": "INFO"
  },
  "mapping": {
      "local": "/tmp/CTM/r_mapping.json"
  },
  "testReport": [{
      "type": "xunit-xml",
      "local": "/home/doergn/sourcecodeRepo/target/surefire-reports"
   }]
}

Note the mapping element, which replaces the sourcecode element from example Code annotated requirement mapping.

Execute CTM with the config file: ctm -c myCTMconfig.json which should generate an output like:

I0817 14:25:19.260275   10311 performance.go:13] Read mapping file took 38.735µs
I0817 14:25:19.260465   10311 xunit.go:127] Parsing /home/doergn/sourcecodeRepo/target/surefire-reports/TEST-com.mycorp.AppTest.xml
I0817 14:25:19.260947   10311 performance.go:13] Scan xunit XML test reports took 509.375µs
I0817 14:25:19.260974   10311 performance.go:13] Create Traces took 1.688µs
I0817 14:25:19.261068   10311 performance.go:13] Create HTML and JSON reports took 87.454µs

The mapping between product requirements and their automated tests is done in the r_mapping.json file. CTM started parsing this file which took 38.735µs
The xunit-xml test report /home/doergn/sourcecodeRepo/target/surefire-reports/TEST-com.mycorp.AppTest.xml is parsed in 509.375µs as indicated in line 2 & 3

The last two log entries Create Traces and Create HTML and JSON reports indicate CTM wrote the overall traceability report (json and html) into the given outputDir:

/tmp/CTM/reports/ctm_report_all.json

{
  "doergn/sourcecodeRepo#1": {
    "link": "https://github.com/doergn/sourcecodeRepo/issues/1",
    "test_cases": [
    {
        "test_fullname": "com.mycorp.AppTest.testApp",
        "test_name": "testApp",
        "test_class": "com.mycorp.AppTest",
        "passed": true,
        "skipped": false
     }]
  }
}

/tmp/CTM/reports/ctm_report_all.html

You could further improve the generated overall traceability report, by adding the source code location to the requirement mapping file:

[
  {
    "source_reference": "com.mycorp.AppTest.testApp()",
    "filelocation": {
      "git": {
        "organization": "doergn",
        "repository": "sourcecodeRepo",
        "branch": "master"
      },
      "relativePath": "./src/test/java/com/mycorp/AppTest.java"
    },
    "github_keys": [
      "doergn/sourcecodeRepo#1"
    ]
  }
]

Note the new filelocation element. git contains the GitHub repository coordinates while relativePath contains the relative location of the automated test source code inside the GitHub repository.

The generated overall traceability report will now also contain a link to the corresponding source code in the GitHub repository:

/tmp/CTM/reports/ctm_report_all.json

{
  "doergn/sourcecodeRepo#1": {
    "link": "https://github.com/doergn/sourcecodeRepo/issues/1",
    "test_cases": [
      {
        "test_fullname": "com.mycorp.AppTest.testApp",
        "test_name": "testApp",
        "test_class": "com.mycorp.AppTest",
        "test_source": "https://github.com/doergn/sourcecodeRepo/blob/master/src/test/java/com/mycorp/AppTest.java",
        "passed": true,
        "skipped": false
      }]
    }
}

Note the new test_source element

/tmp/CTM/reports/ctm_report_all.html Note the new link at the source code

Continuous trace of automated test results per product requirement

In both examples given above, CTM creates an overall traceability report consisting of two files (json and html formated). These files could be further process by additional tools or put on a web server for the team's constant reference.
In order to ease the access to the CTM results, thus raising the team's quality awareness, you can also configure CTM to store it's result in a dedicated GitHub repository, the traceability repository.
All you need is:

an empty GitHub repository (on GitHub or on your Enterprise GitHub instance)
a working git command line client which can be called by CTM and which has access to the above given repository

Once this is set, you can add the traceabilityRepo element to your configuration file like this:

{
  "github": {
    "base_url": "https://github.com"
  },
  "workDir": "/tmp/CTM/workdir",
  "outputDir": "/tmp/CTM/reports",
  "log": {
    "level": "INFO"
  },
  "sourcecode": [{
    "local": "/home/doergn/sourcecodeRepo",
    "git": {
      "organization": "doergn",
      "repository": "sourcecodeRepo",
      "branch": "master"
    },
    "language": "java"
  },
  {
    "local": "/home/doergn/sourcecodeRepo2",
    "git": {
      "organization": "doergn",
      "repository": "sourcecodeRepo2",
      "branch": "master"
    },
    "language": "java"
  }],
  "traceabilityRepo": {
    "git": {
      "organization": "doergn",
      "repository": "traceabilityRepo",
      "branch": "master"
    }
  },
  "testReport": [{
    "type": "xunit-xml",
    "local": "/home/doergn/sourcecodeRepo/target/surefire-reports"
  }]
}

Note the traceabilityRepo element.

Executing CTM will still generate the overall traceability report in json and HTML format, but in additional also maintain the traceability repository. Traceability Example Report

Find further information on the traceability repository in the CTM Guidebook

Working with releases

New releases introduce new product requirements (and new automated tests). You might want to persist the information which product requirement (covered by which automated test results) was delivered with a certain release. Therefore you can create a delivery mapping file, which maps a product requirement to a specific release.

Create a new delivery mapping file, named delivery1_0_0.json which looks like this:
```
{
  "program": "MyProgram",
  "delivery": "1.0.0",
  "github_keys": [
    "doergn/sourcecodeRepo#1"
  ]
}
```
This file describes for CTM that the product requirement defined in doergn/sourcecodeRepo#1 is delivered in release 1.0.0 of a program named MyProgram.

❗️ As the mapping between product requirement and release is done in a delivery mapping file, it does not matter, whether you use code annotated requirement mapping or a requirement mapping file

Make sure to set the GitHub access token in the configuration file:

{
  "github": {
    "base_url": "https://github.com",
    "access_token": "12345",
    "createLinksInBacklogItems": true
  },
  "workDir": "/tmp/CTM/workdir",
  "outputDir": "/tmp/CTM/reports",
  "log": {
    "level": "INFO"
  },
  "mapping": {
    "local": "/tmp/CTM/r_mapping.json"
  },
  "traceabilityRepo": {
    "git": {
      "organization": "doergn",
      "repository": "traceabilityRepo",
      "branch": "master"
    }
  },
  "testReport": [{
    "type": "xunit-xml",
    "local": "/home/doergn/sourcecodeRepo/target/surefire-reports"
  }]
}

Note the access_token element unter github which contains the GitHub personal access token with at least scope repo.

Also note the createLinksInBacklogItems element under github which adds a link to the requirement pointing to traceability repository

Execute CTM with the config file myCTMconfig.json and the new delivery mapping file delivery1_0_0.json:
ctm -c myCTMconfig.json -df delivery1_0_0.json.
The output should look like this:

I0820 17:26:37.847075    5079 performance.go:13] Read mapping file took 34.805µs
I0820 17:26:37.847236    5079 xunit.go:127] Parsing /home/dorgn/sourcecodeRepo/target/surefire-reports/TEST-com.mycorp.AppTest.xml
I0820 17:26:37.847572    5079 performance.go:13] Scan xunit XML test reports took 355.38µs
I0820 17:26:37.847590    5079 performance.go:13] Create Traces took 1.209µs
Counting objects: 11, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (11/11), 889 bytes | 889.00 KiB/s, done.
Total 11 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To github.com:doergn/traceabilityRepo2.git
   bf1503e..4b565e8  master -> master
I0820 17:26:43.623592    5079 performance.go:13] Create GitHub report took 5.775976631s
W0820 17:26:43.623745    5079 github.go:256] Cache file not found. Create new one at /tmp/CTM/workdir/.commentCache
I0820 17:26:44.081163    5079 performance.go:13] Create links in GitHub Backlog items took 457.463117ms
I0820 17:26:44.081733    5079 performance.go:13] Create HTML and JSON reports took 430.903µs

After line four you can see the output of the git command line client, which updates and pushes changes to the traceability repository. Also a GitHub release is created for the new delivery.
Traceability delivery readme A delivery has been created in the traceability repository Traceability release A GitHub release is created for the new delivery

The lines

Cache file not found. Create new one at /tmp/CTM/workdir/.commentCache
Create links in GitHub Backlog items

results from the createLinksInBacklogItems configuration file entry.
The last line Create HTML and JSON reports now indicates the generation of the overall traceability report and the delivery traceability report (both in json and html) in the given outputDir:

/tmp/CTM/reports/ctm_report_all.json

{
    "doergn/sourcecodeRepo#1": {
        "link": "https://github.com/doergn/sourcecodeRepo/issues/1",
        "test_cases": [
            {
                "test_fullname": "com.mycorp.AppTest.testApp",
                "test_name": "testApp",
                "test_class": "com.mycorp.AppTest",
                "test_source": "https://github.com/doergn/sourcecodeRepo/blob/master/src/test/java/com/mycorp/AppTest.java",
                "passed": true,
                "skipped": false
            }
        ]
    },
    "doergn/sourcecodeRepo#2": {
        "link": "https://github.com/doergn/sourcecodeRepo/issues/2",
        "test_cases": [
            {
                "test_fullname": "com.mycorp.AppTest.testApp2",
                "test_name": "testApp2",
                "test_class": "com.mycorp.AppTest",
                "test_source": "https://github.com/doergn/sourcecodeRepo/blob/master/src/test/java/com/mycorp/AppTest.java",
                "passed": true,
                "skipped": false
            }
        ]
    }
}

Note the two product requirements in the json formated overall traceability report

/tmp/CTM/reports/ctm_report_all.html

Note the two product requirements in the html formated overall traceability report

/tmp/CTM/reports/ctm_report_1.0.0.json

{
    "doergn/sourcecodeRepo#1": {
        "link": "https://github.com/doergn/sourcecodeRepo/issues/1",
        "test_cases": [
            {
                "test_fullname": "com.mycorp.AppTest.testApp",
                "test_name": "testApp",
                "test_class": "com.mycorp.AppTest",
                "test_source": "https://github.com/doergn/sourcecodeRepo/blob/master/src/test/java/com/mycorp/AppTest.java",
                "passed": true,
                "skipped": false
            }
         ]
    }
}

Note that only one product requirement is in the json formated delivery traceability report, as we only assigned the product requirement doergn/sourcecodeRepo#1 to release 1.0.0 in the delivery mapping file

/tmp/CTM/reports/ctm_report_1.0.0.html

Note that only one product requirement is in the html formated delivery traceability report, as we only assigned the product requirement doergn/sourcecodeRepo#1 to release 1.0.0 in the delivery mapping file

made with ❤ by SAP

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Getting Started

Getting started with CTM

Example conditions

Requirment mapping in the source code

Requirement mapping via one dedicated file

Continuous trace of automated test results per product requirement

Working with releases

Quick-Navigation

Getting Started

CTM Guidebook

Clone this wiki locally