Automation Testing Framework

With the Automation Testing Framework, you can test the logic of scripts that use the published APIs using common testing frameworks like JUnit or Spock, while running them with the functionality of DaVinci Configurator 6.

How to enable the testing framework

Closure for testing framework
scriptProject {
    cfgPath = file("${cfgPath}")
    classes = ["MyScript"]
    testing {
        bswPath = file("${bswPath}")
    }
}
This testing closure is used to configure the Automation Testing Framework. This method allows to set up the testing environment, such as BSW path and other testing parameters. If this method is called, testing will be automatically enabled for the script project.

Additionally the testing dependencies have to be added.

Managing Dependencies

For adding test dependencies to your project, you should use the testCfg configuration.

Dependencies for testing framework
    testCfgImplementation libs.spock
    testCfgImplementation platform(libs.junitBom)
    testCfgImplementation libs.junitApi
    testCfgImplementation libs.junitEngine

The template contain some of them with a compatible version in the libs.version.toml file, so you can use them directly in your build.gradle file.

Minimal Supported JUnit Platform Version

The minimal supported version of junit platform is 1.10.0, which is contained in junit bom version 5.10.0.

Writing basic tests

You can write tests using a common testing framework of your choice. While Spock is the preferred method, JUnit works as well. These tests must be located in the src/testCfg directory of your script project.

The example GetDpaProjectNameTest shows a Spock test that loads a DaVinci Configurator 6 project using the @TestProjectLoad annotation. All available loading scenarios are described in Loading CFG6 Projects in Tests below. For more control over the test task itself, see Custom Automation Script Tests.

Spock Example with usage of ScriptApi to read project name from DPA file
class GetDpaProjectNameTest extends Specification {

    @Shared
    IProjectRef projectRef = ScriptApi.scriptCode.projects.parameterizeProjectLoad {
        it.projectFile = Paths.get("%CFG_PROJECT_PATH%")
    }

    @Shared
    @TestProjectLoad
    ITestProjectRef projectRefExtShared = ITestProjectRef.forProject(projectRef)

    def "Test read project name"() {
        when:
        String origin = GetProjectName.getDpaProjectName()

        then:
        origin == "WorkshopProject"
    }

    static String getDpaProjectName() {
        def name = ""
        ScriptApi.scriptCode {
            projects.activeProject {
                name = project.getProjectName()
                scriptLogger.info("Work with the project: " + name + ".dvjson")
            }
        }
        return name
    }
}

The %CFG_PROJECT_PATH% placeholder must be replaced with the path of the DaVinci Configurator 6 project you want to test.

Model TestInfrastructure

The ITransactionUndoAllExtension class provides a JUnit Extension, which will undo all transaction after the JUnit test was executed. This can be used as static or instance field to undo for the whole test class or for each test case.
Note: This extension does not work with Spock. 

 public class ProjectLoadTest {
    @RegisterExtension
    static ITransactionUndoAllExtension extension = ITransactionUndoAllExtension.forActiveProject(); // Undo changes after ALL test cases.
 public class ProjectLoadTest {
    @RegisterExtension
    ITransactionUndoAllExtension extension = ITransactionUndoAllExtension.forActiveProject(); // Undo changes after EACH test case.
Usage of the ITransactionUndoAllExtension in a JUnit test
class TransactionUndoAllExtensionTest {
    @RegisterExtension
    ITransactionUndoAllExtension rule = ITransactionUndoAllExtension.forActiveProject();

    @Test
    void testcase() {
        //After this test case all transactions are reverted by the TransactionUndoAllExtension
    }
Setting the UndoHistoryLimit of the ITransactionUndoAllExtension in a JUnit test
class TransactionUndoAllExtensionWithLimitTest {
    @RegisterExtension
    ITransactionUndoAllExtension rule = ITransactionUndoAllExtension.forActiveProject(50);

    @Test
    void testcase() {
        //After this test case all transactions are reverted by the TransactionUndoAllExtension
    }

Loading CFG6 Projects in Tests

Spock


Simple — once per test class (@Shared)
class LoadProjectTest extends Specification {

    @Shared
    @TestProjectLoad
    ITestProjectRef testProjectRef = ITestProjectRef.loadProject("%CFG_PROJECT_PATH%")

    def "Load project via loadProject shorthand"() {
        when:
        def projectName = testProjectRef.project.getProjectName()

        then:
        projectName == "WorkshopProject"
    }
}

Simple — once per test method
class LoadProjectPerTestExampleTest extends Specification {

    @TestProjectLoad
    ITestProjectRef testProjectRef = ITestProjectRef.loadProject("%CFG_PROJECT_PATH%")

    def "Load project per test method via loadProject shorthand"() {
        when:
        def projectName = testProjectRef.project.getProjectName()

        then:
        projectName == "WorkshopProject"
    }
}

Standard — once per test class (@Shared)
class ForProjectSharedExampleTest extends Specification {

    @Shared
    IProjectRef projectRef = ScriptApi.scriptCode.projects.parameterizeProjectLoad {
        it.projectFile = Paths.get("%CFG_PROJECT_PATH%")
    }

    @Shared
    @TestProjectLoad
    ITestProjectRef testProjectRef = ITestProjectRef.forProject(projectRef)

    def "Load project per test class via forProject"() {
        when:
        def projectName = testProjectRef.project.getProjectName()

        then:
        projectName == "WorkshopProject"
    }
}

Standard — once per test method
class ForProjectPerTestExampleTest extends Specification {

    @Shared
    IProjectRef projectRef = ScriptApi.scriptCode.projects.parameterizeProjectLoad {
        it.projectFile = Paths.get("%CFG_PROJECT_PATH%")
    }

    @TestProjectLoad
    ITestProjectRef testProjectRef = ITestProjectRef.forProject(projectRef)

    def "Load project per test method via forProject"() {
        when:
        def projectName = testProjectRef.project.getProjectName()

        then:
        projectName == "WorkshopProject"
    }
}

Advanced — once per class + once per method
class ForProjectAdvancedExampleTest extends Specification {

    @Shared
    IProjectRef projectRef = ScriptApi.scriptCode.projects.parameterizeProjectLoad {
        it.projectFile = Paths.get("%CFG_PROJECT_PATH%")
    }

    @Shared
    @TestProjectLoad
    ITestProjectRef testProjectRefShared = ITestProjectRef.forProject(projectRef)

    @TestProjectLoad
    ITestProjectRef testProjectRefPerTest = ITestProjectRef.forProject(projectRef)

    def "Access project from both class-scoped and method-scoped ref"() {
        expect:
        testProjectRefShared.project.getProjectName() == "WorkshopProject"
        testProjectRefPerTest.project.getProjectName() == "WorkshopProject"
    }
}

Load a Project Once for multiple Tests in a Test Run

When your tests need a DaVinci Configurator 6 project open throughout the entire test run — without reloading it for every test method — you can create a SharedCfgProject class. The class implements the JUnit Platform LauncherSessionListener and TestExecutionListener interfaces. It loads the project once before all tests start and closes it after all tests finish. This works with both Spock and JUnit Jupiter tests in the same test run.

Step 1: Create SharedCfgProject.groovy

Place the following class in src/testCfg/groovy/SharedCfgProject.groovy:

SharedCfgProject.groovy — shared CFG6 project loaded once for all test classes
import com.vector.cfg.automation.api.ScriptApi
import com.vector.cfg.automation.scripting.api.project.IProject
import org.junit.platform.launcher.LauncherSession
import org.junit.platform.launcher.LauncherSessionListener
import org.junit.platform.launcher.TestExecutionListener
import org.junit.platform.launcher.TestPlan

import java.nio.file.Paths
import java.util.concurrent.atomic.AtomicReference

class SharedCfgProject implements LauncherSessionListener, TestExecutionListener {

    private static final AtomicReference<IProject> PROJECT = new AtomicReference<>()

    // Returns false when %CFG_PROJECT_PATH% has not been replaced with an actual path,
    // keeping this class a no-op in projects that do not use a shared CFG6 project.
    private static boolean isConfigured() {
        return !"%CFG_PROJECT_PATH%".contains("%")
    }

    private static IProject initProject() {
        def projectRef = ScriptApi.scriptCode().projects.parameterizeProjectLoad {
            it.projectFile = Paths.get("%CFG_PROJECT_PATH%")
        }
        def project = projectRef.advanced().openProject()
        println "Loading shared CFG6 project used for tests."
        return project
    }

    static synchronized IProject getProject() {
        if (!isConfigured()) {
            return null
        }
        def project = PROJECT.get()
        if (project == null) {
            project = initProject()
            PROJECT.set(project)
        }
        return project
    }

    @Override
    void launcherSessionOpened(LauncherSession session) {
        session.launcher.registerTestExecutionListeners(this)
    }

    @Override
    void testPlanExecutionStarted(TestPlan testPlan) {
        getProject()
    }

    @Override
    void testPlanExecutionFinished(TestPlan testPlan) {
        def project = PROJECT.getAndSet(null)
        if (project == null) {
            return
        }
        try {
            project.close()
        } catch (Exception ignored) {
        }
        println "Shared CFG6 project unloaded after tests."
    }
}

Replace %CFG_PROJECT_PATH% with the path to your DaVinci Configurator 6 project file.

Step 2: Register the service

Create the file src/testCfg/resources/META-INF/services/org.junit.platform.launcher.LauncherSessionListener with the following content:

META-INF/services/org.junit.platform.launcher.LauncherSessionListener
SharedCfgProject

Using a Shared CFG6 Project in Tests

The shared CFG6 project is set as the active project, so your tests can access it using the standard ScriptApi.scriptCode.projects.activeProject.project API — no additional setup is required in individual test classes.

JUnit

Load a CFG6 Project

The IProjectLoadExtension class provides a JUnit 5 Extension, which will load the passed IProjectRef before test execution and close it afterward. This can be used as an JUnit 5 Extension. 

 public class ProjectLoadExtensionTest {
    @RegisterExtension
    static IProjectLoadExtension extension = IProjectLoadExtension.forProject(ScriptApi.scriptCode.projects.parameterizeProjectLoad{
        projectFile "YourProject.dvjson"
    })

    @Test
    public void testcase() {
        //In this test the project was loaded and will be closed after all tests.
    }

For details about the how to load a CFG6 project, or if you want to load only an .arxml file, please refer to the Automation Interface User Documentation.

Custom Automation Script Tests

If you want to have more flexibility in your tests, you can write custom tests that use the AutomationScriptTest task directly. This allows you to specify multiple test classes in different source sets and have more control over the test execution.

Configure a custom AutomationScriptTest task
   import com.vector.cfg.pai.testexecution.AutomationScriptTest
   import org.gradle.api.plugins.JavaPlugin

   def customSourceSetName = "myTest"
   sourceSets {
       create(customSourceSetName){
           compileClasspath += sourceSets.main.output
       }
   }

   configurations {
       myTestImplementation.extendsFrom(implementation)
       myTestCompileOnly.extendsFrom(compileOnly)
   }

   dependencies {
       myTestImplementation libs.spock
       myTestImplementation libs.junitApi
       myTestImplementation libs.junitEngine
       myTestImplementation platform(libs.junitBom)
       myTestRuntimeOnly libs.junitPlatformLauncher
   }

   tasks.register("myTestTaskStandardBsw", AutomationScriptTest){
       it.sourceSetName = customSourceSetName
       it.bswPath.set(myBswPath)
       it.configureCfgPath(myCfgPath)
       it.dependsOn(tasks.named("jar"))
   }

   tasks.register("myTestSecondBsw", AutomationScriptTest){
       it.sourceSetName = customSourceSetName
       it.bswPath.set(mySecondBswPath)
       it.configureCfgPath(myCfgPath)
       it.dependsOn(tasks.named("jar"))
   }

Plugin configuration

Following properties can be set in the testCfg configuration:

The cfgPath property specifies the path to the DaVinci Configurator 6 which should be used to run the tests. The bswPath property specifies the path to the BSW package which is used to run the tests. The additionalVmArgs property allows you to specify additional JVM arguments for the test execution.

Source Set 'testCfg'

The testing framework is based on the testCfg source set[1], which is automatically enabled when you activate the testing framework in your project.

See How to enable the testing framework for more information. In addition to the testCfg source set, the standard test source set can be used without script support.

Gradle Task 'testCfg'

To run the tests, you can use the gradlew testCfg command. This gradle task will execute all tests in the testCfg source set. The testCfg task will automatically get executed when you run the build task.

Test and Debug productive Tasks with the Testing Framework

To test and debug a script task in a fast and convenient way, you can use the testing framework.
For this, a test will be set up that executes the task you want to test or debug. You can set breakpoints in your script and debug the test.
With the testing framework, you have control over the task execution environment. This means you can provide a bsw package, a DaVinci Configurator 6 project and user defined arguments.

Attention: Tests which executes a whole script task can take some time, especially if a project will be loaded. You can classify or disable these test with the common Spock/Junit mechanisms.

Test a DV_APPLICATION task

To call a DV_APPLICATION task:

Simple DV_APPLICATION script task
scriptTask("SimpleTask", DV_APPLICATION) {
    taskDescription 'Task description - Prints "HelloWorld" to console.'
    code {
        scriptLogger.info "!!!Hello World!!!"
        return "SimpleTask executed"
    }
}

You can write a spock test and test/debug the task:

Test for debugging a simple script task
def "Debug simple task"() {
    when:
    def result = ScriptApi.scriptCode().scripts.callScriptTask("SimpleTask")
    then:
    result == "SimpleTask executed"
}

Test a task with userdefined arguments

To call a task with userdefined arguments:

DV_APPLICATION script task with user defined parameters
scriptTask("SimpleTaskWithParams", DV_APPLICATION) {
    def nameArg = newUserDefinedArgument("name", String, "My String Param")
    def value = newUserDefinedArgument("value", Integer, "My Integer Param")

    code {
        return "name: $nameArg.value, value: $value.value"
    }
}

You can write a spock test and test/debug the task:

Test for debugging a script task with user defined parameters
def "Debug task with params"() {
    when:
    def result = ScriptApi.scriptCode().scripts.callScriptTaskWithUserArgs("SimpleTaskWithParams", "--name=Hello --value=42")
    then:
    result == "name: Hello, value: 42"
}

Test a DV_PROJECT task

To call a DV_PROJECT task:

Simple DV_PROJECT script task
scriptTask("ProjectTask", DV_PROJECT) {
    code {
        activeProject().getProjectName()
    }
}

You can write a spock test and test/debug the task:

Test for debugging a script task that requires a project
def "Debug task with project"() {
    when:
    def result = ScriptApi.scriptCode().projects.openProject("%CFG_PROJECT_PATH%") { scripts.callScriptTask("ProjectTask") }
    then:
    result == "WorkshopProject"
}

Attention:
The project will not be saved by default. But if the script contains a saveProject call, the project will be saved and persisted. See