Scenario based Unit Tests for Business Entity FetchData (read) operations

Introduction

Fetching Data (reading data) is a key operation of Business Entities. While the SmartComponent Library based Business Entities provide a lot of (tested) functionality out of the box, the read operations should ideally also be unit tested by developers. There are three main criterias for successful Business Entity read operations:

  • error free based on expected input (FetchDataRequest, Queries)
  • the correct result
    • correct number of records
    • correct values in records including calculated fields
  • acceptable performance

As the nature of FetchData operations are typically very similar, classic Unit Test classes would all contain a large number of almost identically code - for executing a request and asserting the result. The idea of using scenarios is to implement those tests based on a scenario definition in a JSON file rather than in source code. This allows developers to spend the majority of their time on writing those unit tests that are really specific to individual components.

Scenarios are typically defined as JSON files with a .scenario extension:

A simple .scenario file without assertions
{
  "SerializedType": "Consultingwerk.SmartUnit.OERA.RetrieveDataScenario.Scenario",
  "EntityName": "Consultingwerk.OeraTests.SCL1684.CustomerItemBusinessEntity",
  "EntityTables": "eCustomer,eItem",
  "QueryStrings": [
    "FOR EACH eCustomer WHERE eCustomer.CustNum = 2",
    "FOR EACH eItem WHERE eItem.ItemNum = 2"
  ],
  "BatchSize": 100,
  "StopAfter": 5
}

Defining Scenarios from the Business Entity Tester

The Business Entity Tester supports creating scenario definitions that match the current selection criteria:

 

Using the "Save Scenario" and "Open Scenario" developers can save and reopen scenarios in the Business Entity Designer.

Executing Scenarios in an ANT job

SmartUnit supports the execution of .scenario file based FetchData operations through an alternative test runner.

The following sample ANT script components demonstrate the usage of the Scenario based test runner.

    <!-- SmartUnit Macro Definition --> 
    <macrodef name="ScenarioRunner">
        <attribute name="testSuite" default=""/>
        <attribute name="tests" default=""/>
        <attribute name="output" default=""/>
        <attribute name="services" default=""/>
        <attribute name="haltOnError" default="false"/>
        <attribute name="forceXmlXref" default="false"/>  
        <attribute name="importClientLog" default="true"/>
        <attribute name="importStdOut" default="true"/> 
        <element name="options" optional="true" />
        <sequential> 

        <PCTRun procedure="Consultingwerk/SmartUnit/runtest.p" 
            graphicalMode="true" dlcHome="${progress.DLC}" cpinternal="iso8859-1" cpstream="iso8859-1"
            compileUnderscore="true" inputchars="200000" assemblies="../ABL/Assemblies"
            iniFile="../ABL/ini/progress.ini" token="10000" dirSize="500000" stackSize="500000" maximumMemory="65534"
            paramFile="build/build.pf">

            <Profiler enabled="true" outputDir="profiler" coverage="true" />

            <options/>

            <Parameter name="TestSuite" value="@{testSuite}"/>
            <Parameter name="Tests" value="@{tests}"/>
            <Parameter name="Output" value="@{output}"/>
            <Parameter name="TestRunner" value="Consultingwerk.SmartUnit.OERA.RetrieveDataScenario.ScenarioRunner"/>
            <Parameter name="HaltOnError" value="@{haltOnError}"/>
            <Parameter name="ForceXmlXref" value="@{forceXmlXref}"/>
            <Parameter name="Services" value="@{services}"/>
        </PCTRun>
        
        </sequential>
    </macrodef>    

The ScenarioRunner macro supports the following arguments:

ArgumentDescription
testSuiteA name for the current test suite
testsComma delimited list of Packages that contain the .scenario files to be executed
outputThe file name of the JUnit result file (.xml)
servicesServices file to load to initialize the environment
optionsOptions such as PROPATH and Database connections passed to the PCTRun

Following is a sample invokation of the ScenarioRunner macro:

        <ScenarioRunner testSuite="OERA Test Scenarios" 
                   tests="Consultingwerk.OeraTests.*" 
                   output="output\oerascenarios.xml" 
                   haltOnError="false" 
                   forceXmlXref="false">
                   
            <options>
                <propath>
                    <pathelement path="." />
                    <pathelement path="../ABL" />
                    <pathelement path="../ABL/OERA" />
                    <pathelement path="../ABL/src" />
                    <pathelement path="${progress.DLC}/tty/netlib/OpenEdge.Net.pl" />
                </propath>

                <DBConnection dbName="sports2000" dbDir="../DB" singleUser="true">
                    <PCTAlias name="dictdb" />
                </DBConnection>
                <DBConnection dbName="SmartDB" dbDir="../DB" singleUser="true" />
                <DBConnection dbName="icfdb" dbDir="../DB" singleUser="true" />
            </options>
        </ScenarioRunner>

Executing Scenarios from other Unit Tests

The ScenarioRunner class can be invoked directly from unit test methods like shown in the sample below:

    METHOD PUBLIC VOID TestRunScenario():

        DEFINE VARIABLE oRunner AS ScenarioRunner NO-UNDO .
        DEFINE VARIABLE hDataset AS HANDLE NO-UNDO.

        oRunner = NEW ScenarioRunner() .
        oRunner:ExecuteScenario ("Consultingwerk/OeraTests/SCL1684/scl1684.scenario",
                                 OUTPUT DATASET-HANDLE hDataset) .

        hDataset::eCustomer:FIND-FIRST () .
        hDataset::eItem:FIND-FIRST () .

        Assert:Equals(2, hDataset::eCustomer::CustNum) .
        Assert:Equals(2, hDataset::eItem::ItemNum) .

        Assert:Equals(1, BufferHelper:NumRecords(hDataset::eCustomer:HANDLE)) .
        Assert:Equals(1, BufferHelper:NumRecords(hDataset::eItem:HANDLE)) .

        FINALLY:
            GarbageCollectorHelper:DeleteObject(hDataset) .
        END FINALLY.

    END METHOD.

Defining assertions in the Scenario JSON file

Without the definition of assertions, the unit test runner would only return a failure in case of runtime errors caused by the Business Entity read operation. Through assertions defined in the .scenario file, developers can assert the result data in detail.

Developer can implement assertions directly in the .scenario file as a JSON array:

{
    "SerializedType": "Consultingwerk.SmartUnit.OERA.RetrieveDataScenario.Scenario",
    "EntityName": "Consultingwerk.OeraTests.SCL1684.CustomerItemBusinessEntity",
    "EntityTables": "eCustomer,eItem",
    "QueryStrings": [
        "FOR EACH eCustomer WHERE eCustomer.CustNum = 2",
        "FOR EACH eItem WHERE eItem.ItemNum = 2"
    ],
    "BatchSize": 100,
    "StopAfter": 5,
    "Assertions": [
        {
            "NumResults": {
                "eCustomer": 1,
                "eItem": 1
            }
        },
        {
            "CanFind": [
                {
                    "buffer": "eCustomer",
                    "mode": "unique",
                    "where": "eCustomer.CustNum = 2"
                },
                {
                    "buffer": "eItem",
                    "mode": "unique",
                    "where": "eItem.ItemNum = 2"
                },
                {
                    "buffer": "eItem",
                    "mode": "first",
                    "where": "eItem.ItemNum = 2"
                }
            ]
        },
        {
            "CanNotFind": [
                {
                    "buffer": "eCustomer",
                    "mode": "unique",
                    "where": "eCustomer.CustNum = 3"
                }
            ]
        },
        {
            "MaxRuntime": 100
        },
        {
            "MaxReads": {
                "sports2000.customer": 1,
                "sports2000.item": 1,
                "sports2000.employee": 0
            }
        }
    ]
}

Within the Assertions array in the JSON document, the following sections are supported as Array Elements.

For Asserting on the DataSource Query:

{
    "SerializedType": "Consultingwerk.SmartUnit.OERA.RetrieveDataScenario.Scenario",
    "EntityName": "Consultingwerk.SimpleCustomerBusinessEntity.CustomerBusinessEntity",
    "EntityTables": "eCustomer",
    "BatchSize": 100,
    "StopAfter": 5,
    "QueryStrings": [
        "FOR EACH eCustomer WHERE eCustomer.CustNum = 1 or Customer.Country = \"USA\"  BY eCustomer.CustNum"
    ],
    "Assertions": [
        {
            "DataSourceQuery": "FOR EACH Customer WHERE Customer.CustNum = 1 or Customer.Country = \"USA\" BY Customer.CustNum INDEXED-REPOSITION"
        },
        {
            "WholeIndex": [false] 
        },
        {
            "IndexNames": ["CustNum,CountryPost"]
        }
    ]
}

NumResults

The NumResults section contains a JSON object with an INT64 property per Buffer who's num results should be asserted. In the above example, we require exactly one record to be returned for the eCustomer and eItem table.

CanFind

The CanFind section contains a JSON array of objects with a buffer, mode and where properties. This allows to test for records matching the given criteria. As the mode property, we support first and unique. Using the CanFind, developers can also assert for field values using a where criteria like

eCustomer.CustNum = 1 AND eCustomer.Name = "Lift Line Skiin"

CanNotFind

The CanNotFind section is similar to the CanFind section, just requiring that the specified criterias are not met by any record.

MaxRuntime

The MaxRuntime section contains an INT64 value specifying the maximum allowed runtiem for the Business Entity read operation.

MaxReads

The MaxReads section contains a JSON object with an INT64 property per database table. The MaxReads assertion will assert the maximum number of reads in the given database tables.

DataSourceQuery

The resulting query string for the first Entity Table, a single string.

WholeIndex

A JSON Array of boolean values indicating if a WHOLE-INDEX scan is performed for the matching data-source buffer. 

IndexNames

A JSON Array of strings indicating the index names used for the matching data-source buffer.

Using the Generic Mock Data Access factory

The .scenario JSON file may declare a MockDataAccess property. This property contains either a comma delimited list of Data Access mock JSON file names or a JSON Array of Data Access mock JSON file names. 

Loading custom services for the unit test

The .scenario JSON file may declare a LoadServices property. The property may contain a JSON Array of service.xml file names. The services defined in these services.xml files will be loaded before the test is executed. The services are loaded into a custom ServiceContainer only valid for the execution of the unit test which is overloading the default ServiceContainer (Consultingwerk.Framework.FrameworkSettings:ServiceContainer). This allows to load mock services.