Analyzing the Script Agent

From Seapine Labs

If you use another automated testing or scripting tool, you can also schedule and run automated script files through the use of a separate executable, known as a script agent, and a text file for communication between the script agent and TestTrack TCM.

The script agent launches the script and looks for the exit code of the script. Based on the exit code of the script, the script agent returns a status to the text file.

The text file, known as a *trjob file, communicates to TestTrack TCM the status and results of the script.

We are going to disect the script agents available for download here, to help you use it with your automated testing scripts.

This will be not be a line by line description of the source code of each script agent, rather, it will be a general walk through of both agents.

Note: If your script can only be launched through a graphical user interface, you can not use the script agent. The only requirement is for ability to launch the script from the command line or programmatically.

You can find out how to configure the script agent to launch your scripts here.

Contents 1 The Parameter 1.1 The script file path 1.2 Trjob file path 2 Execution command 2.1 The results file 3 Exit Codes 4 Example 4.1 Suggested enhancements to the Script Agent

[edit] The Parameter

When TestTrack TCM starts a script it calls the script agent. TestTrack TCM passes the script agent a single parameter, the full path to the script file.

[edit] The script file path

Under the General category in Tools > Local Options, the TestTrack TCM Automated Scripts Directory is set. This directory is the working area for TestTrack to launch the script file and record the result in the .trjob file.

Under this directory, TestTrack creates a directory called ScheduledScripts, and under it, TestTrack creates a directory with the name of the TestTrack project. If you have multiple projects that have automated scripts, then you will have multiple directories. Under the project folder, a folder is created for each script in each test run. The naming of the folder is TR<test run number>_<script ID>_<Script name>.

For example, if I run the script "LoginTest.vbs" that has a script id of 1, and I run it for test run 28, the folder name is "TR28_1_LoginTest".

TestTrack makes a copy of the script file and places it in this directory. The full path to the script passed to the script agent points to this file. Assuming that the TestTrack Automated Script Directory is set to C:\scripts\ and that we are working with the sample project, the full path in this example is C:\scripts\ScheduledScripts\Sample Project\TR28_1_LoginTest\LoginTest.vbs.

[edit] Trjob file path

TestTrack TCM looks for the trjob file in the same directory where the script is. It assumes the trjob file has the same name as the script. In the example above, TestTrack TCM is looking for a file called LoginExample.trjob in the C:\scripts\ScheduledScripts\Sample Project\TR28_1_LoginTest\ directory.

The script agent, takes the parameter (full path to the script file) and looks for the location of the period (the period preceding the file extension). It then takes the text upto the extension and appends "trjob" to the file.

Note: The current code looks for the first instance of a period. If any of the directories in the script file path has a period, the script agent is likely to not work.

[edit] Execution command

The QA Wizard (and QA Wizard Pro) batch files are stand alone, and that is why in the script agent code it simply sends the full path the script file to the "system". If your script file is not stand alone (you must also tell the system which application is needed to run it), you can edit the command to also include the executable. The command would probably be something like "<Full path to executable> <Full path to script file>".

The execution command is different between the QAWProBatAgent and the QAWBatAgentApp.

[edit] The results file

Both have the "system" run the script, but the QAWProBatAgent also passes a parameter that indicates where to place the report file. The QAWBatAgentApp doesn't do this because the report is always created in a subdirectory under the location of the script file.

The reason why we care about the report is because this is the results file. In the trjob file, the relative path to the report is set for the MainResultsFile. This allows you to view the report from TestTrack. You may have to edit this section of the code to accomodate the way your automated testing application handles reporting.

Note: The script agent does not create a report nor does TestTrack. If your automated test tool does not create a report, or you can't figure out how to programatically figure the path to the report for the script, or the report can not be opened outside of the tool, you may want to simply comment out the section where the path to the results file is set.

[edit] Exit Codes

When the script agent passes the execution command to the system, it captures the exit code to an integer. The script agent uses this integer to determine the status to enter in the trjob file.

If the exit code is 0, then the status of the script in the trjob file is set to Passed. If the exit code is 1, then the status is set to Failed. For any other exit code, the status is set to Indeterminate.

[edit] Example

The likelyhood is that most of you will not want to change the source code and use the script agent as is. Note that TestTrack does install exectuables for each version of the script agent. Under the installation directory, you will see a ScriptAgents directory containing several agents, including QAWAppAgent.exe and QAWProAppAgent.exe.

For this example, we will look at AutomatedQA's TestComplete. This tool allows you to create a batch file that you can use to run the script "stand alone". Below is an example batch file from their documentation:

REM Clears the screen
CLS
@ECHO OFF
REM Launches TestComplete,
REM executes the specified project
REM and closes TestComplete when the run is over
"C:\Program Files\Automated QA\TestComplete 6\Bin\TestComplete.exe" "C:\Program Files\Automated QA\TestComplete 6\Samples\Scripts\Hello\Hello.pjs"   /u:Hello_VBScript /rt:Hello_VB /run /exit /c
IF ERRORLEVEL 3 GOTO CannotRun
IF ERRORLEVEL 2 GOTO Errors
IF ERRORLEVEL 1 GOTO Warnings
IF ERRORLEVEL 0 GOTO Success
 
:CannotRun
ECHO The script cannot be run
GOTO End
 
:Errors
ECHO There are errors
GOTO End
 
:Warnings
ECHO There are warnings
GOTO End
 
:Success
ECHO No errors
GOTO End
 
:End

The batch file calls the TestComplete application and passes parameters including the test suite. Then based on the results (ERROLEVEL), it echoes a status back to the command line.

This works well for us, except for one issue: the exit code. When the script agent runs this batch file, it will always receive an exit code of 0, unless there was an issue runnig the batch file.

A quick and dirty solution is to simply add EXIT # to each ERRORLEVEL section. In this example, "Errors" is mapped to the "Failed" status, "Success" is mapped to the "Passed" status, and "CannotRun" and "Warnings" is mapped to the "Indeterminate" status. This solution does not require changing the source code.

REM Clears the screen
CLS
@ECHO OFF
REM Launches TestComplete,
REM executes the specified project
REM and closes TestComplete when the run is over
"C:\Program Files\Automated QA\TestComplete 6\Bin\TestComplete.exe" "C:\Program Files\Automated QA\TestComplete 6\Samples\Scripts\Hello\Hello.pjs"   /u:Hello_VBScript /rt:Hello_VB /run /exit /c
IF ERRORLEVEL 3 GOTO CannotRun
IF ERRORLEVEL 2 GOTO Errors
IF ERRORLEVEL 1 GOTO Warnings
IF ERRORLEVEL 0 GOTO Success
 
:CannotRun
ECHO The script cannot be run
EXIT 3
GOTO End
 
:Errors
ECHO There are errors
EXIT 1
GOTO End
 
:Warnings
ECHO There are warnings
EXIT 2
GOTO End
 
:Success
ECHO No errors
EXIT 0
GOTO End
 
:End

[edit] Suggested enhancements to the Script Agent

If you use TestComplete and feel comfortable working with C++, there are some improvements that would enhance the script agent:

• Have the script agent use the TestComplete status when updating the trjob file (i.e., instead of "Passed", enter "Success"). This would require adding logic to handle more exit codes.*

• Have the script agent simply read the ECHO output and use this to update the status of the trjob file.*

• Figure the reporting in TestComplete and add a reference to the file in the trjob file.

* Using different script status (Passed, Failed, Indetermined) may break the ability to automatically perform a workflow event based on script status.

How you modify or use the script agent is a really function of how your scripts are setup, and what you're looking to accomplish with the agent. If you want some assistance in this area, Seapine Services can help you with that.