Docs Image

[EUC Score Toolset Documentation Start Page] [Simload Gallery]

Simulated Workloads (Simloads)

Repeatable synthetic user activities specifically designed for user experience benchmarking.

Introduction

Getting started with simulated workloads - overview and launch options.

Side-by-Side File

Understand and overwrite the default settings of a Simload.

Temporary Object

Purpose and structure of the temporary Simload object in the registry.

Introduction

A Simload (= simulated workload) is a compiled executable that can be registered and launched on a guest VM or a physical Windows workstation. Such a Simload simulates (synthetic) user activities representing typical interactive computer work patterns. A Simload launches one or multiple applications, starts logging timestamps, simulates user interactions by controlling the applications, and after a predetermined time terminates all applications under control and stops logging.

A singular Simload type 1 ("Workload") is an individual test scenario for the user in focus. Each type 1 Simload scenario highlights a specific graphic or multimedia format. Most type 1 Simloads require a pre-installed application. Some applications open data files as part of the test scenario. Each type 1 Simload can either be launched in the foreground (using the Run parameter) or in the background of a user session (using the Start/Stop parameters).

A blended Simload type 2 ("Persona") is a sequence of chained or overlayed user activities, orchestrated in such a way they generate the characteristic behavior and load pattern of a predefined interactive user type or persona. Examples for such user types are Author, Knowledge Worker, Power User or Media User. Most type 2 Simloads launch and control multiple applications simultaneously, with a chain of subsequent foreground applications and multiple background applications running in parallel.

A score Simload type 3 runs test applications and measures predefined system metrics used to produces a number (= score) that represents the performance. Typically, each score Simload is associated with a specific theme, such as graphics output performance or application load time. The individual scores collected for each theme can be used to calculate an overall score.

 

Launching Simloads

The type 1 Simload command-line syntax is as follows:

  • SL1-Simload.exe run <seconds> [<x> <y> [<width> <height>]]
  • SL1-Simload.exe start <tempobject> [<x> <y> [<width> <height>]]
  • SL1-Simload.exe stop <tempobject>
  • SL1-Simload.exe info
  • SL1-Simload.exe help

The parameter run launches a type 1 Simload in run mode for a predefined number of seconds and at a configurable screen position (window origin coordinates and window size). A Simload in run mode may include user activities. There can only be one type 1 Simload in run mode up and running at any given time. If width and height are set to 0 (zero), most type 1 Simloads run in full-screen mode.

The parameter start launches the selected type 1 Simload in start/stop mode. This mode requires a unique tempobject parameter that is used to control the application and to terminate it with the stop parameter. Multiple Simloads in start/stop mode can be running simultaneously. If width and height are set to 0 (zero), most Simloads start in full-screen mode. The default behavior of a Simload in start/stop mode is passive, which means that there is no user interaction.

The parameter info shows all relevant object properties as loaded from the temporary object structure in the registry or the side-by-side .ini file. The parameter help shows all command-line parameters.

Examples:

SL1-NotepadEdit run
Runs the SL1-NotepadEdit simload in fullscreen mode for the predefined number of seconds (typically 45 or 60 seconds)
SL1-NotepadEdit run 45 100 100 1200 800
Runs the SL1-NotepadEdit simload at x/y position 100/100 and window size 1200/800 for 45 seconds
SL1-NotepadEdit start MyNotepad 100 100 800 600
Creates a temporary object named MyNotepad and starts the SL1-NotepadEdit simload at x/y position 100/100 and window size 800/600
SL1-NotepadEdit stop MyNotepad
Stops the SL1-NotepadEdit simload and deletes the temporary object named MyNotepad

 

The type 2 and type 3 Simload command-line syntax is as follows:

  • SL2-Simload.exe run <seconds> [<x> <y> [<width> <height>]]
  • SL2-Simload.exe help

Setting window origin coordinates [x, y] and window size values [width, height] is optional. If there is no value set or the values are [0, 0, 0, 0], the Simload uses default positions for the foreground application windows. Setting x, y, width and height to values larger than 0 influences the position and size of the foreground application windows.

Side-by-Side File

Each SimLoad comes with a side-by-side .ini file with the same name as the Simload executable. This .ini files can be used to overwrite the hard-coded Simload default settings. It also acts as a fallback in case general Simload settings and folder locations are not stored in the registry. This is the case when Simloads were deployed by copy and paste rather then running a regular EUCScore installation package.

Please note that a semi-colon at the beginning of a line in an .ini file is interpreted as a full-line comment. This means that the leading semi-colon must be removed to activate the setting defined in a line. It's a best practice to show the hard-coded Simload default settings in side-by-side .ini file comment lines.

Below is the example of the SL1-NotepadEdit.ini file:

[SL1-NotepadEdit]
;sAppName=Notepad.exe
;sAppPath=c:\Windows\System32\
;fAppPathType=2
;sAppParam=
;bEventFile=1
;sDataFile=lordoftherings.txt
;sDataPath=c:\Simloads\Data\Documents\
;fDataPathType=2
;bDataRwAccessRequired=
;sWinTitle=
;sAppControlType=byHandle
;iX=0
;iY=0
;iWidth=0
;iHeight=0
;iTime=45

[Settings]
bLogVerbose=0
bDebug=0
iCountdownScreen=3
bShowControl=1

[Folders]
InstallPath=C:\EUCScore
Data=C:\EUCScore\Data\
Results=C:\EUCScore\Results\
Simloads=C:\EUCScore\Simloads\
Tools=C:\EUCScore\Tools\

[Simloads]
SL1-NotepadEdit=C:\EUCScore\Simloads\SL1-Notepad\SL1-NotepadEdit.exe

 

Most settings in the side-by-side .ini file are self-explanatory, except for fAppPathType, fDataPathType, bDataRwAccessRequired and sAppControlType. Here are the short descriptions:

fAppPathType and fDataPathType are flags that define how sAppPath and sDataPath are interpreted. Allowed values of the application and data path type are 0, 1, 2, 3, 4, 5 or 6.

0 (= None)
There is no application or data file
1 (= Relative)
A relative path that starts from the default data folder stored in HKLM\Software\EUCScore\Folders\Data, HKCU\Software\EUCScore\Folders\Data or in the side-by-side .ini file under [Folders] Data=...
2 (= Absolute)
An absolute path in the local file system
3 (= Simload Folder)
A simload-specific relative path that starts from the location of the simload .executable file that is stored in HKLM\Software\EUCScore\Simloads\<SimloadName>, HKCU\Software\EUCScore\Simloads\<SimloadName> or in the side-by-side .ini file under [Simloads] <SimloadName>=...
4 (= Internet)
The path is a URL located in the internet
5 (= Profile)
A relative path that starts in the MyDocuments folder of a user profile (C:\Users\<UserName>\Documents\)
6 (= Public)
A relative path that starts in the Public Documents folder (C:\Users\Public\Documents\)

When setting bDataRwAccessRequired to 1, the data file is copied to the user's MyDocuments\temp folder prior to opening it and the fDataPathType flag is changed accordingly. This gives the synthetic user full read/write access to the data file and allows individual editing.

The sAppControlType value can be set to byPID, byHandle or byTitle. This value defines how an application is controlled within the simload, for example for user interaction or application termination. In most cases, byHandle is the best choice. Setting this value to byTitle requires a valid sWinTitle value and is used when the process ID and window handle values returned at application start is not working properly. This can happen when an application creates child windows on opening documents or other data files.

IMPORTANT: In case a Simload was deployed to the target system by copy and paste, edit the side-by-side .ini file to make sure the folders under [Folders] and [Simloads] exist in the loacal files system and the folder paths are set correctly.

Temporary Simload Object

Each application launched and controlled by a simload is represented by a temporary object in the user session. All temporary object properties are stored in the registry under HKCU\Software\EUCScore\Temp\<ObjectName>. Temporary objects assigned to a Simload get deleted as soon as the Simload reached the end of its runtime.

TEMP CLASS STRUCTURE: The sTempObjectName string is used to identify the object. It is either identical to sSimloadName or it is a unique custom string in case of Start/Stop.

-- sTempObjectName Name of the temporary object
-- bActive [1] Temporary object is active, [0] marks the object to be terminated
0 sSimloadName Name of the simload stored in the temporary object
1 sStartDate Start date, format YYYY/MM/DD
2 sStartTime Start time, format HH:MM:SS
3 sAppName* Application executable name, including extension
4 sAppPath* Application path
5 fAppPathType* Application path type: None [0], Relative [1], Absolute [2], Simload Folder [3], Internet [4], Profile [5], Public [6]
6 sAppParam* Optional application parameters
7 sEventFilePath Event file w/ timestamps and descriptions, consumable by SxS Player
8 bEventFile* [1] creates an event file, [0] does not
9 sDataFile* Data file name, including extension
10 sDataPath* Data file path
11 fDataPathType* Data file path type: None [0], Relative [1], Absolute [2], Simload Folder [3], Internet [4], Profile [5], Public [6]
12 bDataRwAccessRequired* [1] copy data file to MyDocuments\temp, change fAppPathType to 2, change sDataPath
13 sWinTitle* Title of the application window
14 sWinHandle Handle of the application window
15 sProcessID Application process identifier
16 sAppControlType* byTitle, byHandle, byPID
17 iX* X position of the application window
18 iY* Y position of the application window
19 iWidth* Width of the application window (0 is full screen width)
20 iHeight* Height of the application window (0 is full screen height)
21 iTime* Simload runtime in seconds
22 iAppLaunchTime Application launch time in milliseconds

All structure items with a * can be defined in a Simload-specific side-by-side .ini file.

Changing the bActive value from 1 to 0 during runtime results in the termination of most Simloads within one user interaction cycle.