Docs Image

[EUC Score Toolset Documentation Start Page] [Sample Results]

Sync Player

The Sync Player - sometimes referred to as Side-by-Side Player - is an HTML5 application that runs inside a browser, such as Google Chrome or Microsoft Edge. It was designed to playback videos, show system and user activities, and animate telemetry data side-by-side with a single slider control and in a synchronized fashion. The Sync Player is typically used in conjunction with datasets generated during EUC Score test runs, using screen video recorders and telemetry data collectors.

It is important to note that both the system under test and the primary user's endpoint device act as data or video sources. The two nodes may be at locations far away from each other. This is why data and file timestamps are very important for the synchronisation of the test result visualizations.

 

Sync Player Data Analytics

 

After the Installation of the EUC Score Sync Player installer package, the Sync Player templates, components, and libraries are located in the EUCScore\SyncPlayer subfolder. To build your own customized side-by-side visualizations, create new .html files using the examples as a starting point. For the build automation of complete sets of run visualizations, please refer to the build automation subchapter below.

The top left media tile always includes an MP4 video (= "pacemaker" video). The top right media tile includes either another video or animated text output using .ref file data. The title bar at the top of each media tile includes one line of description text. The individual media tiles are surrounded by a border, each in a different color used for color-coding the data visualization. Important: The video aspect ratio must always be maintained.

The numerical data visualization area is located below the media tiles. It shows eight graphs with timely correlated and animated telemetry data collected on the host side. Each graph shows a title, axis labels and either one or two progressive lines. One or two CSV files created by the Avatar, the Simload Runner or any other performance data collector act as the data source for the numerical data.

The control and status bar in the footer includes video controls (play, pause and progress bar) and status information, such as elapsed time and total time. On the right side is a URL that includes the link to a help page and a button that opens or closes the report popup box.

In singe view, the Sync Player client area is divided into one foreground media output tile and one system and user activity tile on the top (each including a title bar), a numerical data visualization area at the bottom and a control and status bar in the footer.

 

Sync Player Single View

 

In side-by-side view (= double view), the Sync Player client area is divided into two foreground media output tiles on the top (each including a title bar), a numerical data visualization area at the bottom and a control and status bar in the footer. The progressive lines in the data charts are color-coded according to the color of the borders surrounding the media tiles.

 

Sync Player Side-by-Side View

 

Sync Player Clips - Build Automation

Each test run visualization in the Sync Player is encoded as an HTML file. To avoid manual editing, there is a set of build automation PowerShell scripts for an entire test sequence. To see PowerShell script syntax details, use the Get-Help comdlet.

  1. Create-0SimloadFolders.ps1: Create folder structure with individual Simload folders.
  2. Create-1IniSingle-8Charts.ps1: Create .ini file for Sync Player clips in single view mode.
  3. Create-2CsvSingle-8Charts.ps1: Read Simload telemetry data and create new .csv file with eight columns.
  4. Create-3HtmlSingle.ps1: Create .html files for Sync Player clip in single view mode.
  5. Create-4IniDouble-8Charts.ps1: Create .ini files from two single view .ini files for Sync Player clips in side-by-side view mode (= double).
  6. Create-5HtmlDouble.ps1: Create .html files for Sync Player clips in side-by-side view mode (= double).

The Sync Player build scripts import frequently used functions and strings from the EUCScore-PsLib.psm1 module in the root folder. Each start of a build script including its parameters is logged in the BuildScriptLogFile.log file located in the test run target folder.

It is possible to run the same build script multiple times. In the case of INI build scripts, the first run creates a new .ini files with default values. Any subsequent run will inherit the values of the existing .ini file and create a backup file with a timestamp.

The build scripts were used to create the Sample Results

 

Prerequisites

The dataset files of a test sequence and the build scripts need to be located in a root folder. It must also include a _libs subfolder with Java Script libraries (bootstrap, chart, jquery) and CSS files. Another prerequisite is a _templates folder with .html templates used by PowerShell scripts located in the root folder.

Each test sequence is represented by a dedicated data folder in the SyncPlayer root folder, with a descriptive name. The raw data of each test run must be located in a separate subfolder with a name that starts with SL, for example SL1-DominoOpenGL. The file types in each test run subfolder required by the build scripts are the following:

  • One (and only one) MP4 video file created by the screen recorder, ideally with the Simload name in the file name, for example SL1-DominoOpenGL-2022-07-20 09-46-42.mp4.
  • The activity file with the .ref extension created by the Simload. The file name MUST include the test run subfolder name, for example NUC2_Benny_SL1-DominoOpenGL-2022-07-20-09-46-25.ref.
  • The CSV telemetry file created by the Avatar Telemetry Collector or the Simload Runner. The file name MUST start with "Telemetry" and it MUST include the test run subfolder name, for example Telemetry_NUC2_Benny_SL1-DominoOpenGL_2022-07-20-09-47-09.csv.

If any of the prerequisites is not met, the PowerShell scripts may fail and throw error messages. With all result files of a test sequence in one single root folder, the Create-0SimloadFolders script builds the correct subfolder structure.

  • .\Create-0SimloadFolders.ps1 -DataFolder .\TestRun1

 

STEP 1: Create INI file for Single View Mode

In Sync Player Single View Mode, the top left tile contains a test run screen video and the top right tile shows animated text representing timely correlated user activities and session runtime details. The numerical data visualization area shows animated telemetry data collected on the host side.

Run the Create-1IniSingle-8Charts script that creates the .ini file in the test sequence data folder. In the example below, the data folder is named TestRun1 und the .ini file is named TestRun1.ini.

  • .\Create-1IniSingle-8Charts.ps1 -DataFolder .\TestRun1 -IniFile TestRun1.ini

NOTE: Check out the syntax of the script with Get-Help .\Create-1IniSingle-8Charts.ps1

The resulting .ini file looks like this:

[Templates]
SourceFolderTemplates=_templates2
SourceFolderLibs=_libs2
TemplateHTMLVidRef8Charts=Template-VidRef-8Charts.html
TemplateHTMLReport=Template-Report.html

[Common]
TargetPath=TestRun1
TargetFolder=TestRun1
Date=2023-01-05
CountDownInSec=1
LeftTitle=EDIT LEFT TITLE
LeftDescription=EDIT LEFT DESCRIPTION
LeftRef=true
RightTitle=EDIT RIGHT TITLE
RightDescription=EDIT RIGHT DESCRIPTION
RightRef=true
Connection=EDIT PROTOCOL AND NETWORK CONDITIONS
Endpoint=EDIT ENDPOINT SPECIFICATION

[Report]
ReportTitle=TestRun1
TableTitleLeft=Screen Recording
TableTitleColorLeft=#efefef
TableTitleRight=System and User Activities
TableTitleColorRight=#efefef
TableTextLeft=Screen video captured with a frame grabber and recorded with OBS Studio at full HD resolution and 60 frames per second.
TableTextRight=Visulization of the timestamped activities in the reference file created by the Simload.

[Telemetry8Std]
Prefix=STD
Header0=TimeStamp|1000
Header1=CPU|%
Header2=Memory|MBytes
Header3=Network Received|KBytes/sec
Header3=Network Received|KBytes/sec
Header4=Network Sent|KBytes/sec
Header5=CPU Queue Length
Header6=Disk Reads|Bytes/sec
Header7=Disk Writes|Bytes/sec
Header8=Session CPU|%

[Telemetry8GPU]
Prefix=GPU
Header0=TimeStamp|1000
Header1=CPU|%
Header2=Memory|MBytes
Header3=Network Received|KBytes/sec
Header4=Network Sent|KBytes/sec
Header5=GPU 3D|%
Header6=GPU Video Decode|%
Header7=GPU Video Processing|%
Header8=GPU Memory|MBytes

[1]
Name=SL1-BSPBlendingDX11
LengthInSec=45
VideoFile=SL1-BSPBlendingDX11-2022-05-20 09-46-42.mp4
TelemetryFile=Telemetry_NUC2_Benny_SL1-BSPBlendingDX11_2022-05-20-09-47-09.csv
ActivityFile=NUC2_Benny_SL1-BSPBlendingDX11-2022-05-20-09-46-25.ref
ReportFile=SL1-MSEdgeFishbowlHTML5-Report.html
ReportFindings=None
ReportTableCommentLeft=
ReportTableCommentRight=

...

 

The resulting .ini file automatically opens in Notepad, allowing to change the "EDIT..." values in the [Common] section. Add adequate title and description strings. LeftTitel and RightTitel is what you see in the title bar above the top quadrants. LeftDescription, RightDescription, Connection and Endpoint are the strings you see when you click on the information icon in the title bar.

IMPORTANT: Make sure that Header values in the [Telemetry*] sections reflect the headers in the CSV telemetry files.

 

STEP 2: Extract CSV data with eight columns

The Create-2CsvSingle-8Charts script uses the keys in one of the the [Telemetry] sections to create new .csv files in each SL* subfolder, adding the prefix STD or GPU to the file names. These new .csv files only include the selected eight columns.

In the [Telemetry] sections, the value of Header0 is always TimeStamp|1000. The values of Header1 through Header8 must match the EXACT string of a column header in the .csv files. Please check if the column headers configured in [Telemetry8Std] and [Telemetry8GPU] really exist, otherwise the script will break when trying to create STD and GPU .csv files.

  • .\Create-2CsvSingle-8Charts.ps1 -IniPath .\TestRun1\TestRun1.ini -Telemetry Telemetry8Std
  • .\Create-2CsvSingle-8Charts.ps1 -IniPath .\TestRun1\TestRun1.ini -Telemetry Telemetry8GPU

After running these scripts successfully, you'll find new STD-*.csv and GPU-*.csv files in each SL1 subfolder.

 

STEP 3: Create HTML files for Single View Mode

The Create-3HtmlSingle script uses the templates, the .ref files and the new .csv files to create an individual .html file for each test run in a SL* subfolder.

  • .\Create-3HtmlSingle.ps1 -IniPath .\TestRun1\TestRun1.ini -Telemetry Telemetry8Std
  • .\Create-3HtmlSingle.ps1 -IniPath .\TestRun1\TestRun1.ini -Telemetry Telemetry8GPU

In addition, the script creates a LinkList-STD-Foldername.html or LinkList-GPU-Foldername.html file in the Simload root folder.

IMPORTANT: Building the Sync Player single view HTML files requires the _template2 folder with the HTML templates to be located in the same root folder as the PowerShell build scripts. The _libs2 folder is also located in the root folder. It contains all CSS and JavaScript files that are needed during the runtime of a Sync Player clip. The relative path of this library folder must always be maintained. If this is not the case, the clips will not start.

 

STEP 4: Create INI file for Side-by-Side View Mode (= Double)

In Side-by-Side View Mode, both top tiles include test run screen videos with a yellow border on the left side and a red border on the right side. The numerical data visualization area shows animated telemetry data collected on the host side and color-codeed according to the video border colors.

Prerequisite is TWO data folders with their respective single view .ini files. The Create-4IniDouble-8Charts script creates a new data folder with its side-by-side .ini file.

  • .\Create-4IniDouble-8Charts.ps1 -SourceIniPath1 .\TestRun1\TestRun1.ini -SourceIniPath2 .\TestRun2\TestRun2.ini -TargetName Test1-vs-Test2

Please review and if needed edit the resulting .ini file.

[Templates]
SourceFolderTemplates=_templates2
SourceFolderLibs=_libs2
TemplateHTMLVidVid8Charts=Template-VidVid-8Charts.html
TemplateHTMLReport=Template-Report.html

[Common]
TargetFolder=SxS-Test1-vs-Test2
TargetName=Test1-vs-Test2
Date=2023-01-16
LeftCountDownInSec=1
LeftSourceFolder=TestRun1
LeftTitle=Test 1 Left Title
LeftDescription=Test 1 Left Description
LeftConnection=Test 1 Left Connection
LeftEndpoint=Test 1 Left Endpoint
LeftRef=false
RightCountDownInSec=1
RightSourceFolder=TestRun2
RightTitle=Test 2 Left Title
RightDescription=Test 2 Right Description
RightConnection=Test 2 Right Connection
RightEndpoint=Test 2 Right Endpoint
RightRef=false

[Report]
ReportTitle=Test1-vs-Test2
TableTitleLeft=Screen Recording (Left)
TableTitleColorLeft=#ffce56
TableTitleRight=Screen Recording (Right)
TableTitleColorRight=#ff6384
TableTextLeft=Screen video captured with a frame grabber and recorded with OBS Studio at full HD resolution and 60 frames per second.
TableTextRight=Screen video captured with a frame grabber and recorded with OBS Studio at full HD resolution and 60 frames per second.

[Telemetry8Std]
Prefix=STD
Header0=TimeStamp|1000
Header1=CPU|%
Header2=Memory|MBytes
Header3=Network Received|KBytes/sec
Header3=Network Received|KBytes/sec
Header4=Network Sent|KBytes/sec
Header5=CPU Queue Length
Header6=Disk Reads|Bytes/sec
Header7=Disk Writes|Bytes/sec
Header8=Session CPU|%

[Telemetry8GPU]
Prefix=GPU
Header0=TimeStamp|1000
Header1=CPU|%
Header2=Memory|MBytes
Header3=Network Received|KBytes/sec
Header4=Network Sent|KBytes/sec
Header5=GPU 3D|%
Header6=GPU Video Decode|%
Header7=GPU Video Processing|%
Header8=GPU Memory|MBytes

[1]
Name=SL1-BSPBlendingDX11
LengthInSec=45
LeftDelayInSec=0
LeftVideoFile=SL1-BSPBlendingDX11-2022-05-20 09-46-42.mp4
LeftTelemetryFile=Telemetry_NUC2_Benny_SL1-BSPBlendingDX11_2022-05-20-09-47-09.csv
RightDelayInSec=0
RightVideoFile=SL1-BSPBlendingDX11-2022-05-22 13-09-26.mp4
RightTelemetryFile=Telemetry_DUNGEON3_Benny_SL1-BSPBlendingDX11_2022-05-22-13-09-53.csv
ReportFile=SL1-BSPBlendingDX11-Report.html
ReportFindings=None
ReportTableCommentLeft=
ReportTableCommentRight=

...

 

STEP 5: Create HTML files for Side-by-Side View Mode (= Double)

The Create-5HtmlDouble script uses the templates and the .csv files to create an individual .html file for each side-by-side comparison.

  • .\Create-5HtmlDouble.ps1 -IniPath .\SxS-Test1-vs-Test2\Test1-vs-Test2.ini -Telemetry Telemetry8Std
  • .\Create-5HtmlDouble.ps1 -IniPath .\SxS-Test1-vs-Test2\Test1-vs-Test2.ini -Telemetry Telemetry8GPU

In addition, the script creates a LinkList-STD-SxS-Foldername.html or LinkList-GPU-SxS-Foldername.html file in the SxSPlayer root folder.

 

Post-Production

A number of additional PowerShell scripts are provided for post-production purposes.

  • Update-SyncPlayerReport.ps1: Rebuild one or all reports in the test sequence folders defined in the .ini file.
  • Export-SyncPlayerClip.ps1: Export one or all test sequence clips and the runtime libraries to a target folder structure.
  • Convert-CsvFile.ps1: Read a .csv file, create a backup of it, replace strings and save results in a new .csv file with the original name. Typically used to convert the German number format to the US number format by switching periods and commas.