1       Testing tools

Testing tools measures specific user performance in human – computer interaction. Each test is designed to measure one, exactly defined, type of interaction. During the test, several samples are taken, the average value is calculated, and then this average value is recorded in the database as a result of the test for specific person.

Some tests measure two or more values. For example, in the typing speed test the number of typed characters per second and the number of errors the user made during the test are measured. All these values are recorded into the database.

Below is a detailed description of the available tests.

1.1     Time for left mouse button click

This test measures how many times a user can click the left mouse button repeatedly at a given time interval.

Testee should click the "Click Me" button to start the test and in the same time to start time measurement. The button will turn green. Testee should press the same button repeatedly for 10 seconds. After the 10 seconds, the button will automatically become disabled which means the test is completed.

 Click me button

Testee has two option then:

  • To select the "Save" button to save data for a later analysis.
  • To select the "Quit test" button to discard this test result.

Measured values

label

Units

Number of mouse clicks

 

 

Elapsed time

 

 

Values recorded in database tables tests and mouse_click

DB Atribut

Type

User ID

cad_user_id (table tests)

int

Date and time of test

test_date (table tests)

Datetime

Number of clicks in given interval

cclick (table mouse_click)

int

Elapsed time

elapsed_time

ms

Calculated values

   
     

Values (test results) presented on the screen

   

User ID

   

Username

   

Date and time of test

   
     

 

   

Previous results of the testee for mouse clicks

 

Clicks per second

     

Histogram for average speed of mouse click

   

Sample median for average speed of mouse click

   

Sample mean for average speed of mouse click

   

Standard deviation for average speed of mouse click

   

1.2     Mouse movement test

This is compound test that discover testee’s ability to efficiently move the mouse (i.e. cursor). Starting position of the cursor (X0, Y0) is random (actually, position where cursor was positioned last time) and the values of X0 and Y0 are recorded in database. The screen displays the small rectangle which needs to be selected with the mouse. Time of appearance of the first rectangle T0 is recorded. Rectangle is selected only if testee click on some pixel inside rectangle.  After a point inside the rectangle is selected, current time Tc is recorded, new rectangle is drawn on some other random position of the screen and the user needs to re-select it. The procedure is repeated 10 times.

Mouse positionig test

As the size of the distance expressed in pixels can be affected by the resolution that is set on the user's screen, the data is normalized in comparison to the full HD resolution (1920 x 1080) before storing it in the database. Normalized coordinates are calculated using the following forms:

Xn = X * 1920/w

Yn = Y * 1080/h

where w and h represents the width and height of the user's screen.

 

Measured values

label

Units

Time when new rectangle appears on the screen

T0

second

Time when rectangle is selected – current time

Tc

second

X,Y coordinates of starting point

X0 , Y0

pixels

X,Y coordinates of the rectangle

Xr ,Yr

pixels

Consecutive X and Y coordinates over time

Xn ,Yn

pixels

Values recorded in database tables tests and mouse_speed

DB Atribut

Type

User ID

cad_user_id (table tests)

int

Date and time of test

test_date (table tests)

Datetime

Elapsed time from moment when rectangle is drawn until is selected

elapsed_time (table mouse_speed)

int (in ms)

Real distance

distance (table mouse_speed)

int

Short distance

shorter_distance (table mouse_speed)

int

Time spent to select new rectangle (Elapsed time Tr - T0)

Te

second

Distance between the starting point and the new rectangle

Di

pixels

Speed of mouse movement to the particular rectangle

Vi

pixels/s

Calculated values

   

Total distance between all rectangles

Dt

pixels

Averaged speed of mouse movement

Va

pixels/s

Total length of the actual path of the mouse

Lt

pixels

Mouse movement distortion ratio Mdr = Lt / Dt

Mdr

 

Mouse movement efficiency Me = Dt  / Lt

Me

 

Values (test results) presented on the screen

   

User ID

   

Username

   

Date and time of test

   

Averaged speed of mouse movement

   

Mouse movement distortion ratio

   

Previous results of the testee for averaged speed of mouse movement

   

Previous results of the testee for mouse movement distortion ratio

   

Histogram for averaged speed of mouse movement

   

Sample median for averaged speed of mouse movement

   

Sample mean for averaged speed of mouse movement

   

Standard deviation for averaged speed of mouse movement

   

Histogram for mouse movement distortion ratio

   

Sample median for mouse movement distortion ratio

   

Sample mean for mouse movement distortion ratio

   

1.3     Time for left, right, middle and double mouse button clicks

The goal of this application module is to determine the mean time required for one left, right, middle or double mouse click. Double click is performed with the left mouse button. After the test has been initiated, the message is shown containing the name of the mouse button to be pressed. The mouse button to be clicked is determined by random selection. If the user selects the correct button, a message about the selection of a new button is displayed. In total, during the test testee has to click 20 times.

Measured values

label

Units

Time when mouse button name appears on the screen

T0

ms

Time when mouse button is pressed

Tc

ms

Number of errors before exact button is clicked

En

 
 

 

 
     

Values recorded in database table tests and mouse_change_taster

DB Atribut

Type

User ID

cad_user_id (table tests)

int

Date and time of test

test_date (table tests)

Datetime

Mouse button name which should be pressed

taster

varchar

Previously pressed mouse button name

previous_taster

varchar

Elapsed time

time

int

Number of errors for given attempt

number_of_errors

 
     

Calculated values

   

Average time for LEFT, RIGHT, MIDDLE and DOUBLE button clicks (per user and per test)

   

Average number of errors for LEFT, RIGHT, MIDDLE and DOUBLE button clicks (per user and per test)

   
 

 

 
     
 

 

 
 

 

 
     

Values (test results) presented on the screen

   

User ID

   

Username

   

Date and time of test

   
     

 

   

Previous results of the testee for

   
     

Previous results of the testee for averaged time of mouse button clicks (LEFT, RIGHT, MIDDLE and DOUBLE CLICK)

   

Previous results of the testee for number of errors

   

Histogram for averaged time of mouse buttons clicks

   

Sample median for averaged time of mouse button clicks

   

Sample mean for averaged time of mouse buttons clicks

   

Standard deviation for averaged time of mouse buttons clicks

   

Those data that deviated from the average value by 500% were considered to be irregular. Also, users who have incorrectly reported the year (for example year: 5) and users who did not perform the test for 30 seconds were excluded from calculations. The extreme values of time usually appear at the start of the test. This is probably due to the fact that the user need some time to understand how the application works. Also, another cause of extreme values is the lack of user concentration, most frequently caused by:  message on the phone, conversation with another person, not watching the command display or continuous clicking on the left mouse button.

1.4     Time for mouse wheel scroll

This test implies scrolling of the text that is displayed in a separate text box. When scrolling the mouse, the number of rotating steps of the mouse wheel per second is measured. The text should be scrolled in both directions. The test lasts 10 seconds.

Measured values

label

Units

Number of mouse wheel rotating steps

 

 

Elapsed time

 

 
     
 

 

 
     

Values recorded in database tables tests and mouse_scroll

DB Atribut

Type

User ID

cad_user_id (table tests)

int

Date and time of test

test_date (table tests)

Datetime

Number of rotating steps in given interval

count (table mouse_scroll)

int

Elapsed time

elapsed_time

ms

     
 

 

 
     

Calculated values

   

Average number of rotating steps (per user and per test)

   
     
     

Values (test results) presented on the screen

   

User ID

   

Username

   

Date and time of test

   
     

 

   

Previous results of the testee for mouse scrolling

   
     

Histogram for average time of mouse scrolling

   

Sample median for average time of mouse scrolling

   

Sample mean for average time of mouse scrolling

   

Standard deviation for average time of mouse scrolling

   

This test needs to be improved because it does not correspond to the real use of the mouse wheel during the scroll. In a revised test, the user should scroll up to strictly specified lines of text. Now the scrolling goes back and forth, without paying attention to cursor positioning precision.

1.5     Time for left hand keystrokes

This test is used for determining average time needed for pressing one keystroke on the keyboard with left hand.  Keys that are used are: Alt, Ctrl, Shift, Esc. By random principle the program displays the key that has to be pressed. When the test starts, program displays the first key, and only when user presses a correct key, the next key is shown. The test is 30 seconds long.

Measured values

label

Units

Time when key name appears on the screen

 

 

Time when key is pressed

 

 
     
 

 

 
     

Values recorded in database table tests and mouse_change_taster

DB Atribut

Type

User ID

cad_user_id (table tests)

int

Date and time of test

test_date (table tests)

Datetime

Time elapsed since the display of the key name, until it has been pressed

elapsed_time (mili seconds)

int

Previous key that needed to be pressed

previous_taster

String

Key that should have been pressed

taster

String

     
     

Calculated values

   

Average input speed for all keys

 

Keys per second

Average input speed for ESC, ALT, SHIFT and CTRL keys

 

Keys per second

     

Values (test results) presented on the screen

   

User ID

   

Username

   

Date and time of test

   
     

 

   

Previous results of the testee for

   
     

Previous results of the testee for averaged speed of keyboard keys selecting (ALT, SHIFT, CTRL, ESC)

   
     

Histogram for averaged speed of keys selecting

   

Total median value of all users

   

Total average value of all users

   

1.6     Time for combination of keystrokes

Within this test, users were asked to simultaneously press one of the auxiliary keys (Ctrl, Shift or Alt) in combination with a key representing a letter (for example, Ctrl+X, Alt+O, etc.). Upon starting the test, a combination of keys (also known as keyboard shortcuts) that are meant to be selected by the subject is displayed in a specified space. If the subject inputs the designated combination successfully another combination is displayed and so on. As in previous cases, the combinations are chosen randomly, and the time required to enter the corresponding combinations is measured. The test lasts for 30 seconds.

Measured values

label

Units

Time when key combination appears on the screen

 

 

Time when key combination is pressed

 

 
     
 

 

 
     

Values recorded in database tables test

and keyboard_two_tasters

DB Atribut

Type

User ID

cad_user_id (table tests)

int

Date and time of test

test_date (table tests)

Datetime

Time elapsed since the display of the key combination, until it has been pressed

elapsed_time (mili seconds)

int

First key in combination

first_taster

String

Second key in combination

second_taster

Sring

     
     

Calculated values

   
     
     

Values (test results) presented on the screen

   

User ID

   

Username

   

Date and time of test

   
     

 

   

Previous results of the testee for averaged speed of keyboard key combination selecting (ALT, SHIFT, CTRL, ESC)

   
     

Histogram for averaged speed of key combination selecting

   

Total median value of all users

   

Total average value of all users

   
     

During the analysis of this test, all recorded instances of keystroke combinations with a duration time greater than a fivefold value of the average time were discarded. We believe that these occurrences of longer time intervals (which were discarded) were caused due to subjects being distracted by various unforeseen elements, such as: phone calls, communication with other users, etc.

1.7     Time required for typing regular text

Within this test, users were asked to type a pre-prepared specific text using both hands (for example, file name or part name). This test accepts all data typed in text box, with all possible mistakes, which makes this test imperfect. The data obtained are stored in the database for further processing. Before processing, two samples with too many errors were discarded. Data processing consists of calculating average times for one keystroke for every user and then average values for all users.

In addition, the number of errors per ten typed chars entered for each user is counted. The Damerau–Levenshtein algorithm is used to calculate the number of errors. The Damerau–Levenshtein distance between two words is the minimum number of operations (consisting of insertions, deletions or substitutions of a single character, or transposition two adjacent characters) required to change one word into the other. Since this number is usually too small, we counted the number of errors per 10 characters.

Measured values

label

Units

Time when text which should be entered appears on the screen

 

 

Time when enter key is pressed, after entering the text

 

 

Damerau-Levenshtein distance between entered and given text (this is number of errors)

   
 

 

 
     

Values recorded in database table tests and keyboard_text

DB Atribut

Type

User ID

cad_user_id (table tests)

int

Date and time of test

test_date (table tests)

Datetime

Time elapsed since the display of the text, until it has been entered

elapsed_time (mili seconds)

int

Text which should be entered

given_text

String

Text which is actually entered

entered_text

String

Number of errors for each entered text

number_of_errors

int

     

Calculated values

   
     
     

Values (test results) presented on the screen

   

User ID

   

Username

   

Date and time of test

   
     

 

   

Previous results of the testee for averaged speed of entering text from the keyboard

   
     

Histogram for averaged speed of text entering

   

Total median value of all users

   

Total average value of all users

   
     

1.8     Time required for typing numbers

The goal of this test is to see how much time it is needed for one keystroke of a number on the keyboard. By random principle the program displays a number made of three whole digits and one or two decimals separated by comma. Upon starting the test program displays the first number. When the user finishes pressing all number keys (and a comma that separates whole from decimal digits), user has to press Enter so the next number can be displayed. The process is repeated for 5 different numbers.

Data processing consists of calculating average times for one keystroke for every user and then average values for all users.

In addition, the number of errors per ten typed numbers entered for each user is counted. The Damerau–Levenshtein algorithm is used to calculate the number of errors. The Damerau–Levenshtein distance between two words is the minimum number of operations (consisting of insertions, deletions or substitutions of a single character, or transposition two adjacent characters) required to change one word into the other. Since this number is usually too small, we counted the number of errors per 10 typed numbers.

Measured values

label

Units

Time when number which should be entered appears on the screen

 

 

Time when enter key is pressed, after entering the number

 

 

Damerau-Levenshtein distance between entered and given number (this is number of errors)

   
 

 

 
     

Values recorded in database table tests and keyboard_numbers

DB Atribut

Type

User ID

cad_user_id (table tests)

int

Date and time of test

test_date (table tests)

Datetime

Time elapsed since the display of the number, until it has been entered

elapsed_period (mili seconds)

int

Number which should be entered

given_number

String

Text which is actually entered

entered_number

String

Number of errors for each entered number

error_number

int

     

Calculated values

   
     
     

Values (test results) presented on the screen

   

User ID

   

Username

   

Date and time of test

   
     

 

   

Previous results of the testee for averaged speed of entering numbers from the keyboard

 

Numbers per second

     

Histogram for averaged speed of numbers entering

 

Numbers per second

Total median value of all users

 

Numbers per second

Total average value of all users

 

Numbers per second

     

1.9     Time for transfer the right hand from the mouse to the keyboard and vice versa

Within this test, the speed of moving the hand from the mouse to the keyboard and vice versa is determined. The user is shown the rectangle to be selected by the mouse. After choosing the rectangle, he is asked to enter a number in the text input field. After entering the number, the rectangle to be selected is displayed again, and so on. The procedure is repeated 10 times. The time follows from the selection of the rectangle to the entry of the number, as well as the time elapsed from entering the number to the rectangle selection. The user is also required to use only the numeric keypad.

Measured values

label

Units

Time when number which should be entered appears on the screen

 

 

Time when enter key is pressed, after entering the number

 

 

Time when left mouse button is pressed

   
 

 

 
     

Values recorded in database table tests and mouse_keyboard_movement

DB Atribut

Type

User ID

cad_user_id (table tests)

int

Date and time of test

test_date (table tests)

Datetime

Time elapsed since the display of the number, until it has been entered

mouse_to_keyboard_time (mili seconds)

int

The time elapsed since the user presses the enter until the moment he presses the left mouse button

keyboard_to_mouse_time (mili seconds)

int

Value which should be entered

entered_value

String

 

 

 
     

Calculated values

   
     
     

Values (test results) presented on the screen

   

User ID

   

Username

   

Date and time of test

   
     

 

   

Previous results of the testee for averaged times of moving hand from mouse to keyboard

 

seconds

Previous results of the testee for averaged times of moving hand from keyboard to mouse

 

seconds

Histogram for averaged speed of numbers entering

 

Seconds

Total median value of all users

 

seconds

Total average value of all users

 

Numbers per second