User Guide
Version v3.6.0

Test Case Data Format

Alyvix test cases are stored as JSON files, which are then used by all Alyvix applications. Alyvix files represent the unique mechanism for interchange between Alyvix modules, and each file is self-contained, without any external dependencies.

Alyvix and its data structures are open source. The entire JSON file format is described in detail below.

Test Case Files

Each time you exit Designer or Selector by pressing the OK button, or save an object in Editor, the JSON object representing the test case is saved in the current directory under the file name:

<filename>.alyvix

where filename is the value of the -f parameter passed when starting Editor or Designer.

If you then call Alyvix Robot via the command line with that file as a parameter, when it finishes it will create a new file based on the original test case, but with the resulting measures included. The file name will have the timestamp at the moment of execution appended:

<filename>_<full-timestamp>.alyvix

For example:

logintest_20191220_145228_UTC+0100.alyvix

Note

This file is not produced when a test case is executed in Editor via runblue, only when executed directly from Alyvix Robot.

Test Case JSON Structure

A single .alyvix file stores the entire test case and holds all measures resulting from a single execution run.

The following example JSON structure illustrates the high-level structure of the test case object:

 1{
 2   "maps": { },
 3   "objects": {
 4      "<test-case-object-name>": {
 5         "call": { },
 6         "components": {
 7            "<resolution>": {
 8               "groups": [
 9                  { "main": { },
10                    "subs": [ ] },
11                  { "main": { },
12                    "subs": [ ] },
13                  { "main": { },
14                    "subs": [ ] }
15               ],
16               "screen": "<base64>"
17            }
18         },
19         "date_modified": "<timestamp>",
20         "detection": {
21            "break": true,
22            "timeout_s": 10,
23            "type": "appear" },
24         "measure": {
25            "group": "<transaction-group-name>",
26            "output": true,
27            "series": { },
28            "thresholds": { } }
29      }
30   },
31   "script": {
32      "case": [
33         "<test-case-object-name>",
34         { "disable": "<test-case-object-name>" },
35         { "flow": "<test-case-object-name>",
36           "if-true": "<test-case-object-name>" },
37         { "flow": "<test-case-object-name>",
38           "if-false": "<test-case-object-name>" },
39         { "flow": "<test-case-object-name>",
40           "for": "<test-case-map-name>" }
41      ],
42      "sections": {
43         "<section-name>": [
44            "<test-case-object-name>"
45         ],
46         "exit": [],
47         "fail": [] }
48   }
49}

The individual sections of the JSON structure are explained below.

  1. Top level

    The top level corresponds to the principal elements of Alyvix Editor:

    { "maps": {
     "map-name":  { } },
  "objects": {
     "<test-case-object-name>":  { } },
  "script": {
     "case":  [ ],
     "sections":  { } }
}

    1. maps — An ordered set of values that a script can loop over, for instance to insert a sequence of values in multiple text fields

    2. objects — A list of the individual test case objects created with Alyvix Designer. Each test case object is identified uniquely in the list by its object name

    3. script — The scripts created for a test case, both the main script and any section (subroutine) scripts.

  1. Maps

    Each named map under the maps field contains a set of named keys (rows) and a fixed number of cell values, to represent a m x n matrix (columns cannot be named):

    { "<map-name-1>": {
     "key1": [ "key1-val1",
               "key1-val2" ],
     "key2": [ "key2-val1",
               "key2-val2" ] }
}

    When arguments are passed to Alyvix Robot in the command line interface, a new map structure will be added under the name cli, and containing the arguments as keys. Because of this, the map name cli is reserved.

  1. Objects

    The value of each property contained within the objects substructure (shown here as “test-case-object-name”) corresponds to a test case object created in Designer. Those values have the following meanings:

    { "<test-case-object-name>": {
     "call": { },
     "components": { },
     "date_modified": "<timestamp>",
     "detection": { },
     "measure": { }
  }
}

    1. call — The recorded options to start or kill an external application when a test case object is first called

    2. components — The representation for the component tree (detailed below)

    3. date_modified — The time the test case object was last modified (also displayed in Selector)

    4. detection — The test case object’s detection conditions such as timeout, break, and appear/disappear

    5. measure — Details of the last execution via Alyvix Editor or Robot (see below), along with the settings for transaction group, output, and thresholds.

  1. Objects > {object} > Components

    The components section represents the component tree of a given test case object. The example below shows how you can have one component tree at each distinct screen resolution.

    { "components": {
     "<resolution>": {
        "groups": [
           { "main": {
                "detection": { },
                "interactions": { },
                "visuals": { } },
             "subs": [ ] },
           { "main": { },
             "subs": [ ] },
           { "main": { },
             "subs": [ ] } ],
        "screen": "<base64>" } }
}

    <resolution> — The resolution of a screen capture as shown in Alyvix Selector. This value is not selected by the user, but is dependent on the hardware and the settings in use when the screen is captured.

    1. groups — A JSON array of exactly 3 items, each of which corresponds to one of the groups in the component tree.

      1. main — The main component in a group.

        1. detection — Information about the match method to use when trying to detect the region of interest.

        2. interactions — Details on any mouse or keyboard actions to initiate in the event of a positive detection.

        3. visuals — Information about the region of interest and the selection box laid over the screen capture.

      2. subs — A JSON array of exactly 4 items, each of which corresponds to one of the subcomponents in a group. The structure of each element is identical to the structure of the main component above.

    2. screen — The Base64 representation of the screen capture. There is exactly one screen capture for each resolution in a test case object.

  1. Objects > {object} > Measure

    The measure structure holds some of the user-selected options visible in Selector for a given test case object, along with the series array, which contains recorded data resulting from each run of the test case object by Alyvix Robot in the test case’s script within a single session.

    When a test case is executed, its .alyvix file is taken as the starting point, and thus the series structure for all of its test case objects are empty. Each time a test case object is called by the test case script, the results and auxiliary data are added to series as a new array element.

    This example shows the structure and some default values:

    { "measure": {
    "group": "<transaction-group-name>",
    "output": "<true/false>",
    "series": [
        {
            "accuracy_ms": "<ms-amount>",
            "annotation": "<base64>",
            "exit": "<true/false/fail/not_executed>",
            "performance_ms": "<ms-amount>",
            "records": {
                "check": "<true/false>",
                "extract": "<extracted-text>",
                "image": "<base64>",
                "text": "<scraped-text>"
            },
            "resolution": {
                "height": "<pixel-amount>",
                "width": "<pixel-amount>"
            },
            "scaling_factor": "<zoom-amount>",
            "screenshot": "<base64>",
            "timestamp": "<epoch>"
        }
    ],
    "thresholds": {
        "critical_s": 5,
        "warning_s": 3 }
}

    1. group — The transaction group assigned to the test case object.

    2. output — Whether the test case object’s “Measure” checkbox is marked.

    3. series — An array containing the results and auxiliary data for each run of the parent test case object. It contains the following information:

      1. accuracy_ms — Given the performance measurement from the same run (see below), this is the interval (+/-) over which that measurement is guaranteed to have occurred.

      2. annotation — The Base64 representation of the screen when the test case object was being run, overlaid with graphic annotations to indicate those components that matched, and potentially the first component that did not match.

      3. exit — The result type of the most recent run. If the test case object did not match, then the type false or fail is determined by whether the break flag was set.

      4. performance_ms — The number of milliseconds that elapsed from when the run of a test case object began until a match was confirmed, subject to the accuracy interval described above; otherwise -1.

      5. records — Temporary variables that are computed during the matching process, such as a text string extracted by a text component.

      6. resolution — The screen resolution at which the test case object was executed.

      7. scaling_factor — The Windows zoom level at the time when the test case object was executed.

      8. screenshot — The Base64 representation of the screen during the test case object run. It’s similar to the main screen capture, but is made during the execution phase, rather than during the building phase.

      9. timestamp — The time (Unix epoch) when this series element was run; otherwise -1.

    4. thresholds — The number of elapsed seconds after the test case object has been started until a warning and/or critical alert should be triggered. If no values are specified, no alerts will be triggered.


  1. Script

    The script parameter records the main script and any subroutines used to compose more complex interaction behaviors.

    { "case": [
     "<test-case-object-name>",
     { "disable": "<test-case-object-name>" },
     { "flow": "<test-case-object-name>",
       "if-true": "<test-case-object-name>" },
     { "flow": "<test-case-object-name>",
       "if-false": "<test-case-object-name>" },
     { "flow": "<test-case-object-name>",
       "for": "<test-case-map-name>" }
  ],
  "sections": {
     "<section-name>": [
        "<test-case-object-name>",
        "<test-case-object-name>"
     ],
     "exit": [],
     "fail": [] }
}

    1. case — The main script that will be executed by Alyvix Robot. The script is an ordered list of scripting elements, where a simple, enabled element is just the name of a test case object. Other elements have the following meanings:

      1. disable — Skip this test case object and continue with the next.

      2. flow — Indicates the test case object or section that is the value of this key should be executed by Alyvix Robot in the context of a conditional ( if-true or if-false) or a loop ( map) .

      3. if-true — Evaluates whether the detection part of a given test case object would match, but without executing its actions.

      4. if-false — As above, but if the test case object would not match.

      5. for — Indicates there should be a loop over the values in the specified Map.

    2. sections — Corresponds to the named subroutine scripts created in Alyvix Editor. Each section is a parameter where its key is the name of the section and its value is the same as the case value described above.

      1. exit — The (teardown) script to execute when a script has completed, regardless of whether it succeeded or failed.

      2. fail — The (teardown) script to execute when a script has failed during the execution of its test case objects.