Checking Status of Test Run

Chapter Prerequisites

To follow the steps described in this chapter, you need to obtain:

  • A token.
  • An identifier of a test run.

    The following values are used as example values throughout the chapter:

  • token_Qwe12345 as a token.
  • tr_1234 as an identifier of a test run.

The processes of creating and executing the test requests begin when the first baseline request is recorded and could last a significant amount of time after the process of baseline requests recording has been stopped. You could check the state of the test run periodically to get some insights into the performing processes.

How to choose the right period of time to perform check of a test run

You can check the state of the test run in the pre-defined period of time (e.g., 15 seconds). Alternatively, you can employ the estimated time of completion for a test run to determine when the next check is to be done. You can obtain this estimate while checking the state of a test run. See details below.

You could perform a single check of the test run state by issuing the following API call:

GET

Do One-Time Check of Test Run State

https://us1.api.wallarm.com/v1/test_run/test_run_id
API call: GET /v1/test_run/test_run_id
Authorization: Required With the token
HTTP header with the token: X-WallarmAPI-Token Serves to pass the token’s value to the API server
Parameters: test_run_id (required) The identifier of the test run whose state to obtain



Example of a request:

curl --request GET \
  --url https://us1.api.wallarm.com/v1/test_run/tr_1234 \
  --header 'Host: us1.api.wallarm.com' \
  --header 'X-WallarmAPI-Token: token_Qwe12345'

Example of a response:

{
  "status": 200,
  "body": {
    "id": tr_1234,
    "name": "demo-testrun",
    "vulns": [
      {
        "id": vuln_0001,
        "threat": 80,
        "code": "S0001",
        "type": "sqli"
      }
    ],
    "clientid": demo_0000,
    "state": "failed",
    "simple_state": "failed",
    "allowed_actions": [],
    "baseline_check_all_terminated_count": 1,
    "baseline_check_fail_count": 1,
    "baseline_check_tech_fail_count": 0,
    "baseline_check_passed_count": 0,
    "baseline_check_running_count": 0,
    "baseline_check_interrupted_count": 0,
    "sended_requests_count": 70,
    ...
    "start_time": 1555572038,
    "end_time": 1555572309,
    ...
    "domains": [
      "app.example.local"
    ],
    "baseline_count": 1,
    ...    
    "baseline_check_waiting_count": 0,
    "planing_requests_count": 70
  }
}

If the request to the API server is successful, you are presented with the server’s response. The response provides a lot of useful information, including:

  • vulns: an array that contains information about the detected vulnerabilities in the target application. Each of the vulnerability records holds the following data regarding the certain vulnerability:

    • id: an identifier of the vulnerability.

    • threat: the number in the range from 1 to 100, that describes the threat level for the vulnerability. The higher the level, the more severe the vulnerability.

    • code: a code assigned to the vulnerability.

    • type: the vulnerability type. The parameter can take one of the values which are described here.

  • state: the test run’s state. The parameter can take one of the following values:

    • running: the test run is running and executing.
    • paused: the test run execution is paused.
    • interrupted: the test run execution is interrupted (e.g. a new test run for the FAST node was created while the current test run was being conducted by this node).
    • passed: the test run execution has completed successfully (no vulnerabilities were found).
    • failed: the test run execution has completed unsuccessfully (some vulnerabilities were found).
  • baseline_check_all_terminated_count: the number of baseline requests for which all of the test request checks are completed.

  • baseline_check_fail_count: the number of baseline requests for which some of the test request checks are failed (in other words, FAST found a vulnerability).

  • baseline_check_tech_fail_count: the number of baseline requests for which some of the test request checks are failed due to the technical issues (e.g. if the target application was unavailable for some period of time).

  • baseline_check_passed_count: the number of baseline requests for which all of the test request checks are passed (in other words, FAST did not find a vulnerability).

  • baseline_check_running_count: the number of baseline requests for which the test request checks are still in progress.

  • baseline_check_interrupted_count: the number of baseline requests for which all of the test request checks were interrupted (e.g. due to interruption of the test run)

  • sended_requests_count: the total number of test requests that were sent to the target application.

  • start_time and end_time: time when the test run started and ended, respectively. The time is specified in the UNIX time format.

  • domains: the list of the target application’s domain names the baseline requests were targeted to.

  • baseline_count: the number of recorded baseline requests.

  • baseline_check_waiting_count: the number of baseline requests that are waiting to be checked;

  • planing_requests_count: the total number of test requests that are queued to be sent to the target application.

It is possible to make a conclusion regarding the presence or absence of vulnerabilities in the application on the basis of the state and vulns parameters’ values.

Example.

A process, that is querying the test run state by making the API call periodically, could terminate with the exit code 0 if there was the state:passed parameter found in the API server’s response and with the exit code 1 if there was the state:failed parameter found in the API server’s response.

The exit code value could be employed by the CI/CD tool in order to calculate the overall CI/CD job’s status.

It is possible to establish even more complex logic of how the FAST-enabled CI/CD job should interact with the CI/CD tool. To do so, use other pieces of data that could be found in the API server’s response.

Estimates of test run's execution speed and time to completion

There is a separate group of parameters in the API server's response, that allows you to estimate a test run's execution speed and time to completion. The group comprises the following parameters:

  • current_rps—the current speed with which FAST sends requests to the target application (in the moment of obtaining the test run's state).

    This value is the average requests per second (RPS). This average RPS is calculated as the number of requests FAST sent to the target application in the 10 second interval before the test run's state was acquired.

    Example: If the test run's state is acquired in 12:03:01 that the current_rps parameter's value is calculated as (the number of requests sent in [12:02:51-12:03:01] time interval)/10.

  • avg_rps—the average speed with which FAST sends requests to the target application (in the moment of obtaining the test run's state).

    This value is the average number of requests per second (RPS) that FAST sent to the target application in the whole test run's execution time:

    • From the start of the test run's execution to the current moment of time if the test run is still executing (which is equal to current time-start_time).
    • From the start of the test run's execution to the end of the test run's execution if the test run's execution is complete (which is equal to end_time-start_time).

    The value of the avg_rps parameter is calculated as (sended_requests_count/(the whole test run's execution time)).

  • estimated_time_to_completion—the amount of time (in seconds) after which test run's execution is likely to be completed (in the moment of obtaining the test run's state).

    The parameter's value is null if:

    • There are no vulnerability checks in progress yet (e.g., there are no baseline requests recorded for the newly created test run so far).
    • Test run is not executing (i.e., it is in any state, excluding "state":"running").

    The value of the estimated_time_to_completion parameter is calculated as (planing_requests_count/current_rps).

The aforementioned parameters' values are null in the first 10 seconds of a test run's execution.

You can employ the estimated_time_to_completion parameter's value to determine when the next test run's state check is to be done. Note that the value may either increase or decrease.

Example:

To check a test run's state in an estimated_time_to_completion period of time, do the following:

  1. After the test run's execution begins, acquire the test run's state several times. For example, you can do it in the 10 seconds interval. Continue to do so until the estimated_time_to_completion parameter's value is not null.

  2. Perform the next checking of the test run's state after the estimated_time_to_completion seconds.

  3. Repeat the previous step until the test run's execution is complete.

The graphical representation of the estimates

You can obtain the estimates' values by using the Wallarm web interface as well. To do so, log in to the Wallarm portal and navigate to the list of test runs which are executing now:

Test run's speed and execution time estimates

When the test run's execution is complete, you are presented with the average requests per second value:

Average requests per second value

You could go back to the list of FAST integration's steps if necessary.

results matching ""

    No results matching ""