oblt-playwright

A test framework for assessing the search performance of Elastic Observability solution across various deployment types from the end-user perspective, supporting the last released version.

📦 Getting Started

Prerequisites

Datasets

Test suites (APM, Infrastructure, Logs) require a deployment/project to ingest scenario-specific data, examine required datasets accordingly.

Note: APM scenarios are strictly bound to data generated by apmsoak.

Background tasks

In case there is a need to assess the impact of alerting rules execution on the performance, it is recommended to create alerting rules before running tests using the following command:

npx playwright test alerting_rules.api.spec.ts --project api

The following alerting rules will be established to monitor alert conditions with a one-minute check frequency:

• apm.error_rate
• apm.transaction_error_rate
• apm.transaction_duration
• host.memory.threshold
• host.cpu.threshold
• pod.memory.threshold
• pod.outbound_traffic.threshold

Note: To authorize access to Elasticsearch endpoint, pass your API key into API_KEY environmental variable.

🤖 CI

🔒 Environmental variables

GitHub Actions Secrets:

API_KEY_SERVERLESS = 'ApiKey ...'
API_KEY_STATEFUL = 'ApiKey ...'
ELASTICSEARCH_HOST_SERVERLESS = // No trailing slash
ELASTICSEARCH_HOST_STATEFUL = // No trailing slash
KIBANA_HOST_SERVERLESS = 
KIBANA_HOST_STATEFUL = 
KIBANA_PASSWORD_SERVERLESS = 
KIBANA_PASSWORD_STATEFUL = 
KIBANA_USERNAME_SERVERLESS = 
KIBANA_USERNAME_STATEFUL = 
REPORTING_CLUSTER_ES = // No trailing slash
REPORTING_CLUSTER_API_KEY = 'ApiKey ...'

GitHub Actions Variables:

REPORTING_CLUSTER_INDEX = 
// Predefined variables:
TIME_UNIT_HOURS
TIME_UNIT_DAYS
TIME_VALUE_1
TIME_VALUE_3
TIME_VALUE_7
TIME_VALUE_14
TIME_VALUE_29

🚀 Workflows

Currently configured workflows (see ECH | Serverless ) execute each of the test suites (APM, Infrastructure, Logs) on an hourly basis with 20-minute intervals between them.

• APM test suite executes at the top of each hour.
• Infrastructure test suite executes 20 minutes past the hour.
• Logs test suite executes 40 minutes past the hour.

Tests are executed sequentially, with only one Kibana user at a given time.

📊 Test reports

A custom reporter generates JSON files and transmits them to a remote Elasticsearch reporting cluster (specify Elasticsearch endpoint as REPORTING_CLUSTER_ES and set REPORTING_CLUSTER_API_KEY). Make sure to create an index first and pass its name into REPORTING_CLUSTER_INDEX variable. In a newly created index, match startTime field to Date type.

💻 Local usage

🛠️ Installation

Make sure you have Node.js installed.

1. Clone the repository.

git clone https://github.com/elastic/oblt-playwright.git

2. Install dependencies:

npm install

3. Install Chromium:

npx playwright install chromium

⚙️ Setup

🔒 Environmental variables

Create .env file in the root directory with the following environmental variables:

KIBANA_HOST = 
ELASTICSEARCH_HOST = // No trailing slash
KIBANA_USERNAME = 
KIBANA_PASSWORD = 
TIME_UNIT = 'Minutes'
TIME_VALUE = 15
API_KEY = 'ApiKey ...'

📂 Test report location

Create playwright-report directory in the root.

▶️ Running tests

By default, tests run in headless mode. To launch browsers in headed mode, use the --headed flag. It is recommended to run tests during ongoing data collection.

Run a specific suite

Specify script name and project name (stateful or serverless) in the test command. Example:

npx playwright test apm.serverless.spec.ts --project serverless --headed

Run a specific test

Specify test name and project name (stateful or serverless) in the test command. Example:

npx playwright test -g "APM - Services" --project serverless --headed

📚 Additional Resources

• Guide for creating a new test
• Test suites and required datasets
• Step definitions: APM | Infrastructure | Logs