diff --git a/docs/advanced-topics.asciidoc b/docs/advanced-topics.asciidoc deleted file mode 100644 index f2aa3c0d3..000000000 --- a/docs/advanced-topics.asciidoc +++ /dev/null @@ -1,13 +0,0 @@ -[[advanced-topics]] -== Advanced Topics - -* <> -* <> -* <> -* <> - -include::./custom-instrumentation.asciidoc[Custom Instrumentation] -include::./sanitizing-data.asciidoc[Sanitizing Data] -include::./how-the-agent-works.asciidoc[How the Agent works] -include::./run-tests-locally.asciidoc[Run Tests Locally] - diff --git a/docs/aiohttp-server.asciidoc b/docs/aiohttp-server.asciidoc deleted file mode 100644 index 357aa79b3..000000000 --- a/docs/aiohttp-server.asciidoc +++ /dev/null @@ -1,124 +0,0 @@ -[[aiohttp-server-support]] -=== Aiohttp Server support - -Getting Elastic APM set up for your Aiohttp Server project is easy, -and there are various ways you can tweak it to fit to your needs. - -[float] -[[aiohttp-server-installation]] -==== Installation - -Install the Elastic APM agent using pip: - -[source,bash] ----- -$ pip install elastic-apm ----- - -or add `elastic-apm` to your project's `requirements.txt` file. - - -[float] -[[aiohttp-server-setup]] -==== Setup - -To set up the agent, you need to initialize it with appropriate settings. - -The settings are configured either via environment variables, -the application's settings, or as initialization arguments. - -You can find a list of all available settings in the <> page. - -To initialize the agent for your application using environment variables: - -[source,python] ----- -from aiohttp import web - -from elasticapm.contrib.aiohttp import ElasticAPM - -app = web.Application() - -apm = ElasticAPM(app) ----- - -To configure the agent using `ELASTIC_APM` in your application's settings: - -[source,python] ----- -from aiohttp import web - -from elasticapm.contrib.aiohttp import ElasticAPM - -app = web.Application() - -app['ELASTIC_APM'] = { - 'SERVICE_NAME': '', - 'SECRET_TOKEN': '', -} -apm = ElasticAPM(app) ----- - -[float] -[[aiohttp-server-usage]] -==== Usage - -Once you have configured the agent, -it will automatically track transactions and capture uncaught exceptions within aiohttp. - -Capture an arbitrary exception by calling <>: - -[source,python] ----- -try: - 1 / 0 -except ZeroDivisionError: - apm.client.capture_exception() ----- - -Log a generic message with <>: - -[source,python] ----- -apm.client.capture_message('hello, world!') ----- - -[float] -[[aiohttp-server-performance-metrics]] -==== Performance metrics - -If you've followed the instructions above, the agent has already installed our middleware. -This will measure response times, as well as detailed performance data for all supported technologies. - -NOTE: due to the fact that `asyncio` drivers are usually separate from their synchronous counterparts, -specific instrumentation is needed for all drivers. -The support for asynchronous drivers is currently quite limited. - -[float] -[[aiohttp-server-ignoring-specific-views]] -===== Ignoring specific routes - -You can use the <> configuration option to ignore specific routes. -The list given should be a list of regular expressions which are matched against the transaction name: - -[source,python] ----- -app['ELASTIC_APM'] = { - # ... - 'TRANSACTIONS_IGNORE_PATTERNS': ['^OPTIONS ', '/api/'] - # ... -} ----- - -This would ignore any requests using the `OPTIONS` method -and any requests containing `/api/`. - - - -[float] -[[supported-aiohttp-and-python-versions]] -==== Supported aiohttp and Python versions - -A list of supported <> and <> versions can be found on our <> page. - -NOTE: Elastic APM only supports `asyncio` when using Python 3.7+ diff --git a/docs/api.asciidoc b/docs/api.asciidoc deleted file mode 100644 index 327ddee71..000000000 --- a/docs/api.asciidoc +++ /dev/null @@ -1,551 +0,0 @@ -[[api]] -== API reference - -The Elastic APM Python agent has several public APIs. -Most of the public API functionality is not needed when using one of our <>, -but they allow customized usage. - -[float] -[[client-api]] -=== Client API - -The public Client API consists of several methods on the `Client` class. -This API can be used to track exceptions and log messages, -as well as to mark the beginning and end of transactions. - -[float] -[[client-api-init]] -==== Instantiation - -[small]#Added in v1.0.0.# - -To create a `Client` instance, import it and call its constructor: - -[source,python] ----- -from elasticapm import Client - -client = Client({'SERVICE_NAME': 'example'}, **defaults) ----- - - * `config`: A dictionary, with key/value configuration. For the possible configuration keys, see <>. - * `**defaults`: default values for configuration. These can be omitted in most cases, and take the least precedence. - -NOTE: framework integrations like <> and <> -instantiate the client automatically. - -[float] -[[api-get-client]] -===== `elasticapm.get_client()` - -[small]#Added in v6.1.0. - -Retrieves the `Client` singleton. This is useful for many framework integrations, -where the client is instantiated automatically. - -[source,python] ----- -client = elasticapm.get_client() -client.capture_message('foo') ----- - -[float] -[[error-api]] -==== Errors - -[float] -[[client-api-capture-exception]] -===== `Client.capture_exception()` - -[small]#Added in v1.0.0. `handled` added in v2.0.0.# - -Captures an exception object: - -[source,python] ----- -try: - x = int("five") -except ValueError: - client.capture_exception() ----- - - * `exc_info`: A `(type, value, traceback)` tuple as returned by https://docs.python.org/3/library/sys.html#sys.exc_info[`sys.exc_info()`]. If not provided, it will be captured automatically. - * `date`: A `datetime.datetime` object representing the occurrence time of the error. If left empty, it defaults to `datetime.datetime.utcnow()`. - * `context`: A dictionary with contextual information. This dictionary must follow the - {apm-guide-ref}/api-error.html[Context] schema definition. - * `custom`: A dictionary of custom data you want to attach to the event. - * `handled`: A boolean to indicate if this exception was handled or not. - -Returns the id of the error as a string. - -[float] -[[client-api-capture-message]] -===== `Client.capture_message()` - -[small]#Added in v1.0.0.# - -Captures a message with optional added contextual data. Example: - -[source,python] ----- -client.capture_message('Billing process succeeded.') ----- - - * `message`: The message as a string. - * `param_message`: Alternatively, a parameterized message as a dictionary. - The dictionary contains two values: `message`, and `params`. - This allows the APM Server to group messages together that share the same - parameterized message. Example: -+ -[source,python] ----- -client.capture_message(param_message={ - 'message': 'Billing process for %s succeeded. Amount: %s', - 'params': (customer.id, order.total_amount), -}) ----- -+ - * `stack`: If set to `True` (the default), a stacktrace from the call site will be captured. - * `exc_info`: A `(type, value, traceback)` tuple as returned by - https://docs.python.org/3/library/sys.html#sys.exc_info[`sys.exc_info()`]. - If not provided, it will be captured automatically, if `capture_message()` was called in an `except` block. - * `date`: A `datetime.datetime` object representing the occurrence time of the error. - If left empty, it defaults to `datetime.datetime.utcnow()`. - * `context`: A dictionary with contextual information. This dictionary must follow the - {apm-guide-ref}/api-error.html[Context] schema definition. - * `custom`: A dictionary of custom data you want to attach to the event. - -Returns the id of the message as a string. - -NOTE: Either the `message` or the `param_message` argument is required. - -[float] -[[transaction-api]] -==== Transactions - -[float] -[[client-api-begin-transaction]] -===== `Client.begin_transaction()` - -[small]#Added in v1.0.0. `trace_parent` support added in v5.6.0.# - -Begin tracking a transaction. -Should be called e.g. at the beginning of a request or when starting a background task. Example: - -[source,python] ----- -client.begin_transaction('processors') ----- - - * `transaction_type`: (*required*) A string describing the type of the transaction, e.g. `'request'` or `'celery'`. - * `trace_parent`: (*optional*) A `TraceParent` object. See <>. - * `links`: (*optional*) A list of `TraceParent` objects to which this transaction is causally linked. - -[float] -[[client-api-end-transaction]] -===== `Client.end_transaction()` - -[small]#Added in v1.0.0.# - -End tracking the transaction. -Should be called e.g. at the end of a request or when ending a background task. Example: - -[source,python] ----- -client.end_transaction('myapp.billing_process', processor.status) ----- - - * `name`: (*optional*) A string describing the name of the transaction, e.g. `process_order`. - This is typically the name of the view/controller that handles the request, or the route name. - * `result`: (*optional*) A string describing the result of the transaction. - This is typically the HTTP status code, or e.g. `'success'` for a background task. - -NOTE: if `name` and `result` are not set in the `end_transaction()` call, -they have to be set beforehand by calling <> and <> during the transaction. - -[float] -[[traceparent-api]] -==== `TraceParent` - -Transactions can be started with a `TraceParent` object. This creates a -transaction that is a child of the `TraceParent`, which is essential for -distributed tracing. - -[float] -[[api-traceparent-from-string]] -===== `elasticapm.trace_parent_from_string()` - -[small]#Added in v5.6.0.# - -Create a `TraceParent` object from the string representation generated by -`TraceParent.to_string()`: - -[source,python] ----- -parent = elasticapm.trace_parent_from_string('00-03d67dcdd62b7c0f7a675424347eee3a-5f0e87be26015733-01') -client.begin_transaction('processors', trace_parent=parent) ----- - - * `traceparent_string`: (*required*) A string representation of a `TraceParent` object. - - -[float] -[[api-traceparent-from-headers]] -===== `elasticapm.trace_parent_from_headers()` - -[small]#Added in v5.6.0.# - -Create a `TraceParent` object from HTTP headers (usually generated by another -Elastic APM agent): - -[source,python] ----- -parent = elasticapm.trace_parent_from_headers(headers_dict) -client.begin_transaction('processors', trace_parent=parent) ----- - - * `headers`: (*required*) HTTP headers formed as a dictionary. - -[float] -[[api-traceparent-get-header]] -===== `elasticapm.get_trace_parent_header()` - -[small]#Added in v5.10.0.# - -Return the string representation of the current transaction `TraceParent` object: - -[source,python] ----- -elasticapm.get_trace_parent_header() ----- - -[float] -[[api-other]] -=== Other APIs - -[float] -[[api-elasticapm-instrument]] -==== `elasticapm.instrument()` - -[small]#Added in v1.0.0.# - -Instruments libraries automatically. -This includes a wide range of standard library and 3rd party modules. -A list of instrumented modules can be found in `elasticapm.instrumentation.register`. -This function should be called as early as possibly in the startup of your application. -For <>, this is called automatically. Example: - -[source,python] ----- -import elasticapm - -elasticapm.instrument() ----- - -[float] -[[api-set-transaction-name]] -==== `elasticapm.set_transaction_name()` - -[small]#Added in v1.0.0.# - -Set the name of the current transaction. -For supported frameworks, the transaction name is determined automatically, -and can be overridden using this function. Example: - -[source,python] ----- -import elasticapm - -elasticapm.set_transaction_name('myapp.billing_process') ----- - - * `name`: (*required*) A string describing name of the transaction - * `override`: if `True` (the default), overrides any previously set transaction name. - If `False`, only sets the name if the transaction name hasn't already been set. - -[float] -[[api-set-transaction-result]] -==== `elasticapm.set_transaction_result()` - -[small]#Added in v2.2.0.# - -Set the result of the current transaction. -For supported frameworks, the transaction result is determined automatically, -and can be overridden using this function. Example: - -[source,python] ----- -import elasticapm - -elasticapm.set_transaction_result('SUCCESS') ----- - - * `result`: (*required*) A string describing the result of the transaction, e.g. `HTTP 2xx` or `SUCCESS` - * `override`: if `True` (the default), overrides any previously set result. - If `False`, only sets the result if the result hasn't already been set. - -[float] -[[api-set-transaction-outcome]] -==== `elasticapm.set_transaction_outcome()` - -[small]#Added in v5.9.0.# - -Sets the outcome of the transaction. The value can either be `"success"`, `"failure"` or `"unknown"`. -This should only be called at the end of a transaction after the outcome is determined. - -The `outcome` is used for error rate calculations. -`success` denotes that a transaction has concluded successful, while `failure` indicates that the transaction failed -to finish successfully. -If the `outcome` is set to `unknown`, the transaction will not be included in error rate calculations. - -For supported web frameworks, the transaction outcome is set automatically if it has not been set yet, based on the -HTTP status code. -A status code below `500` is considered a `success`, while any value of `500` or higher is counted as a `failure`. - -If your transaction results in an HTTP response, you can alternatively provide the HTTP status code. - -NOTE: While the `outcome` and `result` field look very similar, they serve different purposes. - Other than the `result` field, which canhold an arbitrary string value, - `outcome` is limited to three different values, - `"success"`, `"failure"` and `"unknown"`. - This allows the APM app to perform error rate calculations on these values. - -Example: - -[source,python] ----- -import elasticapm - -elasticapm.set_transaction_outcome("success") - -# Using an HTTP status code -elasticapm.set_transaction_outcome(http_status_code=200) - -# Using predefined constants: - -from elasticapm.conf.constants import OUTCOME - -elasticapm.set_transaction_outcome(OUTCOME.SUCCESS) -elasticapm.set_transaction_outcome(OUTCOME.FAILURE) -elasticapm.set_transaction_outcome(OUTCOME.UNKNOWN) ----- - - * `outcome`: One of `"success"`, `"failure"` or `"unknown"`. Can be omitted if `http_status_code` is provided. - * `http_status_code`: if the transaction represents an HTTP response, its status code can be provided to determine the `outcome` automatically. - * `override`: if `True` (the default), any previously set `outcome` will be overriden. - If `False`, the outcome will only be set if it was not set before. - - -[float] -[[api-get-transaction-id]] -==== `elasticapm.get_transaction_id()` - -[small]#Added in v5.2.0.# - -Get the id of the current transaction. Example: - -[source,python] ----- -import elasticapm - -transaction_id = elasticapm.get_transaction_id() ----- - - -[float] -[[api-get-trace-id]] -==== `elasticapm.get_trace_id()` - -[small]#Added in v5.2.0.# - -Get the `trace_id` of the current transaction's trace. Example: - -[source,python] ----- -import elasticapm - -trace_id = elasticapm.get_trace_id() ----- - - -[float] -[[api-get-span-id]] -==== `elasticapm.get_span_id()` - -[small]#Added in v5.2.0.# - -Get the id of the current span. Example: - -[source,python] ----- -import elasticapm - -span_id = elasticapm.get_span_id() ----- - - -[float] -[[api-set-custom-context]] -==== `elasticapm.set_custom_context()` - -[small]#Added in v2.0.0.# - -Attach custom contextual data to the current transaction and errors. -Supported frameworks will automatically attach information about the HTTP request and the logged in user. -You can attach further data using this function. - -TIP: Before using custom context, ensure you understand the different types of -{apm-guide-ref}/data-model-metadata.html[metadata] that are available. - -Example: - -[source,python] ----- -import elasticapm - -elasticapm.set_custom_context({'billing_amount': product.price * item_count}) ----- - - * `data`: (*required*) A dictionary with the data to be attached. This should be a flat key/value `dict` object. - -NOTE: `.`, `*`, and `"` are invalid characters for key names and will be replaced with `_`. - - -Errors that happen after this call will also have the custom context attached to them. -You can call this function multiple times, new context data will be merged with existing data, -following the `update()` semantics of Python dictionaries. - -[float] -[[api-set-user-context]] -==== `elasticapm.set_user_context()` - -[small]#Added in v2.0.0.# - -Attach information about the currently logged in user to the current transaction and errors. -Example: - -[source,python] ----- -import elasticapm - -elasticapm.set_user_context(username=user.username, email=user.email, user_id=user.id) ----- - - * `username`: The username of the logged in user - * `email`: The email of the logged in user - * `user_id`: The unique identifier of the logged in user, e.g. the primary key value - -Errors that happen after this call will also have the user context attached to them. -You can call this function multiple times, new user data will be merged with existing data, -following the `update()` semantics of Python dictionaries. - - -[float] -[[api-capture-span]] -==== `elasticapm.capture_span` - -[small]#Added in v4.1.0.# - -Capture a custom span. -This can be used either as a function decorator or as a context manager (in a `with` statement). -When used as a decorator, the name of the span will be set to the name of the function. -When used as a context manager, a name has to be provided. - -[source,python] ----- -import elasticapm - -@elasticapm.capture_span() -def coffee_maker(strength): - fetch_water() - - with elasticapm.capture_span('near-to-machine', labels={"type": "arabica"}): - insert_filter() - for i in range(strength): - pour_coffee() - - start_drip() - - fresh_pots() ----- - - * `name`: The name of the span. Defaults to the function name if used as a decorator. - * `span_type`: (*optional*) The type of the span, usually in a dot-separated hierarchy of `type`, `subtype`, and `action`, e.g. `db.mysql.query`. Alternatively, type, subtype and action can be provided as three separate arguments, see `span_subtype` and `span_action`. - * `skip_frames`: (*optional*) The number of stack frames to skip when collecting stack traces. Defaults to `0`. - * `leaf`: (*optional*) if `True`, all spans nested bellow this span will be ignored. Defaults to `False`. - * `labels`: (*optional*) a dictionary of labels. Keys must be strings, values can be strings, booleans, or numerical (`int`, `float`, `decimal.Decimal`). Defaults to `None`. - * `span_subtype`: (*optional*) subtype of the span, e.g. name of the database. Defaults to `None`. - * `span_action`: (*optional*) action of the span, e.g. `query`. Defaults to `None`. - * `links`: (*optional*) A list of `TraceParent` objects to which this span is causally linked. - - -[float] -[[api-async-capture-span]] -==== `elasticapm.async_capture_span` - -[small]#Added in v5.4.0.# - -Capture a custom async-aware span. -This can be used either as a function decorator or as a context manager (in an `async with` statement). -When used as a decorator, the name of the span will be set to the name of the function. -When used as a context manager, a name has to be provided. - -[source,python] ----- -import elasticapm - -@elasticapm.async_capture_span() -async def coffee_maker(strength): - await fetch_water() - - async with elasticapm.async_capture_span('near-to-machine', labels={"type": "arabica"}): - await insert_filter() - async for i in range(strength): - await pour_coffee() - - start_drip() - - fresh_pots() ----- - - * `name`: The name of the span. Defaults to the function name if used as a decorator. - * `span_type`: (*optional*) The type of the span, usually in a dot-separated hierarchy of `type`, `subtype`, and `action`, e.g. `db.mysql.query`. Alternatively, type, subtype and action can be provided as three separate arguments, see `span_subtype` and `span_action`. - * `skip_frames`: (*optional*) The number of stack frames to skip when collecting stack traces. Defaults to `0`. - * `leaf`: (*optional*) if `True`, all spans nested bellow this span will be ignored. Defaults to `False`. - * `labels`: (*optional*) a dictionary of labels. Keys must be strings, values can be strings, booleans, or numerical (`int`, `float`, `decimal.Decimal`). Defaults to `None`. - * `span_subtype`: (*optional*) subtype of the span, e.g. name of the database. Defaults to `None`. - * `span_action`: (*optional*) action of the span, e.g. `query`. Defaults to `None`. - * `links`: (*optional*) A list of `TraceParent` objects to which this span is causally linked. - -NOTE: `asyncio` is only supported for Python 3.7+. - -[float] -[[api-label]] -==== `elasticapm.label()` - -[small]#Added in v5.0.0.# - -Attach labels to the the current transaction and errors. - -TIP: Before using custom labels, ensure you understand the different types of -{apm-guide-ref}/data-model-metadata.html[metadata] that are available. - -Example: - -[source,python] ----- -import elasticapm - -elasticapm.label(ecommerce=True, dollar_value=47.12) ----- - -Errors that happen after this call will also have the labels attached to them. -You can call this function multiple times, new labels will be merged with existing labels, -following the `update()` semantics of Python dictionaries. - -Keys must be strings, values can be strings, booleans, or numerical (`int`, `float`, `decimal.Decimal`) -`.`, `*`, and `"` are invalid characters for label names and will be replaced with `_`. - -WARNING: Avoid defining too many user-specified labels. -Defining too many unique fields in an index is a condition that can lead to a -{ref}/mapping.html#mapping-limit-settings[mapping explosion]. diff --git a/docs/asgi-middleware.asciidoc b/docs/asgi-middleware.asciidoc deleted file mode 100644 index 75607d8bc..000000000 --- a/docs/asgi-middleware.asciidoc +++ /dev/null @@ -1,61 +0,0 @@ -[[asgi-middleware]] -=== ASGI Middleware - -experimental::[] - -Incorporating Elastic APM into your ASGI-based project only requires a few easy -steps. - -NOTE: Several ASGI frameworks are supported natively. -Please check <> for more information - -[float] -[[asgi-installation]] -==== Installation - -Install the Elastic APM agent using pip: - -[source,bash] ----- -$ pip install elastic-apm ----- - -or add `elastic-apm` to your project's `requirements.txt` file. - - -[float] -[[asgi-setup]] -==== Setup - -To set up the agent, you need to initialize it with appropriate settings. - -The settings are configured either via environment variables, or as -initialization arguments. - -You can find a list of all available settings in the -<> page. - -To set up the APM agent, wrap your ASGI app with the `ASGITracingMiddleware`: - -[source,python] ----- -from elasticapm.contrib.asgi import ASGITracingMiddleware - -app = MyGenericASGIApp() # depending on framework - -app = ASGITracingMiddleware(app) - ----- - -Make sure to call <> with an appropriate transaction name in all your routes. - -NOTE: Currently, the agent doesn't support automatic capturing of exceptions. -You can follow progress on this issue on https://github.com/elastic/apm-agent-python/issues/1548[Github]. - -[float] -[[supported-python-versions]] -==== Supported Python versions - -A list of supported <> versions can be found on our <> page. - -NOTE: Elastic APM only supports `asyncio` when using Python 3.7+ diff --git a/docs/configuration.asciidoc b/docs/configuration.asciidoc deleted file mode 100644 index 112d4ca3e..000000000 --- a/docs/configuration.asciidoc +++ /dev/null @@ -1,1387 +0,0 @@ -[[configuration]] -== Configuration - -To adapt the Elastic APM agent to your needs, configure it using environment variables or framework-specific -configuration. - -You can either configure the agent by setting environment variables: -[source,bash] ----- -ELASTIC_APM_SERVICE_NAME=foo python manage.py runserver ----- - -or with inline configuration: - -[source,python] ----- -apm_client = Client(service_name="foo") ----- - -or by using framework specific configuration e.g. in your Django `settings.py` file: - -[source,python] ----- -ELASTIC_APM = { - "SERVICE_NAME": "foo", -} ----- - -The precedence is as follows: - - * <> -(supported options are marked with <>) - * Environment variables - * Inline configuration - * Framework-specific configuration - * Default value - -[float] -[[dynamic-configuration]] -=== Dynamic configuration - -Configuration options marked with the image:./images/dynamic-config.svg[] badge can be changed at runtime -when set from a supported source. - -The Python Agent supports {apm-app-ref}/agent-configuration.html[Central configuration], -which allows you to fine-tune certain configurations from in the APM app. -This feature is enabled in the Agent by default with <>. - -[float] -[[django-configuration]] -=== Django - -To configure Django, add an `ELASTIC_APM` dictionary to your `settings.py`: - -[source,python] ----- -ELASTIC_APM = { - 'SERVICE_NAME': 'my-app', - 'SECRET_TOKEN': 'changeme', -} ----- - -[float] -[[flask-configuration]] -=== Flask - -To configure Flask, add an `ELASTIC_APM` dictionary to your `app.config`: - -[source,python] ----- -app.config['ELASTIC_APM'] = { - 'SERVICE_NAME': 'my-app', - 'SECRET_TOKEN': 'changeme', -} - -apm = ElasticAPM(app) ----- - -[float] -[[core-options]] -=== Core options - -[float] -[[config-service-name]] -==== `service_name` - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_SERVICE_NAME` | `SERVICE_NAME` | `unknown-python-service` | `my-app` -|============ - - -The name of your service. -This is used to keep all the errors and transactions of your service together -and is the primary filter in the Elastic APM user interface. - -While a default is provided, it is essential that you override this default -with something more descriptive and unique across your infrastructure. - -NOTE: The service name must conform to this regular expression: `^[a-zA-Z0-9 _-]+$`. -In other words, the service name must only contain characters from the ASCII -alphabet, numbers, dashes, underscores, and spaces. It cannot be an empty string -or whitespace-only. - -[float] -[[config-server-url]] -==== `server_url` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SERVER_URL` | `SERVER_URL` | `'http://127.0.0.1:8200'` -|============ - -The URL for your APM Server. -The URL must be fully qualified, including protocol (`http` or `https`) and port. -Note: Do not set this if you are using APM in an AWS lambda function. APM Agents are designed to proxy their calls to the APM Server through the lambda extension. Instead, set `ELASTIC_APM_LAMBDA_APM_SERVER`. For more info, see <>. - -[float] -[[config-enabled]] -=== `enabled` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_ENABLED` | `ENABLED` | `true` -|============ - -Enable or disable the agent. -When set to false, the agent will not collect any data or start any background threads. - - -[float] -[[config-recording]] -=== `recording` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_RECORDING` | `RECORDING` | `true` -|============ - -Enable or disable recording of events. -If set to false, then the Python agent does not send any events to the Elastic APM server, -and instrumentation overhead is minimized. The agent will continue to poll the server for configuration changes. - - -[float] -[[logging-options]] -=== Logging Options - -[float] -[[config-log_level]] -==== `log_level` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_LOG_LEVEL` | `LOG_LEVEL` | -|============ - -The `logging.logLevel` at which the `elasticapm` logger will log. The available -options are: - -* `"off"` (sets `logging.logLevel` to 1000) -* `"critical"` -* `"error"` -* `"warning"` -* `"info"` -* `"debug"` -* `"trace"` (sets `logging.log_level` to 5) - -Options are case-insensitive - -Note that this option doesn't do anything with logging handlers. In order -for any logs to be visible, you must either configure a handler -(https://docs.python.org/3/library/logging.html#logging.basicConfig[`logging.basicConfig`] -will do this for you) or set <>. This will also override -any log level your app has set for the `elasticapm` logger. - -[float] -[[config-log_file]] -==== `log_file` - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_LOG_FILE` | `LOG_FILE` | `""` | `"/var/log/elasticapm/log.txt"` -|============ - -This enables the agent to log to a file. This is disabled by default. The agent will log -at the `logging.logLevel` configured with <>. Use -<> to configure the maximum size of the log file. This log -file will automatically rotate. - -Note that setting <> is required for this setting to do -anything. - -If https://github.com/elastic/ecs-logging-python[`ecs_logging`] is installed, -the logs will automatically be formatted as ecs-compatible json. - -[float] -[[config-log_file_size]] -==== `log_file_size` - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_LOG_FILE_SIZE` | `LOG_FILE_SIZE` | `"50mb"` | `"100mb"` -|============ - -The size of the log file if <> is set. - -The agent always keeps one backup file when rotating, so the maximum space that -the log files will consume is twice the value of this setting. - -[float] -[[config-log_ecs_reformatting]] -==== `log_ecs_reformatting` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_LOG_ECS_REFORMATTING` | `LOG_ECS_REFORMATTING` | `"off"` -|============ - -experimental::[] - -Valid options: - -* `"off"` -* `"override"` - -If https://github.com/elastic/ecs-logging-python[`ecs_logging`] is installed, -setting this to `"override"` will cause the agent to automatically attempt to enable -ecs-formatted logging. - -For base `logging` from the standard library, the agent will get the root -logger, find any attached handlers, and for each, set the formatter to -`ecs_logging.StdlibFormatter()`. - -If `structlog` is installed, the agent will override any configured processors -with `ecs_logging.StructlogFormatter()`. - -Note that this is a very blunt instrument that could have unintended side effects. -If problems arise, please apply these formatters manually and leave this setting -as `"off"`. See the -https://www.elastic.co/guide/en/ecs-logging/python/current/installation.html[`ecs_logging` docs] -for more information about using these formatters. - -Also note that this setting does not facilitate shipping logs to Elasticsearch. -We recommend https://www.elastic.co/beats/filebeat[Filebeat] for that purpose. - -[float] -[[other-options]] -=== Other options - -[float] -[[config-transport-class]] -==== `transport_class` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_TRANSPORT_CLASS` | `TRANSPORT_CLASS` | `elasticapm.transport.http.Transport` -|============ - - -The transport class to use when sending events to the APM Server. - -[float] -[[config-service-node-name]] -==== `service_node_name` - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_SERVICE_NODE_NAME` | `SERVICE_NODE_NAME` | `None` | `"redis1"` -|============ - -The name of the given service node. This is optional and if omitted, the APM -Server will fall back on `system.container.id` if available, and -`host.name` if necessary. - -This option allows you to set the node name manually to ensure it is unique and meaningful. - -[float] -[[config-environment]] -==== `environment` - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_ENVIRONMENT` | `ENVIRONMENT` | `None` | `"production"` -|============ - -The name of the environment this service is deployed in, -e.g. "production" or "staging". - -Environments allow you to easily filter data on a global level in the APM app. -It's important to be consistent when naming environments across agents. -See {apm-app-ref}/filters.html#environment-selector[environment selector] in the APM app for more information. - -NOTE: This feature is fully supported in the APM app in Kibana versions >= 7.2. -You must use the query bar to filter for a specific environment in versions prior to 7.2. - -[float] -[[config-cloud-provider]] -==== `cloud_provider` - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_CLOUD_PROVIDER` | `CLOUD_PROVIDER` | `"auto"` | `"aws"` -|============ - -This config value allows you to specify which cloud provider should be assumed -for metadata collection. By default, the agent will attempt to detect the cloud -provider or, if that fails, will use trial and error to collect the metadata. - -Valid options are `"auto"`, `"aws"`, `"gcp"`, and `"azure"`. If this config value is set -to `"none"`, then no cloud metadata will be collected. - -[float] -[[config-secret-token]] -==== `secret_token` - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_SECRET_TOKEN` | `SECRET_TOKEN` | `None` | A random string -|============ - -This string is used to ensure that only your agents can send data to your APM Server. -Both the agents and the APM Server have to be configured with the same secret token. -An example to generate a secure secret token is: - -[source,bash] ----- -python -c "import secrets; print(secrets.token_urlsafe(32))" ----- - -WARNING: Secret tokens only provide any security if your APM Server uses TLS. - -[float] -[[config-api-key]] -==== `api_key` - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_API_KEY` | `API_KEY` | `None` | A base64-encoded string -|============ - -experimental::[] - -// TODO: add link to APM Server API Key docs once the docs are released - -This base64-encoded string is used to ensure that only your agents can send data to your APM Server. -The API key must be created using the {apm-guide-ref}/api-key.html[APM server command-line tool]. - -WARNING: API keys only provide any real security if your APM Server uses TLS. - -[float] -[[config-service-version]] -==== `service_version` -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_SERVICE_VERSION` | `SERVICE_VERSION` | `None` | A string indicating the version of the deployed service -|============ - -A version string for the currently deployed version of the service. -If youre deploys are not versioned, the recommended value for this field is the commit identifier of the deployed revision, e.g. the output of `git rev-parse HEAD`. - -[float] -[[config-framework-name]] -==== `framework_name` -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_FRAMEWORK_NAME` | `FRAMEWORK_NAME` | Depending on framework -|============ - -The name of the used framework. -For Django and Flask, this defaults to `django` and `flask` respectively, -otherwise, the default is `None`. - - -[float] -[[config-framework-version]] -==== `framework_version` -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_FRAMEWORK_VERSION` | `FRAMEWORK_VERSION` | Depending on framework -|============ - -The version number of the used framework. -For Django and Flask, this defaults to the used version of the framework, -otherwise, the default is `None`. - -[float] -[[config-filter-exception-types]] -==== `filter_exception_types` -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_FILTER_EXCEPTION_TYPES` | `FILTER_EXCEPTION_TYPES` | `[]` | `['OperationalError', 'mymodule.SomeoneElsesProblemError']` -| multiple values separated by commas, without spaces ||| -|============ - -A list of exception types to be filtered. -Exceptions of these types will not be sent to the APM Server. - - -[float] -[[config-transaction-ignore-urls]] -==== `transaction_ignore_urls` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_TRANSACTION_IGNORE_URLS` | `TRANSACTION_IGNORE_URLS` | `[]` | `['/api/ping', '/static/*']` -| multiple values separated by commas, without spaces ||| -|============ - -A list of URLs for which the agent should not capture any transaction data. - -Optionally, `*` can be used to match multiple URLs at once. - -[float] -[[config-transactions-ignore-patterns]] -==== `transactions_ignore_patterns` -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_TRANSACTIONS_IGNORE_PATTERNS` | `TRANSACTIONS_IGNORE_PATTERNS` | `[]` | `['^OPTIONS ', 'myviews.Healthcheck']` -| multiple values separated by commas, without spaces ||| -|============ - -A list of regular expressions. -Transactions with a name that matches any of the configured patterns will be ignored and not sent to the APM Server. - -NOTE: as the the name of the transaction can only be determined at the end of the transaction, -the agent might still cause overhead for transactions ignored through this setting. -If agent overhead is a concern, we recommend <> instead. - -[float] -[[config-server-timeout]] -==== `server_timeout` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SERVER_TIMEOUT` | `SERVER_TIMEOUT` | `"5s"` -|============ - -A timeout for requests to the APM Server. -The setting has to be provided in *<>*. -If a request to the APM Server takes longer than the configured timeout, -the request is cancelled and the event (exception or transaction) is discarded. -Set to `None` to disable timeouts. - -WARNING: If timeouts are disabled or set to a high value, -your app could experience memory issues if the APM Server times out. - - -[float] -[[config-hostname]] -==== `hostname` - -[options="header"] -|============ -| Environment | Django/Flask | Default | Example -| `ELASTIC_APM_HOSTNAME` | `HOSTNAME` | `socket.gethostname()` | `app-server01.example.com` -|============ - -The host name to use when sending error and transaction data to the APM Server. - -[float] -[[config-auto-log-stacks]] -==== `auto_log_stacks` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_AUTO_LOG_STACKS` | `AUTO_LOG_STACKS` | `True` -| set to `"true"` / `"false"` ||| -|============ - -If set to `True` (the default), the agent will add a stack trace to each log event, -indicating where the log message has been issued. - -This setting can be overridden on an individual basis by setting the `extra`-key `stack`: - -[source,python] ----- -logger.info('something happened', extra={'stack': False}) ----- - -[float] -[[config-collect-local-variables]] -==== `collect_local_variables` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_COLLECT_LOCAL_VARIABLES` | `COLLECT_LOCAL_VARIABLES` | `errors` -|============ - -Possible values: `errors`, `transactions`, `all`, `off` - -The Elastic APM Python agent can collect local variables for stack frames. -By default, this is only done for errors. - -NOTE: Collecting local variables has a non-trivial overhead. -Collecting local variables for transactions in production environments -can have adverse effects for the performance of your service. - -[float] -[[config-local-var-max-length]] -==== `local_var_max_length` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_LOCAL_VAR_MAX_LENGTH` | `LOCAL_VAR_MAX_LENGTH` | `200` -|============ - -When collecting local variables, they will be converted to strings. -This setting allows you to limit the length of the resulting string. - - -[float] -[[config-local-list-var-max-length]] -==== `local_var_list_max_length` - -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_LOCAL_VAR_LIST_MAX_LENGTH` | `LOCAL_VAR_LIST_MAX_LENGTH` | `10` -|============ - -This setting allows you to limit the length of lists in local variables. - - -[float] -[[config-local-dict-var-max-length]] -==== `local_var_dict_max_length` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_LOCAL_VAR_DICT_MAX_LENGTH` | `LOCAL_VAR_DICT_MAX_LENGTH` | `10` -|============ - -This setting allows you to limit the length of dicts in local variables. - - -[float] -[[config-source-lines-error-app-frames]] -==== `source_lines_error_app_frames` -[float] -[[config-source-lines-error-library-frames]] -==== `source_lines_error_library_frames` -[float] -[[config-source-lines-span-app-frames]] -==== `source_lines_span_app_frames` -[float] -[[config-source-lines-span-library-frames]] -==== `source_lines_span_library_frames` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SOURCE_LINES_ERROR_APP_FRAMES` | `SOURCE_LINES_ERROR_APP_FRAMES` | `5` -| `ELASTIC_APM_SOURCE_LINES_ERROR_LIBRARY_FRAMES` | `SOURCE_LINES_ERROR_LIBRARY_FRAMES` | `5` -| `ELASTIC_APM_SOURCE_LINES_SPAN_APP_FRAMES` | `SOURCE_LINES_SPAN_APP_FRAMES` | `0` -| `ELASTIC_APM_SOURCE_LINES_SPAN_LIBRARY_FRAMES` | `SOURCE_LINES_SPAN_LIBRARY_FRAMES` | `0` -|============ - -By default, the APM agent collects source code snippets for errors. -This setting allows you to modify the number of lines of source code that are being collected. - -We differ between errors and spans, as well as library frames and app frames. - -WARNING: Especially for spans, collecting source code can have a large impact on storage use in your Elasticsearch cluster. - -[float] -[[config-capture-body]] -==== `capture_body` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_CAPTURE_BODY` | `CAPTURE_BODY` | `off` -|============ - -For transactions that are HTTP requests, -the Python agent can optionally capture the request body (e.g. `POST` variables). - -Possible values: `errors`, `transactions`, `all`, `off`. - -If the request has a body and this setting is disabled, the body will be shown as `[REDACTED]`. - -For requests with a content type of `multipart/form-data`, -any uploaded files will be referenced in a special `_files` key. -It contains the name of the field and the name of the uploaded file, if provided. - -WARNING: Request bodies often contain sensitive values like passwords and credit card numbers. -If your service handles data like this, we advise to only enable this feature with care. - -[float] -[[config-capture-headers]] -==== `capture_headers` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_CAPTURE_HEADERS` | `CAPTURE_HEADERS` | `true` -|============ - -For transactions and errors that happen due to HTTP requests, -the Python agent can optionally capture the request and response headers. - -Possible values: `true`, `false` - -WARNING: Request headers often contain sensitive values like session IDs and cookies. -See <> for more information on how to filter out sensitive data. - -[float] -[[config-transaction-max-spans]] -==== `transaction_max_spans` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_TRANSACTION_MAX_SPANS` | `TRANSACTION_MAX_SPANS` | `500` -|============ - -This limits the amount of spans that are recorded per transaction. -This is helpful in cases where a transaction creates a very high amount of spans (e.g. thousands of SQL queries). -Setting an upper limit will prevent edge cases from overloading the agent and the APM Server. - -[float] -[[config-stack-trace-limit]] -==== `stack_trace_limit` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_STACK_TRACE_LIMIT` | `STACK_TRACE_LIMIT` | `50` -|============ - -This limits the number of frames captured for each stack trace. - -Setting the limit to `0` will disable stack trace collection, -while any positive integer value will be used as the maximum number of frames to collect. -To disable the limit and always capture all frames, set the value to `-1`. - - -[float] -[[config-span-stack-trace-min-duration]] -==== `span_stack_trace_min_duration` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SPAN_STACK_TRACE_MIN_DURATION` | `SPAN_STACK_TRACE_MIN_DURATION` | `"5ms"` -|============ - -By default, the APM agent collects a stack trace with every recorded span -that has a duration equal to or longer than this configured threshold. While -stack traces are very helpful to find the exact place in your code from which a -span originates, collecting this stack trace does have some overhead. Tune this -threshold to ensure that you only collect stack traces for spans that -could be problematic. - -To collect traces for all spans, regardless of their length, set the value to `0`. - -To disable stack trace collection for spans completely, set the value to `-1`. - -Except for the special values `-1` and `0`, -this setting should be provided in *<>*. - - -[float] -[[config-span-frames-min-duration]] -==== `span_frames_min_duration` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SPAN_FRAMES_MIN_DURATION` | `SPAN_FRAMES_MIN_DURATION` | `"5ms"` -|============ - -NOTE: This config value is being deprecated. Use -<> instead. - - -[float] -[[config-span-compression-enabled]] -==== `span_compression_enabled` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SPAN_COMPRESSION_ENABLED` | `SPAN_COMPRESSION_ENABLED` | `True` -|============ - -Enable/disable span compression. - -If enabled, the agent will compress very short, repeated spans into a single span, -which is beneficial for storage and processing requirements. -Some information is lost in this process, e.g. exact durations of each compressed span. - -[float] -[[config-span-compression-exact-match-max_duration]] -==== `span_compression_exact_match_max_duration` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION` | `SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION` | `"50ms"` -|============ - -Consecutive spans that are exact match and that are under this threshold will be compressed into a single composite span. -This reduces the collection, processing, and storage overhead, and removes clutter from the UI. -The tradeoff is that the DB statements of all the compressed spans will not be collected. - -Two spans are considered exact matches if the following attributes are identical: - * span name - * span type - * span subtype - * destination resource (e.g. the Database name) - -[float] -[[config-span-compression-same-kind-max-duration]] -==== `span_compression_same_kind_max_duration` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SPAN_COMPRESSION_SAME_KIND_MAX_DURATION` | `SPAN_COMPRESSION_SAME_KIND_MAX_DURATION` | `"0ms"` (disabled) -|============ - -Consecutive spans to the same destination that are under this threshold will be compressed into a single composite span. -This reduces the collection, processing, and storage overhead, and removes clutter from the UI. -The tradeoff is that metadata such as database statements of all the compressed spans will not be collected. - -Two spans are considered to be of the same kind if the following attributes are identical: - * span type - * span subtype - * destination resource (e.g. the Database name) - -[float] -[[config-exit-span-min-duration]] -==== `exit_span_min_duration` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_EXIT_SPAN_MIN_DURATION` | `EXIT_SPAN_MIN_DURATION` | `"0ms"` -|============ - -Exit spans are spans that represent a call to an external service, like a database. -If such calls are very short, they are usually not relevant and can be ignored. - -This feature is disabled by default. - -NOTE: if a span propagates distributed tracing IDs, it will not be ignored, even if it is shorter than the configured threshold. -This is to ensure that no broken traces are recorded. - -[float] -[[config-api-request-size]] -==== `api_request_size` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_API_REQUEST_SIZE` | `API_REQUEST_SIZE` | `"768kb"` -|============ - -The maximum queue length of the request buffer before sending the request to the APM Server. -A lower value will increase the load on your APM Server, -while a higher value can increase the memory pressure of your app. -A higher value also impacts the time until data is indexed and searchable in Elasticsearch. - -This setting is useful to limit memory consumption if you experience a sudden spike of traffic. -It has to be provided in *<>*. - -NOTE: Due to internal buffering of gzip, the actual request size can be a few kilobytes larger than the given limit. -By default, the APM Server limits request payload size to `1 MByte`. - -[float] -[[config-api-request-time]] -==== `api_request_time` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_API_REQUEST_TIME` | `API_REQUEST_TIME` | `"10s"` -|============ - -The maximum queue time of the request buffer before sending the request to the APM Server. -A lower value will increase the load on your APM Server, -while a higher value can increase the memory pressure of your app. -A higher value also impacts the time until data is indexed and searchable in Elasticsearch. - -This setting is useful to limit memory consumption if you experience a sudden spike of traffic. -It has to be provided in *<>*. - -NOTE: The actual time will vary between 90-110% of the given value, -to avoid stampedes of instances that start at the same time. - -[float] -[[config-processors]] -==== `processors` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_PROCESSORS` | `PROCESSORS` | `['elasticapm.processors.sanitize_stacktrace_locals', - 'elasticapm.processors.sanitize_http_request_cookies', - 'elasticapm.processors.sanitize_http_headers', - 'elasticapm.processors.sanitize_http_wsgi_env', - 'elasticapm.processors.sanitize_http_request_body']` -|============ - -A list of processors to process transactions and errors. -For more information, see <>. - -WARNING: We recommend always including the default set of validators if you customize this setting. - -[float] -[[config-sanitize-field-names]] -==== `sanitize_field_names` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SANITIZE_FIELD_NAMES` | `SANITIZE_FIELD_NAMES` | `["password", - "passwd", - "pwd", - "secret", - "*key", - "*token*", - "*session*", - "*credit*", - "*card*", - "*auth*", - "*principal*", - "set-cookie"]` -|============ - -A list of glob-matched field names to match and mask when using processors. -For more information, see <>. - -WARNING: We recommend always including the default set of field name matches -if you customize this setting. - - -[float] -[[config-transaction-sample-rate]] -==== `transaction_sample_rate` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_TRANSACTION_SAMPLE_RATE` | `TRANSACTION_SAMPLE_RATE` | `1.0` -|============ - -By default, the agent samples every transaction (e.g. request to your service). -To reduce overhead and storage requirements, set the sample rate to a value between `0.0` and `1.0`. -We still record overall time and the result for unsampled transactions, but no context information, labels, or spans. - -NOTE: This setting will be automatically rounded to 4 decimals of precision. - -[float] -[[config-include-paths]] -==== `include_paths` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_INCLUDE_PATHS` | `INCLUDE_PATHS` | `[]` -| multiple values separated by commas, without spaces ||| -|============ - -A set of paths, optionally using shell globs -(see https://docs.python.org/3/library/fnmatch.html[`fnmatch`] for a description of the syntax). -These are matched against the absolute filename of every frame, and if a pattern matches, the frame is considered -to be an "in-app frame". - -`include_paths` *takes precedence* over `exclude_paths`. - -[float] -[[config-exclude-paths]] -==== `exclude_paths` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_EXCLUDE_PATHS` | `EXCLUDE_PATHS` | Varies on Python version and implementation -| multiple values separated by commas, without spaces ||| -|============ - -A set of paths, optionally using shell globs -(see https://docs.python.org/3/library/fnmatch.html[`fnmatch`] for a description of the syntax). -These are matched against the absolute filename of every frame, and if a pattern matches, the frame is considered -to be a "library frame". - -`include_paths` *takes precedence* over `exclude_paths`. - -The default value varies based on your Python version and implementation, e.g.: - - * PyPy3: `['\*/lib-python/3/*', '\*/site-packages/*']` - * CPython 2.7: `['\*/lib/python2.7/*', '\*/lib64/python2.7/*']` - -[float] -[[config-debug]] -==== `debug` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_DEBUG` | `DEBUG` | `False` -|============ - -If your app is in debug mode (e.g. in Django with `settings.DEBUG = True` or in Flask with `app.debug = True`), -the agent won't send any data to the APM Server. You can override it by changing this setting to `True`. - - -[float] -[[config-disable-send]] -==== `disable_send` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_DISABLE_SEND` | `DISABLE_SEND` | `False` -|============ - -If set to `True`, the agent won't send any events to the APM Server, independent of any debug state. - - -[float] -[[config-instrument]] -==== `instrument` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_INSTRUMENT` | `INSTRUMENT` | `True` -|============ - -If set to `False`, the agent won't instrument any code. -This disables most of the tracing functionality, but can be useful to debug possible instrumentation issues. - - -[float] -[[config-verify-server-cert]] -==== `verify_server_cert` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_VERIFY_SERVER_CERT` | `VERIFY_SERVER_CERT` | `True` -|============ - -By default, the agent verifies the SSL certificate if an HTTPS connection to the APM Server is used. -Verification can be disabled by changing this setting to `False`. -This setting is ignored when <> is set. - -[float] -[[config-server-cert]] -==== `server_cert` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SERVER_CERT` | `SERVER_CERT` | `None` -|============ - -If you have configured your APM Server with a self-signed TLS certificate, or you -just wish to pin the server certificate, you can specify the path to the PEM-encoded -certificate via the `ELASTIC_APM_SERVER_CERT` configuration. - -NOTE: If this option is set, the agent only verifies that the certificate provided by the APM Server is -identical to the one configured here. Validity of the certificate is not checked. - -[float] -[[config-server-ca-cert-file]] -==== `server_ca_cert_file` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_SERVER_CA_CERT_FILE` | `SERVER_CA_CERT_FILE` | `None` -|============ - -By default, the agent will validate the TLS/SSL certificate of the APM Server using the well-known CAs curated by Mozilla, -and provided by the https://pypi.org/project/certifi/[`certifi`] package. - -You can set this option to the path of a file containing a CA certificate that will be used instead. - -Specifying this option is required when using self-signed certificates, unless server certificate validation is disabled. - -[float] -[[config-use-certifi]] -==== `use_certifi` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_USE_CERTIFI` | `USE_CERTIFI` | `True` -|============ - -By default, the Python Agent uses the https://pypi.org/project/certifi/[`certifi`] certificate store. -To use Python's default mechanism for finding certificates, set this option to `False`. - -[float] -[[config-metrics_interval]] -==== `metrics_interval` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_METRICS_INTERVAL` | `METRICS_INTERVAL` | `30s` -|============ - - -The interval in which the agent collects metrics. A shorter interval increases the granularity of metrics, -but also increases the overhead of the agent, as well as storage requirements. - -It has to be provided in *<>*. - -[float] -[[config-disable_metrics]] -==== `disable_metrics` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_DISABLE_METRICS` | `DISABLE_METRICS` | `None` -|============ - - -A comma-separated list of dotted metrics names that should not be sent to the APM Server. -You can use `*` to match multiple metrics; for example, to disable all CPU-related metrics, -as well as the "total system memory" metric, set `disable_metrics` to: - -.... -"*.cpu.*,system.memory.total" -.... - -NOTE: This setting only disables the *sending* of the given metrics, not collection. - -[float] -[[config-breakdown_metrics]] -==== `breakdown_metrics` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_BREAKDOWN_METRICS` | `BREAKDOWN_METRICS` | `True` -|============ - -Enable or disable the tracking and collection of breakdown metrics. -Setting this to `False` disables the tracking of breakdown metrics, which can reduce the overhead of the agent. - -NOTE: This feature requires APM Server and Kibana >= 7.3. - -[float] -[[config-prometheus_metrics]] -==== `prometheus_metrics` (Beta) - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_PROMETHEUS_METRICS` | `PROMETHEUS_METRICS` | `False` -|============ - -Enable/disable the tracking and collection of metrics from `prometheus_client`. - -See <> for more information. - -NOTE: This feature is currently in beta status. - -[float] -[[config-prometheus_metrics_prefix]] -==== `prometheus_metrics_prefix` (Beta) - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_PROMETHEUS_METRICS_PREFIX` | `PROMETHEUS_METRICS_PREFIX` | `prometheus.metrics.` -|============ - -A prefix to prepend to Prometheus metrics names. - -See <> for more information. - -NOTE: This feature is currently in beta status. - -[float] -[[config-metrics_sets]] -==== `metrics_sets` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_METRICS_SETS` | `METRICS_SETS` | ["elasticapm.metrics.sets.cpu.CPUMetricSet"] -|============ - -List of import paths for the MetricSets that should be used to collect metrics. - -See <> for more information. - -[float] -[[config-central_config]] -==== `central_config` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_CENTRAL_CONFIG` | `CENTRAL_CONFIG` | `True` -|============ - -When enabled, the agent will make periodic requests to the APM Server to fetch updated configuration. - -See <> for more information. - -NOTE: This feature requires APM Server and Kibana >= 7.3. - -[float] -[[config-global_labels]] -==== `global_labels` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_GLOBAL_LABELS` | `GLOBAL_LABELS` | `None` -|============ - -Labels added to all events, with the format `key=value[,key=value[,...]]`. -Any labels set by application via the API will override global labels with the same keys. - -NOTE: This feature requires APM Server >= 7.2. - -[float] -[[config-generic-disable-log-record-factory]] -==== `disable_log_record_factory` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_DISABLE_LOG_RECORD_FACTORY` | `DISABLE_LOG_RECORD_FACTORY` | `False` -|============ - -By default in python 3, the agent installs a <> that -automatically adds tracing fields to your log records. Disable this -behavior by setting this to `True`. - -[float] -[[config-use-elastic-traceparent-header]] -==== `use_elastic_traceparent_header` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_USE_ELASTIC_TRACEPARENT_HEADER` | `USE_ELASTIC_TRACEPARENT_HEADER` | `True` -|============ - -To enable {apm-guide-ref}/apm-distributed-tracing.html[distributed tracing], -the agent sets a number of HTTP headers to outgoing requests made with <>. -These headers (`traceparent` and `tracestate`) are defined in the https://www.w3.org/TR/trace-context-1/[W3C Trace Context] specification. - -Additionally, when this setting is set to `True`, the agent will set `elasticapm-traceparent` for backwards compatibility. - -[float] -[[config-trace-continuation-strategy]] -==== `trace_continuation_strategy` - -<> - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_TRACE_CONTINUATION_STRATEGY` | `TRACE_CONTINUATION_STRATEGY` | `continue` -|============ - -This option allows some control on how the APM agent handles W3C trace-context headers on incoming requests. -By default, the `traceparent` and `tracestate` headers are used per W3C spec for distributed tracing. -However, in certain cases it can be helpful to *not* use the incoming `traceparent` header. -Some example use cases: - -- An Elastic-monitored service is receiving requests with `traceparent` headers from *unmonitored* services. -- An Elastic-monitored service is publicly exposed, and does not want tracing data (trace-ids, sampling decisions) to possibly be spoofed by user requests. - -Valid values are: - -- `'continue'`: The default behavior. An incoming `traceparent` value is used to continue the trace and determine the sampling decision. -- `'restart'`: Always ignores the `traceparent` header of incoming requests. - A new trace-id will be generated and the sampling decision will be made based on <>. - A *span link* will be made to the incoming traceparent. -- `'restart_external'`: If an incoming request includes the `es` vendor flag in `tracestate`, then any 'traceparent' will be considered internal and will be handled as described for `'continue'` above. - Otherwise, any `'traceparent'` is considered external and will be handled as described for `'restart'` above. - -Starting with Elastic Observability 8.2, span links will be visible in trace -views. - -[float] -[[config-use-elastic-excepthook]] -==== `use_elastic_excepthook` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_USE_ELASTIC_EXCEPTHOOK` | `USE_ELASTIC_EXCEPTHOOK` | `False` -|============ - -If set to `True`, the agent will intercept the default `sys.excepthook`, which -allows the agent to collect all uncaught exceptions. - -[float] -[[config-include-process-args]] -==== `include_process_args` - -[options="header"] -|============ -| Environment | Django/Flask | Default -| `ELASTIC_APM_INCLUDE_PROCESS_ARGS` | `INCLUDE_PROCESS_ARGS` | `False` -|============ - -Whether each transaction should have the process arguments attached. Disabled by default to save disk space. - -[float] -[[config-django-specific]] -=== Django-specific configuration - -[float] -[[config-django-transaction-name-from-route]] -==== `django_transaction_name_from_route` - -[options="header"] -|============ -| Environment | Django | Default -| `ELASTIC_APM_DJANGO_TRANSACTION_NAME_FROM_ROUTE` | `DJANGO_TRANSACTION_NAME_FROM_ROUTE` | `False` -|============ - - -By default, we use the function or class name of the view as the transaction name. -Starting with Django 2.2, Django makes the route (e.g. `users//`) available on the `request.resolver_match` object. -If you want to use the route instead of the view name as the transaction name, set this config option to `true`. - -NOTE: in versions previous to Django 2.2, changing this setting will have no effect. - -[float] -[[config-django-autoinsert-middleware]] -==== `django_autoinsert_middleware` - -[options="header"] -|============ -| Environment | Django | Default -| `ELASTIC_APM_DJANGO_AUTOINSERT_MIDDLEWARE` | `DJANGO_AUTOINSERT_MIDDLEWARE` | `True` -|============ - -To trace Django requests, the agent uses a middleware, `elasticapm.contrib.django.middleware.TracingMiddleware`. -By default, this middleware is inserted automatically as the first item in `settings.MIDDLEWARES`. -To disable the automatic insertion of the middleware, change this setting to `False`. - - -[float] -[[config-generic-environment]] -=== Generic Environment variables - -Some environment variables that are not specific to the APM agent can be used to configure the agent. - -[float] -[[config-generic-http-proxy]] -==== `HTTP_PROXY` and `HTTPS_PROXY` - -By using `HTTP_PROXY` and `HTTPS_PROXY`, the agent can be instructed to use a proxy to connect to the APM Server. -If both are set, `HTTPS_PROXY` takes precedence. - -NOTE: The environment variables are case-insensitive. - -[float] -[[config-generic-no-proxy]] -==== `NO_PROXY` - -To instruct the agent to *not* use a proxy, you can use the `NO_PROXY` environment variable. -You can either set it to a comma-separated list of hosts for which no proxy should be used (e.g. `localhost,example.com`) -or use `*` to match any host. - -This is useful if `HTTP_PROXY` / `HTTPS_PROXY` is set for other reasons than agent / APM Server communication. - - -[float] -[[config-ssl-cert-file]] -==== `SSL_CERT_FILE` and `SSL_CERT_DIR` - -To tell the agent to use a different SSL certificate, you can use these environment variables. -See also https://www.openssl.org/docs/manmaster/man7/openssl-env.html#SSL_CERT_DIR-SSL_CERT_FILE[OpenSSL docs]. - -Please note that these variables may apply to other SSL/TLS communication in your service, -not just related to the APM agent. - -NOTE: These environment variables only take effect if <> is set to `False`. - -[float] -[[config-formats]] -=== Configuration formats - -Some options require a unit, either duration or size. -These need to be provided in a specific format. - -[float] -[[config-format-duration]] -==== Duration format - -The _duration_ format is used for options like timeouts. -The unit is provided as a suffix directly after the number–without any separation by whitespace. - -*Example*: `5ms` - -*Supported units* - - * `us` (microseconds) - * `ms` (milliseconds) - * `s` (seconds) - * `m` (minutes) - -[float] -[[config-format-size]] -==== Size format - -The _size_ format is used for options like maximum buffer sizes. -The unit is provided as suffix directly after the number, without and separation by whitespace. - - -*Example*: `10kb` - -*Supported units*: - - * `b` (bytes) - * `kb` (kilobytes) - * `mb` (megabytes) - * `gb` (gigabytes) - -NOTE: We use the power-of-two sizing convention, e.g. `1 kilobyte == 1024 bytes` diff --git a/docs/custom-instrumentation.asciidoc b/docs/custom-instrumentation.asciidoc deleted file mode 100644 index 1db067f72..000000000 --- a/docs/custom-instrumentation.asciidoc +++ /dev/null @@ -1,143 +0,0 @@ -[[instrumenting-custom-code]] -=== Instrumenting custom code - -[float] -[[instrumenting-custom-code-spans]] -==== Creating Additional Spans in a Transaction - -Elastic APM instruments a variety of libraries out of the box, but sometimes you -need to know how long a specific function took or how often it gets -called. - -Assuming you're using one of our <>, you can -apply the `@elasticapm.capture_span()` decorator to achieve exactly that. If -you're not using a supported framework, see -<>. - -`elasticapm.capture_span` can be used either as a decorator or as a context -manager. The following example uses it both ways: - -[source,python] ----- -import elasticapm - -@elasticapm.capture_span() -def coffee_maker(strength): - fetch_water() - - with elasticapm.capture_span('near-to-machine'): - insert_filter() - for i in range(strength): - pour_coffee() - - start_drip() - - fresh_pots() ----- - -Similarly, you can use `elasticapm.async_capture_span` for instrumenting `async` workloads: - -[source,python] ----- -import elasticapm - -@elasticapm.async_capture_span() -async def coffee_maker(strength): - await fetch_water() - - async with elasticapm.async_capture_span('near-to-machine'): - await insert_filter() - async for i in range(strength): - await pour_coffee() - - start_drip() - - fresh_pots() ----- - -NOTE: `asyncio` support is only available in Python 3.7+. - -See <> for more information on `capture_span`. - -[float] -[[instrumenting-custom-code-transactions]] -==== Creating New Transactions - -It's important to note that `elasticapm.capture_span` only works if there is -an existing transaction. If you're not using one of our <>, you need to create a `Client` object and begin and end the -transactions yourself. You can even utilize the agent's -<>! - -To collect the spans generated by the supported libraries, you need -to invoke `elasticapm.instrument()` (just once, at the initialization stage of -your application) and create at least one transaction. It is up to you to -determine what you consider a transaction within your application -- it can -be the whole execution of the script or a part of it. - -The example below will consider the whole execution as a single transaction -with two HTTP request spans in it. The config for `elasticapm.Client` can be -passed in programmatically, and it will also utilize any config environment -variables available to it automatically. - -[source,python] ----- -import requests -import time -import elasticapm - -def main(): - sess = requests.Session() - for url in [ 'https://www.elastic.co', 'https://benchmarks.elastic.co' ]: - resp = sess.get(url) - time.sleep(1) - -if __name__ == '__main__': - client = elasticapm.Client(service_name="foo", server_url="https://example.com:8200") - elasticapm.instrument() # Only call this once, as early as possible. - client.begin_transaction(transaction_type="script") - main() - client.end_transaction(name=__name__, result="success") ----- - -Note that you don't need to do anything to send the data -- the `Client` object -will handle that before the script exits. Additionally, the `Client` object should -be treated as a singleton -- you should only create one instance and store/pass -around that instance for all transaction handling. - -[float] -[[instrumenting-custom-code-distributed-tracing]] -==== Distributed Tracing - -When instrumenting custom code across multiple services, you should propagate -the TraceParent where possible. This allows Elastic APM to bundle the various -transactions into a single distributed trace. The Python Agent will -automatically add TraceParent information to the headers of outgoing HTTP -requests, which can then be used on the receiving end to add that TraceParent -information to new manually-created transactions. - -Additionally, the Python Agent provides utilities for propagating the -TraceParent in string format. - -[source,python] ----- -import elasticapm - -client = elasticapm.Client(service_name="foo", server_url="https://example.com:8200") - -# Retrieve the current TraceParent as a string, requires active transaction -traceparent_string = elasticapm.get_trace_parent_header() - -# Create a TraceParent object from a string and use it for a new transaction -parent = elasticapm.trace_parent_from_string(traceparent_string) -client.begin_transaction(transaction_type="script", trace_parent=parent) -# Do some work -client.end_transaction(name=__name__, result="success") - -# Create a TraceParent object from a dictionary of headers, provided -# automatically by the sending service if it is using an Elastic APM Agent. -parent = elasticapm.trace_parent_from_headers(headers_dict) -client.begin_transaction(transaction_type="script", trace_parent=parent) -# Do some work -client.end_transaction(name=__name__, result="success") ----- diff --git a/docs/django.asciidoc b/docs/django.asciidoc deleted file mode 100644 index 1aa8396f6..000000000 --- a/docs/django.asciidoc +++ /dev/null @@ -1,375 +0,0 @@ -[[django-support]] -=== Django support - -Getting Elastic APM set up for your Django project is easy, and there are various ways you can tweak it to fit to your needs. - -[float] -[[django-installation]] -==== Installation - -Install the Elastic APM agent using pip: - -[source,bash] ----- -$ pip install elastic-apm ----- - -or add it to your project's `requirements.txt` file. - -NOTE: For apm-server 6.2+, make sure you use version 2.0 or higher of `elastic-apm`. - - -NOTE: If you use Django with uwsgi, make sure to -http://uwsgi-docs.readthedocs.org/en/latest/Options.html#enable-threads[enable -threads]. - -[float] -[[django-setup]] -==== Setup - -Set up the Elastic APM agent in Django with these two steps: - -1. Add `elasticapm.contrib.django` to `INSTALLED_APPS` in your settings: - -[source,python] ----- -INSTALLED_APPS = ( - # ... - 'elasticapm.contrib.django', -) ----- - -1. Choose a service name, and set the secret token if needed. - -[source,python] ----- -ELASTIC_APM = { - 'SERVICE_NAME': '', - 'SECRET_TOKEN': '', -} ----- - -or as environment variables: - -[source,shell] ----- -ELASTIC_APM_SERVICE_NAME= -ELASTIC_APM_SECRET_TOKEN= ----- - -You now have basic error logging set up, and everything resulting in a 500 HTTP status code will be reported to the APM Server. - -You can find a list of all available settings in the <> page. - -[NOTE] -==== -The agent only captures and sends data if you have `DEBUG = False` in your settings. -To force the agent to capture data in Django debug mode, set the <> configuration option, e.g.: - -[source,python] ----- -ELASTIC_APM = { - 'SERVICE_NAME': '', - 'DEBUG': True, -} ----- -==== - -[float] -[[django-performance-metrics]] -==== Performance metrics - -In order to collect performance metrics, -the agent automatically inserts a middleware at the top of your middleware list -(`settings.MIDDLEWARE` in current versions of Django, `settings.MIDDLEWARE_CLASSES` in some older versions). -To disable the automatic insertion of the middleware, -see <>. - -NOTE: For automatic insertion to work, -your list of middlewares (`settings.MIDDLEWARE` or `settings.MIDDLEWARE_CLASSES`) must be of type `list` or `tuple`. - -In addition to broad request metrics (what will appear in the APM app as transactions), -the agent also collects fine grained metrics on template rendering, -database queries, HTTP requests, etc. -You can find more information on what we instrument in the <> section. - -[float] -[[django-instrumenting-custom-python-code]] -===== Instrumenting custom Python code - -To gain further insights into the performance of your code, please see -<>. - -[float] -[[django-ignoring-specific-views]] -===== Ignoring specific views - -You can use the `TRANSACTIONS_IGNORE_PATTERNS` configuration option to ignore specific views. -The list given should be a list of regular expressions which are matched against the transaction name as seen in the Elastic APM user interface: - -[source,python] ----- -ELASTIC_APM['TRANSACTIONS_IGNORE_PATTERNS'] = ['^OPTIONS ', 'views.api.v2'] ----- - -This example ignores any requests using the `OPTIONS` method and any requests containing `views.api.v2`. - -[float] -[[django-transaction-name-route]] -===== Using the route as transaction name - -By default, we use the function or class name of the view as the transaction name. -Starting with Django 2.2, Django makes the route (e.g. `users//`) available on the `request.resolver_match` object. -If you want to use the route instead of the view name as the transaction name, you can set the <> config option to `true`. - -[source,python] ----- -ELASTIC_APM['DJANGO_TRANSACTION_NAME_FROM_ROUTE'] = True ----- - -NOTE: in versions previous to Django 2.2, changing this setting will have no effect. - -[float] -[[django-integrating-with-the-rum-agent]] -===== Integrating with the RUM Agent - -To correlate performance measurement in the browser with measurements in your Django app, -you can help the RUM (Real User Monitoring) agent by configuring it with the Trace ID and Span ID of the backend request. -We provide a handy template context processor which adds all the necessary bits into the context of your templates. - -To enable this feature, first add the `rum_tracing` context processor to your `TEMPLATES` setting. -You most likely already have a list of `context_processors`, in which case you can simply append ours to the list. - -[source,python] ----- -TEMPLATES = [ - { - 'BACKEND': 'django.template.backends.django.DjangoTemplates', - 'OPTIONS': { - 'context_processors': [ - # ... - 'elasticapm.contrib.django.context_processors.rum_tracing', - ], - }, - }, -] - ----- - -Then, update the call to initialize the RUM agent (which probably happens in your base template) like this: - -[source,javascript] ----- -elasticApm.init({ - serviceName: "my-frontend-service", - pageLoadTraceId: "{{ apm.trace_id }}", - pageLoadSpanId: "{{ apm.span_id }}", - pageLoadSampled: {{ apm.is_sampled_js }} -}) - ----- - -See the {apm-rum-ref}[JavaScript RUM agent documentation] for more information. - -[float] -[[django-enabling-and-disabling-the-agent]] -==== Enabling and disabling the agent - -The easiest way to disable the agent is to set Django’s `DEBUG` option to `True` in your development configuration. -No errors or metrics will be logged to Elastic APM. - -However, if during debugging you would like to force logging of errors to Elastic APM, then you can set `DEBUG` to `True` inside of the Elastic APM -configuration dictionary, like this: - -[source,python] ----- -ELASTIC_APM = { - # ... - 'DEBUG': True, -} ----- - -[float] -[[django-logging]] -==== Integrating with Python logging - -To easily send Python `logging` messages as "error" objects to Elasticsearch, -we provide a `LoggingHandler` which you can use in your logging setup. -The log messages will be enriched with a stack trace, data from the request, and more. - -NOTE: the intended use case for this handler is to send high priority log messages (e.g. log messages with level `ERROR`) -to Elasticsearch. For normal log shipping, we recommend using {filebeat-ref}[filebeat]. - -If you are new to how the `logging` module works together with Django, read more -https://docs.djangoproject.com/en/2.1/topics/logging/[in the Django documentation]. - -An example of how your `LOGGING` setting could look: - -[source,python] ----- -LOGGING = { - 'version': 1, - 'disable_existing_loggers': True, - 'formatters': { - 'verbose': { - 'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s' - }, - }, - 'handlers': { - 'elasticapm': { - 'level': 'WARNING', - 'class': 'elasticapm.contrib.django.handlers.LoggingHandler', - }, - 'console': { - 'level': 'DEBUG', - 'class': 'logging.StreamHandler', - 'formatter': 'verbose' - } - }, - 'loggers': { - 'django.db.backends': { - 'level': 'ERROR', - 'handlers': ['console'], - 'propagate': False, - }, - 'mysite': { - 'level': 'WARNING', - 'handlers': ['elasticapm'], - 'propagate': False, - }, - # Log errors from the Elastic APM module to the console (recommended) - 'elasticapm.errors': { - 'level': 'ERROR', - 'handlers': ['console'], - 'propagate': False, - }, - }, -} ----- - -With this configuration, logging can be done like this in any module in the `myapp` django app: - -You can now use the logger in any module in the `myapp` Django app, for instance `myapp/views.py`: - -[source,python] ----- -import logging -logger = logging.getLogger('mysite') - -try: - instance = MyModel.objects.get(pk=42) -except MyModel.DoesNotExist: - logger.error( - 'Could not find instance, doing something else', - exc_info=True - ) ----- - -Note that `exc_info=True` adds the exception information to the data that gets sent to Elastic APM. -Without it, only the message is sent. - -[float] -[[django-extra-data]] -===== Extra data - -If you want to send more data than what you get with the agent by default, logging can be done like so: - -[source,python] ----- -import logging -logger = logging.getLogger('mysite') - -try: - instance = MyModel.objects.get(pk=42) -except MyModel.DoesNotExist: - logger.error( - 'There was some crazy error', - exc_info=True, - extra={ - 'datetime': str(datetime.now()), - } - ) ----- - -[float] -[[django-celery-integration]] -==== Celery integration - -For a general guide on how to set up Django with Celery, head over to -Celery's http://celery.readthedocs.org/en/latest/django/first-steps-with-django.html#django-first-steps[Django -documentation]. - -Elastic APM will automatically log errors from your celery tasks, record performance data and keep the trace.id -when the task is launched from an already started Elastic transaction. - -[float] -[[django-logging-http-404-not-found-errors]] -==== Logging "HTTP 404 Not Found" errors - -By default, Elastic APM does not log HTTP 404 errors. If you wish to log -these errors, add -`'elasticapm.contrib.django.middleware.Catch404Middleware'` to -`MIDDLEWARE` in your settings: - -[source,python] ----- -MIDDLEWARE = ( - # ... - 'elasticapm.contrib.django.middleware.Catch404Middleware', - # ... -) ----- - -Note that this middleware respects Django's -https://docs.djangoproject.com/en/1.11/ref/settings/#ignorable-404-urls[`IGNORABLE_404_URLS`] -setting. - -[float] -[[django-disable-agent-during-tests]] -==== Disable the agent during tests - -To prevent the agent from sending any data to the APM Server during tests, set the `ELASTIC_APM_DISABLE_SEND` environment variable to `true`, e.g.: - -[source,python] ----- -ELASTIC_APM_DISABLE_SEND=true python manage.py test ----- - -[float] -[[django-troubleshooting]] -==== Troubleshooting - -Elastic APM comes with a Django command that helps troubleshooting your setup. To check your configuration, run - -[source,bash] ----- -python manage.py elasticapm check ----- - -To send a test exception using the current settings, run - -[source,bash] ----- -python manage.py elasticapm test ----- - -If the command succeeds in sending a test exception, it will print a success message: - -[source,bash] ----- -python manage.py elasticapm test - -Trying to send a test error using these settings: - -SERVICE_NAME: -SECRET_TOKEN: -SERVER: http://127.0.0.1:8200 - -Success! We tracked the error successfully! You should be able to see it in a few seconds. ----- - -[float] -[[supported-django-and-python-versions]] -==== Supported Django and Python versions - -A list of supported <> and <> versions can be found on our <> page. diff --git a/docs/docset.yml b/docs/docset.yml new file mode 100644 index 000000000..2c8aafee1 --- /dev/null +++ b/docs/docset.yml @@ -0,0 +1,494 @@ +project: 'APM Python agent docs' +cross_links: + - apm-agent-rum-js + - apm-aws-lambda + - beats + - docs-content + - ecs + - ecs-logging + - ecs-logging-python + - elasticsearch + - logstash +toc: + - toc: reference + - toc: release-notes +subs: + ref: "https://www.elastic.co/guide/en/elasticsearch/reference/current" + ref-bare: "https://www.elastic.co/guide/en/elasticsearch/reference" + ref-8x: "https://www.elastic.co/guide/en/elasticsearch/reference/8.1" + ref-80: "https://www.elastic.co/guide/en/elasticsearch/reference/8.0" + ref-7x: "https://www.elastic.co/guide/en/elasticsearch/reference/7.17" + ref-70: "https://www.elastic.co/guide/en/elasticsearch/reference/7.0" + ref-60: "https://www.elastic.co/guide/en/elasticsearch/reference/6.0" + ref-64: "https://www.elastic.co/guide/en/elasticsearch/reference/6.4" + xpack-ref: "https://www.elastic.co/guide/en/x-pack/6.2" + logstash-ref: "https://www.elastic.co/guide/en/logstash/current" + kibana-ref: "https://www.elastic.co/guide/en/kibana/current" + kibana-ref-all: "https://www.elastic.co/guide/en/kibana" + beats-ref-root: "https://www.elastic.co/guide/en/beats" + beats-ref: "https://www.elastic.co/guide/en/beats/libbeat/current" + beats-ref-60: "https://www.elastic.co/guide/en/beats/libbeat/6.0" + beats-ref-63: "https://www.elastic.co/guide/en/beats/libbeat/6.3" + beats-devguide: "https://www.elastic.co/guide/en/beats/devguide/current" + auditbeat-ref: "https://www.elastic.co/guide/en/beats/auditbeat/current" + packetbeat-ref: "https://www.elastic.co/guide/en/beats/packetbeat/current" + metricbeat-ref: "https://www.elastic.co/guide/en/beats/metricbeat/current" + filebeat-ref: "https://www.elastic.co/guide/en/beats/filebeat/current" + functionbeat-ref: "https://www.elastic.co/guide/en/beats/functionbeat/current" + winlogbeat-ref: "https://www.elastic.co/guide/en/beats/winlogbeat/current" + heartbeat-ref: "https://www.elastic.co/guide/en/beats/heartbeat/current" + journalbeat-ref: "https://www.elastic.co/guide/en/beats/journalbeat/current" + ingest-guide: "https://www.elastic.co/guide/en/ingest/current" + fleet-guide: "https://www.elastic.co/guide/en/fleet/current" + apm-guide-ref: "https://www.elastic.co/guide/en/apm/guide/current" + apm-guide-7x: "https://www.elastic.co/guide/en/apm/guide/7.17" + apm-app-ref: "https://www.elastic.co/guide/en/kibana/current" + apm-agents-ref: "https://www.elastic.co/guide/en/apm/agent" + apm-android-ref: "https://www.elastic.co/guide/en/apm/agent/android/current" + apm-py-ref: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-py-ref-3x: "https://www.elastic.co/guide/en/apm/agent/python/3.x" + apm-node-ref-index: "https://www.elastic.co/guide/en/apm/agent/nodejs" + apm-node-ref: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-node-ref-1x: "https://www.elastic.co/guide/en/apm/agent/nodejs/1.x" + apm-rum-ref: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-dotnet-ref: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref: "https://www.elastic.co/guide/en/apm/agent/php/current" + apm-ios-ref: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-lambda-ref: "https://www.elastic.co/guide/en/apm/lambda/current" + apm-attacher-ref: "https://www.elastic.co/guide/en/apm/attacher/current" + docker-logging-ref: "https://www.elastic.co/guide/en/beats/loggingplugin/current" + esf-ref: "https://www.elastic.co/guide/en/esf/current" + kinesis-firehose-ref: "https://www.elastic.co/guide/en/kinesis/{{kinesis_version}}" + estc-welcome-current: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome-all: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions" + hadoop-ref: "https://www.elastic.co/guide/en/elasticsearch/hadoop/current" + stack-ref: "https://www.elastic.co/guide/en/elastic-stack/current" + stack-ref-67: "https://www.elastic.co/guide/en/elastic-stack/6.7" + stack-ref-68: "https://www.elastic.co/guide/en/elastic-stack/6.8" + stack-ref-70: "https://www.elastic.co/guide/en/elastic-stack/7.0" + stack-ref-80: "https://www.elastic.co/guide/en/elastic-stack/8.0" + stack-ov: "https://www.elastic.co/guide/en/elastic-stack-overview/current" + stack-gs: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + stack-gs-current: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + javaclient: "https://www.elastic.co/guide/en/elasticsearch/client/java-api/current" + java-api-client: "https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current" + java-rest: "https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current" + jsclient: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + jsclient-current: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + es-ruby-client: "https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current" + es-dotnet-client: "https://www.elastic.co/guide/en/elasticsearch/client/net-api/current" + es-php-client: "https://www.elastic.co/guide/en/elasticsearch/client/php-api/current" + es-python-client: "https://www.elastic.co/guide/en/elasticsearch/client/python-api/current" + defguide: "https://www.elastic.co/guide/en/elasticsearch/guide/2.x" + painless: "https://www.elastic.co/guide/en/elasticsearch/painless/current" + plugins: "https://www.elastic.co/guide/en/elasticsearch/plugins/current" + plugins-8x: "https://www.elastic.co/guide/en/elasticsearch/plugins/8.1" + plugins-7x: "https://www.elastic.co/guide/en/elasticsearch/plugins/7.17" + plugins-6x: "https://www.elastic.co/guide/en/elasticsearch/plugins/6.8" + glossary: "https://www.elastic.co/guide/en/elastic-stack-glossary/current" + upgrade_guide: "https://www.elastic.co/products/upgrade_guide" + blog-ref: "https://www.elastic.co/blog/" + curator-ref: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + curator-ref-current: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + metrics-ref: "https://www.elastic.co/guide/en/metrics/current" + metrics-guide: "https://www.elastic.co/guide/en/metrics/guide/current" + logs-ref: "https://www.elastic.co/guide/en/logs/current" + logs-guide: "https://www.elastic.co/guide/en/logs/guide/current" + uptime-guide: "https://www.elastic.co/guide/en/uptime/current" + observability-guide: "https://www.elastic.co/guide/en/observability/current" + observability-guide-all: "https://www.elastic.co/guide/en/observability" + siem-guide: "https://www.elastic.co/guide/en/siem/guide/current" + security-guide: "https://www.elastic.co/guide/en/security/current" + security-guide-all: "https://www.elastic.co/guide/en/security" + endpoint-guide: "https://www.elastic.co/guide/en/endpoint/current" + sql-odbc: "https://www.elastic.co/guide/en/elasticsearch/sql-odbc/current" + ecs-ref: "https://www.elastic.co/guide/en/ecs/current" + ecs-logging-ref: "https://www.elastic.co/guide/en/ecs-logging/overview/current" + ecs-logging-go-logrus-ref: "https://www.elastic.co/guide/en/ecs-logging/go-logrus/current" + ecs-logging-go-zap-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-go-zerolog-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-java-ref: "https://www.elastic.co/guide/en/ecs-logging/java/current" + ecs-logging-dotnet-ref: "https://www.elastic.co/guide/en/ecs-logging/dotnet/current" + ecs-logging-nodejs-ref: "https://www.elastic.co/guide/en/ecs-logging/nodejs/current" + ecs-logging-php-ref: "https://www.elastic.co/guide/en/ecs-logging/php/current" + ecs-logging-python-ref: "https://www.elastic.co/guide/en/ecs-logging/python/current" + ecs-logging-ruby-ref: "https://www.elastic.co/guide/en/ecs-logging/ruby/current" + ml-docs: "https://www.elastic.co/guide/en/machine-learning/current" + eland-docs: "https://www.elastic.co/guide/en/elasticsearch/client/eland/current" + eql-ref: "https://eql.readthedocs.io/en/latest/query-guide" + extendtrial: "https://www.elastic.co/trialextension" + wikipedia: "https://en.wikipedia.org/wiki" + forum: "https://discuss.elastic.co/" + xpack-forum: "https://discuss.elastic.co/c/50-x-pack" + security-forum: "https://discuss.elastic.co/c/x-pack/shield" + watcher-forum: "https://discuss.elastic.co/c/x-pack/watcher" + monitoring-forum: "https://discuss.elastic.co/c/x-pack/marvel" + graph-forum: "https://discuss.elastic.co/c/x-pack/graph" + apm-forum: "https://discuss.elastic.co/c/apm" + enterprise-search-ref: "https://www.elastic.co/guide/en/enterprise-search/current" + app-search-ref: "https://www.elastic.co/guide/en/app-search/current" + workplace-search-ref: "https://www.elastic.co/guide/en/workplace-search/current" + enterprise-search-node-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/enterprise-search-node/current" + enterprise-search-php-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/php/current" + enterprise-search-python-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/python/current" + enterprise-search-ruby-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/ruby/current" + elastic-maps-service: "https://maps.elastic.co" + integrations-docs: "https://docs.elastic.co/en/integrations" + integrations-devguide: "https://www.elastic.co/guide/en/integrations-developer/current" + time-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#time-units" + byte-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#byte-units" + apm-py-ref-v: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-node-ref-v: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-rum-ref-v: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref-v: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref-v: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref-v: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-ios-ref-v: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-dotnet-ref-v: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref-v: "https://www.elastic.co/guide/en/apm/agent/php/current" + ecloud: "Elastic Cloud" + esf: "Elastic Serverless Forwarder" + ess: "Elasticsearch Service" + ece: "Elastic Cloud Enterprise" + eck: "Elastic Cloud on Kubernetes" + serverless-full: "Elastic Cloud Serverless" + serverless-short: "Serverless" + es-serverless: "Elasticsearch Serverless" + es3: "Elasticsearch Serverless" + obs-serverless: "Elastic Observability Serverless" + sec-serverless: "Elastic Security Serverless" + serverless-docs: "https://docs.elastic.co/serverless" + cloud: "https://www.elastic.co/guide/en/cloud/current" + ess-utm-params: "?page=docs&placement=docs-body" + ess-baymax: "?page=docs&placement=docs-body" + ess-trial: "https://cloud.elastic.co/registration?page=docs&placement=docs-body" + ess-product: "https://www.elastic.co/cloud/elasticsearch-service?page=docs&placement=docs-body" + ess-console: "https://cloud.elastic.co?page=docs&placement=docs-body" + ess-console-name: "Elasticsearch Service Console" + ess-deployments: "https://cloud.elastic.co/deployments?page=docs&placement=docs-body" + ece-ref: "https://www.elastic.co/guide/en/cloud-enterprise/current" + eck-ref: "https://www.elastic.co/guide/en/cloud-on-k8s/current" + ess-leadin: "You can run Elasticsearch on your own hardware or use our hosted Elasticsearch Service that is available on AWS, GCP, and Azure. https://cloud.elastic.co/registration{ess-utm-params}[Try the Elasticsearch Service for free]." + ess-leadin-short: "Our hosted Elasticsearch Service is available on AWS, GCP, and Azure, and you can https://cloud.elastic.co/registration{ess-utm-params}[try it for free]." + ess-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elasticsearch Service\"]" + ece-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud_ece.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elastic Cloud Enterprise\"]" + cloud-only: "This feature is designed for indirect use by https://cloud.elastic.co/registration{ess-utm-params}[Elasticsearch Service], https://www.elastic.co/guide/en/cloud-enterprise/{ece-version-link}[Elastic Cloud Enterprise], and https://www.elastic.co/guide/en/cloud-on-k8s/current[Elastic Cloud on Kubernetes]. Direct use is not supported." + ess-setting-change: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"{ess-trial}\", title=\"Supported on {ess}\"] indicates a change to a supported https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html[user setting] for Elasticsearch Service." + ess-skip-section: "If you use Elasticsearch Service, skip this section. Elasticsearch Service handles these changes for you." + api-cloud: "https://www.elastic.co/docs/api/doc/cloud" + api-ece: "https://www.elastic.co/docs/api/doc/cloud-enterprise" + api-kibana-serverless: "https://www.elastic.co/docs/api/doc/serverless" + es-feature-flag: "This feature is in development and not yet available for use. This documentation is provided for informational purposes only." + es-ref-dir: "'{{elasticsearch-root}}/docs/reference'" + apm-app: "APM app" + uptime-app: "Uptime app" + synthetics-app: "Synthetics app" + logs-app: "Logs app" + metrics-app: "Metrics app" + infrastructure-app: "Infrastructure app" + siem-app: "SIEM app" + security-app: "Elastic Security app" + ml-app: "Machine Learning" + dev-tools-app: "Dev Tools" + ingest-manager-app: "Ingest Manager" + stack-manage-app: "Stack Management" + stack-monitor-app: "Stack Monitoring" + alerts-ui: "Alerts and Actions" + rules-ui: "Rules" + rac-ui: "Rules and Connectors" + connectors-ui: "Connectors" + connectors-feature: "Actions and Connectors" + stack-rules-feature: "Stack Rules" + user-experience: "User Experience" + ems: "Elastic Maps Service" + ems-init: "EMS" + hosted-ems: "Elastic Maps Server" + ipm-app: "Index Pattern Management" + ingest-pipelines: "ingest pipelines" + ingest-pipelines-app: "Ingest Pipelines" + ingest-pipelines-cap: "Ingest pipelines" + ls-pipelines: "Logstash pipelines" + ls-pipelines-app: "Logstash Pipelines" + maint-windows: "maintenance windows" + maint-windows-app: "Maintenance Windows" + maint-windows-cap: "Maintenance windows" + custom-roles-app: "Custom Roles" + data-source: "data view" + data-sources: "data views" + data-source-caps: "Data View" + data-sources-caps: "Data Views" + data-source-cap: "Data view" + data-sources-cap: "Data views" + project-settings: "Project settings" + manage-app: "Management" + index-manage-app: "Index Management" + data-views-app: "Data Views" + rules-app: "Rules" + saved-objects-app: "Saved Objects" + tags-app: "Tags" + api-keys-app: "API keys" + transforms-app: "Transforms" + connectors-app: "Connectors" + files-app: "Files" + reports-app: "Reports" + maps-app: "Maps" + alerts-app: "Alerts" + crawler: "Enterprise Search web crawler" + ents: "Enterprise Search" + app-search-crawler: "App Search web crawler" + agent: "Elastic Agent" + agents: "Elastic Agents" + fleet: "Fleet" + fleet-server: "Fleet Server" + integrations-server: "Integrations Server" + ingest-manager: "Ingest Manager" + ingest-management: "ingest management" + package-manager: "Elastic Package Manager" + integrations: "Integrations" + package-registry: "Elastic Package Registry" + artifact-registry: "Elastic Artifact Registry" + aws: "AWS" + stack: "Elastic Stack" + xpack: "X-Pack" + es: "Elasticsearch" + kib: "Kibana" + esms: "Elastic Stack Monitoring Service" + esms-init: "ESMS" + ls: "Logstash" + beats: "Beats" + auditbeat: "Auditbeat" + filebeat: "Filebeat" + heartbeat: "Heartbeat" + metricbeat: "Metricbeat" + packetbeat: "Packetbeat" + winlogbeat: "Winlogbeat" + functionbeat: "Functionbeat" + journalbeat: "Journalbeat" + es-sql: "Elasticsearch SQL" + esql: "ES|QL" + elastic-agent: "Elastic Agent" + k8s: "Kubernetes" + log-driver-long: "Elastic Logging Plugin for Docker" + security: "X-Pack security" + security-features: "security features" + operator-feature: "operator privileges feature" + es-security-features: "Elasticsearch security features" + stack-security-features: "Elastic Stack security features" + endpoint-sec: "Endpoint Security" + endpoint-cloud-sec: "Endpoint and Cloud Security" + elastic-defend: "Elastic Defend" + elastic-sec: "Elastic Security" + elastic-endpoint: "Elastic Endpoint" + swimlane: "Swimlane" + sn: "ServiceNow" + sn-itsm: "ServiceNow ITSM" + sn-itom: "ServiceNow ITOM" + sn-sir: "ServiceNow SecOps" + jira: "Jira" + ibm-r: "IBM Resilient" + webhook: "Webhook" + webhook-cm: "Webhook - Case Management" + opsgenie: "Opsgenie" + bedrock: "Amazon Bedrock" + gemini: "Google Gemini" + hive: "TheHive" + monitoring: "X-Pack monitoring" + monitor-features: "monitoring features" + stack-monitor-features: "Elastic Stack monitoring features" + watcher: "Watcher" + alert-features: "alerting features" + reporting: "X-Pack reporting" + report-features: "reporting features" + graph: "X-Pack graph" + graph-features: "graph analytics features" + searchprofiler: "Search Profiler" + xpackml: "X-Pack machine learning" + ml: "machine learning" + ml-cap: "Machine learning" + ml-init: "ML" + ml-features: "machine learning features" + stack-ml-features: "Elastic Stack machine learning features" + ccr: "cross-cluster replication" + ccr-cap: "Cross-cluster replication" + ccr-init: "CCR" + ccs: "cross-cluster search" + ccs-cap: "Cross-cluster search" + ccs-init: "CCS" + ilm: "index lifecycle management" + ilm-cap: "Index lifecycle management" + ilm-init: "ILM" + dlm: "data lifecycle management" + dlm-cap: "Data lifecycle management" + dlm-init: "DLM" + search-snap: "searchable snapshot" + search-snaps: "searchable snapshots" + search-snaps-cap: "Searchable snapshots" + slm: "snapshot lifecycle management" + slm-cap: "Snapshot lifecycle management" + slm-init: "SLM" + rollup-features: "data rollup features" + ipm: "index pattern management" + ipm-cap: "Index pattern" + rollup: "rollup" + rollup-cap: "Rollup" + rollups: "rollups" + rollups-cap: "Rollups" + rollup-job: "rollup job" + rollup-jobs: "rollup jobs" + rollup-jobs-cap: "Rollup jobs" + dfeed: "datafeed" + dfeeds: "datafeeds" + dfeed-cap: "Datafeed" + dfeeds-cap: "Datafeeds" + ml-jobs: "machine learning jobs" + ml-jobs-cap: "Machine learning jobs" + anomaly-detect: "anomaly detection" + anomaly-detect-cap: "Anomaly detection" + anomaly-job: "anomaly detection job" + anomaly-jobs: "anomaly detection jobs" + anomaly-jobs-cap: "Anomaly detection jobs" + dataframe: "data frame" + dataframes: "data frames" + dataframe-cap: "Data frame" + dataframes-cap: "Data frames" + watcher-transform: "payload transform" + watcher-transforms: "payload transforms" + watcher-transform-cap: "Payload transform" + watcher-transforms-cap: "Payload transforms" + transform: "transform" + transforms: "transforms" + transform-cap: "Transform" + transforms-cap: "Transforms" + dataframe-transform: "transform" + dataframe-transform-cap: "Transform" + dataframe-transforms: "transforms" + dataframe-transforms-cap: "Transforms" + dfanalytics-cap: "Data frame analytics" + dfanalytics: "data frame analytics" + dataframe-analytics-config: "'{dataframe} analytics config'" + dfanalytics-job: "'{dataframe} analytics job'" + dfanalytics-jobs: "'{dataframe} analytics jobs'" + dfanalytics-jobs-cap: "'{dataframe-cap} analytics jobs'" + cdataframe: "continuous data frame" + cdataframes: "continuous data frames" + cdataframe-cap: "Continuous data frame" + cdataframes-cap: "Continuous data frames" + cdataframe-transform: "continuous transform" + cdataframe-transforms: "continuous transforms" + cdataframe-transforms-cap: "Continuous transforms" + ctransform: "continuous transform" + ctransform-cap: "Continuous transform" + ctransforms: "continuous transforms" + ctransforms-cap: "Continuous transforms" + oldetection: "outlier detection" + oldetection-cap: "Outlier detection" + olscore: "outlier score" + olscores: "outlier scores" + fiscore: "feature influence score" + evaluatedf-api: "evaluate {dataframe} analytics API" + evaluatedf-api-cap: "Evaluate {dataframe} analytics API" + binarysc: "binary soft classification" + binarysc-cap: "Binary soft classification" + regression: "regression" + regression-cap: "Regression" + reganalysis: "regression analysis" + reganalysis-cap: "Regression analysis" + depvar: "dependent variable" + feature-var: "feature variable" + feature-vars: "feature variables" + feature-vars-cap: "Feature variables" + classification: "classification" + classification-cap: "Classification" + classanalysis: "classification analysis" + classanalysis-cap: "Classification analysis" + infer-cap: "Inference" + infer: "inference" + lang-ident-cap: "Language identification" + lang-ident: "language identification" + data-viz: "Data Visualizer" + file-data-viz: "File Data Visualizer" + feat-imp: "feature importance" + feat-imp-cap: "Feature importance" + nlp: "natural language processing" + nlp-cap: "Natural language processing" + apm-agent: "APM agent" + apm-go-agent: "Elastic APM Go agent" + apm-go-agents: "Elastic APM Go agents" + apm-ios-agent: "Elastic APM iOS agent" + apm-ios-agents: "Elastic APM iOS agents" + apm-java-agent: "Elastic APM Java agent" + apm-java-agents: "Elastic APM Java agents" + apm-dotnet-agent: "Elastic APM .NET agent" + apm-dotnet-agents: "Elastic APM .NET agents" + apm-node-agent: "Elastic APM Node.js agent" + apm-node-agents: "Elastic APM Node.js agents" + apm-php-agent: "Elastic APM PHP agent" + apm-php-agents: "Elastic APM PHP agents" + apm-py-agent: "Elastic APM Python agent" + apm-py-agents: "Elastic APM Python agents" + apm-ruby-agent: "Elastic APM Ruby agent" + apm-ruby-agents: "Elastic APM Ruby agents" + apm-rum-agent: "Elastic APM Real User Monitoring (RUM) JavaScript agent" + apm-rum-agents: "Elastic APM RUM JavaScript agents" + apm-lambda-ext: "Elastic APM AWS Lambda extension" + project-monitors: "project monitors" + project-monitors-cap: "Project monitors" + private-location: "Private Location" + private-locations: "Private Locations" + pwd: "YOUR_PASSWORD" + esh: "ES-Hadoop" + default-dist: "default distribution" + oss-dist: "OSS-only distribution" + observability: "Observability" + api-request-title: "Request" + api-prereq-title: "Prerequisites" + api-description-title: "Description" + api-path-parms-title: "Path parameters" + api-query-parms-title: "Query parameters" + api-request-body-title: "Request body" + api-response-codes-title: "Response codes" + api-response-body-title: "Response body" + api-example-title: "Example" + api-examples-title: "Examples" + api-definitions-title: "Properties" + multi-arg: "†footnoteref:[multi-arg,This parameter accepts multiple arguments.]" + multi-arg-ref: "†footnoteref:[multi-arg]" + yes-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png[Yes,20,15]" + no-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png[No,20,15]" + es-repo: "https://github.com/elastic/elasticsearch/" + es-issue: "https://github.com/elastic/elasticsearch/issues/" + es-pull: "https://github.com/elastic/elasticsearch/pull/" + es-commit: "https://github.com/elastic/elasticsearch/commit/" + kib-repo: "https://github.com/elastic/kibana/" + kib-issue: "https://github.com/elastic/kibana/issues/" + kibana-issue: "'{kib-repo}issues/'" + kib-pull: "https://github.com/elastic/kibana/pull/" + kibana-pull: "'{kib-repo}pull/'" + kib-commit: "https://github.com/elastic/kibana/commit/" + ml-repo: "https://github.com/elastic/ml-cpp/" + ml-issue: "https://github.com/elastic/ml-cpp/issues/" + ml-pull: "https://github.com/elastic/ml-cpp/pull/" + ml-commit: "https://github.com/elastic/ml-cpp/commit/" + apm-repo: "https://github.com/elastic/apm-server/" + apm-issue: "https://github.com/elastic/apm-server/issues/" + apm-pull: "https://github.com/elastic/apm-server/pull/" + kibana-blob: "https://github.com/elastic/kibana/blob/current/" + apm-get-started-ref: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-server-ref: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-v: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-m: "https://www.elastic.co/guide/en/apm/server/master" + apm-server-ref-62: "https://www.elastic.co/guide/en/apm/server/6.2" + apm-server-ref-64: "https://www.elastic.co/guide/en/apm/server/6.4" + apm-server-ref-70: "https://www.elastic.co/guide/en/apm/server/7.0" + apm-overview-ref-v: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-overview-ref-70: "https://www.elastic.co/guide/en/apm/get-started/7.0" + apm-overview-ref-m: "https://www.elastic.co/guide/en/apm/get-started/master" + infra-guide: "https://www.elastic.co/guide/en/infrastructure/guide/current" + a-data-source: "a data view" + icon-bug: "pass:[]" + icon-checkInCircleFilled: "pass:[]" + icon-warningFilled: "pass:[]" diff --git a/docs/flask.asciidoc b/docs/flask.asciidoc deleted file mode 100644 index b99ddd198..000000000 --- a/docs/flask.asciidoc +++ /dev/null @@ -1,245 +0,0 @@ -[[flask-support]] -=== Flask support - -Getting Elastic APM set up for your Flask project is easy, -and there are various ways you can tweak it to fit to your needs. - -[float] -[[flask-installation]] -==== Installation - -Install the Elastic APM agent using pip: - -[source,bash] ----- -$ pip install "elastic-apm[flask]" ----- - -or add `elastic-apm[flask]` to your project's `requirements.txt` file. - -NOTE: For apm-server 6.2+, make sure you use version 2.0 or higher of `elastic-apm`. - -NOTE: If you use Flask with uwsgi, make sure to -http://uwsgi-docs.readthedocs.org/en/latest/Options.html#enable-threads[enable -threads]. - -NOTE: If you see an error log that mentions `psutil not found`, you can install -`psutil` using `pip install psutil`, or add `psutil` to your `requirements.txt` -file. - -[float] -[[flask-setup]] -==== Setup - -To set up the agent, you need to initialize it with appropriate settings. - -The settings are configured either via environment variables, -the application's settings, or as initialization arguments. - -You can find a list of all available settings in the <> page. - -To initialize the agent for your application using environment variables: - -[source,python] ----- -from elasticapm.contrib.flask import ElasticAPM - -app = Flask(__name__) - -apm = ElasticAPM(app) ----- - -To configure the agent using `ELASTIC_APM` in your application's settings: - -[source,python] ----- -from elasticapm.contrib.flask import ElasticAPM - -app.config['ELASTIC_APM'] = { - 'SERVICE_NAME': '', - 'SECRET_TOKEN': '', -} -apm = ElasticAPM(app) ----- - -The final option is to initialize the agent with the settings as arguments: - -[source,python] ----- -from elasticapm.contrib.flask import ElasticAPM - -apm = ElasticAPM(app, service_name='', secret_token='') ----- - -[float] -[[flask-debug-mode]] -===== Debug mode - -NOTE: Please note that errors and transactions will only be sent to the APM Server if your app is *not* in -https://flask.palletsprojects.com/en/3.0.x/quickstart/#debug-mode[Flask debug mode]. - -To force the agent to send data while the app is in debug mode, -set the value of `DEBUG` in the `ELASTIC_APM` dictionary to `True`: - -[source,python] ----- -app.config['ELASTIC_APM'] = { - 'SERVICE_NAME': '', - 'SECRET_TOKEN': '', - 'DEBUG': True -} ----- - -[float] -[[flask-building-applications-on-the-fly]] -===== Building applications on the fly? - -You can use the agent's `init_app` hook for adding the application on the fly: - -[source,python] ----- -from elasticapm.contrib.flask import ElasticAPM -apm = ElasticAPM() - -def create_app(): - app = Flask(__name__) - apm.init_app(app, service_name='', secret_token='') - return app ----- - -[float] -[[flask-usage]] -==== Usage - -Once you have configured the agent, -it will automatically track transactions and capture uncaught exceptions within Flask. -If you want to send additional events, -a couple of shortcuts are provided on the ElasticAPM Flask middleware object -by raising an exception or logging a generic message. - -Capture an arbitrary exception by calling `capture_exception`: - -[source,python] ----- -try: - 1 / 0 -except ZeroDivisionError: - apm.capture_exception() ----- - -Log a generic message with `capture_message`: - -[source,python] ----- -apm.capture_message('hello, world!') ----- - -[float] -[[flask-logging]] -==== Shipping Logs to Elasticsearch - -This feature has been deprecated and will be removed in a future version. - -Please see our <> documentation for other supported ways to ship -logs to Elasticsearch. - -Note that you can always send exceptions and messages to the APM Server with -<> and and -<>. - -[source,python] ----- -from elasticapm import get_client - -@app.route('/') -def bar(): - try: - 1 / 0 - except ZeroDivisionError: - get_client().capture_exception() ----- - -[float] -[[flask-extra-data]] -===== Extra data - -In addition to what the agents log by default, you can send extra information: - -[source,python] ----- -@app.route('/') -def bar(): - try: - 1 / 0 - except ZeroDivisionError: - app.logger.error('Math is hard', - exc_info=True, - extra={ - 'good_at_math': False, - } - ) - ) ----- - -[float] -[[flask-celery-tasks]] -===== Celery tasks - -The Elastic APM agent will automatically send errors and performance data from your Celery tasks to the APM Server. - -[float] -[[flask-performance-metrics]] -==== Performance metrics - -If you've followed the instructions above, the agent has already hooked -into the right signals and should be reporting performance metrics. - -[float] -[[flask-ignoring-specific-views]] -===== Ignoring specific routes - -You can use the <> configuration option to ignore specific routes. -The list given should be a list of regular expressions which are matched against the transaction name: - -[source,python] ----- -app.config['ELASTIC_APM'] = { - ... - 'TRANSACTIONS_IGNORE_PATTERNS': ['^OPTIONS ', '/api/'] - ... -} ----- - -This would ignore any requests using the `OPTIONS` method -and any requests containing `/api/`. - - -[float] -[[flask-integrating-with-the-rum-agent]] -===== Integrating with the RUM Agent - -To correlate performance measurement in the browser with measurements in your Flask app, -you can help the RUM (Real User Monitoring) agent by configuring it with the Trace ID and Span ID of the backend request. -We provide a handy template context processor which adds all the necessary bits into the context of your templates. - -The context processor is installed automatically when you initialize `ElasticAPM`. -All that is left to do is to update the call to initialize the RUM agent (which probably happens in your base template) like this: - -[source,javascript] ----- -elasticApm.init({ - serviceName: "my-frontend-service", - pageLoadTraceId: "{{ apm["trace_id"] }}", - pageLoadSpanId: "{{ apm["span_id"]() }}", - pageLoadSampled: {{ apm["is_sampled_js"] }} -}) - ----- - -See the {apm-rum-ref}[JavaScript RUM agent documentation] for more information. - -[float] -[[supported-flask-and-python-versions]] -==== Supported Flask and Python versions - -A list of supported <> and <> versions can be found on our <> page. diff --git a/docs/getting-started.asciidoc b/docs/getting-started.asciidoc deleted file mode 100644 index ec8a88bf8..000000000 --- a/docs/getting-started.asciidoc +++ /dev/null @@ -1,32 +0,0 @@ -[[getting-started]] -== Introduction - -The Elastic APM Python agent sends performance metrics and error logs to the APM Server. -It has built-in support for Django and Flask performance metrics and error logging, as well as generic support of other WSGI frameworks for error logging. - -[float] -[[how-it-works]] -=== How does the Agent work? - -The Python Agent instruments your application to collect APM events in a few different ways: - -To collect data about incoming requests and background tasks, the Agent integrates with <> to make use of hooks and signals provided by the framework. -These framework integrations require limited code changes in your application. - -To collect data from database drivers, HTTP libraries etc., -we instrument certain functions and methods in these libraries. -Instrumentations are set up automatically and do not require any code changes. - -In addition to APM and error data, -the Python agent also collects system and application metrics in regular intervals. -This collection happens in a background thread that is started by the agent. - -More detailed information on how the Agent works can be found in the <>. - -[float] -[[additional-components]] -=== Additional components - -APM Agents work in conjunction with the {apm-guide-ref}/index.html[APM Server], {ref}/index.html[Elasticsearch], and {kibana-ref}/index.html[Kibana]. -The {apm-guide-ref}/index.html[APM Guide] provides details on how these components work together, -and provides a matrix outlining {apm-guide-ref}/agent-server-compatibility.html[Agent and Server compatibility]. diff --git a/docs/grpc.asciidoc b/docs/grpc.asciidoc deleted file mode 100644 index 4b79e15f0..000000000 --- a/docs/grpc.asciidoc +++ /dev/null @@ -1,65 +0,0 @@ -[[grpc-support]] -=== GRPC Support - -Incorporating Elastic APM into your GRPC project only requires a few easy -steps. - -NOTE: currently, only unary-unary RPC calls are instrumented. Streaming requests or responses are not captured. - -[float] -[[grpc-installation]] -==== Installation - -Install the Elastic APM agent using pip: - -[source,bash] ----- -$ pip install elastic-apm ----- - -or add `elastic-apm` to your project's `requirements.txt` file. - - -[float] -[[grpc-setup]] -==== Setup - -Elastic APM can be used both in GRPC server apps, and in GRPC client apps. - -[float] -[[grpc-setup-client]] -===== GRPC Client - -If you use one of our <>, no further steps are needed. - -For other use cases, see <>. -To ensure that our instrumentation is in place, call `elasticapm.instrument()` *before* creating any GRPC channels. - -[float] -[[grpc-setup-server]] -===== GRPC Server - -To set up the agent, you need to initialize it with appropriate settings. - -The settings are configured either via environment variables, or as -initialization arguments. - -You can find a list of all available settings in the -<> page. - -To initialize the agent for your application using environment variables: - -[source,python] ----- -import elasticapm -from elasticapm.contrib.grpc import GRPCApmClient - -elasticapm.instrument() - -client = GRPCApmClient(service_name="my-grpc-server") ----- - - -Once you have configured the agent, it will automatically track transactions -and capture uncaught exceptions within GRPC requests. - diff --git a/docs/how-the-agent-works.asciidoc b/docs/how-the-agent-works.asciidoc deleted file mode 100644 index 796815144..000000000 --- a/docs/how-the-agent-works.asciidoc +++ /dev/null @@ -1,72 +0,0 @@ -[[how-the-agent-works]] -=== How the Agent works - -To gather APM events (called transactions and spans), errors and metrics, -the Python agent instruments your application in a few different ways. -These events, are then sent to the APM Server. -The APM Server converts them to a format suitable for Elasticsearch, and sends them to an Elasticsearch cluster. -You can then use the APM app in Kibana to gain insight into latency issues and error culprits within your application. - -Broadly, we differentiate between three different approaches to collect the necessary data: -framework integration, instrumentation, and background collection. - -[float] -[[how-it-works-framework-integration]] -==== Framework integration - -To collect data about incoming requests and background tasks, -we integrate with frameworks like <>, <> and Celery. -Whenever possible, framework integrations make use of hooks and signals provided by the framework. -Examples of this are: - - * `request_started`, `request_finished`, and `got_request_exception` signals from `django.core.signals` - * `request_started`, `request_finished`, and `got_request_exception` signals from `flask.signals` - * `task_prerun`, `task_postrun`, and `task_failure` signals from `celery.signals` - -Framework integrations require some limited code changes in your app. -E.g. for Django, you need to add `elasticapm.contrib.django` to `INSTALLED_APPS`. - -[float] -[[how-it-works-no-framework]] -==== What if you are not using a framework - -If you're not using a supported framework, for example, a simple Python script, you can still -leverage the agent's <>. Check out -our docs on <>. - -[float] -[[how-it-works-instrumentation]] -==== Instrumentation - -To collect data from database drivers, HTTP libraries etc., -we instrument certain functions and methods in these libraries. -Our instrumentation wraps these callables and collects additional data, like - - * time spent in the call - * the executed query for database drivers - * the fetched URL for HTTP libraries - -We use a 3rd party library, https://github.com/GrahamDumpleton/wrapt[`wrapt`], to wrap the callables. -You can read more on how `wrapt` works in Graham Dumpleton's -excellent series of http://blog.dscpl.com.au/search/label/wrapt[blog posts]. - -Instrumentations are set up automatically and do not require any code changes. -See <> to learn more about which libraries we support. - -[float] -[[how-it-works-background-collection]] -==== Background collection - -In addition to APM and error data, -the Python agent also collects system and application metrics in regular intervals. -This collection happens in a background thread that is started by the agent. - -In addition to the metrics collection background thread, -the agent starts two additional threads per process: - - * a thread to regularly fetch remote configuration from the APM Server - * a thread to process the collected data and send it to the APM Server via HTTP. - -Note that every process that instantiates the agent will have these three threads. -This means that when you e.g. use gunicorn or uwsgi workers, -each worker will have three threads started by the Python agent. diff --git a/docs/images/choose-a-layer.png b/docs/images/choose-a-layer.png new file mode 100644 index 000000000..49cfd9917 Binary files /dev/null and b/docs/images/choose-a-layer.png differ diff --git a/docs/images/config-layer.png b/docs/images/config-layer.png new file mode 100644 index 000000000..ec6c045d3 Binary files /dev/null and b/docs/images/config-layer.png differ diff --git a/docs/images/python-lambda-env-vars.png b/docs/images/python-lambda-env-vars.png new file mode 100644 index 000000000..cbf4a25d2 Binary files /dev/null and b/docs/images/python-lambda-env-vars.png differ diff --git a/docs/index.asciidoc b/docs/index.asciidoc deleted file mode 100644 index 544415367..000000000 --- a/docs/index.asciidoc +++ /dev/null @@ -1,40 +0,0 @@ -include::{asciidoc-dir}/../../shared/versions/stack/current.asciidoc[] -include::{asciidoc-dir}/../../shared/attributes.asciidoc[] - -ifdef::env-github[] -NOTE: For the best reading experience, -please view this documentation at https://www.elastic.co/guide/en/apm/agent/python/current/index.html[elastic.co] -endif::[] - -= APM Python Agent Reference - -NOTE: Python 2.7 reached End of Life on January 1, 2020. -The Elastic APM agent will stop supporting Python 2.7 starting in version 6.0.0. - -include::./getting-started.asciidoc[] - -include::./set-up.asciidoc[] - -include::./supported-technologies.asciidoc[] - -include::./configuration.asciidoc[] - -include::./advanced-topics.asciidoc[] - -include::./api.asciidoc[] - -include::./metrics.asciidoc[] - -include::./opentelemetry.asciidoc[] - -include::./logging.asciidoc[] - -include::./tuning.asciidoc[] - -include::./troubleshooting.asciidoc[] - -include::./upgrading.asciidoc[] - -include::./release-notes.asciidoc[] - -include::./redirects.asciidoc[] diff --git a/docs/lambda/configure-lambda-widget.asciidoc b/docs/lambda/configure-lambda-widget.asciidoc deleted file mode 100644 index 9763f49f8..000000000 --- a/docs/lambda/configure-lambda-widget.asciidoc +++ /dev/null @@ -1,118 +0,0 @@ -++++ -
-
- - - - - - -
-
-++++ - -include::configure-lambda.asciidoc[tag=console-{layer-section-type}] - -++++ -
- - - - - -
-++++ \ No newline at end of file diff --git a/docs/lambda/configure-lambda.asciidoc b/docs/lambda/configure-lambda.asciidoc deleted file mode 100644 index 09377dcee..000000000 --- a/docs/lambda/configure-lambda.asciidoc +++ /dev/null @@ -1,113 +0,0 @@ -// tag::console-with-agent[] - -To configure APM through the AWS Management Console: - -1. Navigate to your function in the AWS Management Console -2. Click on the _Configuration_ tab -3. Click on _Environment variables_ -4. Add the following required variables: - -[source,bash] ----- -AWS_LAMBDA_EXEC_WRAPPER = /opt/python/bin/elasticapm-lambda # use this exact fixed value -ELASTIC_APM_LAMBDA_APM_SERVER = # this is your APM Server URL -ELASTIC_APM_SECRET_TOKEN = # this is your APM secret token -ELASTIC_APM_SEND_STRATEGY = background <1> ----- - --- -include::{apm-aws-lambda-root}/docs/images/images.asciidoc[tag=python-env-vars] --- - -// end::console-with-agent[] - -// tag::cli-with-agent[] - -To configure APM through the AWS command line interface execute the following command: - -[source,bash] ----- -aws lambda update-function-configuration --function-name yourLambdaFunctionName \ - --environment "Variables={AWS_LAMBDA_EXEC_WRAPPER=/opt/python/bin/elasticapm-lambda,ELASTIC_APM_LAMBDA_APM_SERVER=,ELASTIC_APM_SECRET_TOKEN=,ELASTIC_APM_SEND_STRATEGY=background}" <1> ----- - -// end::cli-with-agent[] - -// tag::sam-with-agent[] - -In your SAM `template.yml` file configure the following environment variables: - -[source,yml] ----- -... -Resources: - yourLambdaFunction: - Type: AWS::Serverless::Function - Properties: - ... - Environment: - Variables: - AWS_LAMBDA_EXEC_WRAPPER: /opt/python/bin/elasticapm-lambda - ELASTIC_APM_LAMBDA_APM_SERVER: - ELASTIC_APM_SECRET_TOKEN: - ELASTIC_APM_SEND_STRATEGY: background <1> -... ----- - -// end::sam-with-agent[] - -// tag::serverless-with-agent[] - -In your `serverless.yml` file configure the following environment variables: - -[source,yml] ----- -... -functions: - yourLambdaFunction: - ... - environment: - AWS_LAMBDA_EXEC_WRAPPER: /opt/python/bin/elasticapm-lambda - ELASTIC_APM_LAMBDA_APM_SERVER: - ELASTIC_APM_SECRET_TOKEN: - ELASTIC_APM_SEND_STRATEGY: background <1> -... ----- - -// end::serverless-with-agent[] - -// tag::terraform-with-agent[] -In your Terraform file configure the following environment variables: - -[source,terraform] ----- -... -resource "aws_lambda_function" "your_lambda_function" { - ... - environment { - variables = { - AWS_LAMBDA_EXEC_WRAPPER = /opt/python/bin/elasticapm-lambda - ELASTIC_APM_LAMBDA_APM_SERVER = "" - ELASTIC_APM_SECRET_TOKEN = "" - ELASTIC_APM_SEND_STRATEGY = "background" <1> - } - } -} -... ----- - -// end::terraform-with-agent[] - -// tag::container-with-agent[] -Environment variables configured for an AWS Lambda function are passed to the container running the lambda function. -You can use one of the other options (through AWS Web Console, AWS CLI, etc.) to configure the following environment variables: - -[source,bash] ----- -AWS_LAMBDA_EXEC_WRAPPER = /opt/python/bin/elasticapm-lambda # use this exact fixed value -ELASTIC_APM_LAMBDA_APM_SERVER = # this is your APM Server URL -ELASTIC_APM_SECRET_TOKEN = # this is your APM secret token -ELASTIC_APM_SEND_STRATEGY = background <1> ----- - -// end::container-with-agent[] diff --git a/docs/lambda/python-arn-replacement.asciidoc b/docs/lambda/python-arn-replacement.asciidoc deleted file mode 100644 index 24d9d1a7f..000000000 --- a/docs/lambda/python-arn-replacement.asciidoc +++ /dev/null @@ -1,9 +0,0 @@ -++++ - -++++ \ No newline at end of file diff --git a/docs/logging.asciidoc b/docs/logging.asciidoc deleted file mode 100644 index 8f51edd50..000000000 --- a/docs/logging.asciidoc +++ /dev/null @@ -1,175 +0,0 @@ -[[logs]] -== Logs - -Elastic Python APM Agent provides the following log features: - -- <> : Automatically inject correlation IDs that allow navigation between logs, traces and services. -- <> : Automatically reformat plaintext logs in {ecs-logging-ref}/intro.html[ECS logging] format. - -NOTE: Elastic Python APM Agent does not send the logs to Elasticsearch. It only -injects correlation IDs and reformats the logs. You must use another ingestion -strategy. We recommend https://www.elastic.co/beats/filebeat[Filebeat] for that purpose. - -Those features are part of {observability-guide}/application-logs.html[Application log ingestion strategies]. - -The {ecs-logging-python-ref}/intro.html[`ecs-logging-python`] library can also be used to use the {ecs-logging-ref}/intro.html[ECS logging] format without an APM agent. -When deployed with the Python APM agent, the agent will provide <> IDs. - -[float] -[[log-correlation-ids]] -=== Log correlation - -{apm-guide-ref}/log-correlation.html[Log correlation] allows you to navigate to all logs belonging to a particular trace -and vice-versa: for a specific log, see in which context it has been logged and which parameters the user provided. - -The Agent provides integrations with both the default Python logging library, -as well as http://www.structlog.org/en/stable/[`structlog`]. - -* <> -* <> - -[float] -[[logging-integrations]] -==== Logging integrations - -[float] -[[logging]] -===== `logging` - -We use https://docs.python.org/3/library/logging.html#logging.setLogRecordFactory[`logging.setLogRecordFactory()`] -to decorate the default LogRecordFactory to automatically add new attributes to -each LogRecord object: - -* `elasticapm_transaction_id` -* `elasticapm_trace_id` -* `elasticapm_span_id` - -This factory also adds these fields to a dictionary attribute, -`elasticapm_labels`, using the official ECS https://www.elastic.co/guide/en/ecs/current/ecs-tracing.html[tracing fields]. - -You can disable this automatic behavior by using the -<> setting -in your configuration. - -[float] -[[structlog]] -===== `structlog` - -We provide a http://www.structlog.org/en/stable/processors.html[processor] for -http://www.structlog.org/en/stable/[`structlog`] which will add three new keys -to the event_dict of any processed event: - -* `transaction.id` -* `trace.id` -* `span.id` - -[source,python] ----- -from structlog import PrintLogger, wrap_logger -from structlog.processors import JSONRenderer -from elasticapm.handlers.structlog import structlog_processor - -wrapped_logger = PrintLogger() -logger = wrap_logger(wrapped_logger, processors=[structlog_processor, JSONRenderer()]) -log = logger.new() -log.msg("some_event") ----- - -[float] -===== Use structlog for agent-internal logging - -The Elastic APM Python agent uses logging to log internal events and issues. -By default, it will use a `logging` logger. -If your project uses structlog, you can tell the agent to use a structlog logger -by setting the environment variable `ELASTIC_APM_USE_STRUCTLOG` to `true`. - -[float] -[[log-correlation-in-es]] -=== Log correlation in Elasticsearch - -In order to correlate logs from your app with transactions captured by the -Elastic APM Python Agent, your logs must contain one or more of the following -identifiers: - -* `transaction.id` -* `trace.id` -* `span.id` - -If you're using structured logging, either https://docs.python.org/3/howto/logging-cookbook.html#implementing-structured-logging[with a custom solution] -or with http://www.structlog.org/en/stable/[structlog] (recommended), then this -is fairly easy. Throw the http://www.structlog.org/en/stable/api.html#structlog.processors.JSONRenderer[JSONRenderer] -in, and use {blog-ref}structured-logging-filebeat[Filebeat] -to pull these logs into Elasticsearch. - -Without structured logging the task gets a little trickier. Here we -recommend first making sure your LogRecord objects have the elasticapm -attributes (see <>), and then you'll want to combine some specific -formatting with a Grok pattern, either in Elasticsearch using -{ref}/grok-processor.html[the grok processor], -or in {logstash-ref}/plugins-filters-grok.html[logstash with a plugin]. - -Say you have a https://docs.python.org/3/library/logging.html#logging.Formatter[Formatter] -that looks like this: - -[source,python] ----- -import logging - -fh = logging.FileHandler('spam.log') -formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") -fh.setFormatter(formatter) ----- - -You can add the APM identifiers by simply switching out the `Formatter` object -for the one that we provide: - -[source,python] ----- -import logging -from elasticapm.handlers.logging import Formatter - -fh = logging.FileHandler('spam.log') -formatter = Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") -fh.setFormatter(formatter) ----- - -This will automatically append apm-specific fields to your format string: - -[source,python] ----- -formatstring = "%(asctime)s - %(name)s - %(levelname)s - %(message)s" -formatstring = formatstring + " | elasticapm " \ - "transaction.id=%(elasticapm_transaction_id)s " \ - "trace.id=%(elasticapm_trace_id)s " \ - "span.id=%(elasticapm_span_id)s" ----- - -Then, you could use a grok pattern like this (for the -{ref}/grok-processor.html[Elasticsearch Grok Processor]): - -[source, json] ----- -{ - "description" : "...", - "processors": [ - { - "grok": { - "field": "message", - "patterns": ["%{GREEDYDATA:msg} | elasticapm transaction.id=%{DATA:transaction.id} trace.id=%{DATA:trace.id} span.id=%{DATA:span.id}"] - } - } - ] -} ----- - -[float] -[[log-reformatting]] -=== Log reformatting (experimental) - -Starting in version 6.16.0, the agent can automatically reformat application -logs to ECS format with no changes to dependencies. Prior versions must install -the `ecs_logging` dependency. - -Log reformatting is controlled by the <> configuration option, and is disabled by default. - -The reformatted logs will include both the <> IDs. diff --git a/docs/metrics.asciidoc b/docs/metrics.asciidoc deleted file mode 100644 index 2d7ae6216..000000000 --- a/docs/metrics.asciidoc +++ /dev/null @@ -1,215 +0,0 @@ -[[metrics]] -== Metrics - -With Elastic APM, you can capture system and process metrics. -These metrics will be sent regularly to the APM Server and from there to Elasticsearch - -[float] -[[metric-sets]] -=== Metric sets - -* <> -* <> -* <> -* <> - -[float] -[[cpu-memory-metricset]] -==== CPU/Memory metric set - -`elasticapm.metrics.sets.cpu.CPUMetricSet` - -This metric set collects various system metrics and metrics of the current process. - -NOTE: if you do *not* use Linux, you need to install https://pypi.org/project/psutil/[`psutil`] for this metric set. - - -*`system.cpu.total.norm.pct`*:: -+ --- -type: scaled_float - -format: percent - -The percentage of CPU time in states other than Idle and IOWait, normalized by the number of cores. --- - - -*`system.process.cpu.total.norm.pct`*:: -+ --- -type: scaled_float - -format: percent - -The percentage of CPU time spent by the process since the last event. -This value is normalized by the number of CPU cores and it ranges from 0 to 100%. --- - -*`system.memory.total`*:: -+ --- -type: long - -format: bytes - -Total memory. --- - -*`system.memory.actual.free`*:: -+ --- -type: long - -format: bytes - -Actual free memory in bytes. --- - -*`system.process.memory.size`*:: -+ --- -type: long - -format: bytes - -The total virtual memory the process has. --- - -*`system.process.memory.rss.bytes`*:: -+ --- -type: long - -format: bytes - -The Resident Set Size. The amount of memory the process occupied in main memory (RAM). --- - -[float] -[[cpu-memory-cgroup-metricset]] -===== Linux’s cgroup metrics - -*`system.process.cgroup.memory.mem.limit.bytes`*:: -+ --- -type: long - -format: bytes - -Memory limit for current cgroup slice. --- - -*`system.process.cgroup.memory.mem.usage.bytes`*:: -+ --- -type: long - -format: bytes - -Memory usage in current cgroup slice. --- - - -[float] -[[breakdown-metricset]] -==== Breakdown metric set - -NOTE: Tracking and collection of this metric set can be disabled using the <> setting. - -*`span.self_time`*:: -+ --- -type: simple timer - -This timer tracks the span self-times and is the basis of the transaction breakdown visualization. - -Fields: - -* `sum`: The sum of all span self-times in ms since the last report (the delta) -* `count`: The count of all span self-times since the last report (the delta) - -You can filter and group by these dimensions: - -* `transaction.name`: The name of the transaction -* `transaction.type`: The type of the transaction, for example `request` -* `span.type`: The type of the span, for example `app`, `template` or `db` -* `span.subtype`: The sub-type of the span, for example `mysql` (optional) - --- -[float] -[[prometheus-metricset]] -==== Prometheus metric set (beta) - -beta[] - -If you use https://github.com/prometheus/client_python[`prometheus_client`] to collect metrics, the agent can -collect them as well and make them available in Elasticsearch. - -The following types of metrics are supported: - - * Counters - * Gauges - * Summaries - * Histograms (requires APM Server / Elasticsearch / Kibana 7.14+) - -To use the Prometheus metric set, you have to enable it with the <> configuration option. - -All metrics collected from `prometheus_client` are prefixed with `"prometheus.metrics."`. This can be changed using the <> configuration option. - -[float] -[[prometheus-metricset-beta]] -===== Beta limitations - * The metrics format may change without backwards compatibility in future releases. - -[float] -[[custom-metrics]] -=== Custom Metrics - -Custom metrics allow you to send your own metrics to Elasticsearch. - -The most common way to send custom metrics is with the -<>. However, you can also use your -own metric set. If you collect the metrics manually in your code, you can use -the base `MetricSet` class: - -[source,python] ----- -from elasticapm.metrics.base_metrics import MetricSet - -client = elasticapm.Client() -metricset = client.metrics.register(MetricSet) - -for x in range(10): - metricset.counter("my_counter").inc() ----- - -Alternatively, you can create your own MetricSet class which inherits from the -base class. In this case, you'll usually want to override the `before_collect` -method, where you can gather and set metrics before they are collected and sent -to Elasticsearch. - -You can add your `MetricSet` class as shown in the example above, or you can -add an import string for your class to the <> -configuration option: - -[source,bash] ----- -ELASTIC_APM_METRICS_SETS="elasticapm.metrics.sets.cpu.CPUMetricSet,myapp.metrics.MyMetricSet" ----- - -Your MetricSet might look something like this: - -[source,python] ----- -from elasticapm.metrics.base_metrics import MetricSet - -class MyAwesomeMetricSet(MetricSet): - def before_collect(self): - self.gauge("my_gauge").set(myapp.some_value) ----- - -In the example above, the MetricSet would look up `myapp.some_value` and set -the metric `my_gauge` to that value. This would happen whenever metrics are -collected/sent, which is controlled by the -<> setting. \ No newline at end of file diff --git a/docs/opentelemetry.asciidoc b/docs/opentelemetry.asciidoc deleted file mode 100644 index ee3800376..000000000 --- a/docs/opentelemetry.asciidoc +++ /dev/null @@ -1,76 +0,0 @@ -[[opentelemetry-bridge]] -== OpenTelemetry API Bridge - -The Elastic APM OpenTelemetry bridge allows you to create Elastic APM `Transactions` and `Spans`, -using the OpenTelemetry API. This allows users to utilize the Elastic APM agent's -automatic instrumentations, while keeping custom instrumentations vendor neutral. - -If a span is created while there is no transaction active, it will result in an -Elastic APM {apm-guide-ref}/data-model-transactions.html[`Transaction`]. Inner spans -are mapped to Elastic APM {apm-guide-ref}/data-model-spans.html[`Span`]. - -[float] -[[opentelemetry-getting-started]] -=== Getting started -The first step in getting started with the OpenTelemetry bridge is to install the `opentelemetry` libraries: - -[source,bash] ----- -pip install elastic-apm[opentelemetry] ----- - -Or if you already have installed `elastic-apm`: - - -[source,bash] ----- -pip install opentelemetry-api opentelemetry-sdk ----- - - -[float] -[[opentelemetry-usage]] -=== Usage - -[source,python] ----- -from elasticapm.contrib.opentelemetry import Tracer - -tracer = Tracer(__name__) -with tracer.start_as_current_span("test"): - # Do some work ----- - -or - -[source,python] ----- -from elasticapm.contrib.opentelemetry import trace - -tracer = trace.get_tracer(__name__) -with tracer.start_as_current_span("test"): - # Do some work ----- - - -`Tracer` and `get_tracer()` accept the following optional arguments: - - * `elasticapm_client`: an already instantiated Elastic APM client - * `config`: a configuration dictionary, which will be used to instantiate a new Elastic APM client, - e.g. `{"SERVER_URL": "https://example.org"}`. See <> for more information. - -The `Tracer` object mirrors the upstream interface on the -https://opentelemetry-python.readthedocs.io/en/latest/api/trace.html#opentelemetry.trace.Tracer[OpenTelemetry `Tracer` object.] - - -[float] -[[opentelemetry-caveats]] -=== Caveats -Not all features of the OpenTelemetry API are supported. - -Processors, exporters, metrics, logs, span events, and span links are not supported. - -Additionally, due to implementation details, the global context API only works -when a span is included in the activated context, and tokens are not used. -Instead, the global context works as a stack, and when a context is detached the -previously-active context will automatically be activated. diff --git a/docs/redirects.asciidoc b/docs/redirects.asciidoc deleted file mode 100644 index c924b6efe..000000000 --- a/docs/redirects.asciidoc +++ /dev/null @@ -1,14 +0,0 @@ -["appendix",role="exclude",id="redirects"] -== Deleted pages - -The following pages have moved or been deleted. - -[role="exclude",id="opentracing-bridge"] -=== OpenTracing API - -Refer to <> instead. - -[role="exclude",id="log-correlation"] -=== Log correlation - -Refer to <> instead. diff --git a/docs/reference/advanced-topics.md b/docs/reference/advanced-topics.md new file mode 100644 index 000000000..aade7f2df --- /dev/null +++ b/docs/reference/advanced-topics.md @@ -0,0 +1,16 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/advanced-topics.html +--- + +# Advanced topics [advanced-topics] + +* [Instrumenting custom code](/reference/instrumenting-custom-code.md) +* [Sanitizing data](/reference/sanitizing-data.md) +* [How the Agent works](/reference/how-agent-works.md) +* [Run Tests Locally](/reference/run-tests-locally.md) + + + + + diff --git a/docs/reference/aiohttp-server-support.md b/docs/reference/aiohttp-server-support.md new file mode 100644 index 000000000..fcde2cdab --- /dev/null +++ b/docs/reference/aiohttp-server-support.md @@ -0,0 +1,112 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/aiohttp-server-support.html +--- + +# Aiohttp Server support [aiohttp-server-support] + +Getting Elastic APM set up for your Aiohttp Server project is easy, and there are various ways you can tweak it to fit to your needs. + + +## Installation [aiohttp-server-installation] + +Install the Elastic APM agent using pip: + +```bash +$ pip install elastic-apm +``` + +or add `elastic-apm` to your project’s `requirements.txt` file. + + +## Setup [aiohttp-server-setup] + +To set up the agent, you need to initialize it with appropriate settings. + +The settings are configured either via environment variables, the application’s settings, or as initialization arguments. + +You can find a list of all available settings in the [Configuration](/reference/configuration.md) page. + +To initialize the agent for your application using environment variables: + +```python +from aiohttp import web + +from elasticapm.contrib.aiohttp import ElasticAPM + +app = web.Application() + +apm = ElasticAPM(app) +``` + +To configure the agent using `ELASTIC_APM` in your application’s settings: + +```python +from aiohttp import web + +from elasticapm.contrib.aiohttp import ElasticAPM + +app = web.Application() + +app['ELASTIC_APM'] = { + 'SERVICE_NAME': '', + 'SECRET_TOKEN': '', +} +apm = ElasticAPM(app) +``` + + +## Usage [aiohttp-server-usage] + +Once you have configured the agent, it will automatically track transactions and capture uncaught exceptions within aiohttp. + +Capture an arbitrary exception by calling [`capture_exception`](/reference/api-reference.md#client-api-capture-exception): + +```python +try: + 1 / 0 +except ZeroDivisionError: + apm.client.capture_exception() +``` + +Log a generic message with [`capture_message`](/reference/api-reference.md#client-api-capture-message): + +```python +apm.client.capture_message('hello, world!') +``` + + +## Performance metrics [aiohttp-server-performance-metrics] + +If you’ve followed the instructions above, the agent has already installed our middleware. This will measure response times, as well as detailed performance data for all supported technologies. + +::::{note} +due to the fact that `asyncio` drivers are usually separate from their synchronous counterparts, specific instrumentation is needed for all drivers. The support for asynchronous drivers is currently quite limited. +:::: + + + +### Ignoring specific routes [aiohttp-server-ignoring-specific-views] + +You can use the [`TRANSACTIONS_IGNORE_PATTERNS`](/reference/configuration.md#config-transactions-ignore-patterns) configuration option to ignore specific routes. The list given should be a list of regular expressions which are matched against the transaction name: + +```python +app['ELASTIC_APM'] = { + # ... + 'TRANSACTIONS_IGNORE_PATTERNS': ['^OPTIONS ', '/api/'] + # ... +} +``` + +This would ignore any requests using the `OPTIONS` method and any requests containing `/api/`. + + +## Supported aiohttp and Python versions [supported-aiohttp-and-python-versions] + +A list of supported [aiohttp](/reference/supported-technologies.md#supported-aiohttp) and [Python](/reference/supported-technologies.md#supported-python) versions can be found on our [Supported Technologies](/reference/supported-technologies.md) page. + +::::{note} +Elastic APM only supports `asyncio` when using Python 3.7+ +:::: + + diff --git a/docs/reference/api-reference.md b/docs/reference/api-reference.md new file mode 100644 index 000000000..23cc9cc58 --- /dev/null +++ b/docs/reference/api-reference.md @@ -0,0 +1,463 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/api.html +--- + +# API reference [api] + +The Elastic APM Python agent has several public APIs. Most of the public API functionality is not needed when using one of our [supported frameworks](/reference/supported-technologies.md#framework-support), but they allow customized usage. + + +## Client API [client-api] + +The public Client API consists of several methods on the `Client` class. This API can be used to track exceptions and log messages, as well as to mark the beginning and end of transactions. + + +### Instantiation [client-api-init] + +Added in v1.0.0. + +To create a `Client` instance, import it and call its constructor: + +```python +from elasticapm import Client + +client = Client({'SERVICE_NAME': 'example'}, **defaults) +``` + +* `config`: A dictionary, with key/value configuration. For the possible configuration keys, see [Configuration](/reference/configuration.md). +* `**defaults`: default values for configuration. These can be omitted in most cases, and take the least precedence. + +::::{note} +framework integrations like [Django](/reference/django-support.md) and [Flask](/reference/flask-support.md) instantiate the client automatically. +:::: + + + +#### `elasticapm.get_client()` [api-get-client] + +[small]#Added in v6.1.0. + +Retrieves the `Client` singleton. This is useful for many framework integrations, where the client is instantiated automatically. + +```python +client = elasticapm.get_client() +client.capture_message('foo') +``` + + +### Errors [error-api] + + +#### `Client.capture_exception()` [client-api-capture-exception] + +Added in v1.0.0. `handled` added in v2.0.0. + +Captures an exception object: + +```python +try: + x = int("five") +except ValueError: + client.capture_exception() +``` + +* `exc_info`: A `(type, value, traceback)` tuple as returned by [`sys.exc_info()`](https://docs.python.org/3/library/sys.html#sys.exc_info). If not provided, it will be captured automatically. +* `date`: A `datetime.datetime` object representing the occurrence time of the error. If left empty, it defaults to `datetime.datetime.utcnow()`. +* `context`: A dictionary with contextual information. This dictionary must follow the [Context](docs-content://solutions/observability/apps/elastic-apm-events-intake-api.md#apm-api-error) schema definition. +* `custom`: A dictionary of custom data you want to attach to the event. +* `handled`: A boolean to indicate if this exception was handled or not. + +Returns the id of the error as a string. + + +#### `Client.capture_message()` [client-api-capture-message] + +Added in v1.0.0. + +Captures a message with optional added contextual data. Example: + +```python +client.capture_message('Billing process succeeded.') +``` + +* `message`: The message as a string. +* `param_message`: Alternatively, a parameterized message as a dictionary. The dictionary contains two values: `message`, and `params`. This allows the APM Server to group messages together that share the same parameterized message. Example: + + ```python + client.capture_message(param_message={ + 'message': 'Billing process for %s succeeded. Amount: %s', + 'params': (customer.id, order.total_amount), + }) + ``` + +* `stack`: If set to `True` (the default), a stacktrace from the call site will be captured. +* `exc_info`: A `(type, value, traceback)` tuple as returned by [`sys.exc_info()`](https://docs.python.org/3/library/sys.html#sys.exc_info). If not provided, it will be captured automatically, if `capture_message()` was called in an `except` block. +* `date`: A `datetime.datetime` object representing the occurrence time of the error. If left empty, it defaults to `datetime.datetime.utcnow()`. +* `context`: A dictionary with contextual information. This dictionary must follow the [Context](docs-content://solutions/observability/apps/elastic-apm-events-intake-api.md#apm-api-error) schema definition. +* `custom`: A dictionary of custom data you want to attach to the event. + +Returns the id of the message as a string. + +::::{note} +Either the `message` or the `param_message` argument is required. +:::: + + + +### Transactions [transaction-api] + + +#### `Client.begin_transaction()` [client-api-begin-transaction] + +Added in v1.0.0. `trace_parent` support added in v5.6.0. + +Begin tracking a transaction. Should be called e.g. at the beginning of a request or when starting a background task. Example: + +```python +client.begin_transaction('processors') +``` + +* `transaction_type`: (**required**) A string describing the type of the transaction, e.g. `'request'` or `'celery'`. +* `trace_parent`: (**optional**) A `TraceParent` object. See [TraceParent generation](#traceparent-api). +* `links`: (**optional**) A list of `TraceParent` objects to which this transaction is causally linked. + + +#### `Client.end_transaction()` [client-api-end-transaction] + +Added in v1.0.0. + +End tracking the transaction. Should be called e.g. at the end of a request or when ending a background task. Example: + +```python +client.end_transaction('myapp.billing_process', processor.status) +``` + +* `name`: (**optional**) A string describing the name of the transaction, e.g. `process_order`. This is typically the name of the view/controller that handles the request, or the route name. +* `result`: (**optional**) A string describing the result of the transaction. This is typically the HTTP status code, or e.g. `'success'` for a background task. + +::::{note} +if `name` and `result` are not set in the `end_transaction()` call, they have to be set beforehand by calling [`elasticapm.set_transaction_name()`](#api-set-transaction-name) and [`elasticapm.set_transaction_result()`](#api-set-transaction-result) during the transaction. +:::: + + + +### `TraceParent` [traceparent-api] + +Transactions can be started with a `TraceParent` object. This creates a transaction that is a child of the `TraceParent`, which is essential for distributed tracing. + + +#### `elasticapm.trace_parent_from_string()` [api-traceparent-from-string] + +Added in v5.6.0. + +Create a `TraceParent` object from the string representation generated by `TraceParent.to_string()`: + +```python +parent = elasticapm.trace_parent_from_string('00-03d67dcdd62b7c0f7a675424347eee3a-5f0e87be26015733-01') +client.begin_transaction('processors', trace_parent=parent) +``` + +* `traceparent_string`: (**required**) A string representation of a `TraceParent` object. + + +#### `elasticapm.trace_parent_from_headers()` [api-traceparent-from-headers] + +Added in v5.6.0. + +Create a `TraceParent` object from HTTP headers (usually generated by another Elastic APM agent): + +```python +parent = elasticapm.trace_parent_from_headers(headers_dict) +client.begin_transaction('processors', trace_parent=parent) +``` + +* `headers`: (**required**) HTTP headers formed as a dictionary. + + +#### `elasticapm.get_trace_parent_header()` [api-traceparent-get-header] + +Added in v5.10.0. + +Return the string representation of the current transaction `TraceParent` object: + +```python +elasticapm.get_trace_parent_header() +``` + + +## Other APIs [api-other] + + +### `elasticapm.instrument()` [api-elasticapm-instrument] + +Added in v1.0.0. + +Instruments libraries automatically. This includes a wide range of standard library and 3rd party modules. A list of instrumented modules can be found in `elasticapm.instrumentation.register`. This function should be called as early as possibly in the startup of your application. For [supported frameworks](/reference/supported-technologies.md#framework-support), this is called automatically. Example: + +```python +import elasticapm + +elasticapm.instrument() +``` + + +### `elasticapm.set_transaction_name()` [api-set-transaction-name] + +Added in v1.0.0. + +Set the name of the current transaction. For supported frameworks, the transaction name is determined automatically, and can be overridden using this function. Example: + +```python +import elasticapm + +elasticapm.set_transaction_name('myapp.billing_process') +``` + +* `name`: (**required**) A string describing name of the transaction +* `override`: if `True` (the default), overrides any previously set transaction name. If `False`, only sets the name if the transaction name hasn’t already been set. + + +### `elasticapm.set_transaction_result()` [api-set-transaction-result] + +Added in v2.2.0. + +Set the result of the current transaction. For supported frameworks, the transaction result is determined automatically, and can be overridden using this function. Example: + +```python +import elasticapm + +elasticapm.set_transaction_result('SUCCESS') +``` + +* `result`: (**required**) A string describing the result of the transaction, e.g. `HTTP 2xx` or `SUCCESS` +* `override`: if `True` (the default), overrides any previously set result. If `False`, only sets the result if the result hasn’t already been set. + + +### `elasticapm.set_transaction_outcome()` [api-set-transaction-outcome] + +Added in v5.9.0. + +Sets the outcome of the transaction. The value can either be `"success"`, `"failure"` or `"unknown"`. This should only be called at the end of a transaction after the outcome is determined. + +The `outcome` is used for error rate calculations. `success` denotes that a transaction has concluded successful, while `failure` indicates that the transaction failed to finish successfully. If the `outcome` is set to `unknown`, the transaction will not be included in error rate calculations. + +For supported web frameworks, the transaction outcome is set automatically if it has not been set yet, based on the HTTP status code. A status code below `500` is considered a `success`, while any value of `500` or higher is counted as a `failure`. + +If your transaction results in an HTTP response, you can alternatively provide the HTTP status code. + +::::{note} +While the `outcome` and `result` field look very similar, they serve different purposes. Other than the `result` field, which canhold an arbitrary string value, `outcome` is limited to three different values, `"success"`, `"failure"` and `"unknown"`. This allows the APM app to perform error rate calculations on these values. +:::: + + +Example: + +```python +import elasticapm + +elasticapm.set_transaction_outcome("success") + +# Using an HTTP status code +elasticapm.set_transaction_outcome(http_status_code=200) + +# Using predefined constants: + +from elasticapm.conf.constants import OUTCOME + +elasticapm.set_transaction_outcome(OUTCOME.SUCCESS) +elasticapm.set_transaction_outcome(OUTCOME.FAILURE) +elasticapm.set_transaction_outcome(OUTCOME.UNKNOWN) +``` + +* `outcome`: One of `"success"`, `"failure"` or `"unknown"`. Can be omitted if `http_status_code` is provided. +* `http_status_code`: if the transaction represents an HTTP response, its status code can be provided to determine the `outcome` automatically. +* `override`: if `True` (the default), any previously set `outcome` will be overriden. If `False`, the outcome will only be set if it was not set before. + + +### `elasticapm.get_transaction_id()` [api-get-transaction-id] + +Added in v5.2.0. + +Get the id of the current transaction. Example: + +```python +import elasticapm + +transaction_id = elasticapm.get_transaction_id() +``` + + +### `elasticapm.get_trace_id()` [api-get-trace-id] + +Added in v5.2.0. + +Get the `trace_id` of the current transaction’s trace. Example: + +```python +import elasticapm + +trace_id = elasticapm.get_trace_id() +``` + + +### `elasticapm.get_span_id()` [api-get-span-id] + +Added in v5.2.0. + +Get the id of the current span. Example: + +```python +import elasticapm + +span_id = elasticapm.get_span_id() +``` + + +### `elasticapm.set_custom_context()` [api-set-custom-context] + +Added in v2.0.0. + +Attach custom contextual data to the current transaction and errors. Supported frameworks will automatically attach information about the HTTP request and the logged in user. You can attach further data using this function. + +::::{tip} +Before using custom context, ensure you understand the different types of [metadata](docs-content://solutions/observability/apps/metadata.md) that are available. +:::: + + +Example: + +```python +import elasticapm + +elasticapm.set_custom_context({'billing_amount': product.price * item_count}) +``` + +* `data`: (**required**) A dictionary with the data to be attached. This should be a flat key/value `dict` object. + +::::{note} +`.`, `*`, and `"` are invalid characters for key names and will be replaced with `_`. +:::: + + +Errors that happen after this call will also have the custom context attached to them. You can call this function multiple times, new context data will be merged with existing data, following the `update()` semantics of Python dictionaries. + + +### `elasticapm.set_user_context()` [api-set-user-context] + +Added in v2.0.0. + +Attach information about the currently logged in user to the current transaction and errors. Example: + +```python +import elasticapm + +elasticapm.set_user_context(username=user.username, email=user.email, user_id=user.id) +``` + +* `username`: The username of the logged in user +* `email`: The email of the logged in user +* `user_id`: The unique identifier of the logged in user, e.g. the primary key value + +Errors that happen after this call will also have the user context attached to them. You can call this function multiple times, new user data will be merged with existing data, following the `update()` semantics of Python dictionaries. + + +### `elasticapm.capture_span` [api-capture-span] + +Added in v4.1.0. + +Capture a custom span. This can be used either as a function decorator or as a context manager (in a `with` statement). When used as a decorator, the name of the span will be set to the name of the function. When used as a context manager, a name has to be provided. + +```python +import elasticapm + +@elasticapm.capture_span() +def coffee_maker(strength): + fetch_water() + + with elasticapm.capture_span('near-to-machine', labels={"type": "arabica"}): + insert_filter() + for i in range(strength): + pour_coffee() + + start_drip() + + fresh_pots() +``` + +* `name`: The name of the span. Defaults to the function name if used as a decorator. +* `span_type`: (**optional**) The type of the span, usually in a dot-separated hierarchy of `type`, `subtype`, and `action`, e.g. `db.mysql.query`. Alternatively, type, subtype and action can be provided as three separate arguments, see `span_subtype` and `span_action`. +* `skip_frames`: (**optional**) The number of stack frames to skip when collecting stack traces. Defaults to `0`. +* `leaf`: (**optional**) if `True`, all spans nested bellow this span will be ignored. Defaults to `False`. +* `labels`: (**optional**) a dictionary of labels. Keys must be strings, values can be strings, booleans, or numerical (`int`, `float`, `decimal.Decimal`). Defaults to `None`. +* `span_subtype`: (**optional**) subtype of the span, e.g. name of the database. Defaults to `None`. +* `span_action`: (**optional**) action of the span, e.g. `query`. Defaults to `None`. +* `links`: (**optional**) A list of `TraceParent` objects to which this span is causally linked. + + +### `elasticapm.async_capture_span` [api-async-capture-span] + +Added in v5.4.0. + +Capture a custom async-aware span. This can be used either as a function decorator or as a context manager (in an `async with` statement). When used as a decorator, the name of the span will be set to the name of the function. When used as a context manager, a name has to be provided. + +```python +import elasticapm + +@elasticapm.async_capture_span() +async def coffee_maker(strength): + await fetch_water() + + async with elasticapm.async_capture_span('near-to-machine', labels={"type": "arabica"}): + await insert_filter() + async for i in range(strength): + await pour_coffee() + + start_drip() + + fresh_pots() +``` + +* `name`: The name of the span. Defaults to the function name if used as a decorator. +* `span_type`: (**optional**) The type of the span, usually in a dot-separated hierarchy of `type`, `subtype`, and `action`, e.g. `db.mysql.query`. Alternatively, type, subtype and action can be provided as three separate arguments, see `span_subtype` and `span_action`. +* `skip_frames`: (**optional**) The number of stack frames to skip when collecting stack traces. Defaults to `0`. +* `leaf`: (**optional**) if `True`, all spans nested bellow this span will be ignored. Defaults to `False`. +* `labels`: (**optional**) a dictionary of labels. Keys must be strings, values can be strings, booleans, or numerical (`int`, `float`, `decimal.Decimal`). Defaults to `None`. +* `span_subtype`: (**optional**) subtype of the span, e.g. name of the database. Defaults to `None`. +* `span_action`: (**optional**) action of the span, e.g. `query`. Defaults to `None`. +* `links`: (**optional**) A list of `TraceParent` objects to which this span is causally linked. + +::::{note} +`asyncio` is only supported for Python 3.7+. +:::: + + + +### `elasticapm.label()` [api-label] + +Added in v5.0.0. + +Attach labels to the the current transaction and errors. + +::::{tip} +Before using custom labels, ensure you understand the different types of [metadata](docs-content://solutions/observability/apps/metadata.md) that are available. +:::: + + +Example: + +```python +import elasticapm + +elasticapm.label(ecommerce=True, dollar_value=47.12) +``` + +Errors that happen after this call will also have the labels attached to them. You can call this function multiple times, new labels will be merged with existing labels, following the `update()` semantics of Python dictionaries. + +Keys must be strings, values can be strings, booleans, or numerical (`int`, `float`, `decimal.Decimal`) `.`, `*`, and `"` are invalid characters for label names and will be replaced with `_`. + +::::{warning} +Avoid defining too many user-specified labels. Defining too many unique fields in an index is a condition that can lead to a [mapping explosion](docs-content://manage-data/data-store/mapping.md#mapping-limit-settings). +:::: + + diff --git a/docs/reference/asgi-middleware.md b/docs/reference/asgi-middleware.md new file mode 100644 index 000000000..852f12565 --- /dev/null +++ b/docs/reference/asgi-middleware.md @@ -0,0 +1,66 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/asgi-middleware.html +--- + +# ASGI Middleware [asgi-middleware] + +::::{warning} +This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. +:::: + + +Incorporating Elastic APM into your ASGI-based project only requires a few easy steps. + +::::{note} +Several ASGI frameworks are supported natively. Please check [Supported Technologies](/reference/supported-technologies.md) for more information +:::: + + + +## Installation [asgi-installation] + +Install the Elastic APM agent using pip: + +```bash +$ pip install elastic-apm +``` + +or add `elastic-apm` to your project’s `requirements.txt` file. + + +## Setup [asgi-setup] + +To set up the agent, you need to initialize it with appropriate settings. + +The settings are configured either via environment variables, or as initialization arguments. + +You can find a list of all available settings in the [Configuration](/reference/configuration.md) page. + +To set up the APM agent, wrap your ASGI app with the `ASGITracingMiddleware`: + +```python +from elasticapm.contrib.asgi import ASGITracingMiddleware + +app = MyGenericASGIApp() # depending on framework + +app = ASGITracingMiddleware(app) +``` + +Make sure to call [`elasticapm.set_transaction_name()`](/reference/api-reference.md#api-set-transaction-name) with an appropriate transaction name in all your routes. + +::::{note} +Currently, the agent doesn’t support automatic capturing of exceptions. You can follow progress on this issue on [Github](https://github.com/elastic/apm-agent-python/issues/1548). +:::: + + + +## Supported Python versions [supported-python-versions] + +A list of supported [Python](/reference/supported-technologies.md#supported-python) versions can be found on our [Supported Technologies](/reference/supported-technologies.md) page. + +::::{note} +Elastic APM only supports `asyncio` when using Python 3.7+ +:::: + + diff --git a/docs/reference/azure-functions-support.md b/docs/reference/azure-functions-support.md new file mode 100644 index 000000000..52f582fbc --- /dev/null +++ b/docs/reference/azure-functions-support.md @@ -0,0 +1,53 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/azure-functions-support.html +--- + +# Monitoring Azure Functions [azure-functions-support] + + +## Prerequisites [_prerequisites_2] + +You need an APM Server to which you can send APM data. Follow the [APM Quick start](docs-content://solutions/observability/apps/get-started-with-apm.md) if you have not set one up yet. For the best-possible performance, we recommend setting up APM on {{ecloud}} in the same Azure region as your Azure Functions app. + +::::{note} +Currently, only HTTP and timer triggers are supported. Other trigger types may be captured as well, but the amount of captured contextual data may differ. +:::: + + + +## Step 1: Enable Worker Extensions [_step_1_enable_worker_extensions] + +Elastic APM uses [Worker Extensions](https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=asgi%2Capplication-level&pivots=python-mode-configuration#python-worker-extensions) to instrument Azure Functions. This feature is not enabled by default, and must be enabled in your Azure Functions App. Please follow the instructions in the [Azure docs](https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=asgi%2Capplication-level&pivots=python-mode-configuration#using-extensions). + +Once you have enabled Worker Extensions, these two lines of code will enable Elastic APM’s extension: + +```python +from elasticapm.contrib.serverless.azure import ElasticAPMExtension + +ElasticAPMExtension.configure() +``` + +Put them somewhere at the top of your Python file, before the function definitions. + + +## Step 2: Install the APM Python Agent [_step_2_install_the_apm_python_agent] + +You need to add `elastic-apm` as a dependency for your Functions app. Simply add `elastic-apm` to your `requirements.txt` file. We recommend pinning the version to the current newest version of the agent, and periodically updating the version. + + +## Step 3: Configure APM on Azure Functions [_step_3_configure_apm_on_azure_functions] + +The APM Python agent is configured through [App Settings](https://learn.microsoft.com/en-us/azure/azure-functions/functions-how-to-use-azure-function-app-settings?tabs=portal#settings). These are then picked up by the agent as environment variables. + +For the minimal configuration, you will need the [`ELASTIC_APM_SERVER_URL`](/reference/configuration.md#config-server-url) to set the destination for APM data and a [`ELASTIC_APM_SECRET_TOKEN`](/reference/configuration.md#config-secret-token). If you prefer to use an [APM API key](docs-content://solutions/observability/apps/api-keys.md) instead of the APM secret token, use the [`ELASTIC_APM_API_KEY`](/reference/configuration.md#config-api-key) environment variable instead of `ELASTIC_APM_SECRET_TOKEN` in the following example configuration. + +```bash +$ az functionapp config appsettings set --settings ELASTIC_APM_SERVER_URL=https://example.apm.northeurope.azure.elastic-cloud.com:443 +$ az functionapp config appsettings set --settings ELASTIC_APM_SECRET_TOKEN=verysecurerandomstring +``` + +You can optionally [fine-tune the Python agent](/reference/configuration.md). + +That’s it; Once the agent is installed and working, spans will be captured for [supported technologies](/reference/supported-technologies.md). You can also use [`capture_span`](/reference/api-reference.md#api-capture-span) to capture custom spans, and you can retrieve the `Client` object for capturing exceptions/messages using [`get_client`](/reference/api-reference.md#api-get-client). + diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md new file mode 100644 index 000000000..2930b1587 --- /dev/null +++ b/docs/reference/configuration.md @@ -0,0 +1,1067 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/configuration.html +--- + +# Configuration [configuration] + +To adapt the Elastic APM agent to your needs, configure it using environment variables or framework-specific configuration. + +You can either configure the agent by setting environment variables: + +```bash +ELASTIC_APM_SERVICE_NAME=foo python manage.py runserver +``` + +or with inline configuration: + +```python +apm_client = Client(service_name="foo") +``` + +or by using framework specific configuration e.g. in your Django `settings.py` file: + +```python +ELASTIC_APM = { + "SERVICE_NAME": "foo", +} +``` + +The precedence is as follows: + +* [Central configuration](#config-central_config) (supported options are marked with [![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration)) +* Environment variables +* Inline configuration +* Framework-specific configuration +* Default value + + +## Dynamic configuration [dynamic-configuration] + +Configuration options marked with the ![dynamic config](../images/dynamic-config.svg "") badge can be changed at runtime when set from a supported source. + +The Python Agent supports [Central configuration](docs-content://solutions/observability/apps/apm-agent-central-configuration.md), which allows you to fine-tune certain configurations from in the APM app. This feature is enabled in the Agent by default with [`central_config`](#config-central_config). + + +## Django [django-configuration] + +To configure Django, add an `ELASTIC_APM` dictionary to your `settings.py`: + +```python +ELASTIC_APM = { + 'SERVICE_NAME': 'my-app', + 'SECRET_TOKEN': 'changeme', +} +``` + + +## Flask [flask-configuration] + +To configure Flask, add an `ELASTIC_APM` dictionary to your `app.config`: + +```python +app.config['ELASTIC_APM'] = { + 'SERVICE_NAME': 'my-app', + 'SECRET_TOKEN': 'changeme', +} + +apm = ElasticAPM(app) +``` + + +## Core options [core-options] + + +### `service_name` [config-service-name] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_SERVICE_NAME` | `SERVICE_NAME` | `unknown-python-service` | `my-app` | + +The name of your service. This is used to keep all the errors and transactions of your service together and is the primary filter in the Elastic APM user interface. + +While a default is provided, it is essential that you override this default with something more descriptive and unique across your infrastructure. + +::::{note} +The service name must conform to this regular expression: `^[a-zA-Z0-9 _-]+$`. In other words, the service name must only contain characters from the ASCII alphabet, numbers, dashes, underscores, and spaces. It cannot be an empty string or whitespace-only. +:::: + + + +### `server_url` [config-server-url] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SERVER_URL` | `SERVER_URL` | `'http://127.0.0.1:8200'` | + +The URL for your APM Server. The URL must be fully qualified, including protocol (`http` or `https`) and port. Note: Do not set this if you are using APM in an AWS lambda function. APM Agents are designed to proxy their calls to the APM Server through the lambda extension. Instead, set `ELASTIC_APM_LAMBDA_APM_SERVER`. For more info, see [AWS Lambda](/reference/lambda-support.md). + + +## `enabled` [config-enabled] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_ENABLED` | `ENABLED` | `true` | + +Enable or disable the agent. When set to false, the agent will not collect any data or start any background threads. + + +## `recording` [config-recording] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_RECORDING` | `RECORDING` | `true` | + +Enable or disable recording of events. If set to false, then the Python agent does not send any events to the Elastic APM server, and instrumentation overhead is minimized. The agent will continue to poll the server for configuration changes. + + +## Logging Options [logging-options] + + +### `log_level` [config-log_level] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_LOG_LEVEL` | `LOG_LEVEL` | | + +The `logging.logLevel` at which the `elasticapm` logger will log. The available options are: + +* `"off"` (sets `logging.logLevel` to 1000) +* `"critical"` +* `"error"` +* `"warning"` +* `"info"` +* `"debug"` +* `"trace"` (sets `logging.log_level` to 5) + +Options are case-insensitive + +Note that this option doesn’t do anything with logging handlers. In order for any logs to be visible, you must either configure a handler ([`logging.basicConfig`](https://docs.python.org/3/library/logging.html#logging.basicConfig) will do this for you) or set [`log_file`](#config-log_file). This will also override any log level your app has set for the `elasticapm` logger. + + +### `log_file` [config-log_file] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_LOG_FILE` | `LOG_FILE` | `""` | `"/var/log/elasticapm/log.txt"` | + +This enables the agent to log to a file. This is disabled by default. The agent will log at the `logging.logLevel` configured with [`log_level`](#config-log_level). Use [`log_file_size`](#config-log_file_size) to configure the maximum size of the log file. This log file will automatically rotate. + +Note that setting [`log_level`](#config-log_level) is required for this setting to do anything. + +If [`ecs_logging`](https://github.com/elastic/ecs-logging-python) is installed, the logs will automatically be formatted as ecs-compatible json. + + +### `log_file_size` [config-log_file_size] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_LOG_FILE_SIZE` | `LOG_FILE_SIZE` | `"50mb"` | `"100mb"` | + +The size of the log file if [`log_file`](#config-log_file) is set. + +The agent always keeps one backup file when rotating, so the maximum space that the log files will consume is twice the value of this setting. + + +### `log_ecs_reformatting` [config-log_ecs_reformatting] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_LOG_ECS_REFORMATTING` | `LOG_ECS_REFORMATTING` | `"off"` | + +::::{warning} +This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. +:::: + + +Valid options: + +* `"off"` +* `"override"` + +If [`ecs_logging`](https://github.com/elastic/ecs-logging-python) is installed, setting this to `"override"` will cause the agent to automatically attempt to enable ecs-formatted logging. + +For base `logging` from the standard library, the agent will get the root logger, find any attached handlers, and for each, set the formatter to `ecs_logging.StdlibFormatter()`. + +If `structlog` is installed, the agent will override any configured processors with `ecs_logging.StructlogFormatter()`. + +Note that this is a very blunt instrument that could have unintended side effects. If problems arise, please apply these formatters manually and leave this setting as `"off"`. See the [`ecs_logging` docs](ecs-logging-python://reference/installation.md) for more information about using these formatters. + +Also note that this setting does not facilitate shipping logs to Elasticsearch. We recommend [Filebeat](https://www.elastic.co/beats/filebeat) for that purpose. + + +## Other options [other-options] + + +### `transport_class` [config-transport-class] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_TRANSPORT_CLASS` | `TRANSPORT_CLASS` | `elasticapm.transport.http.Transport` | + +The transport class to use when sending events to the APM Server. + + +### `service_node_name` [config-service-node-name] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_SERVICE_NODE_NAME` | `SERVICE_NODE_NAME` | `None` | `"redis1"` | + +The name of the given service node. This is optional and if omitted, the APM Server will fall back on `system.container.id` if available, and `host.name` if necessary. + +This option allows you to set the node name manually to ensure it is unique and meaningful. + + +### `environment` [config-environment] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_ENVIRONMENT` | `ENVIRONMENT` | `None` | `"production"` | + +The name of the environment this service is deployed in, e.g. "production" or "staging". + +Environments allow you to easily filter data on a global level in the APM app. It’s important to be consistent when naming environments across agents. See [environment selector](docs-content://solutions/observability/apps/filter-application-data.md#apm-filter-your-data-service-environment-filter) in the APM app for more information. + +::::{note} +This feature is fully supported in the APM app in Kibana versions >= 7.2. You must use the query bar to filter for a specific environment in versions prior to 7.2. +:::: + + + +### `cloud_provider` [config-cloud-provider] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_CLOUD_PROVIDER` | `CLOUD_PROVIDER` | `"auto"` | `"aws"` | + +This config value allows you to specify which cloud provider should be assumed for metadata collection. By default, the agent will attempt to detect the cloud provider or, if that fails, will use trial and error to collect the metadata. + +Valid options are `"auto"`, `"aws"`, `"gcp"`, and `"azure"`. If this config value is set to `"none"`, then no cloud metadata will be collected. + + +### `secret_token` [config-secret-token] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_SECRET_TOKEN` | `SECRET_TOKEN` | `None` | A random string | + +This string is used to ensure that only your agents can send data to your APM Server. Both the agents and the APM Server have to be configured with the same secret token. An example to generate a secure secret token is: + +```bash +python -c "import secrets; print(secrets.token_urlsafe(32))" +``` + +::::{warning} +Secret tokens only provide any security if your APM Server uses TLS. +:::: + + + +### `api_key` [config-api-key] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_API_KEY` | `API_KEY` | `None` | A base64-encoded string | + +::::{warning} +This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. +:::: + + +This base64-encoded string is used to ensure that only your agents can send data to your APM Server. The API key must be created using the [APM server command-line tool](docs-content://solutions/observability/apps/api-keys.md). + +::::{warning} +API keys only provide any real security if your APM Server uses TLS. +:::: + + + +### `service_version` [config-service-version] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_SERVICE_VERSION` | `SERVICE_VERSION` | `None` | A string indicating the version of the deployed service | + +A version string for the currently deployed version of the service. If youre deploys are not versioned, the recommended value for this field is the commit identifier of the deployed revision, e.g. the output of `git rev-parse HEAD`. + + +### `framework_name` [config-framework-name] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_FRAMEWORK_NAME` | `FRAMEWORK_NAME` | Depending on framework | + +The name of the used framework. For Django and Flask, this defaults to `django` and `flask` respectively, otherwise, the default is `None`. + + +### `framework_version` [config-framework-version] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_FRAMEWORK_VERSION` | `FRAMEWORK_VERSION` | Depending on framework | + +The version number of the used framework. For Django and Flask, this defaults to the used version of the framework, otherwise, the default is `None`. + + +### `filter_exception_types` [config-filter-exception-types] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_FILTER_EXCEPTION_TYPES` | `FILTER_EXCEPTION_TYPES` | `[]` | `['OperationalError', 'mymodule.SomeoneElsesProblemError']` | +| multiple values separated by commas, without spaces | | | | + +A list of exception types to be filtered. Exceptions of these types will not be sent to the APM Server. + + +### `transaction_ignore_urls` [config-transaction-ignore-urls] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_TRANSACTION_IGNORE_URLS` | `TRANSACTION_IGNORE_URLS` | `[]` | `['/api/ping', '/static/*']` | +| multiple values separated by commas, without spaces | | | | + +A list of URLs for which the agent should not capture any transaction data. + +Optionally, `*` can be used to match multiple URLs at once. + + +### `transactions_ignore_patterns` [config-transactions-ignore-patterns] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_TRANSACTIONS_IGNORE_PATTERNS` | `TRANSACTIONS_IGNORE_PATTERNS` | `[]` | `['^OPTIONS ', 'myviews.Healthcheck']` | +| multiple values separated by commas, without spaces | | | | + +A list of regular expressions. Transactions with a name that matches any of the configured patterns will be ignored and not sent to the APM Server. + +::::{note} +as the the name of the transaction can only be determined at the end of the transaction, the agent might still cause overhead for transactions ignored through this setting. If agent overhead is a concern, we recommend [`transaction_ignore_urls`](#config-transaction-ignore-urls) instead. +:::: + + + +### `server_timeout` [config-server-timeout] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SERVER_TIMEOUT` | `SERVER_TIMEOUT` | `"5s"` | + +A timeout for requests to the APM Server. The setting has to be provided in **[duration format](#config-format-duration)**. If a request to the APM Server takes longer than the configured timeout, the request is cancelled and the event (exception or transaction) is discarded. Set to `None` to disable timeouts. + +::::{warning} +If timeouts are disabled or set to a high value, your app could experience memory issues if the APM Server times out. +:::: + + + +### `hostname` [config-hostname] + +| Environment | Django/Flask | Default | Example | +| --- | --- | --- | --- | +| `ELASTIC_APM_HOSTNAME` | `HOSTNAME` | `socket.gethostname()` | `app-server01.example.com` | + +The host name to use when sending error and transaction data to the APM Server. + + +### `auto_log_stacks` [config-auto-log-stacks] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_AUTO_LOG_STACKS` | `AUTO_LOG_STACKS` | `True` | +| set to `"true"` / `"false"` | | | + +If set to `True` (the default), the agent will add a stack trace to each log event, indicating where the log message has been issued. + +This setting can be overridden on an individual basis by setting the `extra`-key `stack`: + +```python +logger.info('something happened', extra={'stack': False}) +``` + + +### `collect_local_variables` [config-collect-local-variables] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_COLLECT_LOCAL_VARIABLES` | `COLLECT_LOCAL_VARIABLES` | `errors` | + +Possible values: `errors`, `transactions`, `all`, `off` + +The Elastic APM Python agent can collect local variables for stack frames. By default, this is only done for errors. + +::::{note} +Collecting local variables has a non-trivial overhead. Collecting local variables for transactions in production environments can have adverse effects for the performance of your service. +:::: + + + +### `local_var_max_length` [config-local-var-max-length] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_LOCAL_VAR_MAX_LENGTH` | `LOCAL_VAR_MAX_LENGTH` | `200` | + +When collecting local variables, they will be converted to strings. This setting allows you to limit the length of the resulting string. + + +### `local_var_list_max_length` [config-local-list-var-max-length] + +| | | | +| --- | --- | --- | +| Environment | Django/Flask | Default | +| `ELASTIC_APM_LOCAL_VAR_LIST_MAX_LENGTH` | `LOCAL_VAR_LIST_MAX_LENGTH` | `10` | + +This setting allows you to limit the length of lists in local variables. + + +### `local_var_dict_max_length` [config-local-dict-var-max-length] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_LOCAL_VAR_DICT_MAX_LENGTH` | `LOCAL_VAR_DICT_MAX_LENGTH` | `10` | + +This setting allows you to limit the length of dicts in local variables. + + +### `source_lines_error_app_frames` [config-source-lines-error-app-frames] + + +### `source_lines_error_library_frames` [config-source-lines-error-library-frames] + + +### `source_lines_span_app_frames` [config-source-lines-span-app-frames] + + +### `source_lines_span_library_frames` [config-source-lines-span-library-frames] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SOURCE_LINES_ERROR_APP_FRAMES` | `SOURCE_LINES_ERROR_APP_FRAMES` | `5` | +| `ELASTIC_APM_SOURCE_LINES_ERROR_LIBRARY_FRAMES` | `SOURCE_LINES_ERROR_LIBRARY_FRAMES` | `5` | +| `ELASTIC_APM_SOURCE_LINES_SPAN_APP_FRAMES` | `SOURCE_LINES_SPAN_APP_FRAMES` | `0` | +| `ELASTIC_APM_SOURCE_LINES_SPAN_LIBRARY_FRAMES` | `SOURCE_LINES_SPAN_LIBRARY_FRAMES` | `0` | + +By default, the APM agent collects source code snippets for errors. This setting allows you to modify the number of lines of source code that are being collected. + +We differ between errors and spans, as well as library frames and app frames. + +::::{warning} +Especially for spans, collecting source code can have a large impact on storage use in your Elasticsearch cluster. +:::: + + + +### `capture_body` [config-capture-body] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_CAPTURE_BODY` | `CAPTURE_BODY` | `off` | + +For transactions that are HTTP requests, the Python agent can optionally capture the request body (e.g. `POST` variables). + +Possible values: `errors`, `transactions`, `all`, `off`. + +If the request has a body and this setting is disabled, the body will be shown as `[REDACTED]`. + +For requests with a content type of `multipart/form-data`, any uploaded files will be referenced in a special `_files` key. It contains the name of the field and the name of the uploaded file, if provided. + +::::{warning} +Request bodies often contain sensitive values like passwords and credit card numbers. If your service handles data like this, we advise to only enable this feature with care. +:::: + + + +### `capture_headers` [config-capture-headers] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_CAPTURE_HEADERS` | `CAPTURE_HEADERS` | `true` | + +For transactions and errors that happen due to HTTP requests, the Python agent can optionally capture the request and response headers. + +Possible values: `true`, `false` + +::::{warning} +Request headers often contain sensitive values like session IDs and cookies. See [sanitizing data](/reference/sanitizing-data.md) for more information on how to filter out sensitive data. +:::: + + + +### `transaction_max_spans` [config-transaction-max-spans] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_TRANSACTION_MAX_SPANS` | `TRANSACTION_MAX_SPANS` | `500` | + +This limits the amount of spans that are recorded per transaction. This is helpful in cases where a transaction creates a very high amount of spans (e.g. thousands of SQL queries). Setting an upper limit will prevent edge cases from overloading the agent and the APM Server. + + +### `stack_trace_limit` [config-stack-trace-limit] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_STACK_TRACE_LIMIT` | `STACK_TRACE_LIMIT` | `50` | + +This limits the number of frames captured for each stack trace. + +Setting the limit to `0` will disable stack trace collection, while any positive integer value will be used as the maximum number of frames to collect. To disable the limit and always capture all frames, set the value to `-1`. + + +### `span_stack_trace_min_duration` [config-span-stack-trace-min-duration] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SPAN_STACK_TRACE_MIN_DURATION` | `SPAN_STACK_TRACE_MIN_DURATION` | `"5ms"` | + +By default, the APM agent collects a stack trace with every recorded span that has a duration equal to or longer than this configured threshold. While stack traces are very helpful to find the exact place in your code from which a span originates, collecting this stack trace does have some overhead. Tune this threshold to ensure that you only collect stack traces for spans that could be problematic. + +To collect traces for all spans, regardless of their length, set the value to `0`. + +To disable stack trace collection for spans completely, set the value to `-1`. + +Except for the special values `-1` and `0`, this setting should be provided in **[duration format](#config-format-duration)**. + + +### `span_frames_min_duration` [config-span-frames-min-duration] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SPAN_FRAMES_MIN_DURATION` | `SPAN_FRAMES_MIN_DURATION` | `"5ms"` | + +::::{note} +This config value is being deprecated. Use [`span_stack_trace_min_duration`](#config-span-stack-trace-min-duration) instead. +:::: + + + +### `span_compression_enabled` [config-span-compression-enabled] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SPAN_COMPRESSION_ENABLED` | `SPAN_COMPRESSION_ENABLED` | `True` | + +Enable/disable span compression. + +If enabled, the agent will compress very short, repeated spans into a single span, which is beneficial for storage and processing requirements. Some information is lost in this process, e.g. exact durations of each compressed span. + + +### `span_compression_exact_match_max_duration` [config-span-compression-exact-match-max_duration] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION` | `SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION` | `"50ms"` | + +Consecutive spans that are exact match and that are under this threshold will be compressed into a single composite span. This reduces the collection, processing, and storage overhead, and removes clutter from the UI. The tradeoff is that the DB statements of all the compressed spans will not be collected. + +Two spans are considered exact matches if the following attributes are identical: * span name * span type * span subtype * destination resource (e.g. the Database name) + + +### `span_compression_same_kind_max_duration` [config-span-compression-same-kind-max-duration] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SPAN_COMPRESSION_SAME_KIND_MAX_DURATION` | `SPAN_COMPRESSION_SAME_KIND_MAX_DURATION` | `"0ms"` (disabled) | + +Consecutive spans to the same destination that are under this threshold will be compressed into a single composite span. This reduces the collection, processing, and storage overhead, and removes clutter from the UI. The tradeoff is that metadata such as database statements of all the compressed spans will not be collected. + +Two spans are considered to be of the same kind if the following attributes are identical: * span type * span subtype * destination resource (e.g. the Database name) + + +### `exit_span_min_duration` [config-exit-span-min-duration] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_EXIT_SPAN_MIN_DURATION` | `EXIT_SPAN_MIN_DURATION` | `"0ms"` | + +Exit spans are spans that represent a call to an external service, like a database. If such calls are very short, they are usually not relevant and can be ignored. + +This feature is disabled by default. + +::::{note} +if a span propagates distributed tracing IDs, it will not be ignored, even if it is shorter than the configured threshold. This is to ensure that no broken traces are recorded. +:::: + + + +### `api_request_size` [config-api-request-size] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_API_REQUEST_SIZE` | `API_REQUEST_SIZE` | `"768kb"` | + +The maximum queue length of the request buffer before sending the request to the APM Server. A lower value will increase the load on your APM Server, while a higher value can increase the memory pressure of your app. A higher value also impacts the time until data is indexed and searchable in Elasticsearch. + +This setting is useful to limit memory consumption if you experience a sudden spike of traffic. It has to be provided in **[size format](#config-format-size)**. + +::::{note} +Due to internal buffering of gzip, the actual request size can be a few kilobytes larger than the given limit. By default, the APM Server limits request payload size to `1 MByte`. +:::: + + + +### `api_request_time` [config-api-request-time] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_API_REQUEST_TIME` | `API_REQUEST_TIME` | `"10s"` | + +The maximum queue time of the request buffer before sending the request to the APM Server. A lower value will increase the load on your APM Server, while a higher value can increase the memory pressure of your app. A higher value also impacts the time until data is indexed and searchable in Elasticsearch. + +This setting is useful to limit memory consumption if you experience a sudden spike of traffic. It has to be provided in **[duration format](#config-format-duration)**. + +::::{note} +The actual time will vary between 90-110% of the given value, to avoid stampedes of instances that start at the same time. +:::: + + + +### `processors` [config-processors] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_PROCESSORS` | `PROCESSORS` | `['elasticapm.processors.sanitize_stacktrace_locals', 'elasticapm.processors.sanitize_http_request_cookies', 'elasticapm.processors.sanitize_http_headers', 'elasticapm.processors.sanitize_http_wsgi_env', 'elasticapm.processors.sanitize_http_request_body']` | + +A list of processors to process transactions and errors. For more information, see [Sanitizing Data](/reference/sanitizing-data.md). + +::::{warning} +We recommend always including the default set of validators if you customize this setting. +:::: + + + +### `sanitize_field_names` [config-sanitize-field-names] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SANITIZE_FIELD_NAMES` | `SANITIZE_FIELD_NAMES` | `["password", "passwd", "pwd", "secret", "*key", "*token*", "*session*", "*credit*", "*card*", "*auth*", "*principal*", "set-cookie"]` | + +A list of glob-matched field names to match and mask when using processors. For more information, see [Sanitizing Data](/reference/sanitizing-data.md). + +::::{warning} +We recommend always including the default set of field name matches if you customize this setting. +:::: + + + +### `transaction_sample_rate` [config-transaction-sample-rate] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_TRANSACTION_SAMPLE_RATE` | `TRANSACTION_SAMPLE_RATE` | `1.0` | + +By default, the agent samples every transaction (e.g. request to your service). To reduce overhead and storage requirements, set the sample rate to a value between `0.0` and `1.0`. We still record overall time and the result for unsampled transactions, but no context information, labels, or spans. + +::::{note} +This setting will be automatically rounded to 4 decimals of precision. +:::: + + + +### `include_paths` [config-include-paths] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_INCLUDE_PATHS` | `INCLUDE_PATHS` | `[]` | +| multiple values separated by commas, without spaces | | | + +A set of paths, optionally using shell globs (see [`fnmatch`](https://docs.python.org/3/library/fnmatch.html) for a description of the syntax). These are matched against the absolute filename of every frame, and if a pattern matches, the frame is considered to be an "in-app frame". + +`include_paths` **takes precedence** over `exclude_paths`. + + +### `exclude_paths` [config-exclude-paths] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_EXCLUDE_PATHS` | `EXCLUDE_PATHS` | Varies on Python version and implementation | +| multiple values separated by commas, without spaces | | | + +A set of paths, optionally using shell globs (see [`fnmatch`](https://docs.python.org/3/library/fnmatch.html) for a description of the syntax). These are matched against the absolute filename of every frame, and if a pattern matches, the frame is considered to be a "library frame". + +`include_paths` **takes precedence** over `exclude_paths`. + +The default value varies based on your Python version and implementation, e.g.: + +* PyPy3: `['\*/lib-python/3/*', '\*/site-packages/*']` +* CPython 2.7: `['\*/lib/python2.7/*', '\*/lib64/python2.7/*']` + + +### `debug` [config-debug] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_DEBUG` | `DEBUG` | `False` | + +If your app is in debug mode (e.g. in Django with `settings.DEBUG = True` or in Flask with `app.debug = True`), the agent won’t send any data to the APM Server. You can override it by changing this setting to `True`. + + +### `disable_send` [config-disable-send] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_DISABLE_SEND` | `DISABLE_SEND` | `False` | + +If set to `True`, the agent won’t send any events to the APM Server, independent of any debug state. + + +### `instrument` [config-instrument] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_INSTRUMENT` | `INSTRUMENT` | `True` | + +If set to `False`, the agent won’t instrument any code. This disables most of the tracing functionality, but can be useful to debug possible instrumentation issues. + + +### `verify_server_cert` [config-verify-server-cert] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_VERIFY_SERVER_CERT` | `VERIFY_SERVER_CERT` | `True` | + +By default, the agent verifies the SSL certificate if an HTTPS connection to the APM Server is used. Verification can be disabled by changing this setting to `False`. This setting is ignored when [`server_cert`](#config-server-cert) is set. + + +### `server_cert` [config-server-cert] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SERVER_CERT` | `SERVER_CERT` | `None` | + +If you have configured your APM Server with a self-signed TLS certificate, or you just wish to pin the server certificate, you can specify the path to the PEM-encoded certificate via the `ELASTIC_APM_SERVER_CERT` configuration. + +::::{note} +If this option is set, the agent only verifies that the certificate provided by the APM Server is identical to the one configured here. Validity of the certificate is not checked. +:::: + + + +### `server_ca_cert_file` [config-server-ca-cert-file] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_SERVER_CA_CERT_FILE` | `SERVER_CA_CERT_FILE` | `None` | + +By default, the agent will validate the TLS/SSL certificate of the APM Server using the well-known CAs curated by Mozilla, and provided by the [`certifi`](https://pypi.org/project/certifi/) package. + +You can set this option to the path of a file containing a CA certificate that will be used instead. + +Specifying this option is required when using self-signed certificates, unless server certificate validation is disabled. + + +### `use_certifi` [config-use-certifi] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_USE_CERTIFI` | `USE_CERTIFI` | `True` | + +By default, the Python Agent uses the [`certifi`](https://pypi.org/project/certifi/) certificate store. To use Python’s default mechanism for finding certificates, set this option to `False`. + + +### `metrics_interval` [config-metrics_interval] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_METRICS_INTERVAL` | `METRICS_INTERVAL` | `30s` | + +The interval in which the agent collects metrics. A shorter interval increases the granularity of metrics, but also increases the overhead of the agent, as well as storage requirements. + +It has to be provided in **[duration format](#config-format-duration)**. + + +### `disable_metrics` [config-disable_metrics] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_DISABLE_METRICS` | `DISABLE_METRICS` | `None` | + +A comma-separated list of dotted metrics names that should not be sent to the APM Server. You can use `*` to match multiple metrics; for example, to disable all CPU-related metrics, as well as the "total system memory" metric, set `disable_metrics` to: + +``` +"*.cpu.*,system.memory.total" +``` +::::{note} +This setting only disables the **sending** of the given metrics, not collection. +:::: + + + +### `breakdown_metrics` [config-breakdown_metrics] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_BREAKDOWN_METRICS` | `BREAKDOWN_METRICS` | `True` | + +Enable or disable the tracking and collection of breakdown metrics. Setting this to `False` disables the tracking of breakdown metrics, which can reduce the overhead of the agent. + +::::{note} +This feature requires APM Server and Kibana >= 7.3. +:::: + + + +### `prometheus_metrics` (Beta) [config-prometheus_metrics] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_PROMETHEUS_METRICS` | `PROMETHEUS_METRICS` | `False` | + +Enable/disable the tracking and collection of metrics from `prometheus_client`. + +See [Prometheus metric set (beta)](/reference/metrics.md#prometheus-metricset) for more information. + +::::{note} +This feature is currently in beta status. +:::: + + + +### `prometheus_metrics_prefix` (Beta) [config-prometheus_metrics_prefix] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_PROMETHEUS_METRICS_PREFIX` | `PROMETHEUS_METRICS_PREFIX` | `prometheus.metrics.` | + +A prefix to prepend to Prometheus metrics names. + +See [Prometheus metric set (beta)](/reference/metrics.md#prometheus-metricset) for more information. + +::::{note} +This feature is currently in beta status. +:::: + + + +### `metrics_sets` [config-metrics_sets] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_METRICS_SETS` | `METRICS_SETS` | ["elasticapm.metrics.sets.cpu.CPUMetricSet"] | + +List of import paths for the MetricSets that should be used to collect metrics. + +See [Custom Metrics](/reference/metrics.md#custom-metrics) for more information. + + +### `central_config` [config-central_config] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_CENTRAL_CONFIG` | `CENTRAL_CONFIG` | `True` | + +When enabled, the agent will make periodic requests to the APM Server to fetch updated configuration. + +See [Dynamic configuration](#dynamic-configuration) for more information. + +::::{note} +This feature requires APM Server and Kibana >= 7.3. +:::: + + + +### `global_labels` [config-global_labels] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_GLOBAL_LABELS` | `GLOBAL_LABELS` | `None` | + +Labels added to all events, with the format `key=value[,key=value[,...]]`. Any labels set by application via the API will override global labels with the same keys. + +::::{note} +This feature requires APM Server >= 7.2. +:::: + + + +### `disable_log_record_factory` [config-generic-disable-log-record-factory] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_DISABLE_LOG_RECORD_FACTORY` | `DISABLE_LOG_RECORD_FACTORY` | `False` | + +By default in python 3, the agent installs a [LogRecord factory](/reference/logs.md#logging) that automatically adds tracing fields to your log records. Disable this behavior by setting this to `True`. + + +### `use_elastic_traceparent_header` [config-use-elastic-traceparent-header] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_USE_ELASTIC_TRACEPARENT_HEADER` | `USE_ELASTIC_TRACEPARENT_HEADER` | `True` | + +To enable [distributed tracing](docs-content://solutions/observability/apps/traces.md), the agent sets a number of HTTP headers to outgoing requests made with [instrumented HTTP libraries](/reference/supported-technologies.md#automatic-instrumentation-http). These headers (`traceparent` and `tracestate`) are defined in the [W3C Trace Context](https://www.w3.org/TR/trace-context-1/) specification. + +Additionally, when this setting is set to `True`, the agent will set `elasticapm-traceparent` for backwards compatibility. + + +### `trace_continuation_strategy` [config-trace-continuation-strategy] + +[![dynamic config](../images/dynamic-config.svg "") ](#dynamic-configuration) + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_TRACE_CONTINUATION_STRATEGY` | `TRACE_CONTINUATION_STRATEGY` | `continue` | + +This option allows some control on how the APM agent handles W3C trace-context headers on incoming requests. By default, the `traceparent` and `tracestate` headers are used per W3C spec for distributed tracing. However, in certain cases it can be helpful to **not** use the incoming `traceparent` header. Some example use cases: + +* An Elastic-monitored service is receiving requests with `traceparent` headers from **unmonitored** services. +* An Elastic-monitored service is publicly exposed, and does not want tracing data (trace-ids, sampling decisions) to possibly be spoofed by user requests. + +Valid values are: + +* `'continue'`: The default behavior. An incoming `traceparent` value is used to continue the trace and determine the sampling decision. +* `'restart'`: Always ignores the `traceparent` header of incoming requests. A new trace-id will be generated and the sampling decision will be made based on [`transaction_sample_rate`](#config-transaction-sample-rate). A **span link** will be made to the incoming traceparent. +* `'restart_external'`: If an incoming request includes the `es` vendor flag in `tracestate`, then any *traceparent* will be considered internal and will be handled as described for `'continue'` above. Otherwise, any `'traceparent'` is considered external and will be handled as described for `'restart'` above. + +Starting with Elastic Observability 8.2, span links will be visible in trace views. + + +### `use_elastic_excepthook` [config-use-elastic-excepthook] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_USE_ELASTIC_EXCEPTHOOK` | `USE_ELASTIC_EXCEPTHOOK` | `False` | + +If set to `True`, the agent will intercept the default `sys.excepthook`, which allows the agent to collect all uncaught exceptions. + + +### `include_process_args` [config-include-process-args] + +| Environment | Django/Flask | Default | +| --- | --- | --- | +| `ELASTIC_APM_INCLUDE_PROCESS_ARGS` | `INCLUDE_PROCESS_ARGS` | `False` | + +Whether each transaction should have the process arguments attached. Disabled by default to save disk space. + + +## Django-specific configuration [config-django-specific] + + +### `django_transaction_name_from_route` [config-django-transaction-name-from-route] + +| Environment | Django | Default | +| --- | --- | --- | +| `ELASTIC_APM_DJANGO_TRANSACTION_NAME_FROM_ROUTE` | `DJANGO_TRANSACTION_NAME_FROM_ROUTE` | `False` | + +By default, we use the function or class name of the view as the transaction name. Starting with Django 2.2, Django makes the route (e.g. `users//`) available on the `request.resolver_match` object. If you want to use the route instead of the view name as the transaction name, set this config option to `true`. + +::::{note} +in versions previous to Django 2.2, changing this setting will have no effect. +:::: + + + +### `django_autoinsert_middleware` [config-django-autoinsert-middleware] + +| Environment | Django | Default | +| --- | --- | --- | +| `ELASTIC_APM_DJANGO_AUTOINSERT_MIDDLEWARE` | `DJANGO_AUTOINSERT_MIDDLEWARE` | `True` | + +To trace Django requests, the agent uses a middleware, `elasticapm.contrib.django.middleware.TracingMiddleware`. By default, this middleware is inserted automatically as the first item in `settings.MIDDLEWARES`. To disable the automatic insertion of the middleware, change this setting to `False`. + + +## Generic Environment variables [config-generic-environment] + +Some environment variables that are not specific to the APM agent can be used to configure the agent. + + +### `HTTP_PROXY` and `HTTPS_PROXY` [config-generic-http-proxy] + +By using `HTTP_PROXY` and `HTTPS_PROXY`, the agent can be instructed to use a proxy to connect to the APM Server. If both are set, `HTTPS_PROXY` takes precedence. + +::::{note} +The environment variables are case-insensitive. +:::: + + + +### `NO_PROXY` [config-generic-no-proxy] + +To instruct the agent to **not** use a proxy, you can use the `NO_PROXY` environment variable. You can either set it to a comma-separated list of hosts for which no proxy should be used (e.g. `localhost,example.com`) or use `*` to match any host. + +This is useful if `HTTP_PROXY` / `HTTPS_PROXY` is set for other reasons than agent / APM Server communication. + + +### `SSL_CERT_FILE` and `SSL_CERT_DIR` [config-ssl-cert-file] + +To tell the agent to use a different SSL certificate, you can use these environment variables. See also [OpenSSL docs](https://www.openssl.org/docs/manmaster/man7/openssl-env.html#SSL_CERT_DIR-SSL_CERT_FILE). + +Please note that these variables may apply to other SSL/TLS communication in your service, not just related to the APM agent. + +::::{note} +These environment variables only take effect if [`use_certifi`](#config-use-certifi) is set to `False`. +:::: + + + +## Configuration formats [config-formats] + +Some options require a unit, either duration or size. These need to be provided in a specific format. + + +### Duration format [config-format-duration] + +The *duration* format is used for options like timeouts. The unit is provided as a suffix directly after the number–without any separation by whitespace. + +**Example**: `5ms` + +**Supported units** + +* `us` (microseconds) +* `ms` (milliseconds) +* `s` (seconds) +* `m` (minutes) + + +### Size format [config-format-size] + +The *size* format is used for options like maximum buffer sizes. The unit is provided as suffix directly after the number, without and separation by whitespace. + +**Example**: `10kb` + +**Supported units**: + +* `b` (bytes) +* `kb` (kilobytes) +* `mb` (megabytes) +* `gb` (gigabytes) + +::::{note} +We use the power-of-two sizing convention, e.g. `1 kilobyte == 1024 bytes` +:::: + + diff --git a/docs/reference/django-support.md b/docs/reference/django-support.md new file mode 100644 index 000000000..9db46e864 --- /dev/null +++ b/docs/reference/django-support.md @@ -0,0 +1,327 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/django-support.html +--- + +# Django support [django-support] + +Getting Elastic APM set up for your Django project is easy, and there are various ways you can tweak it to fit to your needs. + + +## Installation [django-installation] + +Install the Elastic APM agent using pip: + +```bash +$ pip install elastic-apm +``` + +or add it to your project’s `requirements.txt` file. + +::::{note} +For apm-server 6.2+, make sure you use version 2.0 or higher of `elastic-apm`. +:::: + + +::::{note} +If you use Django with uwsgi, make sure to [enable threads](http://uwsgi-docs.readthedocs.org/en/latest/Options.html#enable-threads). +:::: + + + +## Setup [django-setup] + +Set up the Elastic APM agent in Django with these two steps: + +1. Add `elasticapm.contrib.django` to `INSTALLED_APPS` in your settings: + +```python +INSTALLED_APPS = ( + # ... + 'elasticapm.contrib.django', +) +``` + +1. Choose a service name, and set the secret token if needed. + +```python +ELASTIC_APM = { + 'SERVICE_NAME': '', + 'SECRET_TOKEN': '', +} +``` + +or as environment variables: + +```shell +ELASTIC_APM_SERVICE_NAME= +ELASTIC_APM_SECRET_TOKEN= +``` + +You now have basic error logging set up, and everything resulting in a 500 HTTP status code will be reported to the APM Server. + +You can find a list of all available settings in the [Configuration](/reference/configuration.md) page. + +::::{note} +The agent only captures and sends data if you have `DEBUG = False` in your settings. To force the agent to capture data in Django debug mode, set the [debug](/reference/configuration.md#config-debug) configuration option, e.g.: + +```python +ELASTIC_APM = { + 'SERVICE_NAME': '', + 'DEBUG': True, +} +``` + +:::: + + + +## Performance metrics [django-performance-metrics] + +In order to collect performance metrics, the agent automatically inserts a middleware at the top of your middleware list (`settings.MIDDLEWARE` in current versions of Django, `settings.MIDDLEWARE_CLASSES` in some older versions). To disable the automatic insertion of the middleware, see [django_autoinsert_middleware](/reference/configuration.md#config-django-autoinsert-middleware). + +::::{note} +For automatic insertion to work, your list of middlewares (`settings.MIDDLEWARE` or `settings.MIDDLEWARE_CLASSES`) must be of type `list` or `tuple`. +:::: + + +In addition to broad request metrics (what will appear in the APM app as transactions), the agent also collects fine grained metrics on template rendering, database queries, HTTP requests, etc. You can find more information on what we instrument in the [Automatic Instrumentation](/reference/supported-technologies.md#automatic-instrumentation) section. + + +### Instrumenting custom Python code [django-instrumenting-custom-python-code] + +To gain further insights into the performance of your code, please see [instrumenting custom code](/reference/instrumenting-custom-code.md). + + +### Ignoring specific views [django-ignoring-specific-views] + +You can use the `TRANSACTIONS_IGNORE_PATTERNS` configuration option to ignore specific views. The list given should be a list of regular expressions which are matched against the transaction name as seen in the Elastic APM user interface: + +```python +ELASTIC_APM['TRANSACTIONS_IGNORE_PATTERNS'] = ['^OPTIONS ', 'views.api.v2'] +``` + +This example ignores any requests using the `OPTIONS` method and any requests containing `views.api.v2`. + + +### Using the route as transaction name [django-transaction-name-route] + +By default, we use the function or class name of the view as the transaction name. Starting with Django 2.2, Django makes the route (e.g. `users//`) available on the `request.resolver_match` object. If you want to use the route instead of the view name as the transaction name, you can set the [`django_transaction_name_from_route`](/reference/configuration.md#config-django-transaction-name-from-route) config option to `true`. + +```python +ELASTIC_APM['DJANGO_TRANSACTION_NAME_FROM_ROUTE'] = True +``` + +::::{note} +in versions previous to Django 2.2, changing this setting will have no effect. +:::: + + + +### Integrating with the RUM Agent [django-integrating-with-the-rum-agent] + +To correlate performance measurement in the browser with measurements in your Django app, you can help the RUM (Real User Monitoring) agent by configuring it with the Trace ID and Span ID of the backend request. We provide a handy template context processor which adds all the necessary bits into the context of your templates. + +To enable this feature, first add the `rum_tracing` context processor to your `TEMPLATES` setting. You most likely already have a list of `context_processors`, in which case you can simply append ours to the list. + +```python +TEMPLATES = [ + { + 'BACKEND': 'django.template.backends.django.DjangoTemplates', + 'OPTIONS': { + 'context_processors': [ + # ... + 'elasticapm.contrib.django.context_processors.rum_tracing', + ], + }, + }, +] +``` + +Then, update the call to initialize the RUM agent (which probably happens in your base template) like this: + +```javascript +elasticApm.init({ + serviceName: "my-frontend-service", + pageLoadTraceId: "{{ apm.trace_id }}", + pageLoadSpanId: "{{ apm.span_id }}", + pageLoadSampled: {{ apm.is_sampled_js }} +}) +``` + +See the [JavaScript RUM agent documentation](apm-agent-rum-js://reference/index.md) for more information. + + +## Enabling and disabling the agent [django-enabling-and-disabling-the-agent] + +The easiest way to disable the agent is to set Django’s `DEBUG` option to `True` in your development configuration. No errors or metrics will be logged to Elastic APM. + +However, if during debugging you would like to force logging of errors to Elastic APM, then you can set `DEBUG` to `True` inside of the Elastic APM configuration dictionary, like this: + +```python +ELASTIC_APM = { + # ... + 'DEBUG': True, +} +``` + + +## Integrating with Python logging [django-logging] + +To easily send Python `logging` messages as "error" objects to Elasticsearch, we provide a `LoggingHandler` which you can use in your logging setup. The log messages will be enriched with a stack trace, data from the request, and more. + +::::{note} +the intended use case for this handler is to send high priority log messages (e.g. log messages with level `ERROR`) to Elasticsearch. For normal log shipping, we recommend using [filebeat](beats://reference/filebeat/filebeat-overview.md). +:::: + + +If you are new to how the `logging` module works together with Django, read more [in the Django documentation](https://docs.djangoproject.com/en/2.1/topics/logging/). + +An example of how your `LOGGING` setting could look: + +```python +LOGGING = { + 'version': 1, + 'disable_existing_loggers': True, + 'formatters': { + 'verbose': { + 'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s' + }, + }, + 'handlers': { + 'elasticapm': { + 'level': 'WARNING', + 'class': 'elasticapm.contrib.django.handlers.LoggingHandler', + }, + 'console': { + 'level': 'DEBUG', + 'class': 'logging.StreamHandler', + 'formatter': 'verbose' + } + }, + 'loggers': { + 'django.db.backends': { + 'level': 'ERROR', + 'handlers': ['console'], + 'propagate': False, + }, + 'mysite': { + 'level': 'WARNING', + 'handlers': ['elasticapm'], + 'propagate': False, + }, + # Log errors from the Elastic APM module to the console (recommended) + 'elasticapm.errors': { + 'level': 'ERROR', + 'handlers': ['console'], + 'propagate': False, + }, + }, +} +``` + +With this configuration, logging can be done like this in any module in the `myapp` django app: + +You can now use the logger in any module in the `myapp` Django app, for instance `myapp/views.py`: + +```python +import logging +logger = logging.getLogger('mysite') + +try: + instance = MyModel.objects.get(pk=42) +except MyModel.DoesNotExist: + logger.error( + 'Could not find instance, doing something else', + exc_info=True + ) +``` + +Note that `exc_info=True` adds the exception information to the data that gets sent to Elastic APM. Without it, only the message is sent. + + +### Extra data [django-extra-data] + +If you want to send more data than what you get with the agent by default, logging can be done like so: + +```python +import logging +logger = logging.getLogger('mysite') + +try: + instance = MyModel.objects.get(pk=42) +except MyModel.DoesNotExist: + logger.error( + 'There was some crazy error', + exc_info=True, + extra={ + 'datetime': str(datetime.now()), + } + ) +``` + + +## Celery integration [django-celery-integration] + +For a general guide on how to set up Django with Celery, head over to Celery’s [Django documentation](http://celery.readthedocs.org/en/latest/django/first-steps-with-django.html#django-first-steps). + +Elastic APM will automatically log errors from your celery tasks, record performance data and keep the trace.id when the task is launched from an already started Elastic transaction. + + +## Logging "HTTP 404 Not Found" errors [django-logging-http-404-not-found-errors] + +By default, Elastic APM does not log HTTP 404 errors. If you wish to log these errors, add `'elasticapm.contrib.django.middleware.Catch404Middleware'` to `MIDDLEWARE` in your settings: + +```python +MIDDLEWARE = ( + # ... + 'elasticapm.contrib.django.middleware.Catch404Middleware', + # ... +) +``` + +Note that this middleware respects Django’s [`IGNORABLE_404_URLS`](https://docs.djangoproject.com/en/1.11/ref/settings/#ignorable-404-urls) setting. + + +## Disable the agent during tests [django-disable-agent-during-tests] + +To prevent the agent from sending any data to the APM Server during tests, set the `ELASTIC_APM_DISABLE_SEND` environment variable to `true`, e.g.: + +```python +ELASTIC_APM_DISABLE_SEND=true python manage.py test +``` + + +## Troubleshooting [django-troubleshooting] + +Elastic APM comes with a Django command that helps troubleshooting your setup. To check your configuration, run + +```bash +python manage.py elasticapm check +``` + +To send a test exception using the current settings, run + +```bash +python manage.py elasticapm test +``` + +If the command succeeds in sending a test exception, it will print a success message: + +```bash +python manage.py elasticapm test + +Trying to send a test error using these settings: + +SERVICE_NAME: +SECRET_TOKEN: +SERVER: http://127.0.0.1:8200 + +Success! We tracked the error successfully! You should be able to see it in a few seconds. +``` + + +## Supported Django and Python versions [supported-django-and-python-versions] + +A list of supported [Django](/reference/supported-technologies.md#supported-django) and [Python](/reference/supported-technologies.md#supported-python) versions can be found on our [Supported Technologies](/reference/supported-technologies.md) page. + diff --git a/docs/reference/flask-support.md b/docs/reference/flask-support.md new file mode 100644 index 000000000..e99e32994 --- /dev/null +++ b/docs/reference/flask-support.md @@ -0,0 +1,215 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/flask-support.html +--- + +# Flask support [flask-support] + +Getting Elastic APM set up for your Flask project is easy, and there are various ways you can tweak it to fit to your needs. + + +## Installation [flask-installation] + +Install the Elastic APM agent using pip: + +```bash +$ pip install "elastic-apm[flask]" +``` + +or add `elastic-apm[flask]` to your project’s `requirements.txt` file. + +::::{note} +For apm-server 6.2+, make sure you use version 2.0 or higher of `elastic-apm`. +:::: + + +::::{note} +If you use Flask with uwsgi, make sure to [enable threads](http://uwsgi-docs.readthedocs.org/en/latest/Options.html#enable-threads). +:::: + + +::::{note} +If you see an error log that mentions `psutil not found`, you can install `psutil` using `pip install psutil`, or add `psutil` to your `requirements.txt` file. +:::: + + + +## Setup [flask-setup] + +To set up the agent, you need to initialize it with appropriate settings. + +The settings are configured either via environment variables, the application’s settings, or as initialization arguments. + +You can find a list of all available settings in the [Configuration](/reference/configuration.md) page. + +To initialize the agent for your application using environment variables: + +```python +from elasticapm.contrib.flask import ElasticAPM + +app = Flask(__name__) + +apm = ElasticAPM(app) +``` + +To configure the agent using `ELASTIC_APM` in your application’s settings: + +```python +from elasticapm.contrib.flask import ElasticAPM + +app.config['ELASTIC_APM'] = { + 'SERVICE_NAME': '', + 'SECRET_TOKEN': '', +} +apm = ElasticAPM(app) +``` + +The final option is to initialize the agent with the settings as arguments: + +```python +from elasticapm.contrib.flask import ElasticAPM + +apm = ElasticAPM(app, service_name='', secret_token='') +``` + + +### Debug mode [flask-debug-mode] + +::::{note} +Please note that errors and transactions will only be sent to the APM Server if your app is **not** in [Flask debug mode](https://flask.palletsprojects.com/en/3.0.x/quickstart/#debug-mode). +:::: + + +To force the agent to send data while the app is in debug mode, set the value of `DEBUG` in the `ELASTIC_APM` dictionary to `True`: + +```python +app.config['ELASTIC_APM'] = { + 'SERVICE_NAME': '', + 'SECRET_TOKEN': '', + 'DEBUG': True +} +``` + + +### Building applications on the fly? [flask-building-applications-on-the-fly] + +You can use the agent’s `init_app` hook for adding the application on the fly: + +```python +from elasticapm.contrib.flask import ElasticAPM +apm = ElasticAPM() + +def create_app(): + app = Flask(__name__) + apm.init_app(app, service_name='', secret_token='') + return app +``` + + +## Usage [flask-usage] + +Once you have configured the agent, it will automatically track transactions and capture uncaught exceptions within Flask. If you want to send additional events, a couple of shortcuts are provided on the ElasticAPM Flask middleware object by raising an exception or logging a generic message. + +Capture an arbitrary exception by calling `capture_exception`: + +```python +try: + 1 / 0 +except ZeroDivisionError: + apm.capture_exception() +``` + +Log a generic message with `capture_message`: + +```python +apm.capture_message('hello, world!') +``` + + +## Shipping Logs to Elasticsearch [flask-logging] + +This feature has been deprecated and will be removed in a future version. + +Please see our [Logging](/reference/logs.md) documentation for other supported ways to ship logs to Elasticsearch. + +Note that you can always send exceptions and messages to the APM Server with [`capture_exception`](/reference/api-reference.md#client-api-capture-exception) and and [`capture_message`](/reference/api-reference.md#client-api-capture-message). + +```python +from elasticapm import get_client + +@app.route('/') +def bar(): + try: + 1 / 0 + except ZeroDivisionError: + get_client().capture_exception() +``` + + +### Extra data [flask-extra-data] + +In addition to what the agents log by default, you can send extra information: + +```python +@app.route('/') +def bar(): + try: + 1 / 0 + except ZeroDivisionError: + app.logger.error('Math is hard', + exc_info=True, + extra={ + 'good_at_math': False, + } + ) + ) +``` + + +### Celery tasks [flask-celery-tasks] + +The Elastic APM agent will automatically send errors and performance data from your Celery tasks to the APM Server. + + +## Performance metrics [flask-performance-metrics] + +If you’ve followed the instructions above, the agent has already hooked into the right signals and should be reporting performance metrics. + + +### Ignoring specific routes [flask-ignoring-specific-views] + +You can use the [`TRANSACTIONS_IGNORE_PATTERNS`](/reference/configuration.md#config-transactions-ignore-patterns) configuration option to ignore specific routes. The list given should be a list of regular expressions which are matched against the transaction name: + +```python +app.config['ELASTIC_APM'] = { + ... + 'TRANSACTIONS_IGNORE_PATTERNS': ['^OPTIONS ', '/api/'] + ... +} +``` + +This would ignore any requests using the `OPTIONS` method and any requests containing `/api/`. + + +### Integrating with the RUM Agent [flask-integrating-with-the-rum-agent] + +To correlate performance measurement in the browser with measurements in your Flask app, you can help the RUM (Real User Monitoring) agent by configuring it with the Trace ID and Span ID of the backend request. We provide a handy template context processor which adds all the necessary bits into the context of your templates. + +The context processor is installed automatically when you initialize `ElasticAPM`. All that is left to do is to update the call to initialize the RUM agent (which probably happens in your base template) like this: + +```javascript +elasticApm.init({ + serviceName: "my-frontend-service", + pageLoadTraceId: "{{ apm["trace_id"] }}", + pageLoadSpanId: "{{ apm["span_id"]() }}", + pageLoadSampled: {{ apm["is_sampled_js"] }} +}) +``` + +See the [JavaScript RUM agent documentation](apm-agent-rum-js://reference/index.md) for more information. + + +## Supported Flask and Python versions [supported-flask-and-python-versions] + +A list of supported [Flask](/reference/supported-technologies.md#supported-flask) and [Python](/reference/supported-technologies.md#supported-python) versions can be found on our [Supported Technologies](/reference/supported-technologies.md) page. + diff --git a/docs/reference/how-agent-works.md b/docs/reference/how-agent-works.md new file mode 100644 index 000000000..6c0f597ca --- /dev/null +++ b/docs/reference/how-agent-works.md @@ -0,0 +1,52 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/how-the-agent-works.html +--- + +# How the Agent works [how-the-agent-works] + +To gather APM events (called transactions and spans), errors and metrics, the Python agent instruments your application in a few different ways. These events, are then sent to the APM Server. The APM Server converts them to a format suitable for Elasticsearch, and sends them to an Elasticsearch cluster. You can then use the APM app in Kibana to gain insight into latency issues and error culprits within your application. + +Broadly, we differentiate between three different approaches to collect the necessary data: framework integration, instrumentation, and background collection. + + +## Framework integration [how-it-works-framework-integration] + +To collect data about incoming requests and background tasks, we integrate with frameworks like [Django](/reference/django-support.md), [Flask](/reference/flask-support.md) and Celery. Whenever possible, framework integrations make use of hooks and signals provided by the framework. Examples of this are: + +* `request_started`, `request_finished`, and `got_request_exception` signals from `django.core.signals` +* `request_started`, `request_finished`, and `got_request_exception` signals from `flask.signals` +* `task_prerun`, `task_postrun`, and `task_failure` signals from `celery.signals` + +Framework integrations require some limited code changes in your app. E.g. for Django, you need to add `elasticapm.contrib.django` to `INSTALLED_APPS`. + + +## What if you are not using a framework [how-it-works-no-framework] + +If you’re not using a supported framework, for example, a simple Python script, you can still leverage the agent’s [automatic instrumentation](/reference/supported-technologies.md#automatic-instrumentation). Check out our docs on [instrumenting custom code](/reference/instrumenting-custom-code.md). + + +## Instrumentation [how-it-works-instrumentation] + +To collect data from database drivers, HTTP libraries etc., we instrument certain functions and methods in these libraries. Our instrumentation wraps these callables and collects additional data, like + +* time spent in the call +* the executed query for database drivers +* the fetched URL for HTTP libraries + +We use a 3rd party library, [`wrapt`](https://github.com/GrahamDumpleton/wrapt), to wrap the callables. You can read more on how `wrapt` works in Graham Dumpleton’s excellent series of [blog posts](http://blog.dscpl.com.au/search/label/wrapt). + +Instrumentations are set up automatically and do not require any code changes. See [Automatic Instrumentation](/reference/supported-technologies.md#automatic-instrumentation) to learn more about which libraries we support. + + +## Background collection [how-it-works-background-collection] + +In addition to APM and error data, the Python agent also collects system and application metrics in regular intervals. This collection happens in a background thread that is started by the agent. + +In addition to the metrics collection background thread, the agent starts two additional threads per process: + +* a thread to regularly fetch remote configuration from the APM Server +* a thread to process the collected data and send it to the APM Server via HTTP. + +Note that every process that instantiates the agent will have these three threads. This means that when you e.g. use gunicorn or uwsgi workers, each worker will have three threads started by the Python agent. + diff --git a/docs/reference/index.md b/docs/reference/index.md new file mode 100644 index 000000000..be43e2048 --- /dev/null +++ b/docs/reference/index.md @@ -0,0 +1,28 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/getting-started.html + - https://www.elastic.co/guide/en/apm/agent/python/current/index.html +--- + +# APM Python agent [getting-started] + +The Elastic APM Python agent sends performance metrics and error logs to the APM Server. It has built-in support for Django and Flask performance metrics and error logging, as well as generic support of other WSGI frameworks for error logging. + + +## How does the Agent work? [how-it-works] + +The Python Agent instruments your application to collect APM events in a few different ways: + +To collect data about incoming requests and background tasks, the Agent integrates with [supported technologies](/reference/supported-technologies.md) to make use of hooks and signals provided by the framework. These framework integrations require limited code changes in your application. + +To collect data from database drivers, HTTP libraries etc., we instrument certain functions and methods in these libraries. Instrumentations are set up automatically and do not require any code changes. + +In addition to APM and error data, the Python agent also collects system and application metrics in regular intervals. This collection happens in a background thread that is started by the agent. + +More detailed information on how the Agent works can be found in the [advanced topics](/reference/how-agent-works.md). + + +## Additional components [additional-components] + +APM Agents work in conjunction with the [APM Server](docs-content://solutions/observability/apps/application-performance-monitoring-apm.md), [Elasticsearch](docs-content://get-started/index.md), and [Kibana](docs-content://get-started/the-stack.md). The [APM Guide](docs-content://solutions/observability/apps/application-performance-monitoring-apm.md) provides details on how these components work together, and provides a matrix outlining [Agent and Server compatibility](docs-content://solutions/observability/apps/apm-agent-compatibility.md). + diff --git a/docs/reference/instrumenting-custom-code.md b/docs/reference/instrumenting-custom-code.md new file mode 100644 index 000000000..ad6afc8fb --- /dev/null +++ b/docs/reference/instrumenting-custom-code.md @@ -0,0 +1,118 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/instrumenting-custom-code.html +--- + +# Instrumenting custom code [instrumenting-custom-code] + + +## Creating Additional Spans in a Transaction [instrumenting-custom-code-spans] + +Elastic APM instruments a variety of libraries out of the box, but sometimes you need to know how long a specific function took or how often it gets called. + +Assuming you’re using one of our [supported frameworks](/reference/set-up-apm-python-agent.md), you can apply the `@elasticapm.capture_span()` decorator to achieve exactly that. If you’re not using a supported framework, see [Creating New Transactions](#instrumenting-custom-code-transactions). + +`elasticapm.capture_span` can be used either as a decorator or as a context manager. The following example uses it both ways: + +```python +import elasticapm + +@elasticapm.capture_span() +def coffee_maker(strength): + fetch_water() + + with elasticapm.capture_span('near-to-machine'): + insert_filter() + for i in range(strength): + pour_coffee() + + start_drip() + + fresh_pots() +``` + +Similarly, you can use `elasticapm.async_capture_span` for instrumenting `async` workloads: + +```python +import elasticapm + +@elasticapm.async_capture_span() +async def coffee_maker(strength): + await fetch_water() + + async with elasticapm.async_capture_span('near-to-machine'): + await insert_filter() + async for i in range(strength): + await pour_coffee() + + start_drip() + + fresh_pots() +``` + +::::{note} +`asyncio` support is only available in Python 3.7+. +:::: + + +See [the API docs](/reference/api-reference.md#api-capture-span) for more information on `capture_span`. + + +## Creating New Transactions [instrumenting-custom-code-transactions] + +It’s important to note that `elasticapm.capture_span` only works if there is an existing transaction. If you’re not using one of our [supported frameworks](/reference/set-up-apm-python-agent.md), you need to create a `Client` object and begin and end the transactions yourself. You can even utilize the agent’s [automatic instrumentation](/reference/supported-technologies.md#automatic-instrumentation)! + +To collect the spans generated by the supported libraries, you need to invoke `elasticapm.instrument()` (just once, at the initialization stage of your application) and create at least one transaction. It is up to you to determine what you consider a transaction within your application — it can be the whole execution of the script or a part of it. + +The example below will consider the whole execution as a single transaction with two HTTP request spans in it. The config for `elasticapm.Client` can be passed in programmatically, and it will also utilize any config environment variables available to it automatically. + +```python +import requests +import time +import elasticapm + +def main(): + sess = requests.Session() + for url in [ 'https://www.elastic.co', 'https://benchmarks.elastic.co' ]: + resp = sess.get(url) + time.sleep(1) + +if __name__ == '__main__': + client = elasticapm.Client(service_name="foo", server_url="https://example.com:8200") + elasticapm.instrument() # Only call this once, as early as possible. + client.begin_transaction(transaction_type="script") + main() + client.end_transaction(name=__name__, result="success") +``` + +Note that you don’t need to do anything to send the data — the `Client` object will handle that before the script exits. Additionally, the `Client` object should be treated as a singleton — you should only create one instance and store/pass around that instance for all transaction handling. + + +## Distributed Tracing [instrumenting-custom-code-distributed-tracing] + +When instrumenting custom code across multiple services, you should propagate the TraceParent where possible. This allows Elastic APM to bundle the various transactions into a single distributed trace. The Python Agent will automatically add TraceParent information to the headers of outgoing HTTP requests, which can then be used on the receiving end to add that TraceParent information to new manually-created transactions. + +Additionally, the Python Agent provides utilities for propagating the TraceParent in string format. + +```python +import elasticapm + +client = elasticapm.Client(service_name="foo", server_url="https://example.com:8200") + +# Retrieve the current TraceParent as a string, requires active transaction +traceparent_string = elasticapm.get_trace_parent_header() + +# Create a TraceParent object from a string and use it for a new transaction +parent = elasticapm.trace_parent_from_string(traceparent_string) +client.begin_transaction(transaction_type="script", trace_parent=parent) +# Do some work +client.end_transaction(name=__name__, result="success") + +# Create a TraceParent object from a dictionary of headers, provided +# automatically by the sending service if it is using an Elastic APM Agent. +parent = elasticapm.trace_parent_from_headers(headers_dict) +client.begin_transaction(transaction_type="script", trace_parent=parent) +# Do some work +client.end_transaction(name=__name__, result="success") +``` + diff --git a/docs/reference/lambda-support.md b/docs/reference/lambda-support.md new file mode 100644 index 000000000..168e216d4 --- /dev/null +++ b/docs/reference/lambda-support.md @@ -0,0 +1,262 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/lambda-support.html +sub: + apm-lambda-ext-v: ver-1-5-7 + apm-python-v: ver-6-23-0 +--- + +# Monitoring AWS Lambda Python Functions [lambda-support] + +The Python APM Agent can be used with AWS Lambda to monitor the execution of your AWS Lambda functions. + +:::{note} +The Centralized Agent Configuration on the Elasticsearch APM currently does NOT support AWS Lambda. +::: + + +## Prerequisites [_prerequisites] + +You need an APM Server to send APM data to. Follow the [APM Quick start](docs-content://solutions/observability/apps/application-performance-monitoring-apm.md) if you have not set one up yet. For the best-possible performance, we recommend setting up APM on {{ecloud}} in the same AWS region as your AWS Lambda functions. + +## Step 1: Add the APM Layers to your Lambda function [add_the_apm_layers_to_your_lambda_function] + +Both the [{{apm-lambda-ext}}](apm-aws-lambda://reference/index.md) and the Python APM Agent are added to your Lambda function as [AWS Lambda Layers](https://docs.aws.amazon.com/lambda/latest/dg/invocation-layers.html). Therefore, you need to add the corresponding Layer ARNs (identifiers) to your Lambda function. + +:::::::{tab-set} + +::::::{tab-item} AWS Web Console +To add the layers to your Lambda function through the AWS Management Console: + +1. Navigate to your function in the AWS Management Console +2. Scroll to the Layers section and click the *Add a layer* button ![image of layer configuration section in AWS Console](../images/config-layer.png "") +3. Choose the *Specify an ARN* radio button +4. Copy and paste the following ARNs of the {{apm-lambda-ext}} layer and the APM agent layer in the *Specify an ARN* text input: + * APM Extension layer: + ``` + arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1 <1> + ``` + 1. Replace `{AWS_REGION}` with the AWS region of your Lambda function and `{ARCHITECTURE}` with its architecture. + + * APM agent layer: + ``` + arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1 <1> + ``` + 1. Replace `{AWS_REGION}` with the AWS region of your Lambda function. + + ![image of choosing a layer in AWS Console](../images/choose-a-layer.png "") +5. Click the *Add* button +:::::: + +::::::{tab-item} AWS CLI +To add the Layer ARNs of the {{apm-lambda-ext}} and the APM agent through the AWS command line interface execute the following command: + +```bash +aws lambda update-function-configuration --function-name yourLambdaFunctionName \ +--layers arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1 \ <1> +arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1 <2> +``` +1. Replace `{AWS_REGION}` with the AWS region of your Lambda function and `{ARCHITECTURE}` with its architecture. +2. Replace `{AWS_REGION}` with the AWS region of your Lambda function. +:::::: + +::::::{tab-item} SAM +In your SAM `template.yml` file add the Layer ARNs of the {{apm-lambda-ext}} and the APM agent as follows: + +```yaml +... +Resources: + yourLambdaFunction: + Type: AWS::Serverless::Function + Properties: + ... + Layers: + - arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1 <1> + - arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1 <2> +... +``` +1. Replace `{AWS_REGION}` with the AWS region of your Lambda function and `{ARCHITECTURE}` with its architecture. +2. Replace `{AWS_REGION}` with the AWS region of your Lambda function. +:::::: + +::::::{tab-item} Serverless +In your `serverless.yml` file add the Layer ARNs of the {{apm-lambda-ext}} and the APM agent to your function as follows: + +```yaml +... +functions: + yourLambdaFunction: + handler: ... + layers: + - arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1 <1> + - arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1 <2> +... +``` +1. Replace `{AWS_REGION}` with the AWS region of your Lambda function and `{ARCHITECTURE}` with its architecture. +2. Replace `{AWS_REGION}` with the AWS region of your Lambda function. +:::::: + +::::::{tab-item} Terraform +To add the{{apm-lambda-ext}} and the APM agent to your function add the ARNs to the `layers` property in your Terraform file: + +```yaml +... +resource "aws_lambda_function" "your_lambda_function" { + ... + layers = ["arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-extension-{{apm-lambda-ext-v}}-{ARCHITECTURE}:1", "arn:aws:lambda:{AWS_REGION}:267093732750:layer:elastic-apm-python-{{apm-python-v}}:1"] <1> +} +... +``` +1. Replace `{AWS_REGION}` with the AWS region of your Lambda function and `{ARCHITECTURE}` with its architecture. +:::::: + +::::::{tab-item} Container Image +To add the {{apm-lambda-ext}} and the APM agent to your container-based function extend the Dockerfile of your function image as follows: + +```Dockerfile +FROM docker.elastic.co/observability/apm-lambda-extension-{IMAGE_ARCH}:latest AS lambda-extension <1> +FROM docker.elastic.co/observability/apm-agent-python:latest AS python-agent + +# FROM ... <-- this is the base image of your Lambda function + +COPY --from=lambda-extension /opt/elastic-apm-extension /opt/extensions/elastic-apm-extension +COPY --from=python-agent /opt/python/ /opt/python/ + +# ... +``` +1. Replace `{IMAGE_ARCH}` with the architecture of the image. +:::::: + +::::::: + +## Step 2: Configure APM on AWS Lambda [configure_apm_on_aws_lambda] + +The {{apm-lambda-ext}} and the APM Python agent are configured through environment variables on the AWS Lambda function. + +For the minimal configuration, you will need the *APM Server URL* to set the destination for APM data and an [APM Secret Token](docs-content://solutions/observability/apps/secret-token.md). If you prefer to use an [APM API key](docs-content://solutions/observability/apps/api-keys.md) instead of the APM secret token, use the `ELASTIC_APM_API_KEY` environment variable instead of `ELASTIC_APM_SECRET_TOKEN` in the following configuration. + +For production environments, we recommend [using the AWS Secrets Manager to store your APM authentication key](apm-aws-lambda://reference/aws-lambda-secrets-manager.md) instead of providing the secret value as plaintext in the environment variables. + +:::::::{tab-set} + +::::::{tab-item} AWS Web Console +To configure APM through the AWS Management Console: + +1. Navigate to your function in the AWS Management Console +2. Click on the *Configuration* tab +3. Click on *Environment variables* +4. Add the following required variables: + +```bash +AWS_LAMBDA_EXEC_WRAPPER = /opt/python/bin/elasticapm-lambda <1> +ELASTIC_APM_LAMBDA_APM_SERVER = <2> +ELASTIC_APM_SECRET_TOKEN = <3> +ELASTIC_APM_SEND_STRATEGY = background <4> +``` + +1. Use this exact fixed value. +2. This is your APM Server URL. +3. This is your APM secret token. +4. The [ELASTIC_APM_SEND_STRATEGY](apm-aws-lambda://reference/aws-lambda-config-options.md#_elastic_apm_send_strategy) defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios. + +![Python environment variables configuration section in AWS Console](../images/python-lambda-env-vars.png "") +:::::: + +::::::{tab-item} AWS CLI +To configure APM through the AWS command line interface execute the following command: + +```bash +aws lambda update-function-configuration --function-name yourLambdaFunctionName \ + --environment "Variables={AWS_LAMBDA_EXEC_WRAPPER=/opt/python/bin/elasticapm-lambda,ELASTIC_APM_LAMBDA_APM_SERVER=,ELASTIC_APM_SECRET_TOKEN=,ELASTIC_APM_SEND_STRATEGY=background}" +``` +:::::: + +::::::{tab-item} SAM +In your SAM `template.yml` file configure the following environment variables: + +```yaml +... +Resources: + yourLambdaFunction: + Type: AWS::Serverless::Function + Properties: + ... + Environment: + Variables: + AWS_LAMBDA_EXEC_WRAPPER: /opt/python/bin/elasticapm-lambda + ELASTIC_APM_LAMBDA_APM_SERVER: + ELASTIC_APM_SECRET_TOKEN: + ELASTIC_APM_SEND_STRATEGY: background <1> +... +``` + +1. The [ELASTIC_APM_SEND_STRATEGY](apm-aws-lambda://reference/aws-lambda-config-options.md#_elastic_apm_send_strategy) defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios. + +:::::: + +::::::{tab-item} Serverless +In your `serverless.yml` file configure the following environment variables: + +```yaml +... +functions: + yourLambdaFunction: + ... + environment: + AWS_LAMBDA_EXEC_WRAPPER: /opt/python/bin/elasticapm-lambda + ELASTIC_APM_LAMBDA_APM_SERVER: + ELASTIC_APM_SECRET_TOKEN: + ELASTIC_APM_SEND_STRATEGY: background <1> +... +``` + +1. The [ELASTIC_APM_SEND_STRATEGY](apm-aws-lambda://reference/aws-lambda-config-options.md#_elastic_apm_send_strategy) defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios. + +:::::: + +::::::{tab-item} Terraform +In your Terraform file configure the following environment variables: + +```yaml +... +resource "aws_lambda_function" "your_lambda_function" { + ... + environment { + variables = { + AWS_LAMBDA_EXEC_WRAPPER = /opt/python/bin/elasticapm-lambda + ELASTIC_APM_LAMBDA_APM_SERVER = "" + ELASTIC_APM_SECRET_TOKEN = "" + ELASTIC_APM_SEND_STRATEGY = "background" <1> + } + } +} +... +``` + +1. The [ELASTIC_APM_SEND_STRATEGY](apm-aws-lambda://reference/aws-lambda-config-options.md#_elastic_apm_send_strategy) defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios. + +:::::: + +::::::{tab-item} Container Image +Environment variables configured for an AWS Lambda function are passed to the container running the lambda function. You can use one of the other options (through AWS Web Console, AWS CLI, etc.) to configure the following environment variables: + +```bash +AWS_LAMBDA_EXEC_WRAPPER = /opt/python/bin/elasticapm-lambda <1> +ELASTIC_APM_LAMBDA_APM_SERVER = <2> +ELASTIC_APM_SECRET_TOKEN = <3> +ELASTIC_APM_SEND_STRATEGY = background <4> +``` + +1. Use this exact fixed value. +2. This is your APM Server URL. +3. This is your APM secret token. +4. The [ELASTIC_APM_SEND_STRATEGY](apm-aws-lambda://reference/aws-lambda-config-options.md#_elastic_apm_send_strategy) defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the background strategy in production environments with steady load scenarios. + +:::::: + +::::::: + +You can optionally [fine-tune the Python agent](/reference/configuration.md) or the [configuration of the {{apm-lambda-ext}}](apm-aws-lambda://reference/aws-lambda-config-options.md). + +That’s it. After following the steps above, you’re ready to go! Your Lambda function invocations should be traced from now on. Spans will be captured for [supported technologies](/reference/supported-technologies.md). You can also use [`capture_span`](/reference/api-reference.md#api-capture-span) to capture custom spans, and you can retrieve the `Client` object for capturing exceptions/messages using [`get_client`](/reference/api-reference.md#api-get-client). + diff --git a/docs/reference/logs.md b/docs/reference/logs.md new file mode 100644 index 000000000..dd9335ca2 --- /dev/null +++ b/docs/reference/logs.md @@ -0,0 +1,141 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/logs.html +--- + +# Logs [logs] + +Elastic Python APM Agent provides the following log features: + +* [Log correlation](#log-correlation-ids) : Automatically inject correlation IDs that allow navigation between logs, traces and services. +* [Log reformatting (experimental)](#log-reformatting) : Automatically reformat plaintext logs in [ECS logging](ecs-logging://reference/intro.md) format. + +::::{note} +Elastic Python APM Agent does not send the logs to Elasticsearch. It only injects correlation IDs and reformats the logs. You must use another ingestion strategy. We recommend [Filebeat](https://www.elastic.co/beats/filebeat) for that purpose. +:::: + + +Those features are part of [Application log ingestion strategies](docs-content://solutions/observability/logs/stream-application-logs.md). + +The [`ecs-logging-python`](ecs-logging-python://reference/index.md) library can also be used to use the [ECS logging](ecs-logging://reference/intro.md) format without an APM agent. When deployed with the Python APM agent, the agent will provide [log correlation](#log-correlation-ids) IDs. + + +## Log correlation [log-correlation-ids] + +[Log correlation](docs-content://solutions/observability/apps/logs.md) allows you to navigate to all logs belonging to a particular trace and vice-versa: for a specific log, see in which context it has been logged and which parameters the user provided. + +The Agent provides integrations with both the default Python logging library, as well as [`structlog`](http://www.structlog.org/en/stable/). + +* [Logging integrations](#logging-integrations) +* [Log correlation in Elasticsearch](#log-correlation-in-es) + + +### Logging integrations [logging-integrations] + + +#### `logging` [logging] + +We use [`logging.setLogRecordFactory()`](https://docs.python.org/3/library/logging.html#logging.setLogRecordFactory) to decorate the default LogRecordFactory to automatically add new attributes to each LogRecord object: + +* `elasticapm_transaction_id` +* `elasticapm_trace_id` +* `elasticapm_span_id` + +This factory also adds these fields to a dictionary attribute, `elasticapm_labels`, using the official ECS [tracing fields](ecs://reference/ecs-tracing.md). + +You can disable this automatic behavior by using the [`disable_log_record_factory`](/reference/configuration.md#config-generic-disable-log-record-factory) setting in your configuration. + + +#### `structlog` [structlog] + +We provide a [processor](http://www.structlog.org/en/stable/processors.html) for [`structlog`](http://www.structlog.org/en/stable/) which will add three new keys to the event_dict of any processed event: + +* `transaction.id` +* `trace.id` +* `span.id` + +```python +from structlog import PrintLogger, wrap_logger +from structlog.processors import JSONRenderer +from elasticapm.handlers.structlog import structlog_processor + +wrapped_logger = PrintLogger() +logger = wrap_logger(wrapped_logger, processors=[structlog_processor, JSONRenderer()]) +log = logger.new() +log.msg("some_event") +``` + + +#### Use structlog for agent-internal logging [_use_structlog_for_agent_internal_logging] + +The Elastic APM Python agent uses logging to log internal events and issues. By default, it will use a `logging` logger. If your project uses structlog, you can tell the agent to use a structlog logger by setting the environment variable `ELASTIC_APM_USE_STRUCTLOG` to `true`. + + +## Log correlation in Elasticsearch [log-correlation-in-es] + +In order to correlate logs from your app with transactions captured by the Elastic APM Python Agent, your logs must contain one or more of the following identifiers: + +* `transaction.id` +* `trace.id` +* `span.id` + +If you’re using structured logging, either [with a custom solution](https://docs.python.org/3/howto/logging-cookbook.html#implementing-structured-logging) or with [structlog](http://www.structlog.org/en/stable/) (recommended), then this is fairly easy. Throw the [JSONRenderer](http://www.structlog.org/en/stable/api.html#structlog.processors.JSONRenderer) in, and use [Filebeat](https://www.elastic.co/blog/structured-logging-filebeat) to pull these logs into Elasticsearch. + +Without structured logging the task gets a little trickier. Here we recommend first making sure your LogRecord objects have the elasticapm attributes (see [`logging`](#logging)), and then you’ll want to combine some specific formatting with a Grok pattern, either in Elasticsearch using [the grok processor](elasticsearch://reference/ingestion-tools/enrich-processor/grok-processor.md), or in [logstash with a plugin](logstash://reference/plugins-filters-grok.md). + +Say you have a [Formatter](https://docs.python.org/3/library/logging.html#logging.Formatter) that looks like this: + +```python +import logging + +fh = logging.FileHandler('spam.log') +formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") +fh.setFormatter(formatter) +``` + +You can add the APM identifiers by simply switching out the `Formatter` object for the one that we provide: + +```python +import logging +from elasticapm.handlers.logging import Formatter + +fh = logging.FileHandler('spam.log') +formatter = Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") +fh.setFormatter(formatter) +``` + +This will automatically append apm-specific fields to your format string: + +```python +formatstring = "%(asctime)s - %(name)s - %(levelname)s - %(message)s" +formatstring = formatstring + " | elasticapm " \ + "transaction.id=%(elasticapm_transaction_id)s " \ + "trace.id=%(elasticapm_trace_id)s " \ + "span.id=%(elasticapm_span_id)s" +``` + +Then, you could use a grok pattern like this (for the [Elasticsearch Grok Processor](elasticsearch://reference/ingestion-tools/enrich-processor/grok-processor.md)): + +```json +{ + "description" : "...", + "processors": [ + { + "grok": { + "field": "message", + "patterns": ["%{GREEDYDATA:msg} | elasticapm transaction.id=%{DATA:transaction.id} trace.id=%{DATA:trace.id} span.id=%{DATA:span.id}"] + } + } + ] +} +``` + + +## Log reformatting (experimental) [log-reformatting] + +Starting in version 6.16.0, the agent can automatically reformat application logs to ECS format with no changes to dependencies. Prior versions must install the `ecs_logging` dependency. + +Log reformatting is controlled by the [`log_ecs_reformatting`](/reference/configuration.md#config-log_ecs_reformatting) configuration option, and is disabled by default. + +The reformatted logs will include both the [trace and service correlation](#log-correlation-ids) IDs. + diff --git a/docs/reference/metrics.md b/docs/reference/metrics.md new file mode 100644 index 000000000..4f1ff2b27 --- /dev/null +++ b/docs/reference/metrics.md @@ -0,0 +1,185 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/metrics.html +--- + +# Metrics [metrics] + +With Elastic APM, you can capture system and process metrics. These metrics will be sent regularly to the APM Server and from there to Elasticsearch + + +## Metric sets [metric-sets] + +* [CPU/Memory metric set](#cpu-memory-metricset) +* [Breakdown metric set](#breakdown-metricset) +* [Prometheus metric set (beta)](#prometheus-metricset) +* [Custom Metrics](#custom-metrics) + + +### CPU/Memory metric set [cpu-memory-metricset] + +`elasticapm.metrics.sets.cpu.CPUMetricSet` + +This metric set collects various system metrics and metrics of the current process. + +::::{note} +if you do **not** use Linux, you need to install [`psutil`](https://pypi.org/project/psutil/) for this metric set. +:::: + + +**`system.cpu.total.norm.pct`** +: type: scaled_float + +format: percent + +The percentage of CPU time in states other than Idle and IOWait, normalized by the number of cores. + + +**`system.process.cpu.total.norm.pct`** +: type: scaled_float + +format: percent + +The percentage of CPU time spent by the process since the last event. This value is normalized by the number of CPU cores and it ranges from 0 to 100%. + + +**`system.memory.total`** +: type: long + +format: bytes + +Total memory. + + +**`system.memory.actual.free`** +: type: long + +format: bytes + +Actual free memory in bytes. + + +**`system.process.memory.size`** +: type: long + +format: bytes + +The total virtual memory the process has. + + +**`system.process.memory.rss.bytes`** +: type: long + +format: bytes + +The Resident Set Size. The amount of memory the process occupied in main memory (RAM). + + + +#### Linux’s cgroup metrics [cpu-memory-cgroup-metricset] + +**`system.process.cgroup.memory.mem.limit.bytes`** +: type: long + +format: bytes + +Memory limit for current cgroup slice. + + +**`system.process.cgroup.memory.mem.usage.bytes`** +: type: long + +format: bytes + +Memory usage in current cgroup slice. + + + +### Breakdown metric set [breakdown-metricset] + +::::{note} +Tracking and collection of this metric set can be disabled using the [`breakdown_metrics`](/reference/configuration.md#config-breakdown_metrics) setting. +:::: + + +**`span.self_time`** +: type: simple timer + +This timer tracks the span self-times and is the basis of the transaction breakdown visualization. + +Fields: + +* `sum`: The sum of all span self-times in ms since the last report (the delta) +* `count`: The count of all span self-times since the last report (the delta) + +You can filter and group by these dimensions: + +* `transaction.name`: The name of the transaction +* `transaction.type`: The type of the transaction, for example `request` +* `span.type`: The type of the span, for example `app`, `template` or `db` +* `span.subtype`: The sub-type of the span, for example `mysql` (optional) + + + +### Prometheus metric set (beta) [prometheus-metricset] + +::::{warning} +This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features. +:::: + + +If you use [`prometheus_client`](https://github.com/prometheus/client_python) to collect metrics, the agent can collect them as well and make them available in Elasticsearch. + +The following types of metrics are supported: + +* Counters +* Gauges +* Summaries +* Histograms (requires APM Server / Elasticsearch / Kibana 7.14+) + +To use the Prometheus metric set, you have to enable it with the [`prometheus_metrics`](/reference/configuration.md#config-prometheus_metrics) configuration option. + +All metrics collected from `prometheus_client` are prefixed with `"prometheus.metrics."`. This can be changed using the [`prometheus_metrics_prefix`](/reference/configuration.md#config-prometheus_metrics_prefix) configuration option. + + +#### Beta limitations [prometheus-metricset-beta] + +* The metrics format may change without backwards compatibility in future releases. + + +## Custom Metrics [custom-metrics] + +Custom metrics allow you to send your own metrics to Elasticsearch. + +The most common way to send custom metrics is with the [Prometheus metric set](#prometheus-metricset). However, you can also use your own metric set. If you collect the metrics manually in your code, you can use the base `MetricSet` class: + +```python +from elasticapm.metrics.base_metrics import MetricSet + +client = elasticapm.Client() +metricset = client.metrics.register(MetricSet) + +for x in range(10): + metricset.counter("my_counter").inc() +``` + +Alternatively, you can create your own MetricSet class which inherits from the base class. In this case, you’ll usually want to override the `before_collect` method, where you can gather and set metrics before they are collected and sent to Elasticsearch. + +You can add your `MetricSet` class as shown in the example above, or you can add an import string for your class to the [`metrics_sets`](/reference/configuration.md#config-metrics_sets) configuration option: + +```bash +ELASTIC_APM_METRICS_SETS="elasticapm.metrics.sets.cpu.CPUMetricSet,myapp.metrics.MyMetricSet" +``` + +Your MetricSet might look something like this: + +```python +from elasticapm.metrics.base_metrics import MetricSet + +class MyAwesomeMetricSet(MetricSet): + def before_collect(self): + self.gauge("my_gauge").set(myapp.some_value) +``` + +In the example above, the MetricSet would look up `myapp.some_value` and set the metric `my_gauge` to that value. This would happen whenever metrics are collected/sent, which is controlled by the [`metrics_interval`](/reference/configuration.md#config-metrics_interval) setting. + diff --git a/docs/reference/opentelemetry-api-bridge.md b/docs/reference/opentelemetry-api-bridge.md new file mode 100644 index 000000000..7564f85fa --- /dev/null +++ b/docs/reference/opentelemetry-api-bridge.md @@ -0,0 +1,63 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/opentelemetry-bridge.html +--- + +# OpenTelemetry API Bridge [opentelemetry-bridge] + +The Elastic APM OpenTelemetry bridge allows you to create Elastic APM `Transactions` and `Spans`, using the OpenTelemetry API. This allows users to utilize the Elastic APM agent’s automatic instrumentations, while keeping custom instrumentations vendor neutral. + +If a span is created while there is no transaction active, it will result in an Elastic APM [`Transaction`](docs-content://solutions/observability/apps/transactions.md). Inner spans are mapped to Elastic APM [`Span`](docs-content://solutions/observability/apps/spans.md). + + +## Getting started [opentelemetry-getting-started] + +The first step in getting started with the OpenTelemetry bridge is to install the `opentelemetry` libraries: + +```bash +pip install elastic-apm[opentelemetry] +``` + +Or if you already have installed `elastic-apm`: + +```bash +pip install opentelemetry-api opentelemetry-sdk +``` + + +## Usage [opentelemetry-usage] + +```python +from elasticapm.contrib.opentelemetry import Tracer + +tracer = Tracer(__name__) +with tracer.start_as_current_span("test"): + # Do some work +``` + +or + +```python +from elasticapm.contrib.opentelemetry import trace + +tracer = trace.get_tracer(__name__) +with tracer.start_as_current_span("test"): + # Do some work +``` + +`Tracer` and `get_tracer()` accept the following optional arguments: + +* `elasticapm_client`: an already instantiated Elastic APM client +* `config`: a configuration dictionary, which will be used to instantiate a new Elastic APM client, e.g. `{"SERVER_URL": "https://example.org"}`. See [configuration](/reference/configuration.md) for more information. + +The `Tracer` object mirrors the upstream interface on the [OpenTelemetry `Tracer` object.](https://opentelemetry-python.readthedocs.io/en/latest/api/trace.html#opentelemetry.trace.Tracer) + + +## Caveats [opentelemetry-caveats] + +Not all features of the OpenTelemetry API are supported. + +Processors, exporters, metrics, logs, span events, and span links are not supported. + +Additionally, due to implementation details, the global context API only works when a span is included in the activated context, and tokens are not used. Instead, the global context works as a stack, and when a context is detached the previously-active context will automatically be activated. + diff --git a/docs/reference/performance-tuning.md b/docs/reference/performance-tuning.md new file mode 100644 index 000000000..04ce10e64 --- /dev/null +++ b/docs/reference/performance-tuning.md @@ -0,0 +1,86 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/tuning-and-overhead.html +--- + +# Performance tuning [tuning-and-overhead] + +Using an APM solution comes with certain trade-offs, and the Python agent for Elastic APM is no different. Instrumenting your code, measuring timings, recording context data, etc., all need resources: + +* CPU time +* memory +* bandwidth use +* Elasticsearch storage + +We invested and continue to invest a lot of effort to keep the overhead of using Elastic APM as low as possible. But because every deployment is different, there are some knobs you can turn to adapt it to your specific needs. + + +## Transaction Sample Rate [tuning-sample-rate] + +The easiest way to reduce the overhead of the agent is to tell the agent to do less. If you set the [`transaction_sample_rate`](/reference/configuration.md#config-transaction-sample-rate) to a value below `1.0`, the agent will randomly sample only a subset of transactions. Unsampled transactions only record the name of the transaction, the overall transaction time, and the result: + +| Field | Sampled | Unsampled | +| --- | --- | --- | +| Transaction name | yes | yes | +| Duration | yes | yes | +| Result | yes | yes | +| Context | yes | no | +| Tags | yes | no | +| Spans | yes | no | + +Reducing the sample rate to a fraction of all transactions can make a huge difference in all four of the mentioned resource types. + + +## Transaction Queue [tuning-queue] + +To reduce the load on the APM Server, the agent does not send every transaction up as it happens. Instead, it queues them up and flushes the queue periodically, or when it reaches a maximum size, using a background thread. + +While this reduces the load on the APM Server (and to a certain extent on the agent), holding on to the transaction data in a queue uses memory. If you notice that using the Python agent results in a large increase of memory use, you can use these settings: + +* [`api_request_time`](/reference/configuration.md#config-api-request-time) to reduce the time between queue flushes +* [`api_request_size`](/reference/configuration.md#config-api-request-size) to reduce the maximum size of the queue + +The first setting, `api_request_time`, is helpful if you have a sustained high number of transactions. The second setting, `api_request_size`, can help if you experience peaks of transactions (a large number of transactions in a short period of time). + +Keep in mind that reducing the value of either setting will cause the agent to send more HTTP requests to the APM Server, potentially causing a higher load. + + +## Spans per transaction [tuning-max-spans] + +The average amount of spans per transaction can influence how much time the agent spends in each transaction collecting contextual data for each span, and the storage space needed in Elasticsearch. In our experience, most *usual* transactions should have well below 100 spans. In some cases, however, the number of spans can explode: + +* long-running transactions +* unoptimized code, e.g. doing hundreds of SQL queries in a loop + +To avoid these edge cases overloading both the agent and the APM Server, the agent stops recording spans when a specified limit is reached. You can configure this limit by changing the [`transaction_max_spans`](/reference/configuration.md#config-transaction-max-spans) setting. + + +## Span Stack Trace Collection [tuning-span-stack-trace-collection] + +Collecting stack traces for spans can be fairly costly from a performance standpoint. Stack traces are very useful for pinpointing which part of your code is generating a span; however, these stack traces are less useful for very short spans (as problematic spans tend to be longer). + +You can define a minimal threshold for span duration using the [`span_stack_trace_min_duration`](/reference/configuration.md#config-span-stack-trace-min-duration) setting. If a span’s duration is less than this config value, no stack frames will be collected for this span. + + +## Collecting Frame Context [tuning-frame-context] + +When a stack trace is captured, the agent will also capture several lines of source code around each frame location in the stack trace. This allows the APM app to give greater insight into where exactly the error or span happens. + +There are four settings you can modify to control this behavior: + +* [`source_lines_error_app_frames`](/reference/configuration.md#config-source-lines-error-app-frames) +* [`source_lines_error_library_frames`](/reference/configuration.md#config-source-lines-error-library-frames) +* [`source_lines_span_app_frames`](/reference/configuration.md#config-source-lines-span-app-frames) +* [`source_lines_span_library_frames`](/reference/configuration.md#config-source-lines-span-library-frames) + +As you can see, these settings are divided between app frames, which represent your application code, and library frames, which represent the code of your dependencies. Each of these categories are also split into separate error and span settings. + +Reading source files inside a running application can cause a lot of disk I/O, and sending up source lines for each frame will have a network and storage cost that is quite high. Turning down these limits will help prevent excessive memory usage. + + +## Collecting headers and request body [tuning-body-headers] + +You can configure the Elastic APM agent to capture headers of both requests and responses ([`capture_headers`](/reference/configuration.md#config-capture-headers)), as well as request bodies ([`capture_body`](/reference/configuration.md#config-capture-body)). By default, capturing request bodies is disabled. Enabling it for transactions may introduce noticeable overhead, as well as increased storage use, depending on the nature of your POST requests. In most scenarios, we advise against enabling request body capturing for transactions, and only enable it if necessary for errors. + +Capturing request/response headers has less overhead on the agent, but can have an impact on storage use. If storage use is a problem for you, it might be worth disabling. + diff --git a/docs/reference/run-tests-locally.md b/docs/reference/run-tests-locally.md new file mode 100644 index 000000000..f72432d7e --- /dev/null +++ b/docs/reference/run-tests-locally.md @@ -0,0 +1,72 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/run-tests-locally.html +--- + +# Run Tests Locally [run-tests-locally] + +To run tests locally you can make use of the docker images also used when running the whole test suite with Jenkins. Running the full test suite first does some linting and then runs the actual tests with different versions of Python and different web frameworks. For a full overview of the test matrix and supported versions have a look at [Jenkins Configuration](https://github.com/elastic/apm-agent-python/blob/main/Jenkinsfile). + + +### Pre Commit [pre-commit] + +We run our git hooks on every commit to automatically point out issues in code. Those issues are also detected within the GitHub actions. Please follow the installation steps stated in [https://pre-commit.com/#install](https://pre-commit.com/#install). + + +### Code Linter [coder-linter] + +We run two code linters `isort` and `flake8`. You can trigger each single one locally by running: + +```bash +$ pre-commit run -a isort +``` + +```bash +$ pre-commit run -a flake8 +``` + + +### Code Formatter [coder-formatter] + +We test that the code is formatted using `black`. You can trigger this check by running: + +```bash +$ pre-commit run -a black +``` + + +### Test Documentation [test-documentation] + +We test that the documentation can be generated without errors. You can trigger this check by running: + +```bash +$ ./tests/scripts/docker/docs.sh +``` + + +### Running Tests [running-tests] + +We run the test suite on different combinations of Python versions and web frameworks. For triggering the test suite for a specific combination locally you can run: + +```bash +$ ./tests/scripts/docker/run_tests.sh python-version framework-version +``` + +::::{note} +The `python-version` must be of format `python-version`, e.g. `python-3.6` or `pypy-2`. The `framework` must be of format `framework-version`, e.g. `django-1.10` or `flask-0.12`. +:::: + + +You can also run the unit tests outside of docker, by installing the relevant [requirements file](https://github.com/elastic/apm-agent-python/tree/main/tests/requirements) and then running `py.test` from the project root. + +## Integration testing [_integration_testing] + +Check out [https://github.com/elastic/apm-integration-testing](https://github.com/elastic/apm-integration-testing) for resources for setting up full end-to-end testing environments. For example, to spin up an environment with the [opbeans Django app](https://github.com/basepi/opbeans-python), with version 7.3 of the elastic stack and the apm-python-agent from your local checkout, you might do something like this: + +```bash +$ ./scripts/compose.py start 7.3 \ + --with-agent-python-django --with-opbeans-python \ + --opbeans-python-agent-local-repo=~/elastic/apm-agent-python +``` + + diff --git a/docs/reference/sanic-support.md b/docs/reference/sanic-support.md new file mode 100644 index 000000000..515bf6837 --- /dev/null +++ b/docs/reference/sanic-support.md @@ -0,0 +1,140 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/sanic-support.html +--- + +# Sanic Support [sanic-support] + +Incorporating Elastic APM into your Sanic project only requires a few easy steps. + + +## Installation [sanic-installation] + +Install the Elastic APM agent using pip: + +```bash +$ pip install elastic-apm +``` + +or add `elastic-apm` to your project’s `requirements.txt` file. + + +## Setup [sanic-setup] + +To set up the agent, you need to initialize it with appropriate settings. + +The settings are configured either via environment variables, or as initialization arguments. + +You can find a list of all available settings in the [Configuration](/reference/configuration.md) page. + +To initialize the agent for your application using environment variables: + +```python +from sanic import Sanic +from elasticapm.contrib.sanic import ElasticAPM + +app = Sanic(name="elastic-apm-sample") +apm = ElasticAPM(app=app) +``` + +To configure the agent using initialization arguments and Sanic’s Configuration infrastructure: + +```python +# Create a file named external_config.py in your application +# If you want this module based configuration to be used for APM, prefix them with ELASTIC_APM_ +ELASTIC_APM_SERVER_URL = "https://serverurl.apm.com:443" +ELASTIC_APM_SECRET_TOKEN = "sometoken" +``` + +```python +from sanic import Sanic +from elasticapm.contrib.sanic import ElasticAPM + +app = Sanic(name="elastic-apm-sample") +app.config.update_config("path/to/external_config.py") +apm = ElasticAPM(app=app) +``` + + +## Usage [sanic-usage] + +Once you have configured the agent, it will automatically track transactions and capture uncaught exceptions within sanic. + +Capture an arbitrary exception by calling [`capture_exception`](/reference/api-reference.md#client-api-capture-exception): + +```python +from sanic import Sanic +from elasticapm.contrib.sanic import ElasticAPM + +app = Sanic(name="elastic-apm-sample") +apm = ElasticAPM(app=app) + +try: + 1 / 0 +except ZeroDivisionError: + apm.capture_exception() +``` + +Log a generic message with [`capture_message`](/reference/api-reference.md#client-api-capture-message): + +```python +from sanic import Sanic +from elasticapm.contrib.sanic import ElasticAPM + +app = Sanic(name="elastic-apm-sample") +apm = ElasticAPM(app=app) + +apm.capture_message('hello, world!') +``` + + +## Performance metrics [sanic-performance-metrics] + +If you’ve followed the instructions above, the agent has installed our instrumentation middleware which will process all requests through your app. This will measure response times, as well as detailed performance data for all supported technologies. + +::::{note} +Due to the fact that `asyncio` drivers are usually separate from their synchronous counterparts, specific instrumentation is needed for all drivers. The support for asynchronous drivers is currently quite limited. +:::: + + + +### Ignoring specific routes [sanic-ignoring-specific-views] + +You can use the [`TRANSACTIONS_IGNORE_PATTERNS`](/reference/configuration.md#config-transactions-ignore-patterns) configuration option to ignore specific routes. The list given should be a list of regular expressions which are matched against the transaction name: + +```python +from sanic import Sanic +from elasticapm.contrib.sanic import ElasticAPM + +app = Sanic(name="elastic-apm-sample") +apm = ElasticAPM(app=app, config={ + 'TRANSACTIONS_IGNORE_PATTERNS': ['^GET /secret', '/extra_secret'], +}) +``` + +This would ignore any requests using the `GET /secret` route and any requests containing `/extra_secret`. + + +## Extended Sanic APM Client Usage [extended-sanic-usage] + +Sanic’s contributed APM client also provides a few extendable way to configure selective behaviors to enhance the information collected as part of the transactions being tracked by the APM. + +In order to enable this behavior, the APM Client middleware provides a few callback functions that you can leverage in order to simplify the process of generating additional contexts into the traces being collected. + +| Callback Name | Callback Invocation Format | Expected Return Format | Is Async | +| --- | --- | --- | --- | +| transaction_name_callback | transaction_name_callback(request) | string | false | +| user_context_callback | user_context_callback(request) | (username_string, user_email_string, userid_string) | true | +| custom_context_callback | custom_context_callback(request) or custom_context_callback(response) | dict(str=str) | true | +| label_info_callback | label_info_callback() | dict(str=str) | true | + + +## Supported Sanic and Python versions [supported-stanic-and-python-versions] + +A list of supported [Sanic](/reference/supported-technologies.md#supported-sanic) and [Python](/reference/supported-technologies.md#supported-python) versions can be found on our [Supported Technologies](/reference/supported-technologies.md) page. + +::::{note} +Elastic APM only supports `asyncio` when using Python 3.7+ +:::: + + diff --git a/docs/sanitizing-data.asciidoc b/docs/reference/sanitizing-data.md similarity index 72% rename from docs/sanitizing-data.asciidoc rename to docs/reference/sanitizing-data.md index 4daa9eb8f..b41d89148 100644 --- a/docs/sanitizing-data.asciidoc +++ b/docs/reference/sanitizing-data.md @@ -1,22 +1,21 @@ -[[sanitizing-data]] -=== Sanitizing data +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/sanitizing-data.html +--- -Sometimes it is necessary to sanitize the data sent to Elastic APM, -e.g. remove sensitive data. +# Sanitizing data [sanitizing-data] -To do this with the Elastic APM module, you create a processor. -A processor is a function that takes a `client` instance as well as an event (an error, a transaction, a span, or a metricset), -and returns the modified event. +Sometimes it is necessary to sanitize the data sent to Elastic APM, e.g. remove sensitive data. + +To do this with the Elastic APM module, you create a processor. A processor is a function that takes a `client` instance as well as an event (an error, a transaction, a span, or a metricset), and returns the modified event. To completely drop an event, your processor should return `False` (or any other "falsy" value) instead of the event. -An event will also be dropped if any processor raises an exception while processing it. -A log message with level `WARNING` will be issued in this case. +An event will also be dropped if any processor raises an exception while processing it. A log message with level `WARNING` will be issued in this case. This is an example of a processor that removes the exception stacktrace from an error: -[source,python] ----- +```python from elasticapm.conf.constants import ERROR from elasticapm.processors import for_events @@ -25,16 +24,13 @@ def my_processor(client, event): if 'exception' in event and 'stacktrace' in event['exception']: event['exception'].pop('stacktrace') return event ----- +``` -You can use the `@for_events` decorator to limit for which event type the processor should be called. -Possible choices are `ERROR`, `TRANSACTION`, `SPAN` and `METRICSET`, -all of which are defined in `elasticapm.conf.constants`. +You can use the `@for_events` decorator to limit for which event type the processor should be called. Possible choices are `ERROR`, `TRANSACTION`, `SPAN` and `METRICSET`, all of which are defined in `elasticapm.conf.constants`. To use this processor, update your `ELASTIC_APM` settings like this: -[source,python] ----- +```python ELASTIC_APM = { 'SERVICE_NAME': '', 'SECRET_TOKEN': '', @@ -47,14 +43,16 @@ ELASTIC_APM = { 'elasticapm.processors.sanitize_http_request_body', ), } ----- +``` + +::::{note} +We recommend using the above list of processors that sanitize passwords and secrets in different places of the event object. +:::: -NOTE: We recommend using the above list of processors that sanitize passwords and secrets in different places of the event object. The default set of processors sanitize fields based on a set of defaults defined in `elasticapm.conf.constants`. This set can be configured with the `SANITIZE_FIELD_NAMES` configuration option. For example, if your application produces a sensitive field called `My-Sensitive-Field`, the default processors can be used to automatically sanitize this field. You can specify what fields to santize within default processors like this: -[source,python] ----- +```python ELASTIC_APM = { 'SERVICE_NAME': '', 'SECRET_TOKEN': '', @@ -72,8 +70,12 @@ ELASTIC_APM = { "set-cookie", ), } ----- +``` + +::::{note} +We recommend to use the above list of fields to sanitize various parts of the event object in addition to your specified fields. +:::: -NOTE: We recommend to use the above list of fields to sanitize various parts of the event object in addition to your specified fields. When choosing fields names to sanitize, you can specify values that will match certain wildcards. For example, passing `base` as a field name to be sanitized will also sanitize all fields whose names match the regex pattern `\*base*`. + diff --git a/docs/reference/set-up-apm-python-agent.md b/docs/reference/set-up-apm-python-agent.md new file mode 100644 index 000000000..a74e45208 --- /dev/null +++ b/docs/reference/set-up-apm-python-agent.md @@ -0,0 +1,32 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/set-up.html +--- + +# Set up the APM Python Agent [set-up] + +To get you off the ground, we’ve prepared guides for setting up the Agent with different frameworks: + +* [Django](/reference/django-support.md) +* [Flask](/reference/flask-support.md) +* [aiohttp](/reference/aiohttp-server-support.md) +* [Tornado](/reference/tornado-support.md) +* [Starlette/FastAPI](/reference/starlette-support.md) +* [Sanic](/reference/sanic-support.md) +* [AWS Lambda](/reference/lambda-support.md) +* [Azure Functions](/reference/azure-functions-support.md) +* [Wrapper (Experimental)](/reference/wrapper-support.md) +* [ASGI Middleware](/reference/asgi-middleware.md) + +For custom instrumentation, see [Instrumenting Custom Code](/reference/instrumenting-custom-code.md). + + + + + + + + + + + diff --git a/docs/reference/starlette-support.md b/docs/reference/starlette-support.md new file mode 100644 index 000000000..673ab79db --- /dev/null +++ b/docs/reference/starlette-support.md @@ -0,0 +1,127 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/starlette-support.html +--- + +# Starlette/FastAPI Support [starlette-support] + +Incorporating Elastic APM into your Starlette project only requires a few easy steps. + + +## Installation [starlette-installation] + +Install the Elastic APM agent using pip: + +```bash +$ pip install elastic-apm +``` + +or add `elastic-apm` to your project’s `requirements.txt` file. + + +## Setup [starlette-setup] + +To set up the agent, you need to initialize it with appropriate settings. + +The settings are configured either via environment variables, or as initialization arguments. + +You can find a list of all available settings in the [Configuration](/reference/configuration.md) page. + +To initialize the agent for your application using environment variables, add the ElasticAPM middleware to your Starlette application: + +```python +from starlette.applications import Starlette +from elasticapm.contrib.starlette import ElasticAPM + +app = Starlette() +app.add_middleware(ElasticAPM) +``` + +::::{warning} +`BaseHTTPMiddleware` breaks `contextvar` propagation, as noted [here](https://www.starlette.io/middleware/#limitations). This means the ElasticAPM middleware must be above any `BaseHTTPMiddleware` in the final middleware list. If you’re calling `add_middleware` repeatedly, add the ElasticAPM middleware last. If you’re passing in a list of middleware, ElasticAPM should be first on that list. +:::: + + +To configure the agent using initialization arguments: + +```python +from starlette.applications import Starlette +from elasticapm.contrib.starlette import make_apm_client, ElasticAPM + +apm = make_apm_client({ + 'SERVICE_NAME': '', + 'SECRET_TOKEN': '', + 'SERVER_URL': '', +}) +app = Starlette() +app.add_middleware(ElasticAPM, client=apm) +``` + + +## FastAPI [starlette-fastapi] + +Because FastAPI supports Starlette middleware, using the agent with FastAPI is almost exactly the same as with Starlette: + +```python +from fastapi import FastAPI +from elasticapm.contrib.starlette import ElasticAPM + +app = FastAPI() +app.add_middleware(ElasticAPM) +``` + + +## Usage [starlette-usage] + +Once you have configured the agent, it will automatically track transactions and capture uncaught exceptions within starlette. + +Capture an arbitrary exception by calling [`capture_exception`](/reference/api-reference.md#client-api-capture-exception): + +```python +try: + 1 / 0 +except ZeroDivisionError: + apm.capture_exception() +``` + +Log a generic message with [`capture_message`](/reference/api-reference.md#client-api-capture-message): + +```python +apm.capture_message('hello, world!') +``` + + +## Performance metrics [starlette-performance-metrics] + +If you’ve followed the instructions above, the agent has installed our instrumentation middleware which will process all requests through your app. This will measure response times, as well as detailed performance data for all supported technologies. + +::::{note} +Due to the fact that `asyncio` drivers are usually separate from their synchronous counterparts, specific instrumentation is needed for all drivers. The support for asynchronous drivers is currently quite limited. +:::: + + + +### Ignoring specific routes [starlette-ignoring-specific-views] + +You can use the [`TRANSACTIONS_IGNORE_PATTERNS`](/reference/configuration.md#config-transactions-ignore-patterns) configuration option to ignore specific routes. The list given should be a list of regular expressions which are matched against the transaction name: + +```python +apm = make_apm_client({ + # ... + 'TRANSACTIONS_IGNORE_PATTERNS': ['^GET /secret', '/extra_secret'] + # ... +}) +``` + +This would ignore any requests using the `GET /secret` route and any requests containing `/extra_secret`. + + +## Supported Starlette and Python versions [supported-starlette-and-python-versions] + +A list of supported [Starlette](/reference/supported-technologies.md#supported-starlette) and [Python](/reference/supported-technologies.md#supported-python) versions can be found on our [Supported Technologies](/reference/supported-technologies.md) page. + +::::{note} +Elastic APM only supports `asyncio` when using Python 3.7+ +:::: + + diff --git a/docs/reference/supported-technologies.md b/docs/reference/supported-technologies.md new file mode 100644 index 000000000..355fc865f --- /dev/null +++ b/docs/reference/supported-technologies.md @@ -0,0 +1,631 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/supported-technologies.html +--- + +# Supported technologies [supported-technologies] + +$$$framework-support$$$ +The Elastic APM Python Agent comes with support for the following frameworks: + +* [Django](/reference/django-support.md) +* [Flask](/reference/flask-support.md) +* [Aiohttp Server](#supported-aiohttp) +* [Tornado](#supported-tornado) +* [Starlette/FastAPI](#supported-starlette) +* [Sanic](#supported-sanic) +* [GRPC](#supported-grpc) + +For other frameworks and custom Python code, the agent exposes a set of [APIs](/reference/api-reference.md) for integration. + + +### Python [supported-python] + +The following Python versions are supported: + +* 3.6 +* 3.7 +* 3.8 +* 3.9 +* 3.10 +* 3.11 +* 3.12 + + +### Django [supported-django] + +We support these Django versions: + +* 1.11 +* 2.0 +* 2.1 +* 2.2 +* 3.0 +* 3.1 +* 3.2 +* 4.0 +* 4.2 +* 5.0 + +For upcoming Django versions, we generally aim to ensure compatibility starting with the first Release Candidate. + +::::{note} +we currently don’t support Django running in ASGI mode. +:::: + + + +### Flask [supported-flask] + +We support these Flask versions: + +* 0.10 (Deprecated) +* 0.11 (Deprecated) +* 0.12 (Deprecated) +* 1.0 +* 1.1 +* 2.0 +* 2.1 +* 2.2 +* 2.3 +* 3.0 + + +### Aiohttp Server [supported-aiohttp] + +We support these aiohttp versions: + +* 3.0+ + + +### Tornado [supported-tornado] + +We support these tornado versions: + +* 6.0+ + + +### Sanic [supported-sanic] + +We support these sanic versions: + +* 20.12.2+ + + +### Starlette/FastAPI [supported-starlette] + +We support these Starlette versions: + +* 0.13.0+ + +Any FastAPI version which uses a supported Starlette version should also be supported. + + +### GRPC [supported-grpc] + +We support these `grpcio` versions: + +* 1.24.0+ + + +## Automatic Instrumentation [automatic-instrumentation] + +The Python APM agent comes with automatic instrumentation of various 3rd party modules and standard library modules. + + +### Scheduling [automatic-instrumentation-scheduling] + + +##### Celery [automatic-instrumentation-scheduling-celery] + +We support these Celery versions: + +* 4.x (deprecated) +* 5.x + +Celery tasks will be recorded automatically with Django and Flask only. + + +### Databases [automatic-instrumentation-db] + + +#### Elasticsearch [automatic-instrumentation-db-elasticsearch] + +Instrumented methods: + +* `elasticsearch.transport.Transport.perform_request` +* `elasticsearch.connection.http_urllib3.Urllib3HttpConnection.perform_request` +* `elasticsearch.connection.http_requests.RequestsHttpConnection.perform_request` +* `elasticsearch._async.transport.AsyncTransport.perform_request` +* `elasticsearch_async.connection.AIOHttpConnection.perform_request` + +Additionally, the instrumentation wraps the following methods of the `Elasticsearch` client class: + +* `elasticsearch.client.Elasticsearch.delete_by_query` +* `elasticsearch.client.Elasticsearch.search` +* `elasticsearch.client.Elasticsearch.count` +* `elasticsearch.client.Elasticsearch.update` + +Collected trace data: + +* the query string (if available) +* the `query` element from the request body (if available) +* the response status code +* the count of affected rows (if available) + +We recommend using keyword arguments only with elasticsearch-py, as recommended by [the elasticsearch-py docs](https://elasticsearch-py.readthedocs.io/en/master/api.html#api-documentation). If you are using positional arguments, we will be unable to gather the `query` element from the request body. + + +#### SQLite [automatic-instrumentation-db-sqlite] + +Instrumented methods: + +* `sqlite3.connect` +* `sqlite3.dbapi2.connect` +* `pysqlite2.dbapi2.connect` + +The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. + +Collected trace data: + +* parametrized SQL query + + +#### MySQLdb [automatic-instrumentation-db-mysql] + +Library: `MySQLdb` + +Instrumented methods: + +* `MySQLdb.connect` + +The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. + +Collected trace data: + +* parametrized SQL query + + +#### mysql-connector [automatic-instrumentation-db-mysql-connector] + +Library: `mysql-connector-python` + +Instrumented methods: + +* `mysql.connector.connect` + +The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. + +Collected trace data: + +* parametrized SQL query + + +#### pymysql [automatic-instrumentation-db-pymysql] + +Library: `pymysql` + +Instrumented methods: + +* `pymysql.connect` + +The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. + +Collected trace data: + +* parametrized SQL query + + +#### aiomysql [automatic-instrumentation-db-aiomysql] + +Library: `aiomysql` + +Instrumented methods: + +* `aiomysql.cursors.Cursor.execute` + +Collected trace data: + +* parametrized SQL query + + +#### PostgreSQL [automatic-instrumentation-db-postgres] + +Library: `psycopg2`, `psycopg2-binary` (`>=2.9`) + +Instrumented methods: + +* `psycopg2.connect` + +The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. + +Collected trace data: + +* parametrized SQL query + + +#### aiopg [automatic-instrumentation-db-aiopg] + +Library: `aiopg` (`>=1.0`) + +Instrumented methods: + +* `aiopg.cursor.Cursor.execute` +* `aiopg.cursor.Cursor.callproc` + +Collected trace data: + +* parametrized SQL query + + +#### asyncpg [automatic-instrumentation-db-asyncg] + +Library: `asyncpg` (`>=0.20`) + +Instrumented methods: + +* `asyncpg.connection.Connection.execute` +* `asyncpg.connection.Connection.executemany` + +Collected trace data: + +* parametrized SQL query + + +#### PyODBC [automatic-instrumentation-db-pyodbc] + +Library: `pyodbc`, (`>=4.0`) + +Instrumented methods: + +* `pyodbc.connect` + +The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. + +Collected trace data: + +* parametrized SQL query + + +#### MS-SQL [automatic-instrumentation-db-mssql] + +Library: `pymssql`, (`>=2.1.0`) + +Instrumented methods: + +* `pymssql.connect` + +The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. + +Collected trace data: + +* parametrized SQL query + + +#### MongoDB [automatic-instrumentation-db-mongodb] + +Library: `pymongo`, `>=2.9,<3.8` + +Instrumented methods: + +* `pymongo.collection.Collection.aggregate` +* `pymongo.collection.Collection.bulk_write` +* `pymongo.collection.Collection.count` +* `pymongo.collection.Collection.create_index` +* `pymongo.collection.Collection.create_indexes` +* `pymongo.collection.Collection.delete_many` +* `pymongo.collection.Collection.delete_one` +* `pymongo.collection.Collection.distinct` +* `pymongo.collection.Collection.drop` +* `pymongo.collection.Collection.drop_index` +* `pymongo.collection.Collection.drop_indexes` +* `pymongo.collection.Collection.ensure_index` +* `pymongo.collection.Collection.find_and_modify` +* `pymongo.collection.Collection.find_one` +* `pymongo.collection.Collection.find_one_and_delete` +* `pymongo.collection.Collection.find_one_and_replace` +* `pymongo.collection.Collection.find_one_and_update` +* `pymongo.collection.Collection.group` +* `pymongo.collection.Collection.inline_map_reduce` +* `pymongo.collection.Collection.insert` +* `pymongo.collection.Collection.insert_many` +* `pymongo.collection.Collection.insert_one` +* `pymongo.collection.Collection.map_reduce` +* `pymongo.collection.Collection.reindex` +* `pymongo.collection.Collection.remove` +* `pymongo.collection.Collection.rename` +* `pymongo.collection.Collection.replace_one` +* `pymongo.collection.Collection.save` +* `pymongo.collection.Collection.update` +* `pymongo.collection.Collection.update_many` +* `pymongo.collection.Collection.update_one` + +Collected trace data: + +* database name +* method name + + +#### Redis [automatic-instrumentation-db-redis] + +Library: `redis` (`>=2.8`) + +Instrumented methods: + +* `redis.client.Redis.execute_command` +* `redis.client.Pipeline.execute` + +Collected trace data: + +* Redis command name + + +#### aioredis [automatic-instrumentation-db-aioredis] + +Library: `aioredis` (`<2.0`) + +Instrumented methods: + +* `aioredis.pool.ConnectionsPool.execute` +* `aioredis.commands.transaction.Pipeline.execute` +* `aioredis.connection.RedisConnection.execute` + +Collected trace data: + +* Redis command name + + +#### Cassandra [automatic-instrumentation-db-cassandra] + +Library: `cassandra-driver` (`>=3.4,<4.0`) + +Instrumented methods: + +* `cassandra.cluster.Session.execute` +* `cassandra.cluster.Cluster.connect` + +Collected trace data: + +* CQL query + + +#### Python Memcache [automatic-instrumentation-db-python-memcache] + +Library: `python-memcached` (`>=1.51`) + +Instrumented methods: + +* `memcache.Client.add` +* `memcache.Client.append` +* `memcache.Client.cas` +* `memcache.Client.decr` +* `memcache.Client.delete` +* `memcache.Client.delete_multi` +* `memcache.Client.disconnect_all` +* `memcache.Client.flush_all` +* `memcache.Client.get` +* `memcache.Client.get_multi` +* `memcache.Client.get_slabs` +* `memcache.Client.get_stats` +* `memcache.Client.gets` +* `memcache.Client.incr` +* `memcache.Client.prepend` +* `memcache.Client.replace` +* `memcache.Client.set` +* `memcache.Client.set_multi` +* `memcache.Client.touch` + +Collected trace data: + +* Destination (address and port) + + +#### pymemcache [automatic-instrumentation-db-pymemcache] + +Library: `pymemcache` (`>=3.0`) + +Instrumented methods: + +* `pymemcache.client.base.Client.add` +* `pymemcache.client.base.Client.append` +* `pymemcache.client.base.Client.cas` +* `pymemcache.client.base.Client.decr` +* `pymemcache.client.base.Client.delete` +* `pymemcache.client.base.Client.delete_many` +* `pymemcache.client.base.Client.delete_multi` +* `pymemcache.client.base.Client.flush_all` +* `pymemcache.client.base.Client.get` +* `pymemcache.client.base.Client.get_many` +* `pymemcache.client.base.Client.get_multi` +* `pymemcache.client.base.Client.gets` +* `pymemcache.client.base.Client.gets_many` +* `pymemcache.client.base.Client.incr` +* `pymemcache.client.base.Client.prepend` +* `pymemcache.client.base.Client.quit` +* `pymemcache.client.base.Client.replace` +* `pymemcache.client.base.Client.set` +* `pymemcache.client.base.Client.set_many` +* `pymemcache.client.base.Client.set_multi` +* `pymemcache.client.base.Client.stats` +* `pymemcache.client.base.Client.touch` + +Collected trace data: + +* Destination (address and port) + + +#### kafka-python [automatic-instrumentation-db-kafka-python] + +Library: `kafka-python` (`>=2.0`) + +Instrumented methods: + +* `kafka.KafkaProducer.send`, +* `kafka.KafkaConsumer.poll`, +* `kafka.KafkaConsumer.\__next__` + +Collected trace data: + +* Destination (address and port) +* topic (if applicable) + + +### External HTTP requests [automatic-instrumentation-http] + + +#### Standard library [automatic-instrumentation-stdlib-urllib] + +Library: `urllib2` (Python 2) / `urllib.request` (Python 3) + +Instrumented methods: + +* `urllib2.AbstractHTTPHandler.do_open` / `urllib.request.AbstractHTTPHandler.do_open` + +Collected trace data: + +* HTTP method +* requested URL + + +#### urllib3 [automatic-instrumentation-urllib3] + +Library: `urllib3` + +Instrumented methods: + +* `urllib3.connectionpool.HTTPConnectionPool.urlopen` + +Additionally, we instrumented vendored instances of urllib3 in the following libraries: + +* `requests` +* `botocore` + +Both libraries have "unvendored" urllib3 in more recent versions, we recommend to use the newest versions. + +Collected trace data: + +* HTTP method +* requested URL + + +#### requests [automatic-instrumentation-requests] + +Instrumented methods: + +* `requests.sessions.Session.send` + +Collected trace data: + +* HTTP method +* requested URL + + +#### AIOHTTP Client [automatic-instrumentation-aiohttp-client] + +Instrumented methods: + +* `aiohttp.client.ClientSession._request` + +Collected trace data: + +* HTTP method +* requested URL + + +#### httpx [automatic-instrumentation-httpx] + +Instrumented methods: + +* `httpx.Client.send + +Collected trace data: + +* HTTP method +* requested URL + + +### Services [automatic-instrumentation-services] + + +#### AWS Boto3 / Botocore [automatic-instrumentation-boto3] + +Library: `boto3` (`>=1.0`) + +Instrumented methods: + +* `botocore.client.BaseClient._make_api_call` + +Collected trace data for all services: + +* AWS region (e.g. `eu-central-1`) +* AWS service name (e.g. `s3`) +* operation name (e.g. `ListBuckets`) + +Additionally, some services collect more specific data + + +#### AWS Aiobotocore [automatic-instrumentation-aiobotocore] + +Library: `aiobotocore` (`>=2.2.0`) + +Instrumented methods: + +* `aiobotocore.client.BaseClient._make_api_call` + +Collected trace data for all services: + +* AWS region (e.g. `eu-central-1`) +* AWS service name (e.g. `s3`) +* operation name (e.g. `ListBuckets`) + +Additionally, some services collect more specific data + + +##### S3 [automatic-instrumentation-s3] + +* Bucket name + + +##### DynamoDB [automatic-instrumentation-dynamodb] + +* Table name + + +##### SNS [automatic-instrumentation-sns] + +* Topic name + + +##### SQS [automatic-instrumentation-sqs] + +* Queue name + + +### Template Engines [automatic-instrumentation-template-engines] + + +#### Django Template Language [automatic-instrumentation-dtl] + +Library: `Django` (see [Django](#supported-django) for supported versions) + +Instrumented methods: + +* `django.template.Template.render` + +Collected trace data: + +* template name + + +#### Jinja2 [automatic-instrumentation-jinja2] + +Library: `jinja2` + +Instrumented methods: + +* `jinja2.Template.render` + +Collected trace data: + +* template name + diff --git a/docs/reference/toc.yml b/docs/reference/toc.yml new file mode 100644 index 000000000..601d9bc45 --- /dev/null +++ b/docs/reference/toc.yml @@ -0,0 +1,33 @@ +project: 'APM Python agent reference' +toc: + - file: index.md + - file: set-up-apm-python-agent.md + children: + - file: django-support.md + - file: flask-support.md + - file: aiohttp-server-support.md + - file: tornado-support.md + - file: starlette-support.md + - file: sanic-support.md + - file: lambda-support.md + - file: azure-functions-support.md + - file: wrapper-support.md + - file: asgi-middleware.md + - file: supported-technologies.md + - file: configuration.md + - file: advanced-topics.md + children: + - file: instrumenting-custom-code.md + - file: sanitizing-data.md + - file: how-agent-works.md + - file: run-tests-locally.md + - file: api-reference.md + - file: metrics.md + - file: opentelemetry-api-bridge.md + - file: logs.md + - file: performance-tuning.md + - file: upgrading.md + children: + - file: upgrading-6.x.md + - file: upgrading-5.x.md + - file: upgrading-4.x.md \ No newline at end of file diff --git a/docs/reference/tornado-support.md b/docs/reference/tornado-support.md new file mode 100644 index 000000000..bae66762b --- /dev/null +++ b/docs/reference/tornado-support.md @@ -0,0 +1,108 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/tornado-support.html +--- + +# Tornado Support [tornado-support] + +Incorporating Elastic APM into your Tornado project only requires a few easy steps. + + +## Installation [tornado-installation] + +Install the Elastic APM agent using pip: + +```bash +$ pip install elastic-apm +``` + +or add `elastic-apm` to your project’s `requirements.txt` file. + + +## Setup [tornado-setup] + +To set up the agent, you need to initialize it with appropriate settings. + +The settings are configured either via environment variables, the application’s settings, or as initialization arguments. + +You can find a list of all available settings in the [Configuration](/reference/configuration.md) page. + +To initialize the agent for your application using environment variables: + +```python +import tornado.web +from elasticapm.contrib.tornado import ElasticAPM + +app = tornado.web.Application() +apm = ElasticAPM(app) +``` + +To configure the agent using `ELASTIC_APM` in your application’s settings: + +```python +import tornado.web +from elasticapm.contrib.tornado import ElasticAPM + +app = tornado.web.Application() +app.settings['ELASTIC_APM'] = { + 'SERVICE_NAME': '', + 'SECRET_TOKEN': '', +} +apm = ElasticAPM(app) +``` + + +## Usage [tornado-usage] + +Once you have configured the agent, it will automatically track transactions and capture uncaught exceptions within tornado. + +Capture an arbitrary exception by calling [`capture_exception`](/reference/api-reference.md#client-api-capture-exception): + +```python +try: + 1 / 0 +except ZeroDivisionError: + apm.client.capture_exception() +``` + +Log a generic message with [`capture_message`](/reference/api-reference.md#client-api-capture-message): + +```python +apm.client.capture_message('hello, world!') +``` + + +## Performance metrics [tornado-performance-metrics] + +If you’ve followed the instructions above, the agent has installed our instrumentation within the base RequestHandler class in tornado.web. This will measure response times, as well as detailed performance data for all supported technologies. + +::::{note} +Due to the fact that `asyncio` drivers are usually separate from their synchronous counterparts, specific instrumentation is needed for all drivers. The support for asynchronous drivers is currently quite limited. +:::: + + + +### Ignoring specific routes [tornado-ignoring-specific-views] + +You can use the [`TRANSACTIONS_IGNORE_PATTERNS`](/reference/configuration.md#config-transactions-ignore-patterns) configuration option to ignore specific routes. The list given should be a list of regular expressions which are matched against the transaction name: + +```python +app.settings['ELASTIC_APM'] = { + # ... + 'TRANSACTIONS_IGNORE_PATTERNS': ['^GET SecretHandler', 'MainHandler'] + # ... +} +``` + +This would ignore any requests using the `GET SecretHandler` route and any requests containing `MainHandler`. + + +## Supported tornado and Python versions [supported-tornado-and-python-versions] + +A list of supported [tornado](/reference/supported-technologies.md#supported-tornado) and [Python](/reference/supported-technologies.md#supported-python) versions can be found on our [Supported Technologies](/reference/supported-technologies.md) page. + +::::{note} +Elastic APM only supports `asyncio` when using Python 3.7+ +:::: + + diff --git a/docs/reference/upgrading-4.x.md b/docs/reference/upgrading-4.x.md new file mode 100644 index 000000000..18625231f --- /dev/null +++ b/docs/reference/upgrading-4.x.md @@ -0,0 +1,29 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/upgrading-4.x.html +--- + +# Upgrading to version 4 of the agent [upgrading-4.x] + +4.0 of the Elastic APM Python Agent comes with several backwards incompatible changes. + +## APM Server 6.5 required [upgrading-4.x-apm-server] + +This version of the agent is **only compatible with APM Server 6.5+**. To upgrade, we recommend to first upgrade APM Server, and then the agent. APM Server 6.5+ is backwards compatible with versions 2.x and 3.x of the agent. + + +## Configuration options [upgrading-4.x-configuration] + +Several configuration options have been removed, or renamed + +* `flush_interval` has been removed +* the `flush_interval` and `max_queue_size` settings have been removed. +* new settings introduced: `api_request_time` and `api_request_size`. +* Some settings now require a unit for duration or size. See [size format](configuration.md#config-format-size) and [duration format](configuration.md#config-format-duration). + + +## Processors [upgrading-4.x-processors] + +The method to write processors for sanitizing events has been changed. It will now be called for every type of event (transactions, spans and errors), unless the event types are limited using a decorator. See [Sanitizing data](sanitizing-data.md) for more information. + + diff --git a/docs/reference/upgrading-5.x.md b/docs/reference/upgrading-5.x.md new file mode 100644 index 000000000..ce30414ce --- /dev/null +++ b/docs/reference/upgrading-5.x.md @@ -0,0 +1,19 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/upgrading-5.x.html +--- + +# Upgrading to version 5 of the agent [upgrading-5.x] + +## APM Server 7.3 required for some features [_apm_server_7_3_required_for_some_features] + +APM Server and Kibana 7.3 introduced support for collecting breakdown metrics, and central configuration of APM agents. To use these features, please update the Python agent to 5.0+ and APM Server / Kibana to 7.3+ + + +## Tags renamed to Labels [_tags_renamed_to_labels] + +To better align with other parts of the Elastic Stack and the [Elastic Common Schema](ecs://reference/index.md), we renamed "tags" to "labels", and introduced limited support for typed labels. While tag values were only allowed to be strings, label values can be strings, booleans, or numerical. + +To benefit from this change, ensure that you run at least **APM Server 6.7**, and use `elasticapm.label()` instead of `elasticapm.tag()`. The `tag()` API will continue to work as before, but emit a `DeprecationWarning`. It will be removed in 6.0 of the agent. + + diff --git a/docs/reference/upgrading-6.x.md b/docs/reference/upgrading-6.x.md new file mode 100644 index 000000000..f730bfa27 --- /dev/null +++ b/docs/reference/upgrading-6.x.md @@ -0,0 +1,22 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/upgrading-6.x.html +--- + +# Upgrading to version 6 of the agent [upgrading-6.x] + +## Python 2 no longer supported [_python_2_no_longer_supported] + +Please upgrade to Python 3.6+ to continue to receive regular updates. + + +## `SANITIZE_FIELD_NAMES` changes [_sanitize_field_names_changes] + +If you are using a non-default `sanitize_field_names` config, please note that your entries must be surrounded with stars (e.g. `*secret*`) in order to maintain previous behavior. + + +## Tags removed (in favor of labels) [_tags_removed_in_favor_of_labels] + +Tags were deprecated in the 5.x release (in favor of labels). They have now been removed. + + diff --git a/docs/reference/upgrading.md b/docs/reference/upgrading.md new file mode 100644 index 000000000..83ae39902 --- /dev/null +++ b/docs/reference/upgrading.md @@ -0,0 +1,19 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/upgrading.html +--- + +# Upgrading [upgrading] + +Upgrades between minor versions of the agent, like from 3.1 to 3.2 are always backwards compatible. Upgrades that involve a major version bump often come with some backwards incompatible changes. + +We highly recommend to always pin the version of `elastic-apm` in your `requirements.txt` or `Pipfile`. This avoids automatic upgrades to potentially incompatible versions. + + +## End of life dates [end-of-life-dates] + +We love all our products, but sometimes we must say goodbye to a release so that we can continue moving forward on future development and innovation. Our [End of life policy](https://www.elastic.co/support/eol) defines how long a given release is considered supported, as well as how long a release is considered still in active development or maintenance. + + + + diff --git a/docs/wrapper.asciidoc b/docs/reference/wrapper-support.md similarity index 50% rename from docs/wrapper.asciidoc rename to docs/reference/wrapper-support.md index 4658201c6..a8f01bbbc 100644 --- a/docs/wrapper.asciidoc +++ b/docs/reference/wrapper-support.md @@ -1,58 +1,56 @@ -[[wrapper-support]] -=== Wrapper Support +--- +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/wrapper-support.html +--- -experimental::[] +# Wrapper Support [wrapper-support] -The following frameworks are supported using our new wrapper script for -no-code-changes instrumentation: +::::{warning} +This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. +:::: - * Django - * Flask - * Starlette -Please keep in mind that these instrumentations are a work in progress! We'd -love to have feedback on our -https://github.com/elastic/apm-agent-python/issues/new/choose[issue tracker]. +The following frameworks are supported using our new wrapper script for no-code-changes instrumentation: -[[wrapper-usage]] -==== Usage +* Django +* Flask +* Starlette -When installing the agent, an entrypoint script, `elasticapm-run` is installed -as well. You can use this script to instrument your app (assuming it's using a -supported framework) without changing your code! +Please keep in mind that these instrumentations are a work in progress! We’d love to have feedback on our [issue tracker](https://github.com/elastic/apm-agent-python/issues/new/choose). -[source,bash] ----- +## Usage [wrapper-usage] + +When installing the agent, an entrypoint script, `elasticapm-run` is installed as well. You can use this script to instrument your app (assuming it’s using a supported framework) without changing your code! + +```bash $ elasticapm-run --version elasticapm-run 6.14.0 ----- +``` Alternatively, you can run the entrypoint directly: -[source,bash] ----- +```bash $ python -m elasticapm.instrumentation.wrapper --version elasticapm-run 6.14.0 ----- +``` The `elasticapm-run` script can be used to run any Python script or module: -[source,bash] ----- +```bash $ elasticapm-run flask run $ elasticapm-run python myapp.py ----- +``` Generally, config should be passed in via environment variables. For example, -[source,bash] ----- +```bash $ ELASTIC_APM_SERVICE_NAME=my_flask_app elasticapm-run flask run ----- +``` You can also pass config options as arguments to the script: -[source,bash] ----- +```bash $ elasticapm-run --config "service_name=my_flask_app" --config "debug=true" flask run ----- +``` + + diff --git a/docs/release-notes.asciidoc b/docs/release-notes.asciidoc deleted file mode 100644 index c8d212db6..000000000 --- a/docs/release-notes.asciidoc +++ /dev/null @@ -1,15 +0,0 @@ -:pull: https://github.com/elastic/apm-agent-python/pull/ - -[[release-notes]] -== Release notes - -All notable changes to this project will be documented here. - -* <> -* <> -* <> -* <> -* <> -* <> - -include::../CHANGELOG.asciidoc[] diff --git a/docs/release-notes/breaking-changes.md b/docs/release-notes/breaking-changes.md new file mode 100644 index 000000000..0ea8db644 --- /dev/null +++ b/docs/release-notes/breaking-changes.md @@ -0,0 +1,28 @@ +--- +navigation_title: "Elastic APM Python Agent" +--- + +# Elastic APM Python Agent breaking changes [elastic-apm-python-agent-breaking-changes] +Before you upgrade, carefully review the Elastic APM RPython Agent breaking changes and take the necessary steps to mitigate any issues. + +To learn how to upgrade, check out . + +% ## Next version [elastic-apm-python-agent-nextversion-breaking-changes] +% **Release date:** Month day, year + +% ::::{dropdown} Title of breaking change +% Description of the breaking change. +% For more information, check [PR #](PR link). +% **Impact**
Impact of the breaking change. +% **Action**
Steps for mitigating deprecation impact. +% :::: + +## 6.0.0 [elastic-apm-python-agent-600-breaking-changes] +**Release date:** February 1, 2021 + +* Python 2.7 and 3.5 support has been deprecated. The Python agent now requires Python 3.6+. For more information, check [#1021](https://github.com/elastic/apm-agent-python/pull/1021). +* No longer collecting body for `elasticsearch-py` update and `delete_by_query`. For more information, check [#1013](https://github.com/elastic/apm-agent-python/pull/1013). +* Align `sanitize_field_names` config with the [cross-agent spec](https://github.com/elastic/apm/blob/3fa78e2a1eeea81c73c2e16e96dbf6b2e79f3c64/specs/agents/sanitization.md). If you are using a non-default `sanitize_field_names`, surrounding each of your entries with stars (e.g. `*secret*`) will retain the old behavior. For more information, check [#982](https://github.com/elastic/apm-agent-python/pull/982). +* Remove credit card sanitization for field values. This improves performance, and the security value of this check was dubious anyway. For more information, check [#982](https://github.com/elastic/apm-agent-python/pull/982). +* Remove HTTP querystring sanitization. This improves performance, and is meant to standardize behavior across the agents, as defined in [#334](https://github.com/elastic/apm/pull/334). For more information, check [#982](https://github.com/elastic/apm-agent-python/pull/982). +* Remove `elasticapm.tag()` (deprecated since 5.0.0). For more information, check [#1034](https://github.com/elastic/apm-agent-python/pull/1034). \ No newline at end of file diff --git a/docs/release-notes/deprecations.md b/docs/release-notes/deprecations.md new file mode 100644 index 000000000..b8cc1553a --- /dev/null +++ b/docs/release-notes/deprecations.md @@ -0,0 +1,38 @@ +--- +navigation_title: "Elastic APM Python Agent" +--- + +# Elastic APM Python Agent deprecations [elastic-apm-python-agent-deprecations] +Review the deprecated functionality for your Elastic APM Python Agent version. While deprecations have no immediate impact, we strongly encourage you update your implementation after you upgrade. + +To learn how to upgrade, check out . + +% ## Next version +% **Release date:** Month day, year + +% ::::{dropdown} Deprecation title +% Description of the deprecation. +% For more information, check [PR #](PR link). +% **Impact**
Impact of deprecation. +% **Action**
Steps for mitigating deprecation impact. +% :::: + +## 6.23.0 [elastic-apm-python-agent-6230-deprecations] +**Release date:** July 30, 2024 + +* Python 3.6 support will be removed in version 7.0.0 of the agent. +* The log shipping LoggingHandler will be removed in version 7.0.0 of the agent. +* The log shipping feature in the Flask instrumentation will be removed in version 7.0.0 of the agent. +* The log shipping feature in the Django instrumentation will be removed in version 7.0.0 of the agent. +* The OpenTracing bridge will be removed in version 7.0.0 of the agent. +* Celery 4.0 support is deprecated because it’s not installable anymore with a modern pip. + +## 6.20.0 [elastic-apm-python-agent-6200-deprecations] +**Release date:** January 10, 2024 + +The log shipping LoggingHandler will be removed in version 7.0.0 of the agent. + +## 6.19.0 [elastic-apm-python-agent-6190-deprecations] +**Release date:** October 11, 2023 + +The log shipping feature in the Flask instrumentation will be removed in version 7.0.0 of the agent. \ No newline at end of file diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md new file mode 100644 index 000000000..c229323b0 --- /dev/null +++ b/docs/release-notes/index.md @@ -0,0 +1,545 @@ +--- +navigation_title: "Elastic APM Python Agent" +mapped_pages: + - https://www.elastic.co/guide/en/apm/agent/python/current/release-notes-6.x.html +--- + +# Elastic APM Python Agent release notes [elastic-apm-python-agent-release-notes] + +Review the changes, fixes, and more in each version of Elastic APM Python Agent. + +To check for security updates, go to [Security announcements for the Elastic stack](https://discuss.elastic.co/c/announcements/security-announcements/31). + +% Release notes includes only features, enhancements, and fixes. Add breaking changes, deprecations, and known issues to the applicable release notes sections. + +% ## version.next [elastic-apm-python-agent-versionext-release-notes] +% **Release date:** Month day, year + +% ### Features and enhancements [elastic-apm-python-agent-versionext-features-enhancements] + +% ### Fixes [elastic-apm-python-agent-versionext-fixes] + +## 6.23.0 [elastic-apm-python-agent-6230-release-notes] +**Release date:** July 30, 2024 + +### Features and enhancements [elastic-apm-python-agent-6230-features-enhancements] +* Make published Docker images multi-platform with the addition of linux/arm64 [#2080](https://github.com/elastic/apm-agent-python/pull/2080) + +### Fixes [elastic-apm-python-agent-6230-fixes] +* Fix handling consumer iteration if transaction not sampled in kafka instrumentation [#2075](https://github.com/elastic/apm-agent-python/pull/2075) +* Fix race condition with urllib3 at shutdown [#2085](https://github.com/elastic/apm-agent-python/pull/2085) +* Fix compatibility with setuptools>=72 that removed test command [#2090](https://github.com/elastic/apm-agent-python/pull/2090) + +## 6.22.3 [elastic-apm-python-agent-6223-release-notes] +**Release date:** June 10, 2024 + +### Fixes [elastic-apm-python-agent-6223-fixes] +* Fix outcome in ASGI and Starlette apps on error status codes without an exception [#2060](https://github.com/elastic/apm-agent-python/pull/2060) + +## 6.22.2 [elastic-apm-python-agent-6222-release-notes] +**Release date:** May 20, 2024 + +### Fixes [elastic-apm-python-agent-6222-fixes] +* Fix CI release workflow [#2046](https://github.com/elastic/apm-agent-python/pull/2046) + +## 6.22.1 [elastic-apm-python-agent-6222-release-notes] +**Release date:** May 17, 2024 + +### Features and enhancements [elastic-apm-python-agent-6221-features-enhancements] +* Relax wrapt dependency to only exclude 1.15.0 [#2005](https://github.com/elastic/apm-agent-python/pull/2005) + +## 6.22.0 [elastic-apm-python-agent-6220-release-notes] +**Release date:** April 3, 2024 + +### Features and enhancements [elastic-apm-python-agent-6220-features-enhancements] +* Add ability to override default JSON serialization [#2018](https://github.com/elastic/apm-agent-python/pull/2018) + +## 6.21.4 [elastic-apm-python-agent-6214-release-notes] +**Release date:** March 19, 2024 + +### Fixes [elastic-apm-python-agent-6214-fixes] +* Fix urllib3 2.0.1+ crash with many args [#2002](https://github.com/elastic/apm-agent-python/pull/2002) + +## 6.21.3 [elastic-apm-python-agent-6213-release-notes] +**Release date:** March 8, 2024 + +### Fixes [elastic-apm-python-agent-6213-fixes] +* Fix artifacts download in CI workflows [#1996](https://github.com/elastic/apm-agent-python/pull/1996) + +## 6.21.2 [elastic-apm-python-agent-6212-release-notes] +**Release date:** March 7, 2024 + +### Fixes [elastic-apm-python-agent-6212-fixes] +* Fix artifacts upload in CI build-distribution workflow [#1993](https://github.com/elastic/apm-agent-python/pull/1993) + +## 6.21.1 [elastic-apm-python-agent-6211-release-notes] +**Release date:** March 7, 2024 + +### Fixes [elastic-apm-python-agent-6211-fixes] +* Fix CI release workflow [#1990](https://github.com/elastic/apm-agent-python/pull/1990) + +## 6.21.0 [elastic-apm-python-agent-6210-release-notes] +**Release date:** March 6, 2024 + +### Fixes [elastic-apm-python-agent-6210-fixes] +* Fix starlette middleware setup without client argument [#1952](https://github.com/elastic/apm-agent-python/pull/1952) +* Fix blocking of gRPC stream-to-stream requests [#1967](https://github.com/elastic/apm-agent-python/pull/1967) +* Always take into account body reading time for starlette requests [#1970](https://github.com/elastic/apm-agent-python/pull/1970) +* Make urllib3 transport tests more robust against local env [#1969](https://github.com/elastic/apm-agent-python/pull/1969) +* Clarify starlette integration documentation [#1956](https://github.com/elastic/apm-agent-python/pull/1956) +* Make dbapi2 query scanning for dollar quotes a bit more correct [#1976](https://github.com/elastic/apm-agent-python/pull/1976) +* Normalize headers in AWS Lambda integration on API Gateway v1 requests [#1982](https://github.com/elastic/apm-agent-python/pull/1982) + +## 6.20.0 [elastic-apm-python-agent-6200-release-notes] +**Release date:** January 10, 2024 + +### Features and enhancements [elastic-apm-python-agent-6200-features-enhancements] +* Async support for dbapi2 (starting with psycopg) [#1944](https://github.com/elastic/apm-agent-python/pull/1944) +* Add object name to procedure call spans in dbapi2 [#1938](https://github.com/elastic/apm-agent-python/pull/1938) +* Add support for python 3.10 and 3.11 lambda runtimes + +### Fixes [elastic-apm-python-agent-6200-fixes] +* Fix asyncpg support for 0.29+ [#1935](https://github.com/elastic/apm-agent-python/pull/1935) +* Fix dbapi2 signature extraction to handle square brackets in table name [#1947](https://github.com/elastic/apm-agent-python/pull/1947) + +## 6.19.0 [elastic-apm-python-agent-6190-release-notes] +**Release date:** October 11, 2023 + +### Features and enhancements [elastic-apm-python-agent-6190-features-enhancements] +* Add Python 3.12 support +* Collect the `configured_hostname` and `detected_hostname` separately, and switch to FQDN for the `detected_hostname`. [#1891](https://github.com/elastic/apm-agent-python/pull/1891) +* Improve postgres dollar-quote detection to be much faster [#1905](https://github.com/elastic/apm-agent-python/pull/1905) + +### Fixes [elastic-apm-python-agent-6190-fixes] +* Fix url argument fetching in aiohttp_client instrumentation [#1890](https://github.com/elastic/apm-agent-python/pull/1890) +* Fix a bug in the AWS Lambda instrumentation when `event["headers"] is None` [#1907](https://github.com/elastic/apm-agent-python/pull/1907) +* Fix a bug in AWS Lambda where metadata could be incomplete, causing validation errors with the APM Server [#1914](https://github.com/elastic/apm-agent-python/pull/1914) +* Fix a bug in AWS Lambda where sending the partial transaction would be recorded as an extra span [#1914](https://github.com/elastic/apm-agent-python/pull/1914) + +## 6.18.0 [elastic-apm-python-agent-6180-release-notes] +**Release date:** July 25, 2023 + +### Features and enhancements [elastic-apm-python-agent-6180-features-enhancements] +* Add support for grpc aio server interceptor [#1870](https://github.com/elastic/apm-agent-python/pull/1870) + +### Fixes [elastic-apm-python-agent-6180-fixes] +* Fix a bug in the Elasticsearch client instrumentation which was causing loss of database context (including statement) when interacting with Elastic Cloud [#1878](https://github.com/elastic/apm-agent-python/pull/1878) + +## 6.17.0 [elastic-apm-python-agent-6170-release-notes] +**Release date:** July 3, 2023 + +### Features and enhancements [elastic-apm-python-agent-6170-features-enhancements] +* Add `server_ca_cert_file` option to provide custom CA certificate [#1852](https://github.com/elastic/apm-agent-python/pull/1852) +* Add `include_process_args` option to allow users to opt-in to collecting process args [#1867](https://github.com/elastic/apm-agent-python/pull/1867) + +### Fixes [elastic-apm-python-agent-6170-fixes] +* Fix a bug in the GRPC instrumentation when reaching the maximum amount of spans per transaction [#1861](https://github.com/elastic/apm-agent-python/pull/1861) + +## 6.16.2 [elastic-apm-python-agent-6162-release-notes] +**Release date:** June 12, 2023 + +### Fixes [elastic-apm-python-agent-6162-fixes] +* Fix compatibility issue with older versions of OpenSSL in lambda runtimes [#1847](https://github.com/elastic/apm-agent-python/pull/1847) +* Add `latest` tag to docker images [#1848](https://github.com/elastic/apm-agent-python/pull/1848) +* Fix issue with redacting `user:pass` in URLs in Python 3.11.4 [#1850](https://github.com/elastic/apm-agent-python/pull/1850) + +## 6.16.1 [elastic-apm-python-agent-6161-release-notes] +**Release date:** June 6, 2023 + +### Fixes [elastic-apm-python-agent-6161-fixes] +* Fix release process for docker and the lambda layer [#1845](https://github.com/elastic/apm-agent-python/pull/1845) + +## 6.16.0 [elastic-apm-python-agent-6160-release-notes] +**Release date:** June 5, 2023 + +### Features and enhancements [elastic-apm-python-agent-6160-features-enhancements] +* Add lambda layer for instrumenting AWS Lambda functions [#1826](https://github.com/elastic/apm-agent-python/pull/1826) +* Implement instrumentation of Azure Functions [#1766](https://github.com/elastic/apm-agent-python/pull/1766) +* Add support for Django to wrapper script [#1780](https://github.com/elastic/apm-agent-python/pull/1780) +* Add support for Starlette to wrapper script [#1830](https://github.com/elastic/apm-agent-python/pull/1830) +* Add `transport_json_serializer` configuration option [#1777](https://github.com/elastic/apm-agent-python/pull/1777) +* Add S3 bucket and key name to OTel attributes [#1790](https://github.com/elastic/apm-agent-python/pull/1790) +* Implement partial transaction support in AWS lambda [#1784](https://github.com/elastic/apm-agent-python/pull/1784) +* Add instrumentation for redis.asyncio [#1807](https://github.com/elastic/apm-agent-python/pull/1807) +* Add support for urllib3 v2.0.1+ [#1822](https://github.com/elastic/apm-agent-python/pull/1822) +* Add `service.environment` to log correlation [#1833](https://github.com/elastic/apm-agent-python/pull/1833) +* Add `ecs_logging` as a dependency [#1840](https://github.com/elastic/apm-agent-python/pull/1840) +* Add support for synchronous psycopg3 [#1841](https://github.com/elastic/apm-agent-python/pull/1841) + +### Fixes [elastic-apm-python-agent-6160-fixes] +* Fix spans being dropped if they don’t have a name [#1770](https://github.com/elastic/apm-agent-python/pull/1770) +* Fix AWS Lambda support when `event` is not a dict [#1775](https://github.com/elastic/apm-agent-python/pull/1775) +* Fix deprecation warning with urllib3 2.0.0 pre-release versions [#1778](https://github.com/elastic/apm-agent-python/pull/1778) +* Fix `activation_method` to only send to APM server 8.7.1+ [#1787](https://github.com/elastic/apm-agent-python/pull/1787) +* Fix span.context.destination.service.resource for S3 spans to have an "s3/" prefix. [#1783](https://github.com/elastic/apm-agent-python/pull/1783) + +**Note**: While this is considered a bugfix, it can potentially be a breaking change in the Kibana APM app: It can break the history of the S3-Spans / metrics for users relying on `context.destination.service.resource`. If users happen to run agents both with and without this fix (for same or different languages), the same S3-buckets can appear twice in the service map (with and without s3-prefix). + +* Fix instrumentation to not bubble up exceptions during instrumentation [#1791](https://github.com/elastic/apm-agent-python/pull/1791) +* Fix HTTP transport to not print useless and confusing stack trace [#1809](https://github.com/elastic/apm-agent-python/pull/1809) + +## 6.15.1 [elastic-apm-python-agent-6151-release-notes] +**Release date:** March 6, 2023 + +### Fixes [elastic-apm-python-agent-6151-fixes] +* Fix issue with botocore instrumentation creating spans with an incorrect `service.name` [#1765](https://github.com/elastic/apm-agent-python/pull/1765) +* Fix a bug in the GRPC instrumentation when the agent is disabled or not recording [#1761](https://github.com/elastic/apm-agent-python/pull/1761) + +## 6.15.0 [elastic-apm-python-agent-6150-release-notes] +**Release date:** February 16, 2023 + +### Features and enhancements [elastic-apm-python-agent-6150-features-enhancements] +* Add `service.agent.activation_method` to the metadata [#1743](https://github.com/elastic/apm-agent-python/pull/1743) + +### Fixes [elastic-apm-python-agent-6150-fixes] +* Small fix to underlying Starlette logic to prevent duplicate Client objects [#1735](https://github.com/elastic/apm-agent-python/pull/1735) +* Change `server_url` default to `http://127.0.0.1:8200` to avoid ipv6 ambiguity [#1744](https://github.com/elastic/apm-agent-python/pull/1744) +* Fix an issue in GRPC instrumentation with unsampled transactions [#1740](https://github.com/elastic/apm-agent-python/pull/1740) +* Fix error in async Elasticsearch instrumentation when spans are dropped [#1758](https://github.com/elastic/apm-agent-python/pull/1758) + +## 6.14.0 [elastic-apm-python-agent-6140-release-notes] +**Release date:** January 30, 2023 + +### Features and enhancements [elastic-apm-python-agent-6140-features-enhancements] +* GRPC support [#1703](https://github.com/elastic/apm-agent-python/pull/1703) +* Wrapper script Flask support (experimental) [#1709](https://github.com/elastic/apm-agent-python/pull/1709) + +### Fixes [elastic-apm-python-agent-6140-fixes] +* Fix an async issue with long elasticsearch queries [#1725](https://github.com/elastic/apm-agent-python/pull/1725) +* Fix a minor inconsistency with the W3C tracestate spec [#1728](https://github.com/elastic/apm-agent-python/pull/1728) +* Fix a cold start performance issue with our AWS Lambda integration [#1727](https://github.com/elastic/apm-agent-python/pull/1727) +* Mark `**kwargs` config usage in our AWS Lambda integration as deprecated [#1727](https://github.com/elastic/apm-agent-python/pull/1727) + +## 6.13.2 [elastic-apm-python-agent-6132-release-notes] +**Release date:** November 17, 2022 + +### Fixes [elastic-apm-python-agent-6132-fixes] +* Fix error in Elasticsearch instrumentation when spans are dropped [#1690](https://github.com/elastic/apm-agent-python/pull/1690) +* Lower log level for errors in APM Server version fetching [#1692](https://github.com/elastic/apm-agent-python/pull/1692) +* Fix for missing parent.id when logging from a DroppedSpan under a leaf span [#1695](https://github.com/elastic/apm-agent-python/pull/1695) + +## 6.13.1 [elastic-apm-python-agent-6131-release-notes] +**Release date:** November 3, 2022 + +### Fixes [elastic-apm-python-agent-6131-fixes] +* Fix elasticsearch instrumentation for track_total_hits=False [#1687](https://github.com/elastic/apm-agent-python/pull/1687) + +## 6.13.0 [elastic-apm-python-agent-6130-release-notes] +**Release date:** October 26, 2022 + +### Features and enhancements [elastic-apm-python-agent-6130-features-enhancements] +* Add support for Python 3.11 +* Add backend granularity data to SQL backends as well as Cassandra and pymongo [#1585](https://github.com/elastic/apm-agent-python/pull/1585), [#1639](https://github.com/elastic/apm-agent-python/pull/1639) +* Add support for instrumenting the Elasticsearch 8 Python client [#1642](https://github.com/elastic/apm-agent-python/pull/1642) +* Add `*principal*` to default `sanitize_field_names` configuration [#1664](https://github.com/elastic/apm-agent-python/pull/1664) +* Add docs and better support for custom metrics, including in AWS Lambda [#1643](https://github.com/elastic/apm-agent-python/pull/1643) +* Add support for capturing span links from AWS SQS in AWS Lambda [#1662](https://github.com/elastic/apm-agent-python/pull/1662) + +### Fixes [elastic-apm-python-agent-6130-fixes] +* Fix Django’s `manage.py check` when agent is disabled [#1632](https://github.com/elastic/apm-agent-python/pull/1632) +* Fix an issue with long body truncation for Starlette [#1635](https://github.com/elastic/apm-agent-python/pull/1635) +* Fix an issue with transaction outcomes in Flask for uncaught exceptions [#1637](https://github.com/elastic/apm-agent-python/pull/1637) +* Fix Starlette instrumentation to make sure transaction information is still present during exception handling [#1674](https://github.com/elastic/apm-agent-python/pull/1674) + +## 6.12.0 [elastic-apm-python-agent-6120-release-notes] +**Release date:** September 7, 2022 + +### Features and enhancements [elastic-apm-python-agent-6120-features-enhancements] +* Add redis query to context data for redis instrumentation [#1406](https://github.com/elastic/apm-agent-python/pull/1406) +* Add AWS request ID to all botocore spans (at `span.context.http.request.id`) [#1625](https://github.com/elastic/apm-agent-python/pull/1625) + +### Fixes [elastic-apm-python-agent-6120-fixes] +* Differentiate Lambda URLs from API Gateway in AWS Lambda integration [#1609](https://github.com/elastic/apm-agent-python/pull/1609) +* Restrict the size of Django request bodies to prevent APM Server rejection [#1610](https://github.com/elastic/apm-agent-python/pull/1610) +* Restrict length of `exception.message` for exceptions captured by the agent [#1619](https://github.com/elastic/apm-agent-python/pull/1619) +* Restrict length of Starlette request bodies [#1549](https://github.com/elastic/apm-agent-python/pull/1549) +* Fix error when using elasticsearch(sniff_on_start=True) [#1618](https://github.com/elastic/apm-agent-python/pull/1618) +* Improve handling of ignored URLs and capture_body=off for Starlette [#1549](https://github.com/elastic/apm-agent-python/pull/1549) +* Fix possible error in the transport flush for Lambda functions [#1628](https://github.com/elastic/apm-agent-python/pull/1628) + +## 6.11.0 [elastic-apm-python-agent-6110-release-notes] +**Release date:** August 9, 2022 + +### Features and enhancements [elastic-apm-python-agent-6110-features-enhancements] +* Added lambda support for ELB triggers [#1605](https://github.com/elastic/apm-agent-python/pull/1605) + +## 6.10.2 [elastic-apm-python-agent-6102-release-notes] +**Release date:** August 9, 2022 + +### Fixes [elastic-apm-python-agent-6102-fixes] +* Fixed an issue with non-integer ports in Django [#1590](https://github.com/elastic/apm-agent-python/pull/1590) +* Fixed an issue with non-integer ports in Redis [#1591](https://github.com/elastic/apm-agent-python/pull/1591) +* Fixed a performance issue for local variable shortening via `varmap()` [#1593](https://github.com/elastic/apm-agent-python/pull/1593) +* Fixed `elasticapm.label()` when a Client object is not available [#1596](https://github.com/elastic/apm-agent-python/pull/1596) + +## 6.10.1 [elastic-apm-python-agent-6101-release-notes] +**Release date:** June 30, 2022 + +### Fixes [elastic-apm-python-agent-6101-fixes] +* Fix an issue with Kafka instrumentation and unsampled transactions [#1579](https://github.com/elastic/apm-agent-python/pull/1579) + +## 6.10.0 [elastic-apm-python-agent-6100-release-notes] +**Release date:** June 22, 2022 + +### Features and enhancements [elastic-apm-python-agent-6100-features-enhancements] +* Add instrumentation for [`aiobotocore`](https://github.com/aio-libs/aiobotocore) [#1520](https://github.com/elastic/apm-agent-python/pull/1520) +* Add instrumentation for [`kafka-python`](https://kafka-python.readthedocs.io/en/master/) [#1555](https://github.com/elastic/apm-agent-python/pull/1555) +* Add API for span links, and implement span link support for OpenTelemetry bridge [#1562](https://github.com/elastic/apm-agent-python/pull/1562) +* Add span links to SQS ReceiveMessage call [#1575](https://github.com/elastic/apm-agent-python/pull/1575) +* Add specific instrumentation for SQS delete/batch-delete [#1567](https://github.com/elastic/apm-agent-python/pull/1567) +* Add `trace_continuation_strategy` setting [#1564](https://github.com/elastic/apm-agent-python/pull/1564) + +### Fixes [elastic-apm-python-agent-6100-fixes] +* Fix return for `opentelemetry.Span.is_recording()` [#1530](https://github.com/elastic/apm-agent-python/pull/1530) +* Fix error logging for bad SERVICE_NAME config [#1546](https://github.com/elastic/apm-agent-python/pull/1546) +* Do not instrument old versions of Tornado > 6.0 due to incompatibility [#1566](https://github.com/elastic/apm-agent-python/pull/1566) +* Fix transaction names for class based views in Django 4.0+ [#1571](https://github.com/elastic/apm-agent-python/pull/1571) +* Fix a problem with our logging handler failing to report internal errors in its emitter [#1568](https://github.com/elastic/apm-agent-python/pull/1568) + +## 6.9.1 [elastic-apm-python-agent-691-release-notes] +**Release date:** March 30, 2022 + +### Fixes [elastic-apm-python-agent-691-fixes] +* Fix `otel_attributes`-related regression with older versions of APM Server (<7.16) [#1510](https://github.com/elastic/apm-agent-python/pull/1510) + +## 6.9.0 [elastic-apm-python-agent-690-release-notes] +**Release date:** March 29, 2022 + +### Features and enhancements [elastic-apm-python-agent-690-features-enhancements] +* Add OpenTelemetry API bridge [#1411](https://github.com/elastic/apm-agent-python/pull/1411) +* Change default for `sanitize_field_names` to sanitize `*auth*` instead of `authorization` [#1494](https://github.com/elastic/apm-agent-python/pull/1494) +* Add `span_stack_trace_min_duration` to replace deprecated `span_frames_min_duration` [#1498](https://github.com/elastic/apm-agent-python/pull/1498) +* Enable exact_match span compression by default [#1504](https://github.com/elastic/apm-agent-python/pull/1504) +* Allow parent celery tasks to specify the downstream `parent_span_id` in celery headers [#1500](https://github.com/elastic/apm-agent-python/pull/1500) + +### Fixes [elastic-apm-python-agent-690-fixes] +* Fix Sanic integration to properly respect the `capture_body` config [#1485](https://github.com/elastic/apm-agent-python/pull/1485) +* Lambda fixes to align with the cross-agent spec [#1489](https://github.com/elastic/apm-agent-python/pull/1489) +* Lambda fix for custom `service_name` [#1493](https://github.com/elastic/apm-agent-python/pull/1493) +* Change default for `stack_trace_limit` from 500 to 50 [#1492](https://github.com/elastic/apm-agent-python/pull/1492) +* Switch all duration handling to use `datetime.timedelta` objects [#1488](https://github.com/elastic/apm-agent-python/pull/1488) + +## 6.8.1 [elastic-apm-python-agent-681-release-notes] +**Release date:** March 9, 2022 + +### Fixes [elastic-apm-python-agent-681-fixes] +* Fix `exit_span_min_duration` and disable by default [#1483](https://github.com/elastic/apm-agent-python/pull/1483) + +## 6.8.0 [elastic-apm-python-agent-680-release-notes] +**Release date:** February 22, 2022 + +### Features and enhancements [elastic-apm-python-agent-680-features-enhancements] +* use "unknown-python-service" as default service name if no service name is configured [#1438](https://github.com/elastic/apm-agent-python/pull/1438) +* add transaction name to error objects [#1441](https://github.com/elastic/apm-agent-python/pull/1441) +* don’t send unsampled transactions to APM Server 8.0+ [#1442](https://github.com/elastic/apm-agent-python/pull/1442) +* implement snapshotting of certain configuration during transaction lifetime [#1431](https://github.com/elastic/apm-agent-python/pull/1431) +* propagate traceparent IDs via Celery [#1371](https://github.com/elastic/apm-agent-python/pull/1371) +* removed Python 2 compatibility shims [#1463](https://github.com/elastic/apm-agent-python/pull/1463) + +**Note:** Python 2 support was already removed with version 6.0 of the agent, this now removes unused compatibilit shims. + +### Fixes [elastic-apm-python-agent-680-fixes] +* fix span compression for redis, mongodb, cassandra and memcached [#1444](https://github.com/elastic/apm-agent-python/pull/1444) +* fix recording of status_code for starlette [#1466](https://github.com/elastic/apm-agent-python/pull/1466) +* fix aioredis span context handling [#1462](https://github.com/elastic/apm-agent-python/pull/1462) + +## 6.7.2 [elastic-apm-python-agent-672-release-notes] +**Release date:** December 7, 2021 + +### Fixes [elastic-apm-python-agent-672-fixes] +* fix AttributeError in sync instrumentation of httpx [#1423](https://github.com/elastic/apm-agent-python/pull/1423) +* add setting to disable span compression, default to disabled [#1429](https://github.com/elastic/apm-agent-python/pull/1429) + +## 6.7.1 [elastic-apm-python-agent-671-release-notes] +**Release date:** November 29, 2021 + +### Fixes [elastic-apm-python-agent-671-fixes] +* fix an issue with Sanic exception tracking [#1414](https://github.com/elastic/apm-agent-python/pull/1414) +* asyncpg: Limit SQL queries in context data to 10000 characters [#1416](https://github.com/elastic/apm-agent-python/pull/1416) + +## 6.7.0 [elastic-apm-python-agent-670-release-notes] +**Release date:** November 17, 2021 + +### Features and enhancements [elastic-apm-python-agent-670-features-enhancements] +* Add support for Sanic framework [#1390](https://github.com/elastic/apm-agent-python/pull/1390) + +### Fixes [elastic-apm-python-agent-670-fixes] +* fix compatibility issues with httpx 0.21 [#1403](https://github.com/elastic/apm-agent-python/pull/1403) +* fix `span_compression_exact_match_max_duration` default value [#1407](https://github.com/elastic/apm-agent-python/pull/1407) + +## 6.6.3 [elastic-apm-python-agent-663-release-notes] +**Release date:** November 15, 2021 + +### Fixes [elastic-apm-python-agent-663-fixes] +* fix an issue with `metrics_sets` configuration referencing the `TransactionMetricSet` removed in 6.6.2 [#1397](https://github.com/elastic/apm-agent-python/pull/1397) + +## 6.6.2 [elastic-apm-python-agent-662-release-notes] +**Release date:** November 10, 2021 + +### Fixes [elastic-apm-python-agent-662-fixes] +* Fix an issue where compressed spans would count against `transaction_max_spans` [#1377](https://github.com/elastic/apm-agent-python/pull/1377) +* Make sure HTTP connections are not re-used after a process fork [#1374](https://github.com/elastic/apm-agent-python/pull/1374) +* Fix an issue with psycopg2 instrumentation when multiple hosts are defined [#1386](https://github.com/elastic/apm-agent-python/pull/1386) +* Update the `User-Agent` header to the new [spec](https://github.com/elastic/apm/pull/514) [#1378](https://github.com/elastic/apm-agent-python/pull/1378) +* Improve status_code handling in AWS Lambda integration [#1382](https://github.com/elastic/apm-agent-python/pull/1382) +* Fix `aiohttp` exception handling to allow for non-500 responses including `HTTPOk` [#1384](https://github.com/elastic/apm-agent-python/pull/1384) +* Force transaction names to strings [#1389](https://github.com/elastic/apm-agent-python/pull/1389) +* Remove unused `http.request.socket.encrypted` context field [#1332](https://github.com/elastic/apm-agent-python/pull/1332) +* Remove unused transaction metrics (APM Server handles these metrics instead) [#1388](https://github.com/elastic/apm-agent-python/pull/1388) + +## 6.6.1 [elastic-apm-python-agent-661-release-notes] +**Release date:** November 2, 2021 + +### Fixes [elastic-apm-python-agent-661-fixes] +* Fix some context fields and metadata handling in AWS Lambda support [#1368](https://github.com/elastic/apm-agent-python/pull/1368) + +## 6.6.0 [elastic-apm-python-agent-660-release-notes] +**Release date:** October 18, 2021 + +### Features and enhancements [elastic-apm-python-agent-660-features-enhancements] +* Add experimental support for AWS lambda instrumentation [#1193](https://github.com/elastic/apm-agent-python/pull/1193) +* Add support for span compression [#1321](https://github.com/elastic/apm-agent-python/pull/1321) +* Auto-infer destination resources for easier instrumentation of new resources [#1359](https://github.com/elastic/apm-agent-python/pull/1359) +* Add support for dropped span statistics [#1327](https://github.com/elastic/apm-agent-python/pull/1327) + +### Fixes [elastic-apm-python-agent-660-fixes] +* Ensure that Prometheus histograms are encoded correctly for APM Server [#1354](https://github.com/elastic/apm-agent-python/pull/1354) +* Remove problematic (and duplicate) `event.dataset` from logging integrations [#1365](https://github.com/elastic/apm-agent-python/pull/1365) +* Fix for memcache instrumentation when configured with a unix socket [#1357](https://github.com/elastic/apm-agent-python/pull/1357) + +## 6.5.0 [elastic-apm-python-agent-650-release-notes] +**Release date:** October 4, 2021 + +### Features and enhancements [elastic-apm-python-agent-650-features-enhancements] +* Add instrumentation for Azure Storage (blob/table/fileshare) and Azure Queue [#1316](https://github.com/elastic/apm-agent-python/pull/1316) + +### Fixes [elastic-apm-python-agent-650-fixes] +* Improve span coverage for `asyncpg` [#1328](https://github.com/elastic/apm-agent-python/pull/1328) +* aiohttp: Correctly pass custom client to tracing middleware [#1345](https://github.com/elastic/apm-agent-python/pull/1345) +* Fixed an issue with httpx instrumentation [#1337](https://github.com/elastic/apm-agent-python/pull/1337) +* Fixed an issue with Django 4.0 removing a private method [#1347](https://github.com/elastic/apm-agent-python/pull/1347) + +## 6.4.0 [elastic-apm-python-agent-640-release-notes] +**Release date:** August 31, 2021 + +### Features and enhancements [elastic-apm-python-agent-640-features-enhancements] +* Rename the experimental `log_ecs_formatting` config to `log_ecs_reformatting` [#1300](https://github.com/elastic/apm-agent-python/pull/1300) +* Add support for Prometheus histograms [#1165](https://github.com/elastic/apm-agent-python/pull/1165) + +### Fixes [elastic-apm-python-agent-640-fixes] +* Fixed cookie sanitization when Cookie is capitalized [#1301](https://github.com/elastic/apm-agent-python/pull/1301) +* Fix a bug with exception capturing for bad UUIDs [#1304](https://github.com/elastic/apm-agent-python/pull/1304) +* Fix potential errors in json serialization [#1203](https://github.com/elastic/apm-agent-python/pull/1203) +* Fix an issue with certain aioredis commands [#1308](https://github.com/elastic/apm-agent-python/pull/1308) + +## 6.3.3 [elastic-apm-python-agent-633-release-notes] +**Release date:** July 14, 2021 + +### Fixes [elastic-apm-python-agent-633-fixes] +* ensure that the elasticsearch instrumentation handles DroppedSpans correctly [#1190](https://github.com/elastic/apm-agent-python/pull/1190) + +## 6.3.2 [elastic-apm-python-agent-632-release-notes] +**Release date:** July 7, 2021 + +### Fixes [elastic-apm-python-agent-632-fixes] +* Fix handling of non-http scopes in Starlette/FastAPI middleware [#1187](https://github.com/elastic/apm-agent-python/pull/1187) + +## 6.3.1 [elastic-apm-python-agent-631-release-notes] +**Release date:** July 7, 2021 + +### Fixes [elastic-apm-python-agent-631-fixes] +* Fix issue with Starlette/FastAPI hanging on startup [#1185](https://github.com/elastic/apm-agent-python/pull/1185) + +## 6.3.0 [elastic-apm-python-agent-630-release-notes] +**Release date:** July 6, 2021 + +### Features and enhancements [elastic-apm-python-agent-630-features-enhancements] +* Add additional context information about elasticsearch client requests [#1108](https://github.com/elastic/apm-agent-python/pull/1108) +* Add `use_certifi` config option to allow users to disable `certifi` [#1163](https://github.com/elastic/apm-agent-python/pull/1163) + +### Fixes [elastic-apm-python-agent-630-fixes] +* Fix for Starlette 0.15.0 error collection [#1174](https://github.com/elastic/apm-agent-python/pull/1174) +* Fix for Starlette static files [#1137](https://github.com/elastic/apm-agent-python/pull/1137) + +## 6.2.3 [elastic-apm-python-agent-623-release-notes] +**Release date:** June 28, 2021 + +### Fixes [elastic-apm-python-agent-623-fixes] +* suppress the default_app_config attribute in Django 3.2+ [#1155](https://github.com/elastic/apm-agent-python/pull/1155) +* bump log level for multiple set_client calls to WARNING [#1164](https://github.com/elastic/apm-agent-python/pull/1164) +* fix issue with adding disttracing to SQS messages when dropping spans [#1170](https://github.com/elastic/apm-agent-python/pull/1170) + +## 6.2.2 [elastic-apm-python-agent-622-release-notes] +**Release date:** June 7, 2021 + +### Fixes [elastic-apm-python-agent-622-fixes] +* Fix an attribute access bug introduced in 6.2.0 [#1149](https://github.com/elastic/apm-agent-python/pull/1149) + +## 6.2.1 [elastic-apm-python-agent-621-release-notes] +**Release date:** June 3, 2021 + +### Fixes [elastic-apm-python-agent-621-fixes] +* catch and log exceptions in interval timer threads [#1145](https://github.com/elastic/apm-agent-python/pull/1145) + +## 6.2.0 [elastic-apm-python-agent-620-release-notes] +**Release date:** May 31, 2021 + +### Features and enhancements [elastic-apm-python-agent-620-features-enhancements] +* Added support for aioredis 1.x [#2526](https://github.com/elastic/apm-agent-python/pull/1082) +* Added support for aiomysql [#1107](https://github.com/elastic/apm-agent-python/pull/1107) +* Added Redis pub/sub instrumentation [#1129](https://github.com/elastic/apm-agent-python/pull/1129) +* Added specific instrumentation for AWS SQS [#1123](https://github.com/elastic/apm-agent-python/pull/1123) + +### Fixes [elastic-apm-python-agent-620-fixes] +* ensure metrics are flushed before agent shutdown [#1139](https://github.com/elastic/apm-agent-python/pull/1139) +* added safeguard for exceptions in processors [#1138](https://github.com/elastic/apm-agent-python/pull/1138) +* ensure sockets are closed which were opened for cloud environment detection [#1134](https://github.com/elastic/apm-agent-python/pull/1134) + +## 6.1.3 [elastic-apm-python-agent-613-release-notes] +**Release date:** April 28, 2021 + +### Fixes [elastic-apm-python-agent-613-fixes] +* added destination information to asyncpg instrumentation [#1115](https://github.com/elastic/apm-agent-python/pull/1115) +* fixed issue with collecting request meta data with Django REST Framework [#1117](https://github.com/elastic/apm-agent-python/pull/1117) +* fixed httpx instrumentation for newly released httpx 0.18.0 [#1118](https://github.com/elastic/apm-agent-python/pull/1118) + +## 6.1.2 [elastic-apm-python-agent-612-release-notes] +**Release date:** April 14, 2021 + +### Fixes [elastic-apm-python-agent-612-fixes] +* fixed issue with empty transaction name for the root route with Django [#1095](https://github.com/elastic/apm-agent-python/pull/1095) +* fixed on-the-fly initialisation of Flask apps [#1099](https://github.com/elastic/apm-agent-python/pull/1099) + +## 6.1.1 [elastic-apm-python-agent-611-release-notes] +**Release date:** April 8, 2021 + +### Fixes [elastic-apm-python-agent-611-fixes] +* fixed a validation issue with the newly introduced instrumentation for S3, SNS and DynamoDB [#1090](https://github.com/elastic/apm-agent-python/pull/1090) + +## 6.1.0 [elastic-apm-python-agent-610-release-notes] +**Release date:** March 31, 2021 + +### Features and enhancements [elastic-apm-python-agent-610-features-enhancements] +* Add global access to Client singleton object at `elasticapm.get_client()` [#1043](https://github.com/elastic/apm-agent-python/pull/1043) +* Add `log_ecs_formatting` config option [#1058](https://github.com/elastic/apm-agent-python/pull/1058) [#1063](https://github.com/elastic/apm-agent-python/pull/1063) +* Add instrumentation for httplib2 [#1031](https://github.com/elastic/apm-agent-python/pull/1031) +* Add better instrumentation for some AWS services (S3, SNS, DynamoDB) [#1054](https://github.com/elastic/apm-agent-python/pull/1054) +* Added beta support for collecting metrics from prometheus_client [#1083](https://github.com/elastic/apm-agent-python/pull/1083) + +### Fixes [elastic-apm-python-agent-610-fixes] +* Fix for potential `capture_body: error` hang in Starlette/FastAPI [#1038](https://github.com/elastic/apm-agent-python/pull/1038) +* Fix a rare error around processing stack frames [#1012](https://github.com/elastic/apm-agent-python/pull/1012) +* Fix for Starlette/FastAPI to correctly capture request bodies as strings [#1041](https://github.com/elastic/apm-agent-python/pull/1042) +* Fix transaction names for Starlette Mount routes [#1037](https://github.com/elastic/apm-agent-python/pull/1037) +* Fix for elastic excepthook arguments [#1050](https://github.com/elastic/apm-agent-python/pull/1050) +* Fix issue with remote configuration when resetting config values [#1068](https://github.com/elastic/apm-agent-python/pull/1068) +* Use a label for the elasticapm Django app that is compatible with Django 3.2 validation [#1064](https://github.com/elastic/apm-agent-python/pull/1064) +* Fix an issue with undefined routes in Starlette [#1076](https://github.com/elastic/apm-agent-python/pull/1076) + +## 6.0.0 [elastic-apm-python-agent-600-release-notes] +**Release date:** February 1, 2021 + +### Fixes [elastic-apm-python-agent-600-fixes] +* Fix for GraphQL span spamming from scalar fields with required flag [#1015](https://github.com/elastic/apm-agent-python/pull/1015) + + diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md new file mode 100644 index 000000000..1b1d48435 --- /dev/null +++ b/docs/release-notes/known-issues.md @@ -0,0 +1,19 @@ +--- +navigation_title: "Elastic APM Python Agent" +--- + +# Elastic APM Python Agent known issues [elastic-apm-python-agent-known-issues] + +% Use the following template to add entries to this page. + +% :::{dropdown} Title of known issue +% **Details** +% On [Month/Day/Year], a known issue was discovered that [description of known issue]. + +% **Workaround** +% Workaround description. + +% **Resolved** +% On [Month/Day/Year], this issue was resolved. + +::: diff --git a/docs/release-notes/toc.yml b/docs/release-notes/toc.yml new file mode 100644 index 000000000..a41006794 --- /dev/null +++ b/docs/release-notes/toc.yml @@ -0,0 +1,5 @@ +toc: + - file: index.md + - file: known-issues.md + - file: breaking-changes.md + - file: deprecations.md \ No newline at end of file diff --git a/docs/run-tests-locally.asciidoc b/docs/run-tests-locally.asciidoc deleted file mode 100644 index fd3aa1eea..000000000 --- a/docs/run-tests-locally.asciidoc +++ /dev/null @@ -1,78 +0,0 @@ -[[run-tests-locally]] -=== Run Tests Locally - -To run tests locally you can make use of the docker images also used when running the whole test suite with Jenkins. -Running the full test suite first does some linting and then runs the actual tests with different versions of Python and different web frameworks. -For a full overview of the test matrix and supported versions have a look at -https://github.com/elastic/apm-agent-python/blob/main/Jenkinsfile[Jenkins Configuration]. - -[float] -[[pre-commit]] -==== Pre Commit -We run our git hooks on every commit to automatically point out issues in code. Those issues are also detected within the GitHub actions. -Please follow the installation steps stated in https://pre-commit.com/#install. - -[float] -[[coder-linter]] -==== Code Linter -We run two code linters `isort` and `flake8`. You can trigger each single one locally by running: - -[source,bash] ----- -$ pre-commit run -a isort ----- - -[source,bash] ----- -$ pre-commit run -a flake8 ----- - -[float] -[[coder-formatter]] -==== Code Formatter -We test that the code is formatted using `black`. You can trigger this check by running: - -[source,bash] ----- -$ pre-commit run -a black ----- - -[float] -[[test-documentation]] -==== Test Documentation -We test that the documentation can be generated without errors. You can trigger this check by running: -[source,bash] ----- -$ ./tests/scripts/docker/docs.sh ----- - -[float] -[[running-tests]] -==== Running Tests -We run the test suite on different combinations of Python versions and web frameworks. For triggering the test suite for a specific combination locally you can run: - -[source,bash] ----- -$ ./tests/scripts/docker/run_tests.sh python-version framework-version ----- -NOTE: The `python-version` must be of format `python-version`, e.g. `python-3.6` or `pypy-2`. -The `framework` must be of format `framework-version`, e.g. `django-1.10` or `flask-0.12`. - -You can also run the unit tests outside of docker, by installing the relevant -https://github.com/elastic/apm-agent-python/tree/main/tests/requirements[requirements file] -and then running `py.test` from the project root. - -==== Integration testing - -Check out https://github.com/elastic/apm-integration-testing for resources for -setting up full end-to-end testing environments. For example, to spin up -an environment with the https://github.com/basepi/opbeans-python[opbeans Django app], -with version 7.3 of the elastic stack and the apm-python-agent from your local -checkout, you might do something like this: - -[source,bash] ----- -$ ./scripts/compose.py start 7.3 \ - --with-agent-python-django --with-opbeans-python \ - --opbeans-python-agent-local-repo=~/elastic/apm-agent-python ----- diff --git a/docs/sanic.asciidoc b/docs/sanic.asciidoc deleted file mode 100644 index 83f8fd540..000000000 --- a/docs/sanic.asciidoc +++ /dev/null @@ -1,179 +0,0 @@ -[[sanic-support]] -=== Sanic Support - -Incorporating Elastic APM into your Sanic project only requires a few easy -steps. - -[float] -[[sanic-installation]] -==== Installation - -Install the Elastic APM agent using pip: - -[source,bash] ----- -$ pip install elastic-apm ----- - -or add `elastic-apm` to your project's `requirements.txt` file. - - -[float] -[[sanic-setup]] -==== Setup - -To set up the agent, you need to initialize it with appropriate settings. - -The settings are configured either via environment variables, or as -initialization arguments. - -You can find a list of all available settings in the -<> page. - -To initialize the agent for your application using environment variables: - -[source,python] ----- -from sanic import Sanic -from elasticapm.contrib.sanic import ElasticAPM - -app = Sanic(name="elastic-apm-sample") -apm = ElasticAPM(app=app) ----- - -To configure the agent using initialization arguments and Sanic's Configuration infrastructure: - -[source,python] ----- -# Create a file named external_config.py in your application -# If you want this module based configuration to be used for APM, prefix them with ELASTIC_APM_ -ELASTIC_APM_SERVER_URL = "https://serverurl.apm.com:443" -ELASTIC_APM_SECRET_TOKEN = "sometoken" ----- - -[source,python] ----- -from sanic import Sanic -from elasticapm.contrib.sanic import ElasticAPM - -app = Sanic(name="elastic-apm-sample") -app.config.update_config("path/to/external_config.py") -apm = ElasticAPM(app=app) ----- - -[float] -[[sanic-usage]] -==== Usage - -Once you have configured the agent, it will automatically track transactions -and capture uncaught exceptions within sanic. - -Capture an arbitrary exception by calling -<>: - -[source,python] ----- -from sanic import Sanic -from elasticapm.contrib.sanic import ElasticAPM - -app = Sanic(name="elastic-apm-sample") -apm = ElasticAPM(app=app) - -try: - 1 / 0 -except ZeroDivisionError: - apm.capture_exception() ----- - -Log a generic message with <>: - -[source,python] ----- -from sanic import Sanic -from elasticapm.contrib.sanic import ElasticAPM - -app = Sanic(name="elastic-apm-sample") -apm = ElasticAPM(app=app) - -apm.capture_message('hello, world!') ----- - -[float] -[[sanic-performance-metrics]] -==== Performance metrics - -If you've followed the instructions above, the agent has installed our -instrumentation middleware which will process all requests through your app. -This will measure response times, as well as detailed performance data for -all supported technologies. - -NOTE: Due to the fact that `asyncio` drivers are usually separate from their -synchronous counterparts, specific instrumentation is needed for all drivers. -The support for asynchronous drivers is currently quite limited. - -[float] -[[sanic-ignoring-specific-views]] -===== Ignoring specific routes - -You can use the -<> -configuration option to ignore specific routes. The list given should be a -list of regular expressions which are matched against the transaction name: - -[source,python] ----- -from sanic import Sanic -from elasticapm.contrib.sanic import ElasticAPM - -app = Sanic(name="elastic-apm-sample") -apm = ElasticAPM(app=app, config={ - 'TRANSACTIONS_IGNORE_PATTERNS': ['^GET /secret', '/extra_secret'], -}) ----- - -This would ignore any requests using the `GET /secret` route -and any requests containing `/extra_secret`. - -[float] -[[extended-sanic-usage]] -==== Extended Sanic APM Client Usage - -Sanic's contributed APM client also provides a few extendable way to configure selective behaviors to enhance the -information collected as part of the transactions being tracked by the APM. - -In order to enable this behavior, the APM Client middleware provides a few callback functions that you can leverage -in order to simplify the process of generating additional contexts into the traces being collected. -[cols="1,1,1,1"] -|=== -| Callback Name | Callback Invocation Format | Expected Return Format | Is Async - -| transaction_name_callback -| transaction_name_callback(request) -| string -| false - -| user_context_callback -| user_context_callback(request) -| (username_string, user_email_string, userid_string) -| true - -| custom_context_callback -| custom_context_callback(request) or custom_context_callback(response) -| dict(str=str) -| true - -| label_info_callback -| label_info_callback() -| dict(str=str) -| true -|=== - -[float] -[[supported-stanic-and-python-versions]] -==== Supported Sanic and Python versions - -A list of supported <> and -<> versions can be found on our -<> page. - -NOTE: Elastic APM only supports `asyncio` when using Python 3.7+ diff --git a/docs/serverless-azure-functions.asciidoc b/docs/serverless-azure-functions.asciidoc deleted file mode 100644 index b137c91c7..000000000 --- a/docs/serverless-azure-functions.asciidoc +++ /dev/null @@ -1,61 +0,0 @@ -[[azure-functions-support]] -=== Monitoring Azure Functions - -[float] -==== Prerequisites - -You need an APM Server to which you can send APM data. -Follow the {apm-guide-ref}/apm-quick-start.html[APM Quick start] if you have not set one up yet. -For the best-possible performance, we recommend setting up APM on {ecloud} in the same Azure region as your Azure Functions app. - -NOTE: Currently, only HTTP and timer triggers are supported. -Other trigger types may be captured as well, but the amount of captured contextual data may differ. - -[float] -==== Step 1: Enable Worker Extensions - -Elastic APM uses https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=asgi%2Capplication-level&pivots=python-mode-configuration#python-worker-extensions[Worker Extensions] -to instrument Azure Functions. -This feature is not enabled by default, and must be enabled in your Azure Functions App. -Please follow the instructions in the https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=asgi%2Capplication-level&pivots=python-mode-configuration#using-extensions[Azure docs]. - -Once you have enabled Worker Extensions, these two lines of code will enable Elastic APM's extension: - -[source,python] ----- -from elasticapm.contrib.serverless.azure import ElasticAPMExtension - -ElasticAPMExtension.configure() ----- - -Put them somewhere at the top of your Python file, before the function definitions. - -[float] -==== Step 2: Install the APM Python Agent - -You need to add `elastic-apm` as a dependency for your Functions app. -Simply add `elastic-apm` to your `requirements.txt` file. -We recommend pinning the version to the current newest version of the agent, and periodically updating the version. - -[float] -==== Step 3: Configure APM on Azure Functions - -The APM Python agent is configured through https://learn.microsoft.com/en-us/azure/azure-functions/functions-how-to-use-azure-function-app-settings?tabs=portal#settings[App Settings]. -These are then picked up by the agent as environment variables. - -For the minimal configuration, you will need the <> to set the destination for APM data and a <>. -If you prefer to use an {apm-guide-ref}/api-key.html[APM API key] instead of the APM secret token, use the <> environment variable instead of `ELASTIC_APM_SECRET_TOKEN` in the following example configuration. - -[source,bash] ----- -$ az functionapp config appsettings set --settings ELASTIC_APM_SERVER_URL=https://example.apm.northeurope.azure.elastic-cloud.com:443 -$ az functionapp config appsettings set --settings ELASTIC_APM_SECRET_TOKEN=verysecurerandomstring ----- - -You can optionally <>. - -That's it; Once the agent is installed and working, spans will be captured for -<>. You can also use -<> to capture custom spans, and -you can retrieve the `Client` object for capturing exceptions/messages -using <>. diff --git a/docs/serverless-lambda.asciidoc b/docs/serverless-lambda.asciidoc deleted file mode 100644 index 732abb2b4..000000000 --- a/docs/serverless-lambda.asciidoc +++ /dev/null @@ -1,53 +0,0 @@ -[[lambda-support]] -=== Monitoring AWS Lambda Python Functions -:layer-section-type: with-agent -:apm-aws-repo-dir: ./lambda - -The Python APM Agent can be used with AWS Lambda to monitor the execution of your AWS Lambda functions. - -``` -Note: The Centralized Agent Configuration on the Elasticsearch APM currently does NOT support AWS Lambda. -``` - - -[float] -==== Prerequisites - -You need an APM Server to send APM data to. Follow the {apm-guide-ref}/apm-quick-start.html[APM Quick start] if you have not set one up yet. For the best-possible performance, we recommend setting up APM on {ecloud} in the same AWS region as your AWS Lambda functions. - -[float] -==== Step 1: Select the AWS Region and Architecture - -include::{apm-aws-lambda-root}/docs/lambda-selector/lambda-attributes-selector.asciidoc[] - -[float] -==== Step 2: Add the APM Layers to your Lambda function - -include::{apm-aws-lambda-root}/docs/lambda-selector/extension-arn-replacement.asciidoc[] -include::./lambda/python-arn-replacement.asciidoc[] - -Both the {apm-lambda-ref}/aws-lambda-arch.html[{apm-lambda-ext}] and the Python APM Agent are added to your Lambda function as https://docs.aws.amazon.com/lambda/latest/dg/invocation-layers.html[AWS Lambda Layers]. Therefore, you need to add the corresponding Layer ARNs (identifiers) to your Lambda function. - -include::{apm-aws-lambda-root}/docs/add-extension/add-extension-layer-widget.asciidoc[] - -[float] -==== Step 3: Configure APM on AWS Lambda - -The {apm-lambda-ext} and the APM Python agent are configured through environment variables on the AWS Lambda function. - -For the minimal configuration, you will need the _APM Server URL_ to set the destination for APM data and an _{apm-guide-ref}/secret-token.html[APM Secret Token]_. -If you prefer to use an {apm-guide-ref}/api-key.html[APM API key] instead of the APM secret token, use the `ELASTIC_APM_API_KEY` environment variable instead of `ELASTIC_APM_SECRET_TOKEN` in the following configuration. - -For production environments, we recommend {apm-lambda-ref}/aws-lambda-secrets-manager.html[using the AWS Secrets Manager to store your APM authentication key] instead of providing the secret value as plaintext in the environment variables. - -include::./lambda/configure-lambda-widget.asciidoc[] -<1> The {apm-lambda-ref}/aws-lambda-config-options.html#_elastic_apm_send_strategy[`ELASTIC_APM_SEND_STRATEGY`] defines when APM data is sent to your Elastic APM backend. To reduce the execution time of your lambda functions, we recommend to use the `background` strategy in production environments with steady load scenarios. - -You can optionally <> or the {apm-lambda-ref}/aws-lambda-config-options.html[configuration of the {apm-lambda-ext}]. - -That's it. After following the steps above, you're ready to go! Your Lambda -function invocations should be traced from now on. Spans will be captured for -<>. You can also use -<> to capture custom spans, and you can -retrieve the `Client` object for capturing exceptions/messages using -<>. diff --git a/docs/set-up.asciidoc b/docs/set-up.asciidoc deleted file mode 100644 index 58e74294b..000000000 --- a/docs/set-up.asciidoc +++ /dev/null @@ -1,37 +0,0 @@ -[[set-up]] -== Set up the Agent - -To get you off the ground, we’ve prepared guides for setting up the Agent with different frameworks: - - * <> - * <> - * <> - * <> - * <> - * <> - * <> - * <> - * <> - * <> - -For custom instrumentation, see <>. - -include::./django.asciidoc[] - -include::./flask.asciidoc[] - -include::./aiohttp-server.asciidoc[] - -include::./tornado.asciidoc[] - -include::./starlette.asciidoc[] - -include::./sanic.asciidoc[] - -include::./serverless-lambda.asciidoc[] - -include::./serverless-azure-functions.asciidoc[] - -include::./wrapper.asciidoc[] - -include::./asgi-middleware.asciidoc[] diff --git a/docs/starlette.asciidoc b/docs/starlette.asciidoc deleted file mode 100644 index 941bf6d7a..000000000 --- a/docs/starlette.asciidoc +++ /dev/null @@ -1,152 +0,0 @@ -[[starlette-support]] -=== Starlette/FastAPI Support - -Incorporating Elastic APM into your Starlette project only requires a few easy -steps. - -[float] -[[starlette-installation]] -==== Installation - -Install the Elastic APM agent using pip: - -[source,bash] ----- -$ pip install elastic-apm ----- - -or add `elastic-apm` to your project's `requirements.txt` file. - - -[float] -[[starlette-setup]] -==== Setup - -To set up the agent, you need to initialize it with appropriate settings. - -The settings are configured either via environment variables, or as -initialization arguments. - -You can find a list of all available settings in the -<> page. - -To initialize the agent for your application using environment variables, add -the ElasticAPM middleware to your Starlette application: - -[source,python] ----- -from starlette.applications import Starlette -from elasticapm.contrib.starlette import ElasticAPM - -app = Starlette() -app.add_middleware(ElasticAPM) ----- - -WARNING: `BaseHTTPMiddleware` breaks `contextvar` propagation, as noted -https://www.starlette.io/middleware/#limitations[here]. This means the -ElasticAPM middleware must be above any `BaseHTTPMiddleware` in the final -middleware list. If you're calling `add_middleware` repeatedly, add the -ElasticAPM middleware last. If you're passing in a list of middleware, -ElasticAPM should be first on that list. - -To configure the agent using initialization arguments: - -[source,python] ----- -from starlette.applications import Starlette -from elasticapm.contrib.starlette import make_apm_client, ElasticAPM - -apm = make_apm_client({ - 'SERVICE_NAME': '', - 'SECRET_TOKEN': '', - 'SERVER_URL': '', -}) -app = Starlette() -app.add_middleware(ElasticAPM, client=apm) ----- - -[float] -[[starlette-fastapi]] -==== FastAPI - -Because FastAPI supports Starlette middleware, using the agent with FastAPI -is almost exactly the same as with Starlette: - -[source,python] ----- -from fastapi import FastAPI -from elasticapm.contrib.starlette import ElasticAPM - -app = FastAPI() -app.add_middleware(ElasticAPM) ----- - -[float] -[[starlette-usage]] -==== Usage - -Once you have configured the agent, it will automatically track transactions -and capture uncaught exceptions within starlette. - -Capture an arbitrary exception by calling -<>: - -[source,python] ----- -try: - 1 / 0 -except ZeroDivisionError: - apm.capture_exception() ----- - -Log a generic message with <>: - -[source,python] ----- -apm.capture_message('hello, world!') ----- - -[float] -[[starlette-performance-metrics]] -==== Performance metrics - -If you've followed the instructions above, the agent has installed our -instrumentation middleware which will process all requests through your app. -This will measure response times, as well as detailed performance data for -all supported technologies. - -NOTE: Due to the fact that `asyncio` drivers are usually separate from their -synchronous counterparts, specific instrumentation is needed for all drivers. -The support for asynchronous drivers is currently quite limited. - -[float] -[[starlette-ignoring-specific-views]] -===== Ignoring specific routes - -You can use the -<> -configuration option to ignore specific routes. The list given should be a -list of regular expressions which are matched against the transaction name: - -[source,python] ----- -apm = make_apm_client({ - # ... - 'TRANSACTIONS_IGNORE_PATTERNS': ['^GET /secret', '/extra_secret'] - # ... -}) ----- - -This would ignore any requests using the `GET /secret` route -and any requests containing `/extra_secret`. - - -[float] -[[supported-starlette-and-python-versions]] -==== Supported Starlette and Python versions - -A list of supported <> and -<> versions can be found on our -<> page. - -NOTE: Elastic APM only supports `asyncio` when using Python 3.7+ diff --git a/docs/supported-technologies.asciidoc b/docs/supported-technologies.asciidoc deleted file mode 100644 index 50198a102..000000000 --- a/docs/supported-technologies.asciidoc +++ /dev/null @@ -1,683 +0,0 @@ -[[supported-technologies]] -== Supported Technologies - -[[framework-support]] -The Elastic APM Python Agent comes with support for the following frameworks: - - * <> - * <> - * <> - * <> - * <> - * <> - * <> - -For other frameworks and custom Python code, the agent exposes a set of <> for integration. - -[float] -[[supported-python]] -=== Python - -The following Python versions are supported: - - * 3.6 - * 3.7 - * 3.8 - * 3.9 - * 3.10 - * 3.11 - * 3.12 - -[float] -[[supported-django]] -=== Django - -We support these Django versions: - - * 1.11 - * 2.0 - * 2.1 - * 2.2 - * 3.0 - * 3.1 - * 3.2 - * 4.0 - * 4.2 - * 5.0 - -For upcoming Django versions, we generally aim to ensure compatibility starting with the first Release Candidate. - -NOTE: we currently don't support Django running in ASGI mode. - -[float] -[[supported-flask]] -=== Flask - -We support these Flask versions: - - * 0.10 (Deprecated) - * 0.11 (Deprecated) - * 0.12 (Deprecated) - * 1.0 - * 1.1 - * 2.0 - * 2.1 - * 2.2 - * 2.3 - * 3.0 - -[float] -[[supported-aiohttp]] -=== Aiohttp Server - -We support these aiohttp versions: - - * 3.0+ - -[float] -[[supported-tornado]] -=== Tornado - -We support these tornado versions: - - * 6.0+ - - -[float] -[[supported-sanic]] -=== Sanic - -We support these sanic versions: - - * 20.12.2+ - - -[float] -[[supported-starlette]] -=== Starlette/FastAPI - -We support these Starlette versions: - - * 0.13.0+ - -Any FastAPI version which uses a supported Starlette version should also -be supported. - -[float] -[[supported-grpc]] -=== GRPC - -We support these `grpcio` versions: - - * 1.24.0+ - - -[float] -[[automatic-instrumentation]] -== Automatic Instrumentation - -The Python APM agent comes with automatic instrumentation of various 3rd party modules and standard library modules. - -[float] -[[automatic-instrumentation-scheduling]] -=== Scheduling - -[float] -[[automatic-instrumentation-scheduling-celery]] -===== Celery - -We support these Celery versions: - -* 4.x (deprecated) -* 5.x - -Celery tasks will be recorded automatically with Django and Flask only. - -[float] -[[automatic-instrumentation-db]] -=== Databases - -[float] -[[automatic-instrumentation-db-elasticsearch]] -==== Elasticsearch - -Instrumented methods: - - * `elasticsearch.transport.Transport.perform_request` - * `elasticsearch.connection.http_urllib3.Urllib3HttpConnection.perform_request` - * `elasticsearch.connection.http_requests.RequestsHttpConnection.perform_request` - * `elasticsearch._async.transport.AsyncTransport.perform_request` - * `elasticsearch_async.connection.AIOHttpConnection.perform_request` - -Additionally, the instrumentation wraps the following methods of the `Elasticsearch` client class: - - * `elasticsearch.client.Elasticsearch.delete_by_query` - * `elasticsearch.client.Elasticsearch.search` - * `elasticsearch.client.Elasticsearch.count` - * `elasticsearch.client.Elasticsearch.update` - -Collected trace data: - - * the query string (if available) - * the `query` element from the request body (if available) - * the response status code - * the count of affected rows (if available) - -We recommend using keyword arguments only with elasticsearch-py, as recommended by -https://elasticsearch-py.readthedocs.io/en/master/api.html#api-documentation[the elasticsearch-py docs]. -If you are using positional arguments, we will be unable to gather the `query` -element from the request body. - -[float] -[[automatic-instrumentation-db-sqlite]] -==== SQLite - -Instrumented methods: - - * `sqlite3.connect` - * `sqlite3.dbapi2.connect` - * `pysqlite2.dbapi2.connect` - -The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. - -Collected trace data: - - * parametrized SQL query - - -[float] -[[automatic-instrumentation-db-mysql]] -==== MySQLdb - -Library: `MySQLdb` - -Instrumented methods: - - * `MySQLdb.connect` - -The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. - -Collected trace data: - - * parametrized SQL query - -[float] -[[automatic-instrumentation-db-mysql-connector]] -==== mysql-connector - -Library: `mysql-connector-python` - -Instrumented methods: - - * `mysql.connector.connect` - -The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. - -Collected trace data: - - * parametrized SQL query - -[float] -[[automatic-instrumentation-db-pymysql]] -==== pymysql - -Library: `pymysql` - -Instrumented methods: - - * `pymysql.connect` - -The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. - -Collected trace data: - - * parametrized SQL query - -[float] -[[automatic-instrumentation-db-aiomysql]] -==== aiomysql - -Library: `aiomysql` - -Instrumented methods: - - * `aiomysql.cursors.Cursor.execute` - -Collected trace data: - - * parametrized SQL query - -[float] -[[automatic-instrumentation-db-postgres]] -==== PostgreSQL - -Library: `psycopg2`, `psycopg2-binary` (`>=2.9`) - -Instrumented methods: - - * `psycopg2.connect` - -The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. - -Collected trace data: - - * parametrized SQL query - -[float] -[[automatic-instrumentation-db-aiopg]] -==== aiopg - -Library: `aiopg` (`>=1.0`) - -Instrumented methods: - - * `aiopg.cursor.Cursor.execute` - * `aiopg.cursor.Cursor.callproc` - -Collected trace data: - - * parametrized SQL query - -[float] -[[automatic-instrumentation-db-asyncg]] -==== asyncpg - -Library: `asyncpg` (`>=0.20`) - -Instrumented methods: - - * `asyncpg.connection.Connection.execute` - * `asyncpg.connection.Connection.executemany` - - -Collected trace data: - - * parametrized SQL query - -[float] -[[automatic-instrumentation-db-pyodbc]] -==== PyODBC - -Library: `pyodbc`, (`>=4.0`) - -Instrumented methods: - - * `pyodbc.connect` - -The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. - -Collected trace data: - - * parametrized SQL query - -[float] -[[automatic-instrumentation-db-mssql]] -==== MS-SQL - -Library: `pymssql`, (`>=2.1.0`) - -Instrumented methods: - - * `pymssql.connect` - -The instrumented `connect` method returns a wrapped connection/cursor which instruments the actual `Cursor.execute` calls. - -Collected trace data: - - * parametrized SQL query - -[float] -[[automatic-instrumentation-db-mongodb]] -==== MongoDB - -Library: `pymongo`, `>=2.9,<3.8` - -Instrumented methods: - - * `pymongo.collection.Collection.aggregate` - * `pymongo.collection.Collection.bulk_write` - * `pymongo.collection.Collection.count` - * `pymongo.collection.Collection.create_index` - * `pymongo.collection.Collection.create_indexes` - * `pymongo.collection.Collection.delete_many` - * `pymongo.collection.Collection.delete_one` - * `pymongo.collection.Collection.distinct` - * `pymongo.collection.Collection.drop` - * `pymongo.collection.Collection.drop_index` - * `pymongo.collection.Collection.drop_indexes` - * `pymongo.collection.Collection.ensure_index` - * `pymongo.collection.Collection.find_and_modify` - * `pymongo.collection.Collection.find_one` - * `pymongo.collection.Collection.find_one_and_delete` - * `pymongo.collection.Collection.find_one_and_replace` - * `pymongo.collection.Collection.find_one_and_update` - * `pymongo.collection.Collection.group` - * `pymongo.collection.Collection.inline_map_reduce` - * `pymongo.collection.Collection.insert` - * `pymongo.collection.Collection.insert_many` - * `pymongo.collection.Collection.insert_one` - * `pymongo.collection.Collection.map_reduce` - * `pymongo.collection.Collection.reindex` - * `pymongo.collection.Collection.remove` - * `pymongo.collection.Collection.rename` - * `pymongo.collection.Collection.replace_one` - * `pymongo.collection.Collection.save` - * `pymongo.collection.Collection.update` - * `pymongo.collection.Collection.update_many` - * `pymongo.collection.Collection.update_one` - -Collected trace data: - - * database name - * method name - - -[float] -[[automatic-instrumentation-db-redis]] -==== Redis - -Library: `redis` (`>=2.8`) - -Instrumented methods: - - * `redis.client.Redis.execute_command` - * `redis.client.Pipeline.execute` - -Collected trace data: - - * Redis command name - - -[float] -[[automatic-instrumentation-db-aioredis]] -==== aioredis - -Library: `aioredis` (`<2.0`) - -Instrumented methods: - - * `aioredis.pool.ConnectionsPool.execute` - * `aioredis.commands.transaction.Pipeline.execute` - * `aioredis.connection.RedisConnection.execute` - -Collected trace data: - - * Redis command name - -[float] -[[automatic-instrumentation-db-cassandra]] -==== Cassandra - -Library: `cassandra-driver` (`>=3.4,<4.0`) - -Instrumented methods: - - * `cassandra.cluster.Session.execute` - * `cassandra.cluster.Cluster.connect` - -Collected trace data: - - * CQL query - -[float] -[[automatic-instrumentation-db-python-memcache]] -==== Python Memcache - -Library: `python-memcached` (`>=1.51`) - -Instrumented methods: - -* `memcache.Client.add` -* `memcache.Client.append` -* `memcache.Client.cas` -* `memcache.Client.decr` -* `memcache.Client.delete` -* `memcache.Client.delete_multi` -* `memcache.Client.disconnect_all` -* `memcache.Client.flush_all` -* `memcache.Client.get` -* `memcache.Client.get_multi` -* `memcache.Client.get_slabs` -* `memcache.Client.get_stats` -* `memcache.Client.gets` -* `memcache.Client.incr` -* `memcache.Client.prepend` -* `memcache.Client.replace` -* `memcache.Client.set` -* `memcache.Client.set_multi` -* `memcache.Client.touch` - -Collected trace data: - -* Destination (address and port) - -[float] -[[automatic-instrumentation-db-pymemcache]] -==== pymemcache - -Library: `pymemcache` (`>=3.0`) - -Instrumented methods: - -* `pymemcache.client.base.Client.add` -* `pymemcache.client.base.Client.append` -* `pymemcache.client.base.Client.cas` -* `pymemcache.client.base.Client.decr` -* `pymemcache.client.base.Client.delete` -* `pymemcache.client.base.Client.delete_many` -* `pymemcache.client.base.Client.delete_multi` -* `pymemcache.client.base.Client.flush_all` -* `pymemcache.client.base.Client.get` -* `pymemcache.client.base.Client.get_many` -* `pymemcache.client.base.Client.get_multi` -* `pymemcache.client.base.Client.gets` -* `pymemcache.client.base.Client.gets_many` -* `pymemcache.client.base.Client.incr` -* `pymemcache.client.base.Client.prepend` -* `pymemcache.client.base.Client.quit` -* `pymemcache.client.base.Client.replace` -* `pymemcache.client.base.Client.set` -* `pymemcache.client.base.Client.set_many` -* `pymemcache.client.base.Client.set_multi` -* `pymemcache.client.base.Client.stats` -* `pymemcache.client.base.Client.touch` - -Collected trace data: - -* Destination (address and port) - -[float] -[[automatic-instrumentation-db-kafka-python]] -==== kafka-python - -Library: `kafka-python` (`>=2.0`) - -Instrumented methods: - - * `kafka.KafkaProducer.send`, - * `kafka.KafkaConsumer.poll`, - * `kafka.KafkaConsumer.\\__next__` - -Collected trace data: - - * Destination (address and port) - * topic (if applicable) - - -[float] -[[automatic-instrumentation-http]] -=== External HTTP requests - -[float] -[[automatic-instrumentation-stdlib-urllib]] -==== Standard library - -Library: `urllib2` (Python 2) / `urllib.request` (Python 3) - -Instrumented methods: - - * `urllib2.AbstractHTTPHandler.do_open` / `urllib.request.AbstractHTTPHandler.do_open` - -Collected trace data: - - * HTTP method - * requested URL - -[float] -[[automatic-instrumentation-urllib3]] -==== urllib3 - -Library: `urllib3` - -Instrumented methods: - - * `urllib3.connectionpool.HTTPConnectionPool.urlopen` - -Additionally, we instrumented vendored instances of urllib3 in the following libraries: - - * `requests` - * `botocore` - -Both libraries have "unvendored" urllib3 in more recent versions, we recommend to use the newest versions. - -Collected trace data: - - * HTTP method - * requested URL - -[float] -[[automatic-instrumentation-requests]] -==== requests - -Instrumented methods: - - * `requests.sessions.Session.send` - -Collected trace data: - - * HTTP method - * requested URL - -[float] -[[automatic-instrumentation-aiohttp-client]] -==== AIOHTTP Client - -Instrumented methods: - - * `aiohttp.client.ClientSession._request` - -Collected trace data: - - * HTTP method - * requested URL - -[float] -[[automatic-instrumentation-httpx]] -==== httpx - -Instrumented methods: - - * `httpx.Client.send - -Collected trace data: - - * HTTP method - * requested URL - - -[float] -[[automatic-instrumentation-services]] -=== Services - -[float] -[[automatic-instrumentation-boto3]] -==== AWS Boto3 / Botocore - -Library: `boto3` (`>=1.0`) - -Instrumented methods: - - * `botocore.client.BaseClient._make_api_call` - -Collected trace data for all services: - - * AWS region (e.g. `eu-central-1`) - * AWS service name (e.g. `s3`) - * operation name (e.g. `ListBuckets`) - -Additionally, some services collect more specific data - -[float] -[[automatic-instrumentation-aiobotocore]] -==== AWS Aiobotocore - -Library: `aiobotocore` (`>=2.2.0`) - -Instrumented methods: - - * `aiobotocore.client.BaseClient._make_api_call` - -Collected trace data for all services: - - * AWS region (e.g. `eu-central-1`) - * AWS service name (e.g. `s3`) - * operation name (e.g. `ListBuckets`) - -Additionally, some services collect more specific data - -[float] -[[automatic-instrumentation-s3]] -===== S3 - - * Bucket name - -[float] -[[automatic-instrumentation-dynamodb]] -===== DynamoDB - - * Table name - - -[float] -[[automatic-instrumentation-sns]] -===== SNS - - * Topic name - -[float] -[[automatic-instrumentation-sqs]] -===== SQS - - * Queue name - -[float] -[[automatic-instrumentation-template-engines]] -=== Template Engines - -[float] -[[automatic-instrumentation-dtl]] -==== Django Template Language - -Library: `Django` (see <> for supported versions) - -Instrumented methods: - - * `django.template.Template.render` - -Collected trace data: - - * template name - -[float] -[[automatic-instrumentation-jinja2]] -==== Jinja2 - -Library: `jinja2` - -Instrumented methods: - - * `jinja2.Template.render` - -Collected trace data: - - * template name diff --git a/docs/tornado.asciidoc b/docs/tornado.asciidoc deleted file mode 100644 index c7281761f..000000000 --- a/docs/tornado.asciidoc +++ /dev/null @@ -1,125 +0,0 @@ -[[tornado-support]] -=== Tornado Support - -Incorporating Elastic APM into your Tornado project only requires a few easy -steps. - -[float] -[[tornado-installation]] -==== Installation - -Install the Elastic APM agent using pip: - -[source,bash] ----- -$ pip install elastic-apm ----- - -or add `elastic-apm` to your project's `requirements.txt` file. - - -[float] -[[tornado-setup]] -==== Setup - -To set up the agent, you need to initialize it with appropriate settings. - -The settings are configured either via environment variables, -the application's settings, or as initialization arguments. - -You can find a list of all available settings in the -<> page. - -To initialize the agent for your application using environment variables: - -[source,python] ----- -import tornado.web -from elasticapm.contrib.tornado import ElasticAPM - -app = tornado.web.Application() -apm = ElasticAPM(app) ----- - -To configure the agent using `ELASTIC_APM` in your application's settings: - -[source,python] ----- -import tornado.web -from elasticapm.contrib.tornado import ElasticAPM - -app = tornado.web.Application() -app.settings['ELASTIC_APM'] = { - 'SERVICE_NAME': '', - 'SECRET_TOKEN': '', -} -apm = ElasticAPM(app) ----- - -[float] -[[tornado-usage]] -==== Usage - -Once you have configured the agent, it will automatically track transactions -and capture uncaught exceptions within tornado. - -Capture an arbitrary exception by calling -<>: - -[source,python] ----- -try: - 1 / 0 -except ZeroDivisionError: - apm.client.capture_exception() ----- - -Log a generic message with <>: - -[source,python] ----- -apm.client.capture_message('hello, world!') ----- - -[float] -[[tornado-performance-metrics]] -==== Performance metrics - -If you've followed the instructions above, the agent has installed our -instrumentation within the base RequestHandler class in tornado.web. This will -measure response times, as well as detailed performance data for all supported -technologies. - -NOTE: Due to the fact that `asyncio` drivers are usually separate from their -synchronous counterparts, specific instrumentation is needed for all drivers. -The support for asynchronous drivers is currently quite limited. - -[float] -[[tornado-ignoring-specific-views]] -===== Ignoring specific routes - -You can use the -<> -configuration option to ignore specific routes. The list given should be a -list of regular expressions which are matched against the transaction name: - -[source,python] ----- -app.settings['ELASTIC_APM'] = { - # ... - 'TRANSACTIONS_IGNORE_PATTERNS': ['^GET SecretHandler', 'MainHandler'] - # ... -} ----- - -This would ignore any requests using the `GET SecretHandler` route -and any requests containing `MainHandler`. - - -[float] -[[supported-tornado-and-python-versions]] -==== Supported tornado and Python versions - -A list of supported <> and <> versions can be found on our <> page. - -NOTE: Elastic APM only supports `asyncio` when using Python 3.7+ diff --git a/docs/troubleshooting.asciidoc b/docs/troubleshooting.asciidoc deleted file mode 100644 index 40b8ed8fe..000000000 --- a/docs/troubleshooting.asciidoc +++ /dev/null @@ -1,172 +0,0 @@ -[[troubleshooting]] -== Troubleshooting - -Below are some resources and tips for troubleshooting and debugging the -python agent. - -* <> -* <> -* <> -* <> - -[float] -[[easy-fixes]] -=== Easy Fixes - -Before you try anything else, go through the following sections to ensure that -the agent is configured correctly. This is not an exhaustive list, but rather -a list of common problems that users run into. - -[float] -[[debug-mode]] -==== Debug Mode - -Most frameworks support a debug mode. Generally, this mode is intended for -non-production environments and provides detailed error messages and logging of -potentially sensitive data. Because of these security issues, the agent will -not collect traces if the app is in debug mode by default. - -You can override this behavior with the <> configuration. - -Note that configuration of the agent should occur before creation of any -`ElasticAPM` objects: - -[source,python] ----- -app = Flask(__name__) -app.config["ELASTIC_APM"] = {"DEBUG": True} -apm = ElasticAPM(app, service_name="flask-app") ----- - -[float] -[[psutil-metrics]] -==== `psutil` for Metrics - -To get CPU and system metrics on non-Linux systems, `psutil` must be -installed. The agent should automatically show a warning on start if it is -not installed, but sometimes this warning can be suppressed. Install `psutil` -and metrics should be collected by the agent and sent to the APM Server. - -[source,bash] ----- -python3 -m pip install psutil ----- - -[float] -[[apm-server-credentials]] -==== Credential issues - -In order for the agent to send data to the APM Server, it may need an -<> or a <>. Double -check your APM Server settings and make sure that your credentials are -configured correctly. Additionally, check that <> -is correct. - -[float] -[[django-test]] -=== Django `check` and `test` - -When used with Django, the agent provides two management commands to help debug -common issues. Head over to the <> -for more information. - -[float] -[[agent-logging]] -=== Agent logging - -To get the agent to log more data, all that is needed is a -https://docs.python.org/3/library/logging.html#handler-objects[Handler] which -is attached either to the `elasticapm` logger or to the root logger. - -Note that if you attach the handler to the root logger, you also need to -explicitly set the log level of the `elasticapm` logger: - -[source,python] ----- -import logging -apm_logger = logging.getLogger("elasticapm") -apm_logger.setLevel(logging.DEBUG) ----- - -[float] -[[django-agent-logging]] -==== Django - -The simplest way to log more data from the agent is to add a console logging -Handler to the `elasticapm` logger. Here's a (very simplified) example: - -[source,python] ----- -LOGGING = { - 'handlers': { - 'console': { - 'level': 'DEBUG', - 'class': 'logging.StreamHandler' - } - }, - 'loggers': { - 'elasticapm': { - 'level': 'DEBUG', - 'handlers': ['console'] - }, - }, -} ----- - -[float] -[[flask-agent-logging]] -==== Flask - -Flask https://flask.palletsprojects.com/en/1.1.x/logging/[recommends using `dictConfig()`] -to set up logging. If you're using this format, adding logging for the agent -will be very similar to the <>. - -Otherwise, you can use the <>. - -[float] -[[generic-agent-logging]] -==== Generic instructions - -Creating a console Handler and adding it to the `elasticapm` logger is easy: - -[source,python] ----- -import logging - -elastic_apm_logger = logging.getLogger("elasticapm") -console_handler = logging.StreamHandler() -console_handler.setLevel(logging.DEBUG) -elastic_apm_logger.addHandler(console_handler) ----- - -You can also just add the console Handler to the root logger. This will apply -that handler to all log messages from all modules. - -[source,python] ----- -import logging - -logger = logging.getLogger() -console_handler = logging.StreamHandler() -console_handler.setLevel(logging.DEBUG) -logger.addHandler(console_handler) ----- - -See the https://docs.python.org/3/library/logging.html[python logging docs] -for more details about Handlers (and information on how to format your logs -using Formatters). - -[float] -[[disable-agent]] -=== Disable the Agent - -In the unlikely event the agent causes disruptions to a production application, -you can disable the agent while you troubleshoot. - -If you have access to <>, -you can disable the recording of events by setting <> to `false`. -When changed at runtime from a supported source, there's no need to restart your application. - -If that doesn't work, or you don't have access to dynamic configuration, you can disable the agent by setting -<> to `false`. -You'll need to restart your application for the changes to take effect. diff --git a/docs/tuning.asciidoc b/docs/tuning.asciidoc deleted file mode 100644 index 6030f29cc..000000000 --- a/docs/tuning.asciidoc +++ /dev/null @@ -1,115 +0,0 @@ -[[tuning-and-overhead]] -== Performance tuning - -Using an APM solution comes with certain trade-offs, and the Python agent for Elastic APM is no different. -Instrumenting your code, measuring timings, recording context data, etc., all need resources: - - * CPU time - * memory - * bandwidth use - * Elasticsearch storage - -We invested and continue to invest a lot of effort to keep the overhead of using Elastic APM as low as possible. -But because every deployment is different, there are some knobs you can turn to adapt it to your specific needs. - -[float] -[[tuning-sample-rate]] -=== Transaction Sample Rate - -The easiest way to reduce the overhead of the agent is to tell the agent to do less. -If you set the <> to a value below `1.0`, -the agent will randomly sample only a subset of transactions. -Unsampled transactions only record the name of the transaction, the overall transaction time, and the result: - -[options="header"] -|============ -| Field | Sampled | Unsampled -| Transaction name | yes | yes -| Duration | yes | yes -| Result | yes | yes -| Context | yes | no -| Tags | yes | no -| Spans | yes | no -|============ - -Reducing the sample rate to a fraction of all transactions can make a huge difference in all four of the mentioned resource types. - -[float] -[[tuning-queue]] -=== Transaction Queue - -To reduce the load on the APM Server, the agent does not send every transaction up as it happens. -Instead, it queues them up and flushes the queue periodically, or when it reaches a maximum size, using a background thread. - -While this reduces the load on the APM Server (and to a certain extent on the agent), -holding on to the transaction data in a queue uses memory. -If you notice that using the Python agent results in a large increase of memory use, -you can use these settings: - - * <> to reduce the time between queue flushes - * <> to reduce the maximum size of the queue - -The first setting, `api_request_time`, is helpful if you have a sustained high number of transactions. -The second setting, `api_request_size`, can help if you experience peaks of transactions -(a large number of transactions in a short period of time). - -Keep in mind that reducing the value of either setting will cause the agent to send more HTTP requests to the APM Server, -potentially causing a higher load. - -[float] -[[tuning-max-spans]] -=== Spans per transaction - -The average amount of spans per transaction can influence how much time the agent spends in each transaction collecting contextual data for each span, -and the storage space needed in Elasticsearch. -In our experience, most _usual_ transactions should have well below 100 spans. -In some cases, however, the number of spans can explode: - - * long-running transactions - * unoptimized code, e.g. doing hundreds of SQL queries in a loop - -To avoid these edge cases overloading both the agent and the APM Server, -the agent stops recording spans when a specified limit is reached. -You can configure this limit by changing the <> setting. - -[float] -[[tuning-span-stack-trace-collection]] -=== Span Stack Trace Collection - -Collecting stack traces for spans can be fairly costly from a performance standpoint. -Stack traces are very useful for pinpointing which part of your code is generating a span; -however, these stack traces are less useful for very short spans (as problematic spans tend to be longer). - -You can define a minimal threshold for span duration -using the <> setting. -If a span's duration is less than this config value, no stack frames will be collected for this span. - -[float] -[[tuning-frame-context]] -=== Collecting Frame Context - -When a stack trace is captured, the agent will also capture several lines of source code around each frame location in the stack trace. This allows the APM app to give greater insight into where exactly the error or span happens. - -There are four settings you can modify to control this behavior: - -* <> -* <> -* <> -* <> - -As you can see, these settings are divided between app frames, which represent your application code, and library frames, which represent the code of your dependencies. Each of these categories are also split into separate error and span settings. - -Reading source files inside a running application can cause a lot of disk I/O, and sending up source lines for each frame will have a network and storage cost that is quite high. Turning down these limits will help prevent excessive memory usage. - -[float] -[[tuning-body-headers]] -=== Collecting headers and request body - -You can configure the Elastic APM agent to capture headers of both requests and responses (<>), -as well as request bodies (<>). -By default, capturing request bodies is disabled. -Enabling it for transactions may introduce noticeable overhead, as well as increased storage use, depending on the nature of your POST requests. -In most scenarios, we advise against enabling request body capturing for transactions, and only enable it if necessary for errors. - -Capturing request/response headers has less overhead on the agent, but can have an impact on storage use. -If storage use is a problem for you, it might be worth disabling. diff --git a/docs/upgrading.asciidoc b/docs/upgrading.asciidoc deleted file mode 100644 index 509116a06..000000000 --- a/docs/upgrading.asciidoc +++ /dev/null @@ -1,81 +0,0 @@ -[[upgrading]] -== Upgrading - -Upgrades between minor versions of the agent, like from 3.1 to 3.2 are always backwards compatible. -Upgrades that involve a major version bump often come with some backwards incompatible changes. - -We highly recommend to always pin the version of `elastic-apm` in your `requirements.txt` or `Pipfile`. -This avoids automatic upgrades to potentially incompatible versions. - -[float] -[[end-of-life-dates]] -=== End of life dates - -We love all our products, but sometimes we must say goodbye to a release so that we can continue moving -forward on future development and innovation. -Our https://www.elastic.co/support/eol[End of life policy] defines how long a given release is considered supported, -as well as how long a release is considered still in active development or maintenance. - -[[upgrading-6.x]] -=== Upgrading to version 6 of the agent - -==== Python 2 no longer supported - -Please upgrade to Python 3.6+ to continue to receive regular updates. - -==== `SANITIZE_FIELD_NAMES` changes - -If you are using a non-default `sanitize_field_names` config, please note -that your entries must be surrounded with stars (e.g. `*secret*`) in order to -maintain previous behavior. - -==== Tags removed (in favor of labels) - -Tags were deprecated in the 5.x release (in favor of labels). They have now been -removed. - -[[upgrading-5.x]] -=== Upgrading to version 5 of the agent - -==== APM Server 7.3 required for some features - -APM Server and Kibana 7.3 introduced support for collecting breakdown metrics, and central configuration of APM agents. -To use these features, please update the Python agent to 5.0+ and APM Server / Kibana to 7.3+ - -==== Tags renamed to Labels - -To better align with other parts of the Elastic Stack and the {ecs-ref}/index.html[Elastic Common Schema], -we renamed "tags" to "labels", and introduced limited support for typed labels. -While tag values were only allowed to be strings, label values can be strings, booleans, or numerical. - -To benefit from this change, ensure that you run at least *APM Server 6.7*, and use `elasticapm.label()` instead of `elasticapm.tag()`. -The `tag()` API will continue to work as before, but emit a `DeprecationWarning`. It will be removed in 6.0 of the agent. - -[[upgrading-4.x]] -=== Upgrading to version 4 of the agent - -4.0 of the Elastic APM Python Agent comes with several backwards incompatible changes. - -[[upgrading-4.x-apm-server]] -==== APM Server 6.5 required -This version of the agent is *only compatible with APM Server 6.5+*. -To upgrade, we recommend to first upgrade APM Server, and then the agent. -APM Server 6.5+ is backwards compatible with versions 2.x and 3.x of the agent. - -[[upgrading-4.x-configuration]] -==== Configuration options - -Several configuration options have been removed, or renamed - - * `flush_interval` has been removed - * the `flush_interval` and `max_queue_size` settings have been removed. - * new settings introduced: `api_request_time` and `api_request_size`. - * Some settings now require a unit for duration or size. See <> and <>. - -[[upgrading-4.x-processors]] -==== Processors - -The method to write processors for sanitizing events has been changed. -It will now be called for every type of event (transactions, spans and errors), -unless the event types are limited using a decorator. -See <> for more information.