Merge pull request #370 from ScalaConsultants/350-decomission-agent

Remove agent project

CONTRIBUTORS.md

@@ -1,68 +1,48 @@
-
-
 # How to contribute
 
-Mesmer is a project to extract and export akka specific metrics.
-
-Projects is comprised of 3 major parts:
-- java agent (called `agent` from now on) - a piece of technology allowing to modify classes during load.
-- akka extension (called `extension` from now on)  - this is where all custom metrics computation happens.
-- application that is being instrumented - in repo we have the "example" app, but this in theory this could be any application that uses akka.
-
-The most interesting parts are java agent and akka extension. They have to work in tandem - `agent` creates a foundation on which `extension` can build and calculate metrics.
-It's good to use an example to get a better understanding: `agent` allows to trigger an event whenever akka persistence query starts and completes.
-We've found places where those events should be triggered by looking at Akka source code and then modified necessary classes during load).
-`extension` then captures those events, calculates the duration between the two, and passes this calculated metric (request time) in opentelemetry instrument. There will be more about opentelemetry role in the product later in this document.
-
-
-## Extension
-
-It's a part that is responsible for calculating and exporting metrics. The fact that it's an Akka extension is an implementation detail, but it's worth knowing that.
-Akka extensions are plugins that grant additional functionality to Akka and can be enabled with configuration - this is the approach that we took.
-At the start up of the extension we check for configured metrics that were enabled and start only those monitors that are required.
-In our example monitors are akka system actors, but this is also an implementation detail.
-As an example `io.scalac.mesmer.extension.PersistenceEventsActor$` is a monitor that listens for Akka Persistence events.
-For conveyor of messages we use Reception
-
-
-## Opentelemetry
-
-It's a fusion of opentracing and openCensus. It's an specification and an implementation for creating, aggregating and publishing traces and metrics.
-We use this extensively as it's makes publishing metrics to different backends (prometheus, datadog etc) easy and can take care of some aggregations (mostly histograms ATM).
-Links to read more:
-- https://opentelemetry.io/docs/instrumentation/java/
-- https://github.com/open-telemetry/opentelemetry-specification
-- https://github.com/open-telemetry/opentelemetry-java
-
+Mesmer instruments Metrics using an OpenTelemetry Agent and our custom Opentelemetry Extension. Additionally, for Akka
+Metrics we use an extra piece: an Akka Extension.
 
-## Agent
+To learn more:
 
-This is probably the most complicated part of mesmer development (there is an ongoing [proposal](https://github.com/ScalaConsultants/mesmer-akka-agent/discussions/272) to switch from our custom agent to opentelemetry one which would simplify development and enable tracing).
-In some cases it requires writing `.java` files as scala doesn't allow some operations that are needed for class instrumentations.
-We use [ByteBuddy](https://bytebuddy.net/#/) as a preferred tool for bytecode generation - it operates on a higher level than ASM and has an `Advice` abstraction - this can be thing as a blueprint that will be injected into loaded class file.
+- [OpenTelemetry Javaagent](https://github.com/open-telemetry/opentelemetry-java-instrumentation#about)
+- [OpenTelemetry Extension](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/examples/extension/README.md)
+- [Akka Extension](https://doc.akka.io/docs/akka/current/extending-akka.html)
 
-Links to read more:
-- https://bytebuddy.net/#/tutorial
-- https://www.youtube.com/watch?v=oflzFGONG08 - introduction to bytebuddy and advices by it's creator Rafael Winterhalter
+In a bird's eye view, the project works as follows: the OpenTelemetry Agent picks up our OTEL Extension to install all
+bytecode modifications that need to be done to auto-instrument an application (ByteBuddy Advice classes + helper code).
+The Advice classes send _Events_ to the Akka Extension that is essentially an Event Bus. Then the Akka Extension
+processes the events and calls Opentelemetry API to report various metrics for you.
 
 ## How to add a new metric
 
 Although adding a new metrics might be a little tricky, there are a couple of steps that are worth following:
 
 * choose a metric that you want to instrument
-* check which api might expose this metric
-* start with tests first - check if you have an application that use chosen API e.g. if it's an Akka library, see if our example app uses that.
-If not, create a simple example app for testing (consider contributing it to the `examples` directory in Mesmer)
-* choose what is the kind of metric e.g. is a gauge, counter or histogram?
-* dive into library source code - find fragments of code that you can hook on to extract a metric.
-This is a tricky part and in many cases you might be forced to change previously made decision (e.g. change the metric type).
-This might get overwhelming quickly, so try to deliver a working PoC rather than figuring out all the details upfront.
-* write `agent` instrumentation code - this varies from metric to metric, but there are a couple rules of thumb
-  * don't use mutable state - currently preferred approach is to fire an event with `EventBus`
-  * use `Advice` rather than `Intercept` API
-  * use java classes as last resort and try to make it as thin as possible - it's usually better to write logic in scala functions
-* write monitor classes - current approach is not to use `opentelemetry` monitors directly but to use our `wrappers` which allowed for better testability.
-Check `io.scalac.mesmer.extension.metric` package in `extension` for current implementations.
-* write an event handling class - currently those are solely actors, but this might change in the future. This is the part that should take
-previously defined monitor as a parameter and submit values to it after it's done doing calculations.
-* remember about tests for both `extension` and `agent` part
+* check which API of the instrumented library might expose this metric
+* start with the tests first - check if you have an application that uses the chosen API. For exampleL if it's an Akka
+  library, see if our example app uses that. If not, create a simple example app for testing (consider contributing it
+  to the `example` module in Mesmer)
+* answer the following question: what is the kind of the metric you wish to add? Is it a gauge, counter or histogram?
+* dive into library source code - find fragments of code that you can hook on to extract a metric. This is a tricky part
+  and in many cases you might be forced to change previously made decision (e.g. change the metric type). This might get
+  overwhelming quickly, so try to deliver a working PoC rather than figuring out all the details upfront.
+* write the byte buddy instrumentation code in the OTEL Extension. You will need to add an InstrumentationModule. See
+  [MesmerAkkaPersistenceInstrumentationModule](https://github.com/ScalaConsultants/mesmer/blob/main/otel-extension/src/main/java/io/scalac/mesmer/otelextension/MesmerAkkaPersistenceInstrumentationModule.java)
+  as an example.
+* Adding instrumentation code varies from metric to metric, but there are a couple of rules of thumb:
+    * don't use mutable state - currently preferred approach is to fire an event with `EventBus`. You can also use
+      a [VirtualField](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation-api/src/main/java/io/opentelemetry/instrumentation/api/field/VirtualField.java)
+      to store some state (attributes, context).
+    * use
+      ByteBuddy's [Advice API](https://javadoc.io/static/net.bytebuddy/byte-buddy/1.10.2/net/bytebuddy/asm/Advice.html)
+    * use Java classes as last resort and try to make it as thin as possible - it's usually better to write logic with
+      Scala code
+* write monitor classes - current approach is not to use `opentelemetry` monitors directly but to use our `wrappers`
+  which allowed for better testability. Check `io.scalac.mesmer.extension.metric` package in `extension` for current
+  implementations.
+* write an event handling class - currently those are solely actors, but this might change in the future. This is the
+  part that should take previously defined monitor as a parameter and submit values to it after it's done doing
+  calculations.
+* remember about testing: for example see
+  the [AkkaPersistenceAgentTest](https://github.com/ScalaConsultants/mesmer/blob/main/otel-extension/src/test/scala/io/scalac/mesmer/instrumentation/akka/persistence/AkkaPersistenceAgentTest.scala)

README.md

@@ -2,9 +2,9 @@
 | --- | --- | --- |
 | ![Scala CI][Badge-CI] | [![Release Artifacts][badge-releases]][link-releases] | [![Snapshot Artifacts][badge-snapshots]][link-snapshots] |
 
-# Mesmer Akka Agent
+# Mesmer
 
-Mesmer Akka Agent is an [OpenTelemetry](https://opentelemetry.io/) instrumentation library for [Akka](https://akka.io/)
+Mesmer is an [OpenTelemetry](https://opentelemetry.io/) instrumentation library for [Akka](https://akka.io/)
 applications.
 
 ## Compatibility
@@ -19,8 +19,8 @@ Mesmer has been tested with:
 
 Mesmer consists of two parts:
 
-- Akka extension - that runs in the background and is responsible for exporting the metrics to your chosen backend
-- JVM agent - that instruments Akka classes to expose metrics for the extension
+- Akka Extension - that runs in the background and is responsible for exporting the metrics to your chosen backend
+- OpenTelemetry Agent Extension - that instruments Akka classes to expose metrics for the Extension
 
 Both parts need to be included in the application for Mesmer to work.
 
@@ -36,50 +36,32 @@ Add this entry to your `application.conf`:
 
     akka.actor.typed.extensions= ["io.scalac.mesmer.extension.AkkaMonitoring"]
 
-### JVM agent:
+### OpenTelemetry Extension:
 
-Download the latest agent jar from https://github.com/ScalaConsultants/mesmer-akka-agent/releases and add a parameter
-when running your JVM:
-
-    java -javaagent {PATH_TO_JAR}/mesmer-akka-agent.jar
-
-where `PATH_TO_JAR` is your **absolute** path to the Mesmer agent jar.
-
-### Exporter:
-
-Since neither the Mesmer agent nor the Mesmer Akka Extension set up the Open Telemetry SDK, you need to do that in your
-app. Only then your application will send your metrics to the Open Telemetry Collector.
-
-For example, you can set up your project with an OTLP Metrics Exporter. To do that you will need to implement an Open
-Telemetry Configuration Provider:
-
-1. add the necessary dependencies in your build.sbt file:
+Download the latest OTEL Extension jar from https://github.com/ScalaConsultants/mesmer/releases and add a parameter when
+running your JVM:
 
 ```
-"io.opentelemetry" % "opentelemetry-exporter-otlp-metrics" % <version>,
-"io.opentelemetry" % "opentelemetry-sdk" % <version>
+    java -javaagent:opentelemetry-javaagent110.jar \ -- this is the OpenTelemetry Agent
+    -Dotel.javaagent.extensions=mesmer-otel-extension.jar -- this is our OTEL Agent Extension
 ```
 
-2. implement the `OpenTelemetryProvider` interface similarly to what's done in `OpenTelemetrySetup` class (the example
-   application),
-3. don't forget to make the service provider discoverable - add
-   the `META-INF/services/io.scalac.mesmer.extension.opentelemetry.OpenTelemetryProvider` in the application code,
-   pointing to your OpenTelemetry Provider class.
+### Exporter:
 
-## Supported metrics
+Mesmer itself uses only OpenTelemetry API - it's the user's responsibility to setup the OpenTelemetry SDK.
 
-Mesmer currently supports the following Akka modules:
+We highly recommend using
+the [OpenTelemetry Sdk Autoconfigure](https://github.com/open-telemetry/opentelemetry-java/tree/main/sdk-extensions/autoconfigure)
+artifact. We use it in our example application too. It will set up the OpenTelemetry SDK and an Exporter for you and
+will provide you with sensible default settings for it.
 
-- Akka (core)
-- Akka Cluster
-- Akka Persistence
-- Akka Streams
+## Supported metrics
 
-For a detailed list of metrics go to [metrics](metrics.md)
+For a detailed list of supported metrics go to [supported_metrics.md](supported_metrics.md)
 
 # Local testing
 
-`example` subproject contains a test application that uses Akka Cluster sharding with Mesmer Akka Agent extension.
+`example` subproject contains a test application that uses Akka Cluster sharding with Mesmer Akka extension.
 Go [here](example/README.md) for more information.
 
 # Contributor setup
@@ -90,7 +72,7 @@ Go [here](example/README.md) for more information.
     - Download "google-java-format" plugin and use it
     - Go to "Editor" -> "Code Style" -> "YAML". Uncheck "Indent sequence value" and "Brackets" (in the "Spaces" menu)
 
-[Badge-CI]: https://github.com/ScalaConsultants/mesmer-akka-agent/workflows/Scala%20CI/badge.svg
+[Badge-CI]: https://github.com/ScalaConsultants/mesmer/workflows/Scala%20CI/badge.svg
 
 [badge-releases]: https://img.shields.io/nexus/r/https/oss.sonatype.org/io.scalac/mesmer-akka-extension_2.13 "Sonatype Releases"