Skip to content

Commit

Permalink
updated documentation, use go templates for connection variables
Browse files Browse the repository at this point in the history
  • Loading branch information
@kndndrj
kndndrj committed Dec 29, 2023
1 parent 6c99272 commit 571c8dd
Show file tree
Hide file tree
Showing 7 changed files with 140 additions and 183 deletions.
200 changes: 77 additions & 123 deletions ARCHITECTURE.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,156 +2,110 @@

The plugin is created from 2 parts:

- Lua frontend
- Go backend
- Go backend that interacts with databases,
- Lua frontend which wraps the backend with neat nvim integration.

These two parts should have clearly defined borders and not "leak"
responsibilities.

## Lua Architecture

The following diagram shows a high level overview of lua packages. Note that a
lot of connections are removed for diagram clarity.

```
│ ┌─────────────┐ ┌─────────┐
│ "dbee" ┌────────────────────► ├─────────────► │
│ │ │ handler │ mapping │ Go │
│ │ ┌──► ├─────────────► │
│ │ │ └─────────────┘ └─────────┘
│ ┌────────────┐ │ │
│ │ │ │ │
┌───────────────┐ │ │ drawer ├──┤ │
│ │ │ │ │ │ │
│ projector │ │ └────────────┘ │ ┌────────────┐ │
│ extension ├────────► │ │ │ │
│ │ │ └──► editor ├─┘
└───────────────┘ │ │ │
│ └────────────┘
│
│
│
│ ┌─────────────┐ ┌────────────┐
│ │ │ │ │
│ │ install │ │ loader │
│ │ │ │ │
│ └─────────────┘ └────────────┘
│
tiles
┌──────────┐
┌─────►│ Result ├──────┐
│ └──────────┘ │
│ │
│ ┌──────────┐ │
├─────►│ Editor ├──────┤ core
┌──────────┐ ┌──────────┐ │ └──────────┘ │ ┌──────────┐
│ API ├──►│ entry ├──┤ ├────►│ Handler ├───► Go
└──────────┘ └──────────┘ │ ┌──────────┐ │ └──────────┘
├─────►│ Drawer ├──────┤
│ └──────────┘ │
│ │
│ ┌──────────┐ │
└─────►│ Call Log ├──────┘
└──────────┘
┌──────────┐ ┌──────────┐
│ sources │ │ layouts │
└──────────┘ └──────────┘
┌──────────┐
│ install │
└──────────┘
```

Description:

- The "dbee" package consists of 3 major functional packages, 1 install package
and additional "helper" packages.
- The "dbee" package consists of 2 major functional packages, 1 tiles (ui)
package and the other handler (core) package.

- `handler` or core package is a wrapper around the go backend handler. The
only extra thing lua handler does on top is information about sources.
- `tiles` or ui package consists of the following packages:
- `Drawer` represents the tree view. It uses the handler and editor to
provide the view of connections and notes.
- `Editor` represents the notepad view. It manages notes per namespace
(namespace is an arbitrary name - Drawer uses it to have connection-local
notes).
- `Result` represents the results view.
- `Call Log` represents the history of calls vie view and supports managing
past calls.

- `install` package is independent of the other packages and is used for
installation of the compiled go binary using the manifest generated by the CI
pipeline.
- `loader` package is also independent and is used as the default loading and
saving method in the config, which is later consumed by the handler
- `drawer` is the "tree" view in UI - it consumes the editor (to provide
scratchpad view and to manage scratchpads) and the handler (for managing
connections and providing layout of each database).
- `editor` is the "scratchpad" window in the UI and is used for managing
scratchpads. It also consumes the handler in order to execute selected
queries.
- `handler` is a mapping to Go backend - it holds all connections (a connection
is an instance of the database) and manages them all calls to backend go
through here. Handler also manages the "results" window in UI. handler is also
responsible for providing layouts of databases to drawer.
- `projector extension` acts as an outside "client" and registers dbee as an
output for nvim-projector.

- `sources` package holds an implementation of some of the most common sources.

- `layouts` package holds the implementation of the default window layout.

## Go Architecture

As We said, the Go backend is accessed exclusively through `handler` in lua. The
handler reads configurations for databases, remembers them and registers them in
the backend as well. This way, both `handler` and go's `main()` function have
knowledge of active connections.
way the communication workd both ways is that lua can call the handler method
directly and go triggers events that lua then listens to.

When a handler wants to perform an action on a specific connection, it sends the
connection's id in the request to go.
An example of this event based message passing is executing a query on
connection:

```
│
│ "main.go"
│
│ ┌──────────────────────────────┐
│ │ map of │
│ │ connections │ ┌──────┐
│ │ │ │┼┼┼┼┼┼│
│ │ ┌──────────────────────────┐ │ └──────┤
│ │ │ postgres_xyz │ │ ┌────────► DB │
│ │ │ │ │ │ ┌──────┤
│ │ │ ┌─────────────────────┐ │ │ │ │┼┼┼┼┼┼│
│ │ │ │clients (postgres) ├────────┘ └──────┘
│ │ │ └────────┬────────────┘ │ │
│ │ │ │ │ │
│ │ │ iter ┌─ ── ── ── ── ── ─iter ── ── ── ── ── ── ──
│ │ │ │ │ │ │ │
│ │ │ ┌─────────▼──────▼─────┐ │ │
│ │ │ │cache │ │ │ │
│ │ │ │ │ │ │ ┌─────────────┐
│ │ │ │ - read first page ──────────────buffer────► │ │
┌─────────────────┐ │ │ │ │ - read rest in bg │ │ │ │ │
│ │ │ │ │ │ ┌─────────────json──────► │ │
│ handler.lua ├─────────► │ │ │ │ │ │ │ │ outputs │
│ │ │ │ │ │ - idle ... ──────┼─────────────csv───────► │ │
└─────────────────┘ │ │ │ │ │ │ │ │ │ │
│ │ │ │ └─────────────buffer────► │ │
│ │ │ │ │ │ │ └─────────────┘
│ │ │ │ │ │ │ │
│ │ │ │ │ │ │ ┌─────────────┐
│ │ │ │ - flush ─────────────────────────────────► history ├ ─┘
│ │ │ │ │ │ │ │ output │
│ │ │ └──────────────────────┘ │ │ └─────────────┘
│ │ │ │ │
│ │ └──────────────────────────┘ │
│ │ │
│ │ ┌──────────────────────────┐ │
│ │ │ sqlite_123 │ │
│ │ │ ┌────────────────┐ │ │
│ │ │ │clients (sqlite)│ │ │
│ │ │ └────────────────┘ │ │
│ │ │ │ │
│ │ └──────────────────────────┘ │
│ │ │
│ │ ... │
│ │ │
│ └──────────────────────────────┘
│
```
- lua registers an event listener to display results.
- lua calls go execute method, which returns call details immediately.
- lua then waits for call to yield some results, to display them.

Now that we have a map of connections, we can focus on how each connection is
structured:
One way of looking at the handler package in go is that it's just an
implementation specific use case of the `core` go package.

There are 2 primarily used result types in the connection:
### Core package

- `IterResult` - result in a form of an iterator returned by the specific
driver.
- `Result` - full in-memory result set, which is sent in full or in chunks to
outputs.
Here is the [godoc](https://pkg.go.dev/github.com/kndndrj/nvim-dbee/dbee/core)
of the core package.

Each connection has a specific driver implementation (`clients`) which are
required to implement "query" functionality and return an `IterResult` which is
an iterator over the results returned from the database.
Main construct is a `Connection`. It takes the parameters to connect to the
database and an adapter for the database. Adapter is a provider for specific
databases, which can return a database driver and returns common database
queries (helpers).

Then the client's internal `cache` reads the ammount of results required for the
first page (specified as a parameter) in the main thread and it immediately
sends it (drained first page `Result`) to the specified output (`buffer`) - this
makes an impression that results are available right away.
Then the connection can execute queries using a driver. This procudes a `Call`.
A call represents a single call to the database and holds it's state.

As soon as the first page is read, a background process is spawned which drains
the remaining results to connections's internal cache.
Database call returns a result, which transforms the iterator returned from
driver to different formats using a formatter.

When the iterator is drained and all results are in cache, we can do a few
things with it:
#### Adapters package

- send a chunk of `Result` to an output (e.g. page of the result) - usually used
by `buffer` output
- send a full `Result` to an output - useful for "saving" (e.g. json or csv
output)
One of the subpackages of core package is `adapters`. It contains implemetations
of multiple database drivers and adapters. One special thing it does is that it
has it's own method to create a connection. This is done so that individual
adapters can register themselves on startup in their init functions (so that we
can exclude some adapters on certain architectures/os-es).

When a user makes a new request to the client (executes a query), the in-memory
`Result` is flushed to history output, which saves it on disk for possible
latter consumption.
#### Builders package

Note: *`buffer`, `json` and `csv` all implement the same `Output` interface and
could be used interchangably. Simillarly, `clients` and `history` implement the
same `Input` interface.*
Another subpackage of `core`, which holds convenience functions for creating
some of the most used constructs. An example are multiple implementations of the
`ResultStream` interface.
Loading

0 comments on commit 571c8dd

Please sign in to comment.