Skip to content

Commit

Permalink
Add docs for Alluxio file system
Browse files Browse the repository at this point in the history
Co-authored-by: Manfred Moser <manfred@simpligility.ca>
  • Loading branch information
jja725 and mosabua committed Sep 27, 2024
1 parent 8994e73 commit 3ff0c42
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 0 deletions.
3 changes: 3 additions & 0 deletions docs/src/main/sphinx/object-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ compatible replacements:
* [](/object-storage/file-system-azure)
* [](/object-storage/file-system-gcs)
* [](/object-storage/file-system-s3)
* [](/object-storage/file-system-alluxio)

The native support is available in all four connectors, but must be activated
for use.
Expand Down Expand Up @@ -90,6 +91,7 @@ Trino also provides the following additional support and features for object
storage:

* [](/object-storage/file-system-cache)
* [](/object-storage/file-system-alluxio)
* [](/object-storage/metastores)
* [](/object-storage/file-formats)

Expand All @@ -106,6 +108,7 @@ storage:
/object-storage/legacy-gcs
/object-storage/legacy-s3
/object-storage/file-system-cache
/object-storage/file-system-alluxio
/object-storage/metastores
/object-storage/file-formats
```
44 changes: 44 additions & 0 deletions docs/src/main/sphinx/object-storage/file-system-alluxio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Alluxio file system support

Trino includes a native implementation to access
[Alluxio](https://docs.alluxio.io/os/user/stable/en/Overview.html) as a file
system with a catalog using the Delta Lake, Hive, Hudi, or Iceberg connectors.
An Alluxio cluster acts as caching layer for one or more Trino catalogs and even
clusters or other systems, in front of the actual object storage.

For comparison the [](/object-storage/file-system-cache) caches data locally on
the Trino cluster nodes and is separate for each catalog and cluster.

Enable the Alluxio file system with `fs.alluxio.enabled=true` in your catalog
properties file.

## Configuration

Use the following properties to configure general aspects of Alluxio file system
support in your catalog properties file:

:::{list-table}
:widths: 40, 60
:header-rows: 1

* - Property
- Description
* - `fs.alluxio.enabled`
- Activate the Alluxio file system support. Defaults to `false`.
:::

## Alluxio client configuration

The Alluxio cluster connection is configured in the `alluxio-site.properties`
properties file. The same config file must be located in `/opt/alluxio/conf` on
all Trino cluster nodes. Follow the [Alluxio client configuration
documentation](https://docs.alluxio.io/os/user/stable/en/operation/Configuration.html)
for more details.

Example content of `alluxio-site.properties`:

```properties
alluxio.master.hostname=127.0.0.1
alluxio.master.port=19998
alluxio.security.authentication.type=NOSASL
```

0 comments on commit 3ff0c42

Please sign in to comment.