Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs for Alluxio file system #23269

Merged
merged 1 commit into from
Sep 27, 2024
Merged

Add docs for Alluxio file system #23269

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/src/main/sphinx/object-storage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ compatible replacements:
* [](/object-storage/file-system-azure)
* [](/object-storage/file-system-gcs)
* [](/object-storage/file-system-s3)
* [](/object-storage/file-system-alluxio)

The native support is available in all four connectors, but must be activated
for use.
Expand Down Expand Up @@ -90,6 +91,7 @@ Trino also provides the following additional support and features for object
storage:

* [](/object-storage/file-system-cache)
* [](/object-storage/file-system-alluxio)
* [](/object-storage/metastores)
* [](/object-storage/file-formats)

Expand All @@ -106,6 +108,7 @@ storage:
/object-storage/legacy-gcs
/object-storage/legacy-s3
/object-storage/file-system-cache
/object-storage/file-system-alluxio
/object-storage/metastores
/object-storage/file-formats
```
44 changes: 44 additions & 0 deletions docs/src/main/sphinx/object-storage/file-system-alluxio.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Alluxio file system support

Trino includes a native implementation to access
[Alluxio](https://docs.alluxio.io/os/user/stable/en/Overview.html) as a file
system with a catalog using the Delta Lake, Hive, Hudi, or Iceberg connectors.
An Alluxio cluster acts as caching layer for one or more Trino catalogs and even
clusters or other systems, in front of the actual object storage.

For comparison the [](/object-storage/file-system-cache) caches data locally on
the Trino cluster nodes and is separate for each catalog and cluster.

Enable the Alluxio file system with `fs.alluxio.enabled=true` in your catalog
properties file.

## Configuration

Use the following properties to configure general aspects of Alluxio file system
support in your catalog properties file:

:::{list-table}
:widths: 40, 60
:header-rows: 1

* - Property
- Description
* - `fs.alluxio.enabled`
- Activate the Alluxio file system support. Defaults to `false`.
:::

## Alluxio client configuration

The Alluxio cluster connection is configured in the `alluxio-site.properties`
properties file. The same config file must be located in `/opt/alluxio/conf` on
all Trino cluster nodes. Follow the [Alluxio client configuration
documentation](https://docs.alluxio.io/os/user/stable/en/operation/Configuration.html)
for more details.

Example content of `alluxio-site.properties`:

```properties
alluxio.master.hostname=127.0.0.1
alluxio.master.port=19998
alluxio.security.authentication.type=NOSASL
```
Loading