From 3ff0c426e8a656d8dec3aef663533b9c8922487e Mon Sep 17 00:00:00 2001 From: Jianjian Date: Mon, 2 Sep 2024 22:12:29 -0700 Subject: [PATCH] Add docs for Alluxio file system Co-authored-by: Manfred Moser --- docs/src/main/sphinx/object-storage.md | 3 ++ .../object-storage/file-system-alluxio.md | 44 +++++++++++++++++++ 2 files changed, 47 insertions(+) create mode 100644 docs/src/main/sphinx/object-storage/file-system-alluxio.md diff --git a/docs/src/main/sphinx/object-storage.md b/docs/src/main/sphinx/object-storage.md index 8fc3a5a8d7c4..3d0290a232ed 100644 --- a/docs/src/main/sphinx/object-storage.md +++ b/docs/src/main/sphinx/object-storage.md @@ -60,6 +60,7 @@ compatible replacements: * [](/object-storage/file-system-azure) * [](/object-storage/file-system-gcs) * [](/object-storage/file-system-s3) +* [](/object-storage/file-system-alluxio) The native support is available in all four connectors, but must be activated for use. @@ -90,6 +91,7 @@ Trino also provides the following additional support and features for object storage: * [](/object-storage/file-system-cache) +* [](/object-storage/file-system-alluxio) * [](/object-storage/metastores) * [](/object-storage/file-formats) @@ -106,6 +108,7 @@ storage: /object-storage/legacy-gcs /object-storage/legacy-s3 /object-storage/file-system-cache +/object-storage/file-system-alluxio /object-storage/metastores /object-storage/file-formats ``` diff --git a/docs/src/main/sphinx/object-storage/file-system-alluxio.md b/docs/src/main/sphinx/object-storage/file-system-alluxio.md new file mode 100644 index 000000000000..066f1ca0986a --- /dev/null +++ b/docs/src/main/sphinx/object-storage/file-system-alluxio.md @@ -0,0 +1,44 @@ +# Alluxio file system support + +Trino includes a native implementation to access +[Alluxio](https://docs.alluxio.io/os/user/stable/en/Overview.html) as a file +system with a catalog using the Delta Lake, Hive, Hudi, or Iceberg connectors. +An Alluxio cluster acts as caching layer for one or more Trino catalogs and even +clusters or other systems, in front of the actual object storage. + +For comparison the [](/object-storage/file-system-cache) caches data locally on +the Trino cluster nodes and is separate for each catalog and cluster. + +Enable the Alluxio file system with `fs.alluxio.enabled=true` in your catalog +properties file. + +## Configuration + +Use the following properties to configure general aspects of Alluxio file system +support in your catalog properties file: + +:::{list-table} +:widths: 40, 60 +:header-rows: 1 + +* - Property + - Description +* - `fs.alluxio.enabled` + - Activate the Alluxio file system support. Defaults to `false`. +::: + +## Alluxio client configuration + +The Alluxio cluster connection is configured in the `alluxio-site.properties` +properties file. The same config file must be located in `/opt/alluxio/conf` on +all Trino cluster nodes. Follow the [Alluxio client configuration +documentation](https://docs.alluxio.io/os/user/stable/en/operation/Configuration.html) +for more details. + +Example content of `alluxio-site.properties`: + +```properties +alluxio.master.hostname=127.0.0.1 +alluxio.master.port=19998 +alluxio.security.authentication.type=NOSASL +```