|
1 | | -hdfs-mount |
2 | | -========== |
| 1 | +What is hopsfs-mount |
| 2 | +==================== |
3 | 3 |
|
4 | | -[](https://travis-ci.org/Microsoft/hdfs-mount) |
| 4 | +Allows to mount remote HopsFS as a local Linux filesystem and allow arbitrary applications / shell scripts to access HopsFS as normal files and directories in efficient and secure way. |
5 | 5 |
|
6 | | -Allows to mount remote HDFS as a local Linux filesystem and allow arbitrary applications / shell scripts to access HDFS as normal files and directories in efficient and secure way. |
| 6 | +Usage |
| 7 | +----- |
7 | 8 |
|
8 | | -Features (Planned) |
9 | | ------------------- |
10 | | -* High performance |
11 | | - * directly interfacing Linux kernel for FUSE and HDFS using protocol buffers (requires no JavaVM) |
12 | | - * designed and optimized for throughput-intensive workloads (throughput is traded for latency whenever possible) |
13 | | - * full streaming and automatic read-ahead support |
14 | | - * concurrent operations |
15 | | - * In-memory metadata caching (very fast ls!) |
16 | | -* High stability and robust failure-handling behavior |
17 | | - * automatic retries and failover, all configurable |
18 | | - * optional lazy mounting, before HDFS becomes available |
19 | | -* Support for both reads and writes |
20 | | - * support for random writes [slow, but functionally correct] |
21 | | - * support for file truncations |
22 | | -* Optionally expands ZIP archives with extracting content on demand |
23 | | - * this provides an effective solution to "millions of small files on HDFS" problem |
24 | | -* CoreOS and Docker-friendly |
25 | | - * optionally packagable as a statically-linked self-contained executable |
| 9 | +``` |
| 10 | +Usage of ./hopsfs-mount: |
| 11 | + ./hopsfs-mount [Options] Namenode:Port MountPoint |
26 | 12 |
|
27 | | -Current state |
28 | | -------------- |
29 | | -"Alpha", under active development. Basic R/O scenarios, key R/O throughout optimizations and ZIP support are implemented and outperform existing HDFS/FUSE solutions. |
30 | | -If you want to use the component - come back in few weeks |
31 | | -If you want to help - contact authors |
32 | | - |
33 | | -Building |
34 | | --------- |
35 | | -Ensure that you cloned the git repository recursively, since it contains submodules. |
36 | | -Run 'make' to build and 'make test' to run unit test. |
37 | | -Please use Go version at least 1.6beta2. This version contains bugfix for handling zip64 archives necessary for hdfs-mount to operate normally. |
| 13 | +Options: |
| 14 | + -allowedPrefixes string |
| 15 | + Comma-separated list of allowed path prefixes on the remote file system, if specified the mount point will expose access to those prefixes only (default "*") |
| 16 | + -clientCertificate string |
| 17 | + Client certificate location (default "/srv/hops/super_crypto/hdfs/hdfs_certificate_bundle.pem") |
| 18 | + -clientKey string |
| 19 | + Client key location (default "/srv/hops/super_crypto/hdfs/hdfs_priv.pem") |
| 20 | + -fuse.debug |
| 21 | + log FUSE processing details |
| 22 | + -lazy |
| 23 | + Allows to mount HopsFS filesystem before HopsFS is available |
| 24 | + -logFile string |
| 25 | + Log file path. By default the log is written to console |
| 26 | + -logLevel string |
| 27 | + logs to be printed. error, warn, info, debug, trace (default "error") |
| 28 | + -readOnly |
| 29 | + Enables mount with readonly |
| 30 | + -retryMaxAttempts int |
| 31 | + Maxumum retry attempts for failed operations (default 10) |
| 32 | + -retryMaxDelay duration |
| 33 | + maximum delay between retries (default 1m0s) |
| 34 | + -retryMinDelay duration |
| 35 | + minimum delay between retries (note, first retry always happens immediatelly) (default 1s) |
| 36 | + -retryTimeLimit duration |
| 37 | + time limit for all retry attempts for failed operations (default 5m0s) |
| 38 | + -rootCABundle string |
| 39 | + Root CA bundle location (default "/srv/hops/super_crypto/hdfs/hops_root_ca.pem") |
| 40 | + -srcDir string |
| 41 | + HopsFS src directory (default "/") |
| 42 | + -stageDir string |
| 43 | + stage directory for writing files (default "/tmp") |
| 44 | + -tls |
| 45 | + Enables tls connections |
| 46 | +``` |
38 | 47 |
|
39 | 48 | Other Platforms |
40 | 49 | --------------- |
|
0 commit comments