Skip to content

Commit 36af974

Browse files
milenkovicmmilenkovicm
authored
doc: update docker related documentation (#1204)
1 parent 6a600e1 commit 36af974

File tree

3 files changed

+22
-47
lines changed

3 files changed

+22
-47
lines changed

‎docker-compose.yml

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -16,25 +16,18 @@
1616
# under the License.
1717
version: '3.3'
1818
services:
19-
etcd:
20-
image: quay.io/coreos/etcd:v3.4.9
21-
command: "etcd -advertise-client-urls http://etcd:2379 -listen-client-urls http://0.0.0.0:2379"
22-
ports:
23-
- "2379:2379"
2419
ballista-scheduler:
2520
image: ballista-scheduler
2621
build:
2722
dockerfile: dev/docker/ballista-scheduler.Dockerfile
2823
context: .
29-
command: "--cluster-backend etcd --etcd-urls etcd:2379 --bind-host 0.0.0.0"
24+
command: "--bind-host 0.0.0.0"
3025
ports:
3126
- "50050:50050"
3227
environment:
3328
- RUST_LOG=ballista=info
3429
volumes:
3530
- ./benchmarks/data:/data
36-
depends_on:
37-
- etcd
3831
healthcheck:
3932
test: ["CMD", "nc", "-z", "ballista-scheduler", "50050"]
4033
interval: 5s

‎docs/source/user-guide/deployment/docker-compose.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -26,24 +26,24 @@ Docker Compose is a convenient way to launch a cluster when testing locally.
2626
Run the following commands to download the [official Docker image](https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-standalone):
2727

2828
```bash
29-
docker pull ghcr.io/apache/datafusion-ballista-standalone:0.12.0-rc4
29+
docker pull ghcr.io/apache/datafusion-ballista-standalone:latest
3030
```
3131

3232
Altenatively run the following commands to clone the source repository and build the Docker images from source:
3333

3434
```bash
35-
git clone git@github.com:apache/datafusion-ballista.git -b 0.12.0
35+
git clone git@github.com:apache/datafusion-ballista.git -b latest
3636
cd datafusion-ballista
3737
./dev/build-ballista-docker.sh
3838
```
3939

4040
This will create the following images:
4141

42-
- `apache/datafusion-ballista-benchmarks:0.12.0`
43-
- `apache/datafusion-ballista-cli:0.12.0`
44-
- `apache/datafusion-ballista-executor:0.12.0`
45-
- `apache/datafusion-ballista-scheduler:0.12.0`
46-
- `apache/datafusion-ballista-standalone:0.12.0`
42+
- `apache/datafusion-ballista-benchmarks:latest`
43+
- `apache/datafusion-ballista-cli:latest`
44+
- `apache/datafusion-ballista-executor:latest`
45+
- `apache/datafusion-ballista-scheduler:latest`
46+
- `apache/datafusion-ballista-standalone:latest`
4747

4848
## Start a Cluster
4949

@@ -75,5 +75,5 @@ The scheduler listens on port 50050 and this is the port that clients will need
7575
## Connect from the Ballista CLI
7676

7777
```shell
78-
docker run --network=host -it apache/datafusion-ballista-cli:0.12.0 --host localhost --port 50050
78+
docker run --network=host -it apache/datafusion-ballista-cli:latest --host localhost --port 50050
7979
```

‎docs/source/user-guide/deployment/docker.md

Lines changed: 13 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -24,24 +24,24 @@
2424
Run the following commands to download the [official Docker image](https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-standalone):
2525

2626
```bash
27-
docker pull ghcr.io/apache/datafusion-ballista-standalone:0.12.0-rc4
27+
docker pull ghcr.io/apache/datafusion-ballista-standalone:latest
2828
```
2929

3030
Altenatively run the following commands to clone the source repository and build the Docker images from source:
3131

3232
```bash
33-
git clone git@github.com:apache/datafusion-ballista.git -b 0.12.0
33+
git clone git@github.com:apache/datafusion-ballista.git
3434
cd datafusion-ballista
3535
./dev/build-ballista-docker.sh
3636
```
3737

3838
This will create the following images:
3939

40-
- `apache/datafusion-ballista-benchmarks:0.12.0`
41-
- `apache/datafusion-ballista-cli:0.12.0`
42-
- `apache/datafusion-ballista-executor:0.12.0`
43-
- `apache/datafusion-ballista-scheduler:0.12.0`
44-
- `apache/datafusion-ballista-standalone:0.12.0`
40+
- `apache/datafusion-ballista-benchmarks:latest`
41+
- `apache/datafusion-ballista-cli:latest`
42+
- `apache/datafusion-ballista-executor:latest`
43+
- `apache/datafusion-ballista-scheduler:latest`
44+
- `apache/datafusion-ballista-standalone:latest`
4545

4646
## Start a Cluster
4747

@@ -51,7 +51,7 @@ Start a scheduler using the following syntax:
5151

5252
```bash
5353
docker run --network=host \
54-
-d apache/datafusion-ballista-scheduler:0.12.0 \
54+
-d apache/datafusion-ballista-scheduler:latest \
5555
--bind-port 50050
5656
```
5757

@@ -60,7 +60,7 @@ Run `docker ps` to check that the process is running:
6060
```
6161
$ docker ps
6262
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
63-
a756055576f3 apache/datafusion-ballista-scheduler:0.12.0 "/root/scheduler-ent…" 8 seconds ago Up 8 seconds xenodochial_carson
63+
a756055576f3 apache/datafusion-ballista-scheduler:latest "/root/scheduler-ent…" 8 seconds ago Up 8 seconds xenodochial_carson
6464
```
6565

6666
Run `docker logs CONTAINER_ID` to check the output from the process:
@@ -82,7 +82,7 @@ Start one or more executor processes. Each executor process will need to listen
8282

8383
```bash
8484
docker run --network=host \
85-
-d apache/datafusion-ballista-executor:0.12.0 \
85+
-d apache/datafusion-ballista-executor:latest \
8686
--external-host localhost --bind-port 50051
8787
```
8888

@@ -91,8 +91,8 @@ Use `docker ps` to check that both the scheduler and executor(s) are now running
9191
```
9292
$ docker ps
9393
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
94-
fb8b530cee6d apache/datafusion-ballista-executor:0.12.0 "/root/executor-entr…" 2 seconds ago Up 1 second gallant_galois
95-
a756055576f3 apache/datafusion-ballista-scheduler:0.12.0 "/root/scheduler-ent…" 8 seconds ago Up 8 seconds xenodochial_carson
94+
fb8b530cee6d apache/datafusion-ballista-executor:latest "/root/executor-entr…" 2 seconds ago Up 1 second gallant_galois
95+
a756055576f3 apache/datafusion-ballista-scheduler:latest "/root/scheduler-ent…" 8 seconds ago Up 8 seconds xenodochial_carson
9696
```
9797

9898
Use `docker logs CONTAINER_ID` to check the output from the executor(s):
@@ -106,26 +106,8 @@ $ docker logs fb8b530cee6d
106106
2024-02-03T14:50:24.063281Z INFO tokio-runtime-worker ThreadId(47) ballista_executor::execution_loop: Starting poll work loop with scheduler
107107
```
108108

109-
### Using etcd as a Backing Store
110-
111-
_NOTE: This functionality is currently experimental_
112-
113-
Ballista can optionally use [etcd](https://etcd.io/) as a backing store for the scheduler. Use the following commands
114-
to launch the scheduler with this option enabled.
115-
116-
```bash
117-
docker run --network=host \
118-
-d apache/datafusion-ballista-scheduler:0.12.0 \
119-
--bind-port 50050 \
120-
--config-backend etcd \
121-
--etcd-urls etcd:2379
122-
```
123-
124-
Please refer to the [etcd](https://etcd.io/) website for installation instructions. Etcd version 3.4.9 or later is
125-
recommended.
126-
127109
## Connect from the CLI
128110

129111
```shell
130-
docker run --network=host -it apache/datafusion-ballista-cli:0.12.0 --host localhost --port 50050
112+
docker run --network=host -it apache/datafusion-ballista-cli:latest --host localhost --port 50050
131113
```

0 commit comments

Comments
 (0)