Skip to content

Commit

Permalink
Merge branch 'main' into navarone/update-docker-documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
navarone-feekery authored May 29, 2024
2 parents e839aff + b4e87a6 commit 8a7cebf
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 4 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Elastic Open Web Crawler

This repository contains code for the Elastic Open Web Crawler.
This is a tool to allow users to ingest content into Elasticsearch easily from the web.
This is a tool to allow users to easily ingest content into Elasticsearch from the web.

## How it works

Expand All @@ -23,7 +23,7 @@ Crawler has a Dockerfile that can be built and run locally.
1. Build the image `docker build -t crawler-image .`
2. Run the container `docker run -i -d --name crawler crawler-image`
- `-i` allows the container to stay alive so CLI commands can be executed inside it
- `-d` allows the container to run "detached" so you don't have to dedicated a terminal window to it
- `-d` allows the container to run "detached" so you don't have to dedicate a terminal window to it
3. Confirm that Crawler commands are working `docker exec -it crawler bin/crawler version`
4. Execute other CLI commands from outside of the container by prepending `docker exec -it crawler <command>`.
- See [Crawling content](#crawling-content) for examples.
Expand Down
3 changes: 2 additions & 1 deletion config/crawler.yml.example
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@
## Elasticsearch connection settings. These can be defined for all crawlers in `config/elasticsearch.yml`,
##
#elasticsearch:
# host: http://localhost:9200
# host: http://localhost
# port: 9200
# username: elastic
# password: changeme
# api_key: 1234
Expand Down
3 changes: 2 additions & 1 deletion config/examples/parks-australia.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ max_indexed_links_count: 5
max_headings_count: 5

elasticsearch:
host: http://localhost:9200
host: http://localhost
port: 9200
username: elastic
password: changeme
bulk_api:
Expand Down

0 comments on commit 8a7cebf

Please sign in to comment.