Update scripts and README (#32)

The script files were a bit bloated as we haven't looked at them since
porting over base crawler. There were also limitations (e.g. forced env
managers) that I don't think would be healthy for an open-code repo.

- Remove script funcs and script files that are not used
- Change echo coloring for progress statements from yellow to blue
    - Yellow should be for warnings only 
- Remove hard requirement for `rbenv` and `jenv` when running `make
install`
- Users can opt-in to this by passing in env var
`CRAWLER_MANAGE_ENV=true`
- Add currently enabled Java and Ruby versions to bash output when
running `make install`
- Remove detailed instructions for env management from README.md
- Instead we just have a recommendation to use an env manager, and links
to official docs

.buildkite/Dockerfile

@@ -3,7 +3,4 @@ RUN apt-get update && \
     apt-get install -y --no-install-recommends \
     libicu-dev netbase make
 
-# used for skipping jenv/rbenv setup
-ENV IS_DOCKER=1
-
 ENTRYPOINT [ "/bin/bash" ]

Jarfile

@@ -2,7 +2,7 @@
 # our java dependencies into vendor/jars (see https://github.com/mkristian/jar-dependencies for details)
 #
 # If you update this file, please run the following command to update the jars cache:
-#    rm -rf Jars.lock vendor/jars && script/development exec script/vendor_jars
+#    make clean install
 #
 # When adding a new dependency, please explain what it is and why we're adding it in a comment.
 #---------------------------------------------------------------------------------------------------

README.md

@@ -30,42 +30,36 @@ Crawler has a Dockerfile that can be built and run locally.
 
 #### Running from source
 
-Crawler uses `jenv` and `rbenv` to manage both java and ruby versions when running from source.
-
-1. Install `jenv` and `rbenv`
-    - [Official documentation for installing jenv](https://www.jenv.be/)
-    - [Official documentation for installing rbenv](https://github.com/rbenv/rbenv?tab=readme-ov-file#installation)
-2. Install the required java version (check the file `.java-version`)
-   - Crawler was developed using OpenJDK, so we recommend using an OpenJDK version of Java
-     - [Instructions for installing OpenJDK](https://openjdk.org/install/)
-   - Mac users can also use `brew` to install
-        ```bash
-        # install with brew
-        $ brew install openjdk@21
-
-        # create symlink
-        $ sudo ln -sfn \
-            /opt/homebrew/opt/openjdk@21/libexec/openjdk.jdk \
-            /Library/Java/JavaVirtualMachines/openjdk-21.jdk
-        ```
-3. Add Java version to `jenv`
-    ```bash
-    # add to jenv and update JAVA_HOME
-    $ jenv add /Library/Java/JavaVirtualMachines/openjdk-21.jdk/Contents/Home
-    $ export JAVA_HOME=$(/usr/libexec/java_home -v21)
-
-    # check java version has been correctly set by `.java-version` file
-    $ java --version
-    ```
-4. Install the required jruby version
-    ```bash
-    # rbenv is easier to use and can install a version based on `.ruby-version` file
-    $ rbenv install
-
-    # check ruby version
-    $ ruby --version
-    ```
-5. Run `make install` to install Crawler dependencies
+Crawler uses both JRuby and Java.
+We recommend using version managers for both.
+When developing Crawler we use `rbenv` and `jenv`.
+There are instructions for setting up these env managers here:
+
+- [Official documentation for installing jenv](https://www.jenv.be/)
+- [Official documentation for installing rbenv](https://github.com/rbenv/rbenv?tab=readme-ov-file#installation)
+
+Go to the root of the Crawler directory and check the expected Java and Ruby versions are being used:
+
+```bash
+# should output the same version as `.ruby-version`
+$ ruby --version
+
+# should output the same version as `.java-version`
+$ java --version
+```
+
+If the versions seem correct, you can install dependencies:
+
+```bash
+$ make install
+```
+
+You can also use the env variable `CRAWLER_MANAGE_ENV` to have the install script automatically check whether `rbenv` and `jenv` are installed, and that the correct versions are running on both:
+Doing this requires that you use both `rbenv` and `jenv` in your local setup.
+
+```bash
+$ CRAWLER_MANAGE_ENV=true make install
+```
 
 Crawler should now be functional.
 See [Configuring Crawlers](#configuring-crawlers) to begin crawling web content.

script/bundle

@@ -16,7 +16,7 @@ function bundle_command() {
     set +x
     echo
     red_echo "ERROR: Bundle command failed!"
-    yellow_echo "Try to run ./script/setup-rubies and then retry this command"
+    yellow_echo "Try to run 'make install' and then retry this command"
     echo
     exit 42
   fi
@@ -25,10 +25,10 @@ function bundle_command() {
 BUNDLER_VERSION="$(cat .bundler-version)"
 BUNDLER_CONSTRAINT="~> $BUNDLER_VERSION"
 
-yellow_echo "Bundling jruby gems..."
+blue_echo "Bundling jruby gems..."
 bundle_command config cache_all true
 
-yellow_echo "Running the bundle command..."
+blue_echo "Running the bundle command..."
 bundle_command "$@"
 
 green_echo "Done!"

script/environment

@@ -5,20 +5,4 @@ source "$(dirname $0)/functions.sh"
 set -e
 
 load_version_constraints
-
-echo "JRuby version required: ${JRUBY_VERSION}"
-echo "Java version required: ${JAVA_VERSION}"
-
-if [ -n "${IS_DOCKER}" ] && [ "${IS_DOCKER}" = "1" ]; then
-  echo "Skipping jenv and rbenv setup because it isn't required for Docker setups."
-else
-  jenv_init
-  ensure_java_installed "$JAVA_VERSION"
-  jenv shell "$JAVA_VERSION"
-
-  rbenv_init
-  ensure_jruby_installed "$JRUBY_VERSION"
-  rbenv shell "$JRUBY_VERSION"
-fi
-
 check_bundle