README.md

Groq

Connection to Groq

Login to the Groq login node from your local machine. Once you are on the login node, ssh to one of the Groq nodes.

local > ssh ALCFUserID@groq.ai.alcf.anl.gov

groq-login > ssh groq-r01-gn-01.ai.alcf.anl.gov
# or
groq-login > ssh groq-r01-gn-09.ai.alcf.anl.gov
# or any node with hostname of form groq-r01-gn-0[1-9].ai.alcf.anl.gov

Prerequisite: Create Virtual Environment

Install Miniconda

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

PyTorch virtual environment

export PYTHON_VERSION=3.10.12
conda create -n groqflow python=$PYTHON_VERSION -y
conda activate groqflow

Install Groqflow

# Alter this if you have cloned groqflow to some other location.
cd ~/groqflow
if [ -d "groqflow.egg-info" ]; then rm -r groqflow.egg-info; fi
pip install --upgrade pip
pip list --format=freeze > frozen.txt
pip install -r frozen.txt -e .
pushd . 
cd demo_helpers
if [ -d "groqflow_demo_helpers.egg-info" ]; then rm -r groqflow_demo_helpers.egg-info; fi
pip install -e .
popd
pip install soundfile
pip install datasets==2.21.0

Use GroqFlow

conda activate groqflow

Job Queuing and Submission

Groq jobs in the AI Testbed's Groqrack are managed by the PBS job scheduler.

• qsub : to submit a batch job using a script
• qstat: to display queue information
• qdel: to delete (cancel) a job:
• qhold: to hold a job

Schedule Interactive Job

Following command gives a single Groq node interactively for 1 hour

qsub -I -l walltime=1:00:00 

Other flags that can be used

-l ncpus=1 
-l groq_accelerator=1

Hands-on Example

• GPT-2

Homework

Run MiniLM example with custom input instead of dummy input. Submit proof (contents printed out to your terminal, path to a logfile or screenshot) that you were able to successfully follow the instructions and execute.

Additional Examples (Optional)

• MiniLMv2
• ResNet50
• Bert

Useful Resources

• ALCF Groq Documenation
• Groq Documentation
• Groq Examples Repository