used for finetuning LLM in big project, contact rytian@wisc.edu for more info
not sure of its limit, my maximum was 200k msgs in one run
- get discord auth token open discord in browser open network tab in inspect elements type in any channel find typing request in network tab copy value in authorization under headers store token in private/.env as DISCORD_AUTHORIZATION_TOKEN
- setup ids channel ids can be found in discord url save ids to a file
- run python scraper.py < {ids-file} > {output-file}