Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hallvoice V3 #659

Closed
wants to merge 86 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
0002f57
Added venv and __pycache__ in .gitignore
42-julien451 Feb 8, 2024
14ecfb8
renaming a bunch of vars
42-julien451 Mar 29, 2024
c1682bb
Added 42api conf
42-julien451 Mar 29, 2024
f6c6a9e
Making hallvoice object oriented
42-julien451 Apr 4, 2024
de4d923
Making hallvoice object-oriented
42-julien451 Apr 4, 2024
0757a37
cache is now in redis
42-julien451 Apr 4, 2024
b2f1f44
Added redis and requests
42-julien451 Apr 4, 2024
71fceaa
playsound replaced by pygame.mixer
42-julien451 Apr 5, 2024
906abd6
Added redis cache and checks
42-julien451 Apr 5, 2024
b149888
Object-oriented hallvoice
42-julien451 Apr 5, 2024
2a7efc9
Object-oriented hallvoice
42-julien451 Apr 5, 2024
34d6f5d
added login.list to gitignore
42-julien451 Apr 5, 2024
8879e80
Added two script that can pre-fill the redis cache
42-julien451 Apr 5, 2024
9bcc72d
Added timestamp to prints
42-julien451 Apr 8, 2024
f6a24fc
Wrong identation...
42-julien451 Apr 8, 2024
a118a28
Removed useless comments...
42-julien451 Apr 8, 2024
1a2400f
Reduce nb of threads + 404 and 429 error handling
42-julien451 Apr 8, 2024
cc0ef4b
Dockerized! (WIP)
42-julien451 Apr 8, 2024
de03779
Added prints and comments
42-julien451 Apr 8, 2024
ce48498
much better code, require list of firstname
42-julien451 Apr 8, 2024
b7ddbed
Renamed config.ini to avoid push mistakes...
42-julien451 Apr 8, 2024
de827dd
added file firstname.list
42-julien451 Apr 8, 2024
fcbb33e
Added a tiny \t
42-julien451 Apr 9, 2024
5f01281
added method to properly close kafka stream
42-julien451 Apr 9, 2024
c3127a1
Added API to purge cache from given login
42-julien451 Apr 9, 2024
4c3323a
Renamed Dockerfile
42-julien451 Apr 9, 2024
fa1ec4e
Added a dockerfile for the purge API
42-julien451 Apr 9, 2024
0e8961d
Renamed APIHV
42-julien451 Apr 9, 2024
41635d3
Moved all python files to src folder
42-julien451 Apr 9, 2024
2e724e1
Finished dockerfiles to be tested
42-julien451 Apr 9, 2024
3246f5b
Putting those files to where it was, otherwise CI will break
42-julien451 Apr 10, 2024
1b7989e
Copy requirement.txt and update path
42-julien451 Apr 10, 2024
8610003
New porte.sh
42-julien451 Apr 10, 2024
5e04322
Added systemctl service file
42-julien451 Apr 10, 2024
3c3b5a4
Added new folder logs
42-julien451 Apr 10, 2024
983c72f
Logs to file
42-julien451 Apr 10, 2024
97fddd9
People now have to be in specific network to purge cache
42-julien451 Apr 10, 2024
96b1d7a
Added logs folder to gitignore
42-julien451 Apr 10, 2024
80a7b83
Custom hallvoice can now take a list of txt
42-julien451 Apr 10, 2024
0f719c5
Renamed because exemple == french but example == english
42-julien451 Apr 10, 2024
8346954
wrong path
42-julien451 Apr 10, 2024
a003795
Added an example json file
42-julien451 Apr 10, 2024
29597e9
New readme!
42-julien451 Apr 10, 2024
6d9d84d
Added USB devices, and purge_api service, WIP, to be tested
42-julien451 Apr 10, 2024
5348691
Now this scripts need conf file in arg
42-julien451 Apr 10, 2024
006bcc5
explicit vars types
42-julien451 Apr 10, 2024
12bd515
explicit vars types
42-julien451 Apr 11, 2024
4bc11f5
Change little things
42-julien451 Apr 11, 2024
a9735bb
Norming the folder names and JSON files are now beautifullized
42-julien451 Apr 12, 2024
1ebe218
Added script to beautifullize json files
42-julien451 Apr 12, 2024
5b1b5d8
Added script to rename folder names
42-julien451 Apr 12, 2024
ef57881
Try except when error on mp3
42-julien451 Apr 12, 2024
2e57ecb
Redis is now in conf file
42-julien451 Apr 12, 2024
7765676
Added a clear note to NOT USER DOCKER
42-julien451 Apr 12, 2024
8c64c17
Added redis TTL in conf file
42-julien451 Apr 12, 2024
3720009
Added redis TTL in conf file
42-julien451 Apr 12, 2024
f89a1da
Added redis TTL in conf file
42-julien451 Apr 12, 2024
a5cf1fb
Added stderr to logfile
42-julien451 Apr 12, 2024
cb6793a
added seconds to logfile name
42-julien451 Apr 12, 2024
170d594
WTF :eyes: :eyes: :eyes:
42-julien451 Apr 12, 2024
8cb817b
Changed kafka_conf to only kafka
42-julien451 Apr 12, 2024
a2b5c78
This need more work
42-julien451 Apr 12, 2024
259b7af
Proper comments
42-julien451 Apr 12, 2024
a4788c0
Added a final comment
42-julien451 Apr 12, 2024
5f21c5a
Cmon bruh, you forgot an `s`...
42-julien451 Apr 12, 2024
f30b330
Added and modified comments
42-julien451 Apr 16, 2024
f94a575
Added and modified comments and example
42-julien451 Apr 16, 2024
5fc40a5
Handle an AttributeError exception
42-julien451 Apr 16, 2024
f35fc47
Actually wait and retry when 42API is rate-limited
42-julien451 Apr 16, 2024
2d67ea6
Error handling
42-julien451 Apr 16, 2024
d06c795
Option to log file in stdout or to file
42-julien451 Apr 16, 2024
b54ece0
This was not nice
42-julien451 Apr 18, 2024
59ec440
Moved CTRL-C Handling to remove finally block
42-julien451 Apr 18, 2024
09866c6
improved main
42-julien451 Apr 18, 2024
a9bd8b3
The service-file now output logs to file
42-julien451 Apr 18, 2024
26ba998
Added networks checks for purgeAPI
42-julien451 Apr 18, 2024
56bc28a
useless return removed
42-julien451 Apr 18, 2024
4af3720
cache folder return
42-julien451 Apr 23, 2024
5f22c01
inverted 2 lines
42-julien451 Apr 23, 2024
0d1d88e
This will not work on python3.11, it's for python3.12
42-julien451 Apr 23, 2024
c306293
Added changelogs
42-julien451 Apr 23, 2024
befea15
not an underscore...
42-julien451 Apr 23, 2024
3f96589
redis host and port not hard-codded
42-julien451 Apr 23, 2024
0463838
redis host and port not hard-codded
42-julien451 Apr 23, 2024
916fb9b
Now docker is working!
42-julien451 Apr 23, 2024
464fd80
Removed entrypoint.sh
42-julien451 Apr 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
7 changes: 6 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
cache/
logs/
config.json
config.ini
.env
.envrc
.DS_Store
.idea
.idea
__pycache__
venv
login.list
firstname.list
5 changes: 5 additions & 0 deletions Dockerfile.API
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
FROM python:3.12
LABEL authors="julien451"
RUN pip3 install --no-cache redis

ENTRYPOINT ["python3", "/hallvoice/purgeAPI.py", "/hallvoice/config/config.ini"]
13 changes: 13 additions & 0 deletions Dockerfile.porte
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
FROM debian:bookworm-slim
LABEL authors="julien451"

COPY requirements.txt .
COPY asound.conf /etc/asound.conf
RUN apt-get update --allow-releaseinfo-change
RUN apt-get install alsa-utils python3-pip libglib2.0-0 libsm6 libxrender1 libxext6 --no-install-suggests --no-install-recommends -y
RUN pip3 install --no-cache -r requirements.txt --break-system-packages

COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh

ENTRYPOINT ["python3", "/hallvoice/porte.py", "/hallvoice/conf/config.ini"]
80 changes: 78 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,89 @@
# Porte42 - Hall Voice
Bonjour la porte

## Requirements
# Requirments
## Studs
- Buy Hall Voice Change on [intra shop](https://shop.intra.42.fr/)
- Make a [Pull Request](https://help.github.com/en/articles/creating-a-pull-request-from-a-fork) to add your files
- Add a .json file with your configuration
- Add a .json file with your configuration (Also see `custom/_Example.json`)
- No sound during more than 10 seconds
- No more than 20 sounds
- No sound over 1 Mo
- No troll, insults, too loud sound, etc

PR might be refused if one of those requirements are missing

You can find an example PR [here](https://github.com/42Paris/hall-voice/pull/3/files)

## Staff
- Buy one RPI and a pretty good audio speaker
- In Raspbian, clone this repository
- Goto the repository and type
- `pip3 install -r requirements.txt`
- Install a Redis server
- `sudo apt-get install redis`
- How to run?
- You can use systemctl
- `sudo cp hallvoice.service /etc/systemd/system`
- `sudo systemctl enable --now hallvoice.service`
- You can use Docker (WIP DO NOT USE ME, OR THE RPI WILL BURN)
- `docker-compose build --no-cache`
- `docker-compose up -d`
- Copy and fill-up the config file in
- `cp config/config.ini.example config/config.ini`
- Kafka:
- The RPI must be in bocal's network so it can access Redpanda
- You messages must be JSONised like so
- `{"building":"<where>","firstname":"<name>","kind":"<in/out>","login":"<login>"}`
- You can pre-fill the redis cache with
- `goinfre-firstname-redis.py`
- to get all firstname from list of logins in `login.list`
- `goinfre-tts-redis.py`
- to cache all gTTS from a list of firstname in `firstname.list`
- This can take a very long time, generate one tts per second to avoid Google rate-limit
- All data stored in redis have a TTL of 6 months

# QnA
## Studs
- Q: Will the hallvoice use Usualname if I have one?
- A: Yes! Hallvoice use 42API to get your Usualname from your login, if there is no Usualname find, legal firstname will be used and cached
- Q: I juste set a Usualname but the old legal firstname is still in use, how to "force" the hallvoice to use the my Usualname?
- A: Firstname is cached in Redis, you need to ask the bocal in [Slack](https://42born2code.slack.com/archives/C7P0Z4F3L) #42paris_staff_it or [Discord](https://discord.com/channels/774300457157918772/839426887171440671) #staff_it to purge it.
- Q: I bought hallvoice, how to use it?
- A: As written in `Requirements`: You can find an example [PR](https://help.github.com/en/articles/creating-a-pull-request-from-a-fork) [here](https://github.com/42Paris/hall-voice/pull/3/files), also see `custom/_Example.json`
- Q: I want to change a song, do I have to buy hallvoice again?
- A: No
- Q: What's the TTL (Time to live) for the cache?
- A: TTL for redis cache is 6 month, the staff can change it in the config file
- Q: Code is garbage, can I improve it?
- I guess... Yes you can? You can try to do a [PR](https://help.github.com/en/articles/creating-a-pull-request-from-a-fork)

## Staff
- Q: I am the bocal, how to purge cache from a login?
- A: From bocal's network, make GET call at `http://<rpi-IP>/?purge=<login>`
- Q: Something's f*cky, need to debug!
- A: Logs can be found in folder `logs/<date>-hallvoice.log`
- Q: How to update?
- A: Hmmm, Are you pisciner? A simple `git pull` should do the trick :)

# Fun fact
- When redis cache is fully pre-filled:
- it consume ~300M of disk space
- 39888 TTS cached (2493 firstname * 16 welcome and goodbye message)
- ~5300 login is used to cache firstnames
- Volume is at maximum in `alsamixer`

# Change logs
- Hallvoice V2
- Now works with kafka
- Hallvoice V3
- Still in python but now code is object oriented
- Redis cache to store the firstname from 42API
- Redis cache to store
- Norm and sanitize JSON files
- Hallvoice V3.1
- Dockerized!

###### Made with love by 42Paris team SI

###### Soon, Hallvoice V4 in Golang!
5 changes: 4 additions & 1 deletion cache/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
# cache
# Cache
You can bind this foler to your docker container

Default location for redis cache is `/var/lib/redis/dump.rdb`
23 changes: 16 additions & 7 deletions config.ini → config/config.ini.example
Original file line number Diff line number Diff line change
@@ -1,12 +1,21 @@
[kafka_conf]
kafka_servers = 127.0.0.1
topic = <topic-name>
group_id = <group-name>
username = <user-name>
password = <password>
[kafka]
kafka_servers = <redpanda>
topic = <topic>
group_id = <group_id>
username = <user>
password = <passwd>

[redis]
host = 127.0.0.1
port = 6379
ttl = 15778800 # 6 Months

[42api]
uid = <API42-UID-key>
secret = <API42-Top-Secret-Do_no_push_me>

[building]
name = <Where am I?>
name = <where>

[welcome]
message1 = Bonjour <name>
Expand Down
14 changes: 6 additions & 8 deletions custom/30_1.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
{
"welcome":
{
"mp3": "trentinou/in"
},
"goodbye":
{
"mp3": "trentinou/out"
}
"welcome": {
"mp3": "trentinou/in"
},
"goodbye": {
"mp3": "trentinou/out"
}
}
25 changes: 25 additions & 0 deletions custom/_Example.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"_comment": "To make your custom mp3 files working, please use the following JSON template",
"_comment": "First, you have to explicitly have a welcome and goodbye key",
"welcome":
{
"_comment": "Choses between `mp3` OR `txt`",
"_comment": "When `mp3` is used, Only ONE path is required, it could be a folder, or a single mp3 file",
"mp3":"mylogin/in",
"_comment": "mp3 files, MUST be in `./mp3/<yourlogin>/in`"
},
"goodbye":
{
"_comment": "Again, choses between `mp3` or `txt`",
"_comment": "When, `txt` is used, it generate a TTS with google (python3-gTTS)",
"_comment": "You can make one like so",

"txt": "Aurevoir hallvoice!",
"_comment": "OR you can make a list, randominette will be use to choose a random txt",
"txt": ["Bonsoir hallvoice", "Adieu hallvoice", "Tchao le hallvoice"],
"_comment": "By, default TTS is in french, you can use another language with key `lang`",
"lang": "zh-CN",
"_comment": "List of available language here: https://gtts.readthedocs.io/en/latest/_modules/gtts/lang.html"
},
"_comment": "FOR THE LOVE OF GOD, please remove comments before pushing your json :)"
}
16 changes: 7 additions & 9 deletions custom/aalliot.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
{
"welcome":
{
"mp3":"aalliot/"
},
"goodbye":
{
"mp3":"aalliot/"
}
}
"welcome": {
"mp3": "aalliot/"
},
"goodbye": {
"mp3": "aalliot/"
}
}
13 changes: 5 additions & 8 deletions custom/aaltun.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
{
"welcome":
{
"mp3": "aaltun/in"
"welcome": {
"mp3": "aaltun/in"
},
"goodbye":
{
"mp3": "aaltun/out"
"goodbye": {
"mp3": "aaltun/out"
}
}

}
16 changes: 7 additions & 9 deletions custom/aasli.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
{
"welcome":
{
"mp3": "aasli/in/"
},
"goodbye":
{
"mp3": "aasli/out/"
}
}
"welcome": {
"mp3": "aasli/in/"
},
"goodbye": {
"mp3": "aasli/out/"
}
}
16 changes: 7 additions & 9 deletions custom/aaverty.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
{
"welcome":
{
"mp3":"aaverty/"
},
"goodbye":
{
"mp3":"aaverty/"
}
}
"welcome": {
"mp3": "aaverty/"
},
"goodbye": {
"mp3": "aaverty/"
}
}
16 changes: 7 additions & 9 deletions custom/abasdere.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
{
"welcome":
{
"mp3":"abasdere/in"
},
"goodbye":
{
"mp3":"abasdere/out"
}
}
"welcome": {
"mp3": "abasdere/in"
},
"goodbye": {
"mp3": "abasdere/out"
}
}
16 changes: 7 additions & 9 deletions custom/abemorea.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
{
"welcome":
{
"mp3": "abemorea/entree"
},
"goodbye":
{
"mp3": "abemorea/sortie"
}
}
"welcome": {
"mp3": "abemorea/in"
},
"goodbye": {
"mp3": "abemorea/out"
}
}
12 changes: 5 additions & 7 deletions custom/ablanar.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
{
"welcome":
{
"mp3": "ablanar/in"
"welcome": {
"mp3": "ablanar/in"
},
"goodbye":
{
"mp3": "ablanar/out"
"goodbye": {
"mp3": "ablanar/out"
}
}
}
17 changes: 7 additions & 10 deletions custom/ablevin.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@

{
"welcome":
{
"mp3": "ablevin/in"
},
"goodbye":
{
"mp3": "ablevin/out"
}
}
"welcome": {
"mp3": "ablevin/in"
},
"goodbye": {
"mp3": "ablevin/out"
}
}
16 changes: 7 additions & 9 deletions custom/abonard.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
{
"welcome":
{
"mp3":"abonard/entree/"
},
"goodbye":
{
"mp3":"abonard/sortie/"
}
}
"welcome": {
"mp3": "abonard/in/"
},
"goodbye": {
"mp3": "abonard/out/"
}
}
16 changes: 7 additions & 9 deletions custom/abonneca.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
{
"welcome":
{
"mp3": "abonneca/in"
},
"goodbye":
{
"mp3": "abonneca/out"
}
}
"welcome": {
"mp3": "abonneca/in"
},
"goodbye": {
"mp3": "abonneca/out"
}
}
Loading
Loading