Skip to content

FastAPI port of API/Studio, second time around #360

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Apr 17, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 7 additions & 10 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,30 +25,26 @@ jobs:
run: |
python -m pip install --upgrade pip
pip install -e .[test]
pip install gunicorn
pip install pip-licenses
if pip-licenses | grep -v 'Artistic License' | grep -v LGPL | grep GNU; then echo 'Please avoid introducing *GPL dependencies'; false; fi
pip install coverage
- name: Ensure browser is installed
run: python -m playwright install --with-deps chromium
- name: Launch the API
run: |
coverage run --parallel-mode \
-m gunicorn --worker-class eventlet -w 1 g2p.app:APP --no-sendfile --bind 127.0.0.1:5000 --daemon
run: coverage run run_studio.py &
- name: Run dev suite
run: coverage run --parallel-mode run_tests.py dev
run: coverage run run_tests.py dev
- name: Run test-studio
run: |
coverage run --parallel-mode g2p/tests/test_studio.py
run: coverage run g2p/tests/test_studio.py
- name: Run generate-mapping
shell: bash
run: |
coverage run --parallel-mode $(which g2p) generate-mapping --from crg --to eng
coverage run $(which g2p) generate-mapping --from crg --to eng
git status --porcelain=v1 | grep 'M g2p/mappings/langs/generated/config-g2p.yaml' || { echo 'g2p generate-mapping did not update generated/config-g2p.yaml as expected'; false; }
- name: Post test analyses
run: |
pkill coverage
sleep 5
pkill -INT coverage
sleep 10
coverage combine
coverage report
coverage xml
Expand Down Expand Up @@ -92,6 +88,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
# Note: this is where we also test that the g2p library still works on 3.7
python-version: "3.7"
- name: Install dependencies
run: |
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,4 @@ COPY Dockerfile /g2p
RUN pip3 install -e /g2p

# Comment this out if you just want to install g2p in the container without running the studio.
CMD gunicorn --worker-class eventlet -w 1 g2p.app:APP --no-sendfile --bind 0.0.0.0:8000
CMD gunicorn --worker-class uvicorn.workers.UvicornWorker -w 1 g2p.app:APP --bind 0.0.0.0:8000
2 changes: 1 addition & 1 deletion Procfile
Original file line number Diff line number Diff line change
@@ -1 +1 @@
web: gunicorn --worker-class eventlet -w 1 g2p.app:APP --no-sendfile
web: gunicorn --worker-class uvicorn.workers.UvicornWorker -w 1 g2p.app:APP
19 changes: 9 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,9 +200,8 @@ Gen DB: this is the part of the textual database that is generated when running

Compiled DB: this contains the same info as Text DB + Gen DB, but in a format optimized for fast reading by the machine. This is what any program using `g2p` reads: `g2p convert`, `readalongs align`, `convertextract`, and also `g2p generate-mapping`. It consists of these files:
* g2p/mappings/langs/langs.json.gz
* g2p/mappings/langs/network.pkl
* g2p/mappings/langs/static/languages-network.json
* g2p/mappings/langs/static/swagger.json
* g2p/mappings/langs/network.json.gz
* g2p/static/languages-network.json

So, when you write a new g2p mapping for a language, say `lll`, and you want to be able to convert text from `lll` to `eng-ipa` or `eng-arpabet`, you need to do the following:
1. Write the mapping from `lll` to `lll-ipa` in g2p/mappings/langs/lll/. You've just updated Text DB.
Expand All @@ -214,19 +213,19 @@ Once you have the Compiled DB, it is then possible to use the `g2p convert` comm

## Studio

You can also run the `g2p Studio` which is a web interface for creating custom lookup tables to be used with g2p. To run the `g2p Studio` either visit https://g2p-studio.herokuapp.com/ or run it locally using `python run_studio.py`.
You can also run the `g2p Studio` which is a web interface for
creating custom lookup tables to be used with g2p. To run the `g2p
Studio` either visit https://g2p-studio.herokuapp.com/ or run it
locally with `python run_studio.py`.

## API for Developers

There is also a REST API available for use in your own applications.
To launch it from the command-line use `python run_studio.py` or
`flask --app g2p.app run`. The API documentation will be viewable
`uvicorn g2p.app:APP`. The API documentation will be viewable
(with the ability to use it interactively) at
http://localhost:5000/docs - an OpenAPI definition is also available
at http://localhost:5000/static/swagger.json .

You can see the list of URLs served by the API using `flask --app
g2p.app routes`.
http://localhost:5000/api/v1/docs - an OpenAPI definition is also available
at http://localhost:5000/api/v1/openapi.json .

## Maintainers

Expand Down
16 changes: 4 additions & 12 deletions docs/migration-2.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,22 +100,14 @@ been using them and should not be construed as public API documentation.

## Some CLI commands no longer exist

Several commands for the `g2p` command-line have been removed as they
were duplicates of other functionality:
Several commands for the `g2p` command-line have been removed:

- run
- routes
- shell

To run the `g2p` API server, you can use:

flask --app g2p.app run

Likewise, for `routes` and `shell`, you can use:

flask --app g2p.app routes
flask --app g2p.app shell

To run G2P Studio, use:
To run the `g2p` API server and G2P Studio, you can use:

python run_studio.py

It does not seem that any equivalents of `routes` or `shell` exist.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here this is conflating the migration from 1.x to 2.0 and from 2.0 to what will probably have to be 2.1 or possibly even 3.0 if this is not backwards compatible. While I don't think we need a separate migrating to 2.1 file, I think this section should distinguish between migrating to 2.0 (previous text) and to 2.1 new text like you're writing here.
Side note: losing routes is unfortunate. But I assume there's some other form of auto-generated API documentation?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, there's the auto-generated online API documentation, which is arguably more useful anyway...

Loading