Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to pyproject.toml #17116

Merged
merged 38 commits into from
Feb 18, 2025
Merged

Migrate to pyproject.toml #17116

merged 38 commits into from
Feb 18, 2025

Conversation

zzstoatzz
Copy link
Collaborator

@zzstoatzz zzstoatzz commented Feb 12, 2025

This PR migrates our library configuration from setup.cfg and setup.py to pyproject.toml.

As part of this migration, we were also able to migrate our build system from setuptools to hatchling, which has proven to be more ergonomic so far. This PR also moves us away from using versioneer to versioningit instead, which is more customizable and integrates nicely with hatchling.

Moving to pyproject.toml allow us to greatly improve the developer experience by enabling new uv features. Developers now can run uv sync to automatically create a virtual environment and install the necessary dependencies. The use of uv sync also simplifies our CI workflows. We set up local sources for all integrations, which allow developers to easily install a local editable version of prefect when working on an integration by running uv sync in an integrations sub-directory.

The contribution docs have also been updated to reflect the new capabilities that uv offers for our project.

Closes #8007
Closes #17124

@zzstoatzz zzstoatzz changed the title no prefect 4 yet working on pyproject.toml Feb 12, 2025
@zzstoatzz zzstoatzz added the development Tech debt, refactors, CI, tests, and other related work. label Feb 12, 2025
@github-actions github-actions bot added the upstream dependency An upstream issue caused by a bug in one of our dependencies label Feb 12, 2025
Copy link

codspeed-hq bot commented Feb 12, 2025

CodSpeed Performance Report

Merging #17116 will improve performances by 10.05%

Comparing pyproject-v2 (88aaee9) with main (1262485)

Summary

⚡ 2 improvements

Benchmarks breakdown

Benchmark BASE HEAD Change
bench_import_prefect_flow 1.7 s 1.4 s +21.63%
bench_task_decorator 485.7 µs 441.3 µs +10.05%

@zzstoatzz zzstoatzz force-pushed the pyproject-v2 branch 2 times, most recently from 07430d3 to b933628 Compare February 12, 2025 22:16
@github-actions github-actions bot added the bug Something isn't working label Feb 13, 2025
@zzstoatzz zzstoatzz changed the title working on pyproject.toml migrate to pyproject.toml Feb 13, 2025
@zzstoatzz zzstoatzz force-pushed the pyproject-v2 branch 3 times, most recently from 05f4e92 to 4609191 Compare February 14, 2025 21:46
@zzstoatzz zzstoatzz marked this pull request as ready for review February 14, 2025 21:59
@desertaxle desertaxle changed the title migrate to pyproject.toml Migrate to pyproject.toml Feb 14, 2025
@github-actions github-actions bot added the docs label Feb 14, 2025
@@ -31,8 +31,8 @@ def _read_toml_file(path: Path) -> dict[str, Any]:
"""use ttl cache to cache toml files"""
modified_time = path.stat().st_mtime
cache_key = f"toml_file:{path}:{modified_time}"
if cache_key in _file_cache:
return _file_cache[cache_key]
if value := _file_cache.get(cache_key):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eliminates a possible key missing error from the duplicate check

Comment on lines +51 to +93
client = [
# Client dependencies
# If you modify this list, make the same modification in client/pyproject.toml
"anyio>=4.4.0,<5.0.0",
"asgi-lifespan>=1.0,<3.0",
"cachetools>=5.3,<6.0",
"cloudpickle>=2.0,<4.0",
"coolname>=1.0.4,<3.0.0",
"croniter>=1.0.12,<7.0.0",
"exceptiongroup>=1.0.0",
"fastapi>=0.111.0,<1.0.0",
"fsspec>=2022.5.0",
"graphviz>=0.20.1",
"griffe>=0.49.0,<2.0.0",
"httpcore>=1.0.5,<2.0.0",
"httpx[http2]>=0.23,!=0.23.2",
"importlib_metadata>=4.4;python_version<'3.10'",
"jsonpatch>=1.32,<2.0",
"jsonschema>=4.0.0,<5.0.0",
"opentelemetry-api>=1.27.0,<2.0.0",
"orjson>=3.7,<4.0",
"packaging>=21.3,<24.3",
"pathspec>=0.8.0",
"pendulum>=3.0.0,<4",
"prometheus-client>=0.20.0",
"pydantic>=2.9,<3.0.0,!=2.10.0",
"pydantic_core>=2.12.0,<3.0.0",
"pydantic_extra_types>=2.8.2,<3.0.0",
"pydantic_settings>2.2.1",
"python-dateutil>=2.8.2,<3.0.0",
"python-slugify>=5.0,<9.0",
"python-socks[asyncio]>=2.5.3,<3.0",
"pyyaml>=5.4.1,<7.0.0",
"rfc3339-validator>=0.1.4,<0.2.0",
"rich>=11.0,<14.0",
"ruamel.yaml>=0.17.0",
"sniffio>=1.3.0,<2.0.0",
"toml>=0.10.0",
"typing_extensions>=4.5.0,<5.0.0",
"ujson>=5.8.0,<6.0.0",
"uvicorn>=0.14.0,!=0.29.0",
"websockets>=10.4,<14.0",
]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as previously in requirements-client.txt

Comment on lines +220 to +240
[tool.hatch.version]
source = "versioningit"

[tool.versioningit.vcs]
match = ["[0-9]*.[0-9]*.[0-9]*", "[0-9]*.[0-9]*.[0-9]*.dev[0-9]*"]
default-tag = "0.0.0"

[tool.versioningit.write]
method = { module = "write_build_info", value = "write_build_info", module-dir = "tools" }
path = "src/prefect/_build_info.py"

[tool.versioningit.format]
distance = "{base_version}+{distance}.{vcs}{rev}"
dirty = "{base_version}+{distance}.{vcs}{rev}.dirty"
distance-dirty = "{base_version}+{distance}.{vcs}{rev}.dirty"

[tool.hatch.build]
artifacts = ["src/prefect/_build_info.py"]

[tool.hatch.build.targets.sdist]
include = ["/src/prefect", "/README.md", "/LICENSE", "/pyproject.toml"]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

important build config

Copy link
Member

@cicdw cicdw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one question, otherwise this is looking good to me -- have you seen noticeable speed up in CI with these things?

MANIFEST.in Show resolved Hide resolved
Copy link
Member

@cicdw cicdw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

amazing, great work!

@zzstoatzz zzstoatzz merged commit ed3c9aa into main Feb 18, 2025
121 checks passed
@zzstoatzz zzstoatzz deleted the pyproject-v2 branch February 18, 2025 18:19
This was referenced Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working development Tech debt, refactors, CI, tests, and other related work. docs upstream dependency An upstream issue caused by a bug in one of our dependencies
Projects
None yet
Development

Successfully merging this pull request may close these issues.

fix get_client overloads Migrate from setup.cfg to pyproject.toml
3 participants