Enables python 3.13 #871

rudolfix · 2025-01-08T11:04:50Z

This PR makes pendulum work on Python 3.13. We are using it a lot in our library: https://github.com/dlt-hub/dlt and initially considered forking pendulum to make it work on Python 3.13. Maybe there's a chance to merge this into main project?

I'm sure some of the changes are going too far. We are open to revert them if there's an interest from maintainers to merge this PR :)

We are running this PR with our own tests on all basic architectures and we do not see any problems

O3 is bumped again to make it build on all Windows architectures
More architectures are added to release build
TimeMachine is used only in tests and present only in test deps.

happy to revert/change

part doing the actual release commented out in github actions
codespeed disabled (could not make it work on our bracnch)

Enables py 3.13

…hon-pendulum-master

ashb

Looks like something equivalent was merged in #863?

.github/workflows/tests.yml

pyproject.toml

rudolfix · 2025-01-14T09:51:01Z

Looks like something equivalent was merged in #863?

this is a continuation of #863. We had to upgrade O3 version again because we could not build Windows wheels. This again required us to handle some deprecations. I think @Secrus was able to merge the relevant commit.

We'll come back with more fixes. We will be testing updated wheels via our private PyPI and then update this PR.

Secrus · 2025-01-18T01:08:15Z

We have merged some updates to Poetry and CI setup. @rudolfix could you rebase the changes?

rudolfix · 2025-01-20T15:28:10Z

@Secrus pendulum is segfaulting on alpine linux (musllinux). it is due to mimalloc being used (and not disabled). do you have anything against removing mimalloc from the rust extensions? I'm not sure what are the expected speedups from using it.

also we added any wheel for platforms that we do not have wheels with rust built. this will disable speedups automatically. I hope that is OK. PR will come soon

Secrus · 2025-01-31T11:42:26Z

@rudolfix I am not against it, but I don't really know the low-level semantics, so if you could TL;DR what removing it would cause, I would be grateful.

rudolfix · 2025-02-03T11:36:28Z

@Secrus I did some investigations. in case of pendulum there's no reason to use mimalloc
https://github.com/microsoft/mimalloc?tab=readme-ov-file#benchmark-results-on-a-16-core-amd-5950x-zen3

it makes some sense on heavy multithreaded code where you allocate and deallocate on different thread. our case is exactly cfrac case (a lot of small allocations, single thread). so we can skip it.

Release workflow needs a lot of updates, some of actions are removed. We have a working one. I'll add this to the PR

Secrus

@rudolfix looks like we can drop mimalloc. Added some extra review comments.

pyproject.toml

.github/workflows/release.yml

pyproject.toml

Secrus · 2025-02-03T14:13:30Z

rust/src/python/parsing.rs

@@ -7,6 +7,7 @@ use pyo3::types::PyTime;
 use crate::parsing::Parser;
 use crate::python::types::{Duration, FixedTimezone};

+#[allow(deprecated)]


Why is that needed? Shouldn't we instead migrate off of the deprecated stuff? If the migration is too extensive, please add a TODO comment here.

There are more deprecations in O3 0.23, I was able to fix most of them, but here the whole function must be IMO refactored. TODO added

ramwin · 2025-02-17T17:35:34Z

Can we just provide a pure python version without using rust?

potiuk · 2025-02-20T13:08:02Z

Thanks @rudolfix (hello friend!) for raising the issue. I am eyeing now at Python 3.13 support for Airflow apache/airflow#46891 - and of course (following @ashb involvement) - it turns out that Pendulum and Python 3.13 is not a great match for now. Putting aside the discusssion whether it works - which I have not run my tests yet on - seems that at least Pendulum compiles on Python 3.13 - having cargo needed to do it is for sure a pain for users to install, so having some correctness fixes but also Python 3.12 wheels is absolutely needed #879

So @Secrus @sdispater -> now there are both Airflow and dlthub people interested. Can we help somohow? Are there any blockers to make it happen? As mentioned in #879 - we wiil shortly need to make decisions what to do with our dependencies that hold us back so it woudl be great to know if there is a way we can help to bring Pendulum up to Python 3.13 level.

Secrus · 2025-02-20T15:01:25Z

@potiuk all the changes needed for Python 3.13 support are contained within this PR. Once it is merged, I plan on releasing Pendulum 3.1, which will include wheels for Python 3.13.

potiuk · 2025-02-20T15:34:07Z

@potiuk all the changes needed for Python 3.13 support are contained within this PR. Once it is merged, I plan on releasing Pendulum 3.1, which will include wheels for Python 3.13.

Fantastic news ! Thank you. If there is any thing needed - I am happy to help. I can probably very easily (providing that Github URL package installation works for pendulum) - test Pendulum from this PR in Airflow test-suite in my Python 3.13 PR. Would that be helpful?

Secrus · 2025-02-21T08:43:16Z

@potiuk, if you have such a possibility, verifying that everything works as intended would be helpful. Thank you in advance

potiuk · 2025-02-21T17:08:11Z

@potiuk, if you have such a possibility, verifying that everything works as intended would be helpful. Thank you in advance

Yep - I will do some testing this weekend and come back to you :)

rudolfix · 2025-02-25T14:05:31Z

@Secrus @potiuk I'm on it again.

…ed github actions, uses new PyPI deployment workflow

codspeed-hq · 2025-02-25T17:42:26Z

CodSpeed Performance Report

Merging #871 will not alter performance

_{Comparing dlt-hub:enables-python-3.13 (4e513e5) with master (7642246)}

Summary

✅ 1 untouched benchmarks

rudolfix · 2025-02-25T17:50:15Z

@Secrus I was able to merge our newest version and run all the CI workflows. I did all the things in your review as well. thx!

@potiuk this may be interesting to you:
We had to do a few changes to release script in order to release pendulum on all platforms. dlt is already using version in this PR via our private PyPI, we also released our own fork on PyPI (which we'll abandon once this is mered).
it seems that it works fine.
changes in the release script:

build actions had to be updated, old ones were removed and stopped working. that had some consequences how files are copied into a single package
we added none build that will be used on all platforms where rust build is not supported, in that build extensions are disabled
we are using https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/ for the final publish. it is easy to setup. maybe you want to change to it? if not I can paste the old step there.

potiuk · 2025-02-26T11:29:54Z

we are using https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/ for the final publish. it is easy to setup. maybe you want to change to it? if not I can paste the old step there.

We are in a process of switching to releases via Trusted Publishing for the whole ASF (and we are leading it, but we want to do it a lot more robust and reusable, so it takes more time)

But I do not really need to publish things - I can simply build it in our CI, our CI build anyhow has all the tools to build pendulum in our CI before the .whl is published, so no worries. I even started it on weekend with this in our hatch_build.py:

https://github.com/apache/airflow/pull/46891/files#diff-d56d1b58c0ca28bdddd10c2cee2ab3cb4312766ea5de51f210dbc15a0fea42faR403

"pendulum @ git+https://github.com/python-pendulum/pendulum.git@76422468575b5817f7a58f3d05c26b480bf671a9"

But I had other issues with the build and other things to look at - so I am only on it now :)

potiuk · 2025-02-26T11:51:02Z

Yeah - apache/airflow#46891 and while it will still have some failures, I hope we will see if there will be any related to pendulum - but yeah - once I have cargo in my image (I have because some other packages for Python 3.13 need it) - Pendulum nicely installs with github-url installatoin.

rudolfix added 14 commits November 9, 2024 06:20

bumps to newest pyO3, converts Py into Bound

2832e5d

disables pypi release

b0aeb53

linter fixes

b0f928d

fixes linting

07da530

Merge pull request #1 from dlt-hub/enables-py-3.13

55f0f5e

Enables py 3.13

fixes release builds

64680e3

fixes package

aa0472f

Merge branch 'master' of github.com:python-pendulum/pendulum into pyt…

2242207

…hon-pendulum-master

Merge branch 'python-pendulum-master'

24f40f6

adds patchelf

d652bdd

make formats

adc533a

bump pyo3

f93c927

cleansup some rust deprecations

d29dee6

uses install poetry action to get poetry 1.8.5

be8936f

ashb reviewed Jan 13, 2025

View reviewed changes

.github/workflows/tests.yml Outdated Show resolved Hide resolved

pyproject.toml Outdated Show resolved Hide resolved

b0661 mentioned this pull request Jan 17, 2025

Update README.md pendulum Akkudoktor-EOS/EOS#386

Closed

Secrus reviewed Feb 3, 2025

View reviewed changes

Merge remote-tracking branch 'upstream/master' into enables-python-3.13

935539a

rudolfix added 5 commits February 25, 2025 16:02

correctly skips extensions

ddd0522

removes mimalloc

4a0c1e4

reverts unwanted pyproject changes

d6dec62

updates release.yml: adds none build without speedups, replaces remov…

b0755e8

…ed github actions, uses new PyPI deployment workflow

adds todo on rust o3 deprecation

073d3c4

rudolfix force-pushed the enables-python-3.13 branch 2 times, most recently from 68b925d to 4f0976c Compare February 25, 2025 17:37

fixes codespeed workflow

4e513e5

rudolfix force-pushed the enables-python-3.13 branch from 4f0976c to 4e513e5 Compare February 25, 2025 17:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enables python 3.13 #871

Enables python 3.13 #871

rudolfix commented Jan 8, 2025 •

edited

Loading

ashb left a comment •

edited

Loading

rudolfix commented Jan 14, 2025

Secrus commented Jan 18, 2025

rudolfix commented Jan 20, 2025

Secrus commented Jan 31, 2025

rudolfix commented Feb 3, 2025

Secrus left a comment

Secrus Feb 3, 2025

rudolfix Feb 25, 2025

ramwin commented Feb 17, 2025

potiuk commented Feb 20, 2025

Secrus commented Feb 20, 2025

potiuk commented Feb 20, 2025

Secrus commented Feb 21, 2025

potiuk commented Feb 21, 2025

rudolfix commented Feb 25, 2025

codspeed-hq bot commented Feb 25, 2025

rudolfix commented Feb 25, 2025

potiuk commented Feb 26, 2025

potiuk commented Feb 26, 2025

Enables python 3.13 #871

Are you sure you want to change the base?

Enables python 3.13 #871

Conversation

rudolfix commented Jan 8, 2025 • edited Loading

ashb left a comment • edited Loading

Choose a reason for hiding this comment

rudolfix commented Jan 14, 2025

Secrus commented Jan 18, 2025

rudolfix commented Jan 20, 2025

Secrus commented Jan 31, 2025

rudolfix commented Feb 3, 2025

Secrus left a comment

Choose a reason for hiding this comment

Secrus Feb 3, 2025

Choose a reason for hiding this comment

rudolfix Feb 25, 2025

Choose a reason for hiding this comment

ramwin commented Feb 17, 2025

potiuk commented Feb 20, 2025

Secrus commented Feb 20, 2025

potiuk commented Feb 20, 2025

Secrus commented Feb 21, 2025

potiuk commented Feb 21, 2025

rudolfix commented Feb 25, 2025

codspeed-hq bot commented Feb 25, 2025

CodSpeed Performance Report

Merging #871 will not alter performance

Summary

rudolfix commented Feb 25, 2025

potiuk commented Feb 26, 2025

potiuk commented Feb 26, 2025

rudolfix commented Jan 8, 2025 •

edited

Loading

ashb left a comment •

edited

Loading