Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SaferRemoveIndexConcurrently #18

Merged
merged 6 commits into from
Sep 3, 2024

Conversation

marcelofern
Copy link
Contributor

@marcelofern marcelofern commented Aug 26, 2024

Description

Adds a new Django migration operation class SaferRemoveIndexConcurrently. The name is self-explanatory, but here's a screenshot of the documentation (built with make docs) with more details (also available in the diff on the ante-penultimate commit).

image

Copy link

github-actions bot commented Aug 27, 2024

Coverage Report Results

Name Stmts Miss Branch BrPart Cover
src/django_pg_migration_tools/__init__.py 0 0 0 0 100%
src/django_pg_migration_tools/management/__init__.py 0 0 0 0 100%
src/django_pg_migration_tools/management/commands/__init__.py 0 0 0 0 100%
src/django_pg_migration_tools/management/commands/migrate_with_timeouts.py 25 0 10 0 100%
src/django_pg_migration_tools/operations.py 67 0 6 0 100%
src/django_pg_migration_tools/timeouts.py 73 5 36 2 92%
TOTAL 165 5 52 2 96%

@marcelofern marcelofern force-pushed the add-safer-remove-index-concurrently branch 2 times, most recently from e356bbf to d3ee1f5 Compare August 27, 2024 23:45
In future commits, we want to have a new class called
SaferRemoveIndexConcurrently, which will basically be the same as
SaferAddIndexConcurrently but with opposite behaviour.

These functions that are now being moved to the base class will be
useful in this future SaferRemoveIndexConcurrently class and are thus
extracted in this commit.

The diff might look a bit strange, but essentially this commit moves the
database_forwards/backwards functions out (along with the auxiliary
functions that make it work) into the new BaseIndexOperation class,
and calls it from the existing SaferAddIndexConcurrently class.
Given that the BaseIndexOperation class will cater for both
SaferAddIndexConcurrently and SaferRemoveIndexConcurrently, we need to
change the way the index argument is passed.

This is because SaferAddIndexConcurrently is instantiated like this:

SaferAddIndexConcurrently(
    model_name="foo",
    index=models.Index(
        fields=["bar"],
        name="foo_bar_idx",
    ),
)

Whereas SaferRemoveIndexConcurrently will be instantiated like this (and
thus doesn't have an index obj at instantiation time):

SaferRemoveIndexConcurrently(
    model_name="foo",
    name="foo_bar_idx",
)
So that they can be run with other index names later.
Not using RunSQL as a base means that I can simplify much of the code in
this class. Plus, I can also make the BaseIndexOperation more generic so
that it can be easily used by SaferRemoveIndexConcurrently

Note that the `hint` argument isn't available on Django's original
AddIndexConcurrently class, and it is not necessary any longer as we
don't use the router but the model itself to check if the migration can
happen. Thus, the `hint` argument has been removed from the
implementation and documentation.
@marcelofern marcelofern force-pushed the add-safer-remove-index-concurrently branch from 8375415 to ae504a1 Compare August 28, 2024 00:18
@marcelofern marcelofern marked this pull request as ready for review August 28, 2024 00:33
@@ -32,7 +32,7 @@ def safer_create_index(
to_state: migrations.state.ProjectState,
index: models.Index,
model: type[models.Model],
operation: SaferAddIndexConcurrently,
operation: SaferAddIndexConcurrently | SaferRemoveIndexConcurrently,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this instead be a Protocol class as the type you expect?

Copy link
Contributor Author

@marcelofern marcelofern Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be only two classes that create/drop indexes, so I'm not sure there're gains in using structural typing here if we're not aiming for flexibility. On the side, nominal typing provides stronger contracts too.

pyproject.toml Outdated
@@ -21,7 +21,7 @@ where = ["src"]

[project]
name = "django_pg_migration_tools"
version = "0.1.2"
version = "0.1.3"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd encourage the previous commit updating dependencies and the part of this commit that changes the version number be a separate PR dedicated to releasing a new version rather than bundling it with this change.

Copy link
Contributor Author

@marcelofern marcelofern Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Are there any pros/cons to either approach I wouldn't be aware of? I thought that if a release is linked to a single change it would be best to keep it in the same PR so that it's easier to link to the scoped work from the tags page. Similar to how some repos create a single release PR over multiple aggregated feature branches. Not particularly fussy about that either though.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, certainly not a hard requirement, but my own sensibility is that releases should be separate as a general habit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a separated PR for the release. To be merged after this one is merged.

self, app_label: str, state: migrations.state.ProjectState

class SaferAddIndexConcurrently(
BaseIndexOperation, psql_operations.AddIndexConcurrently
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there anything stopping us from not using inheritence for composition? So that the SaferAddIndexConcurrently instance creates it's own instance of AddIndexConcurrently that it delegates to rather than being a subclass of AddIndexConcurrently?

Copy link
Contributor Author

@marcelofern marcelofern Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd surely prefer composition if the code action was "AddIndexConcurrently, please go do something for me".

But in this case the code action is more like "AddIndexConcurrently, I need your interface so that Django's migration apparatus can understand me as an index creation/deletion operation".

This includes having methods that will register application database state alterations for example. But more importantly, I'm interested in picking up changes from upstream on that specific interface.

For example, a recent change in Django (new in 5.1) has added the "category" attribute to operations. In this case, category = OperationCategory.ADDITION (or REMOVAL), wouldn't be picked up by our classes unless we inherit from upstream. Otherwise, we'd have to keep different interfaces for different Django versions, which is less desirable.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair enough, inheritence sounds reasonable then.

@marcelofern marcelofern requested a review from delfick August 28, 2024 03:12
@marcelofern marcelofern force-pushed the add-safer-remove-index-concurrently branch from ae504a1 to ae35bef Compare September 1, 2024 21:55
Copy link

@timb07 timb07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; just a couple of copy/paste typos to fix.

"charmodel",
"char_field_idx",
)
# Proceed to try and add the index:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy/paste: "remove" instead of "add"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx: 54d6cf9

How to use
----------

1. Remove the index to the relevant model:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy/paste: should be "… from the relevant model"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx: 50d6377

This class is the analogous of SaferAddIndexConcurrently, but does the
direct oposite. I.e., it swaps the database_forwards operation by the
database_backwards and vice-versa.
@marcelofern marcelofern force-pushed the add-safer-remove-index-concurrently branch from 50d6377 to 68aa12a Compare September 3, 2024 02:30
@marcelofern marcelofern merged commit 81233e6 into main Sep 3, 2024
10 checks passed
@marcelofern marcelofern deleted the add-safer-remove-index-concurrently branch September 3, 2024 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants