Skip to content

Commit

Permalink
Add documentation on how to use SaferRemoveIndexConcurrently
Browse files Browse the repository at this point in the history
  • Loading branch information
marcelofern committed Aug 27, 2024
1 parent 763782e commit d3ee1f5
Showing 1 changed file with 94 additions and 0 deletions.
94 changes: 94 additions & 0 deletions docs/usage/operations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -109,3 +109,97 @@ Class Definitions
index=models.Index(fields=["foo"], name="foo_idx"),
),
]
.. py:class:: SaferRemoveIndexConcurrently(model_name: str, name: str)
Performs DROP INDEX CONCURRENTLY IF EXISTS without a lock_timeout
value to guarantee the index removal won't be affected by any pre-set
value of lock_timeout.

:param model_name: Model name in lowercase without underscores.
:type model_name: str
:param name: The name of the index to be deleted.
:type name: str

**Why use SaferRemoveIndexConcurrently?**
-----------------------------------------

Django already provides its own ``RemoveIndexConcurrently`` class, which is
available through *django.contrib.postres.operations*.

Django's operation performs a naive:

.. code-block:: sql
DROP INDEX CONCURRENTLY IF EXISTS <idx_name>;
Which has a few problems:

1. It might time out if an existing value of lock_timeout is pre-set.
2. If the operation started but failed because of a lock_timeout error,
the existing index won't be removed and it will be marked as INVALID.

Point 2. is usually not widely known. Even with the CONCURRENTLY
condition, the index removal needs to wait on locks, potentially being
interrupted by a lock_timeout thus leaving the existing index marked as
INVALID.

This custom ``SaferRemoveIndexConcurrently`` operation addresses theses
problems by running the following operations:

.. code-block:: sql
-- Necessary so that we can reset the value later.
SHOW lock_timeout;
-- Necessary to avoid lock timeouts. This is a safe operation as
-- DROP INDEX CONCURRENTLY does not lock concurrent selects, inserts,
-- updates, or deletes.
SET lock_timeout = 0;
-- Drop the index
DROP INDEX CONCURRENTLY IF EXISTS foo_idx;
-- Reset lock_timeout to its original value ("1s" as an example).
SET lock_timeout = '1s';
How to use
----------

1. Remove the index to the relevant model:

.. code-block:: diff
class Meta:
indexes = (
- # Existing index being removed.
- models.Index(fields=["foo"], name="foo_idx"),
# Another existing index not being removed.
models.Index(fields=["bar"], name="bar_idx"),
2. Make the new migration:

.. code-block:: bash
./manage.py makemigrations
3. Swap the ``RemoveIndex`` class for ``SaferRemoveIndexConcurrently``.
Remember to use a non-atomic migration.

.. code-block:: diff
+ from django_pg_migration_tools import operations
from django.db import migrations, models
class Migration(migrations.Migration):
+ atomic = False
dependencies = [("myapp", "0042_dependency")]
operations = [
- migrations.RemoveIndex(
+ operations.SaferRemoveIndexConcurrently(
model_name="mymodel",
name="foo_idx",
),
]

0 comments on commit d3ee1f5

Please sign in to comment.