feat: Migrate organizations list to control #92442

dcramer · 2025-05-28T21:18:15Z

WIP

Here's the planned changes:

Add list_organizations RPC method
Migrate /api/0/organizations to list_organizations
Introduce context for RPC calls that give full auth/tenant information (request.auth/request.is_active_superuser concerns)
Utilize control when possible, and fall back to local
Introduce concept of degraded responses (metadata in responses that showcase soft errors)

dcramer · 2025-05-28T21:19:13Z

src/sentry/organizations/services/organization/methods/list_organizations.py

+        order_by = "-date_added"
+        paginator_cls = DateTimePaginator
+
+    # return paginate(


this would need migrated, shoudl be easy enough

dcramer · 2025-05-28T21:19:29Z

src/sentry/organizations/services/organization/methods/list_organizations.py

+    show: Show | None = None,
+    sort_by: SortBy | None = "date",
+    # actor specific stuff
+    actor_user_id: int,


preferably this is a similar construct to 'request.auth' but something all RPC endpoints gain access to - "the current actor"

dcramer · 2025-05-28T21:19:41Z

src/sentry/organizations/services/organization/methods/list_organizations.py

+    ALL = "all"
+
+
+def list_organizations(


broken out into its own module just because the giant impl file is insanity

dcramer · 2025-05-28T21:20:35Z

src/sentry/organizations/services/organization/methods/list_organizations.py

+        order_by = "-member_count"
+        paginator_cls = OffsetPaginator
+    elif sort_by == "projects":
+        queryset = queryset.annotate(project_count=Count("project"))


confirm: are projects on control? i think they should be

Projects are not available in Control Silo:

sentry/src/sentry/models/project.py

Line 228 in 59c3578

class Project(Model):

Similarly, organizations are not fully available in Control Silo either:

sentry/src/sentry/models/organization.py

Line 144 in 59bd9f3

class Organization(ReplicatedRegionModel):

We do have a thin replicated version of organizations in Control though (OrganizationMapping)

I don't have the full context on that particular decision, though @markstory could probably weigh in on this.

Projects are not available in Control Silo:

Projects are organization scoped, and needed to stay in the region for existing transactions and application logic to work. Because all of this logic will need to be region scoped (because of organization) you'll also be able to use projects.

dcramer · 2025-05-28T21:21:09Z

src/sentry/organizations/services/organization/methods/list_organizations.py

+    ALL = "all"
+
+
+def list_organizations(


the API would call this RPC method, and if its fails remotely (e.g. on control), could try/catch onto calling local?

the API would call this RPC method, and if its fails remotely (e.g. on control), could try/catch onto calling local?

That isn't typically how RPC services work. Instead they choose a mode (rpc proxy or local) based on the silo mode that the application is deployed in.

codecov · 2025-05-28T21:37:02Z

❌ 8 Tests Failed:

Tests completed	Failed	Passed	Skipped
26009	8	26001	207

View the top 3 failed test(s) by shortest run time

tests.sentry.api.endpoints.test_organization_index.OrganizationsListTest::test_status_query

Stack Traces | 1.38s run time

#x1B[1m#x1B[.../api/endpoints/test_organization_index.py#x1B[0m:85: in test_status_query
    response = self.get_success_response(qs_params={"query": "status:pending_deletion"})
#x1B[1m#x1B[.../sentry/testutils/cases.py#x1B[0m:616: in get_success_response
    assert_status_code(response, status.HTTP_200_OK)
#x1B[1m#x1B[.../sentry/testutils/asserts.py#x1B[0m:45: in assert_status_code
    assert minimum <= response.status_code < maximum, (
#x1B[1m#x1B[31mE   AssertionError: (500, b'{"detail":"Internal Error","errorId":null}')#x1B[0m
#x1B[1m#x1B[31mE   assert 500 < 201#x1B[0m
#x1B[1m#x1B[31mE    +  where 500 = <Response status_code=500, "application/json">.status_code#x1B[0m

tests.sentry.api.endpoints.test_organization_index.OrganizationsListTest::test_show_all_without_superuser

Stack Traces | 2.13s run time

#x1B[1m#x1B[.../api/endpoints/test_organization_index.py#x1B[0m:56: in test_show_all_without_superuser
    response = self.get_success_response(qs_params={"show": "all"})
#x1B[1m#x1B[.../sentry/testutils/cases.py#x1B[0m:616: in get_success_response
    assert_status_code(response, status.HTTP_200_OK)
#x1B[1m#x1B[.../sentry/testutils/asserts.py#x1B[0m:45: in assert_status_code
    assert minimum <= response.status_code < maximum, (
#x1B[1m#x1B[31mE   AssertionError: (500, b'{"detail":"Internal Error","errorId":null}')#x1B[0m
#x1B[1m#x1B[31mE   assert 500 < 201#x1B[0m
#x1B[1m#x1B[31mE    +  where 500 = <Response status_code=500, "application/json">.status_code#x1B[0m

tests.sentry.api.endpoints.test_organization_index.OrganizationsListTest::test_member_id_query

Stack Traces | 2.47s run time

#x1B[1m#x1B[.../api/endpoints/test_organization_index.py#x1B[0m:99: in test_member_id_query
    response = self.get_success_response(qs_params={"member": 1})
#x1B[1m#x1B[.../sentry/testutils/cases.py#x1B[0m:616: in get_success_response
    assert_status_code(response, status.HTTP_200_OK)
#x1B[1m#x1B[.../sentry/testutils/asserts.py#x1B[0m:45: in assert_status_code
    assert minimum <= response.status_code < maximum, (
#x1B[1m#x1B[31mE   AssertionError: (500, b'{"detail":"Internal Error","errorId":null}')#x1B[0m
#x1B[1m#x1B[31mE   assert 500 < 201#x1B[0m
#x1B[1m#x1B[31mE    +  where 500 = <Response status_code=500, "application/json">.status_code#x1B[0m

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

dcramer · 2025-05-28T22:37:04Z

@markstory is there a reason I cant make this new method be:

list_organizations(organization_service, ...)

And not even add it to the existing mega class?

dcramer · 2025-05-28T22:58:49Z

@markstory how do i know if somethings on control vs region? i see both org/project with @region_silo_model

GabeVillalobos · 2025-05-28T23:01:34Z

I'm curious what the intention is behind this change, and how the full implementation will look.

In particular, these parts:

Utilize control when possible, and fall back to local
Introduce concept of degraded responses (metadata in responses that showcase soft errors)

Is the idea to have a control endpoint fan out to all available regions a user has data in, in order to retrieve the full list of organizations that user belongs to?

dcramer · 2025-05-28T23:08:14Z

I'm curious what the intention is behind this change, and how the full implementation will look.

In particular, these parts:

Utilize control when possible, and fall back to local
Introduce concept of degraded responses (metadata in responses that showcase soft errors)

Is the idea to have a control endpoint fan out to all available regions a user has data in, in order to retrieve the full list of organizations that user belongs to?

The intention behind this change is to fix what is a broken API experience today. Right now the API no longer works for any metadata endpoint, as you'd have to query every region silo that might possibly exist, which is particularly a problem for core user and organization-style metadata. This exposed itself in the MCP server when users with DE-only orgs received empty responses.

My original thinking was:

This data must live on control, as its sentry metadata [clearly this is not how the world exists, which is a separate problem]
We dont want to rely on control being online if we can avoid it, so if control is offline, read only the local data

I might have to revisit that plan. I'm also going to push us to revisit why its structured this way in the first place as it creates a lot of undue complexity, but I'm sure there was a reason.

If you designed this from the ground up you'd have the following system:

a control plane - core system metadata. This would have information like user accounts, organizations, some other stuff.
the region silos

This is kind of what we have, but clearly not all the way given data appears to almost never? live in the control plane.

You'd then design two APIs: a control plane API (e.g. the /organizations endpoint) and the individual tenants APIs (the region silo endpoints). This is generally speaking how you'd want to design any of these tenant based systems, otherwise you have no way to actually identify data without magic knowledge in a system (e.g. I "know" there is DE and US tenants).

We are half way there. Organization responses give you their regionUrl. The problem is those responses cannot be achieved without.. regional responses.

I think the tl;dr is organization needs migrated to control, just like user is.

GabeVillalobos · 2025-05-28T23:36:11Z

My original thinking was:

This data must live on control, as its sentry metadata [clearly this is not how the world exists, which is a separate problem]
We dont want to rely on control being online if we can avoid it, so if control is offline, read only the local data

Got it. I think it makes sense to revisit this decision, since this has caused a fair bit of friction in some of our other APIs (organization/subscription provisioning are the most obvious two I can think of). I do want to call out that this may be a considerable amount of work to actually achieve.

In the meantime, we do support a control-silo endpoint that supplies all available region URLs a user has data in via the https://sentry.sentry.io/api/0/users/me/regions/ endpoint as a point of discovery/bridge for client fan-outs, which is what we use in the CLI to list all available orgs. It's not perfect, but it's allowed us to sidestep this concern for now.

markstory · 2025-05-29T14:02:00Z

src/sentry/organizations/services/organization/methods/list_organizations.py

+    ALL = "all"
+
+
+def list_organizations(


the API would call this RPC method, and if its fails remotely (e.g. on control), could try/catch onto calling local?

That isn't typically how RPC services work. Instead they choose a mode (rpc proxy or local) based on the silo mode that the application is deployed in.

markstory · 2025-05-29T14:07:36Z

src/sentry/organizations/services/organization/methods/list_organizations.py

+        order_by = "-member_count"
+        paginator_cls = OffsetPaginator
+    elif sort_by == "projects":
+        queryset = queryset.annotate(project_count=Count("project"))


Projects are not available in Control Silo:

Projects are organization scoped, and needed to stay in the region for existing transactions and application logic to work. Because all of this logic will need to be region scoped (because of organization) you'll also be able to use projects.

markstory · 2025-05-29T14:08:15Z

src/sentry/organizations/services/organization/methods/list_organizations.py

+                {"organization": serialize(org), "singleOwner": org.has_single_owner()}
+            )
+
+        return Response(org_results)


I don't think this will work as RPC services need to serialize their return values into JSON via pydantic models.

markstory · 2025-05-29T14:10:03Z

src/sentry/api/endpoints/organization_index.py

-            order_by=order_by,
-            on_results=lambda x: serialize(x, request.user),
-            paginator_cls=paginator_cls,
+        result = organization_service.list_organizations(


Another wrinkle we'll run into with having org list be available on control is that we don't have a way for an endpoint to be partially in control and partially in region. The other methods on this endpoint need to stay in the region because they mutate the organization. Django might not like having two endpoints bound to the same URL as well.

dcramer · 2025-05-30T18:47:28Z

Quick update here - I chatted more with Mark on this. I'm not totally sure what the right direction is right now. I think the outcome is clear to me: things like /organizations should just list all orgs you're a member of - no matter the region.

I have temporarily worked around this in the MCP, but all versions of fixing this in Sentry are a little complex. I think it MUST get fixed, but I also have a hard time saying it MUST get fixed tomorrow.

IMO the best course of action while this is in conversation: what do we want this to function like, write that down, then we work towards it when time permits.

github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label May 28, 2025

dcramer commented May 28, 2025

View reviewed changes

vercel bot deployed to Preview May 28, 2025 21:20 View deployment

dcramer commented May 28, 2025

View reviewed changes

dcramer force-pushed the feat/list-organizations-control-endpoint branch from 91f08bc to 9ebb327 Compare May 28, 2025 21:43

vercel bot deployed to Preview May 28, 2025 21:44 View deployment

dcramer force-pushed the feat/list-organizations-control-endpoint branch from 9ebb327 to 46392e1 Compare May 28, 2025 21:46

vercel bot deployed to Preview May 28, 2025 21:48 View deployment

feat: Migrate organizations list to control

b79c998

dcramer force-pushed the feat/list-organizations-control-endpoint branch from 46392e1 to b79c998 Compare May 28, 2025 21:56

vercel bot deployed to Preview May 28, 2025 21:58 View deployment

sentaur-athena requested review from a team and GabeVillalobos May 28, 2025 22:07

dcramer mentioned this pull request May 28, 2025

List Organizations returns empty result (CallToolSuccess) getsentry/sentry-mcp#227

Closed

markstory reviewed May 29, 2025

View reviewed changes

Uh oh!

feat: Migrate organizations list to control #92442

Are you sure you want to change the base?

feat: Migrate organizations list to control #92442

Uh oh!

Conversation

dcramer commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 8 Tests Failed:

Uh oh!

dcramer commented May 28, 2025

Uh oh!

dcramer commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GabeVillalobos commented May 28, 2025

Uh oh!

dcramer commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GabeVillalobos commented May 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dcramer commented May 30, 2025

Uh oh!

Uh oh!

dcramer commented May 28, 2025 •

edited

Loading

codecov bot commented May 28, 2025 •

edited

Loading

dcramer commented May 28, 2025 •

edited

Loading

dcramer commented May 28, 2025 •

edited

Loading