No pagination links or total meta in included resource collection

[bjornharrtell]

DESCRIPTION

It appears it's not possible to get pagination links or total meta in included resource collection. Perhaps it's more of missing feature than bug...

STEPS TO REPRODUCE

Running the example JsonApiDotNetCoreExample and making request http://localhost:14140/api/people?fields[people]=assignedTodoItems&include=assignedTodoItems&page[size]=2,assignedTodoItems:2 returns payload:

{
  "links": {
    "self": "/api/people?fields[people]=assignedTodoItems&include=assignedTodoItems&page[size]=2,assignedTodoItems:2",
    "describedby": "/swagger/v1/swagger.json",
    "first": "/api/people?fields%5Bpeople%5D=assignedTodoItems&include=assignedTodoItems&page%5Bsize%5D=2,assignedTodoItems%3A2",
    "last": "/api/people?fields%5Bpeople%5D=assignedTodoItems&include=assignedTodoItems&page%5Bsize%5D=2,assignedTodoItems%3A2&page%5Bnumber%5D=25",
    "next": "/api/people?fields%5Bpeople%5D=assignedTodoItems&include=assignedTodoItems&page%5Bsize%5D=2,assignedTodoItems%3A2&page%5Bnumber%5D=2"
  },
  "data": [
    {
      "type": "people",
      "id": "1",
      "relationships": {
        "assignedTodoItems": {
          "links": {
            "self": "/api/people/1/relationships/assignedTodoItems",
            "related": "/api/people/1/assignedTodoItems"
          },
          "data": [
            {
              "type": "todoItems",
              "id": "376"
            },
            {
              "type": "todoItems",
              "id": "76"
            }
          ]
        }
      },
      "links": {
        "self": "/api/people/1"
      }
    },
    {
      "type": "people",
      "id": "2",
      "relationships": {
        "assignedTodoItems": {
          "links": {
            "self": "/api/people/2/relationships/assignedTodoItems",
            "related": "/api/people/2/assignedTodoItems"
          },
          "data": []
        }
      },
      "links": {
        "self": "/api/people/2"
      }
    }
  ],
  "included": [
    {
      "type": "todoItems",
      "id": "376",
      "attributes": {
        "description": "TodoItem376",
        "priority": "High",
        "durationInHours": 375,
        "createdAt": "2025-06-05T15:08:00.899032+00:00",
        "modifiedAt": null
      },
      "relationships": {
        "owner": {
          "links": {
            "self": "/api/todoItems/376/relationships/owner",
            "related": "/api/todoItems/376/owner"
          }
        },
        "assignee": {
          "links": {
            "self": "/api/todoItems/376/relationships/assignee",
            "related": "/api/todoItems/376/assignee"
          }
        },
        "tags": {
          "links": {
            "self": "/api/todoItems/376/relationships/tags",
            "related": "/api/todoItems/376/tags"
          }
        }
      },
      "links": {
        "self": "/api/todoItems/376"
      }
    },
    {
      "type": "todoItems",
      "id": "76",
      "attributes": {
        "description": "TodoItem076",
        "priority": "High",
        "durationInHours": 75,
        "createdAt": "2025-06-05T15:08:00.898895+00:00",
        "modifiedAt": null
      },
      "relationships": {
        "owner": {
          "links": {
            "self": "/api/todoItems/76/relationships/owner",
            "related": "/api/todoItems/76/owner"
          }
        },
        "assignee": {
          "links": {
            "self": "/api/todoItems/76/relationships/assignee",
            "related": "/api/todoItems/76/assignee"
          }
        },
        "tags": {
          "links": {
            "self": "/api/todoItems/76/relationships/tags",
            "related": "/api/todoItems/76/tags"
          }
        }
      },
      "links": {
        "self": "/api/todoItems/76"
      }
    }
  ],
  "meta": {
    "total": 50
  }
}

EXPECTED BEHAVIOR

I'd expect pagination links and total meta for any included collection data.

ACTUAL BEHAVIOR

Pagination links and total meta appears not to be available for included collection data.

VERSIONS USED

• JsonApiDotNetCore version: Run from source at 3c6d3be

[bkoelman]

As indicated at #1731 (comment), there are additional costs to fetching this information. I suppose it could be implemented, though I never expected users to actually paginate through related resources, because it returns a lot of duplicate information. It would be more optimal to fetch the related resources directly and paginate through them.

For example, to fetch the next nested page by sending GET http://localhost:14140/api/people?fields[people]=assignedTodoItems&include=assignedTodoItems&page[size]=2,assignedTodoItems:2&page[number]=1,assignedTodoItems:2, it would return all top-level people again.

Can you explain the scenario where this would be useful?

[bjornharrtell]

The scenario is when data is typically few related entities and if the UX is to do a table listing of resources that also includes typically few related details up front without additional interaction from the user. This UX isn't well suited if related details do not fit in a page... but I can at least imagine that if there are some (rare?) instances where the related details are too many to fit in a page it would be presented in another way and require user interaction (and subsequent requests) to show the full related details. But since paging links or metadata is not supplied it's not known if there are more items... unless pessimistic presentation where only up to pagesize -1 related details are presented... which is kind of an ugly hack in my opinion. I hope that rambling makes some sense. :)

I suppose I would say that having a page of data without any info that there is additional data is at least less useful than if the info was there, and one could perhaps argue that the value of paging related included resources is perhaps in question... or at least difficult to fully grasp how to use properly when it doesn't behave like primary resource paging. I know (and agree with) that the intent is a DDoS protection though.

I'm willing to accept that the current behavior is the least worst / best possible behavior at this time, but it should perhaps be documented as such and motivated why.

[bkoelman]

Well, that's trading a potential extra query (namely, only when the page is full) for always doing an extra query to determine the related count. This multiplies per top-level resource. So, with a page size of 10 and one included relationship, it means 10 extra database queries (even if only two top-level resources have a full page). It gets worse when multiple relationships are included. For two included to-many relationships, the number of queries multiplies by 2. For nested includes, it explodes.

Even if we could pack them into a single SQL query, the database costs would be similarly high for fetching data that's unlikely going to be needed by the API consumer. Doesn't sound like a good tradeoff to me.

If you know upfront that the number of related resources is small, consider increasing the page size. Either globally, or per resource type via a resource definition. There's some ongoing discussion at #1731 on whether we should offer additional ways to control page size.

But I don't think there's anything wrong with expecting the API consumer to send an extra request when a full page is encountered. It enables them to decide whether to do so (or not). When done so, it returns the total count.

For example, the client sends:

GET /api/people?include=assignedTodoItems HTTP/1.1

Assuming the response for the third person has a full page, the client now sends:

GET /api/todoItems?filter=equals(assignee.id,'3')&page[number]=2 HTTP/1.1

which returns the second page of the todo-items assigned to the third person. That response includes the total.

The client can optimize further. Let's assume that for three people the page was full in the first response. Then it can send:

GET /api/todoItems?filter=any(assignee.id,'3','5','8')&include=assignee&fields[people]=id&page[number]=2 HTTP/1.1

which returns the second page of assigned todo-items for all three people. Because the person ID is included, the client can correlate back which items belong to what person.