Skip to content

Add docs for index component selector api convention #128786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jbaiera
Copy link
Member

@jbaiera jbaiera commented Jun 2, 2025

Adds documentation for index component selectors to the reference docs.

This documentation references and will be referenced by new docs located at elastic/docs-content#1368

@jbaiera jbaiera added >docs General docs changes :Data Management/Data streams Data streams and their lifecycles v8.19.0 v9.1.0 labels Jun 2, 2025
@elasticsearchmachine elasticsearchmachine added Team:Data Management Meta label for data/management team Team:Docs Meta label for docs team labels Jun 2, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-docs (Team:Docs)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general shape and structure LGTM. I left some suggestions, but this is a hard topic to talk about with simplicity, so I wouldn't take any of them as dogmatic.

There are currently two selector suffixes supported by {{es}} APIs:

`::data`
: (Indices, data streams) This selector references the default component present on all indices and data streams which holds the regular data contained therein.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this sentence could be clearer, I think we can simplify it (e.g.: ::data refers to backing indices containing regular data, not failures.)

: (Indices, data streams) This selector references the default component present on all indices and data streams which holds the regular data contained therein.

`::failures`
: (Data streams only) This component references data stored in a data stream's [failure store](docs-content://manage-data/data-store/data-streams/failure-store.md). This component is only present on data streams.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can say it references "data", because that doesn't work for non-data APIs. For example GET /myds::failures/_settings returns the settings for the failure store indices, and not the data stored in those failure store indices.


As an example, [search]({{es-apis}}group/endpoint-search), [field capabilities]({{es-apis}}operation/operation-field-caps), and [index stats]({{es-apis}}operation/operation-indices-stats) APIs can all report results from a different component rather than from the default data.

Selectors are applied to an index pattern at the end and can be combined with other index pattern syntax like [date math](#api-date-math-index-names) and wildcards.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Selectors are applied … at the end …

applied at the end of what? Is it that they're the last thing resolved, or at the end of the name pattern itself?

Selectors are applied to an index pattern at the end and can be combined with other index pattern syntax like [date math](#api-date-math-index-names) and wildcards.

```console
# Search for regular data on a data stream
Copy link
Member

@dakrone dakrone Jun 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "regular" may mislead here (and the word "failures" is hard also, because a user could be indexing failures successfully). Perhaps something like

Suggested change
# Search for regular data on a data stream
# Search the data (non-failure) backing indices of a data stream

Maybe that's too specific though? (I think the one two lines below this suffers from the same problem)

Copy link
Contributor

@kilfoyle kilfoyle Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Search for regular data on a data stream
# Search for non-failure data in a data stream

Would this work? I'd probably avoid mentioning the backing indices, just to keep things simple.


There are currently two selector suffixes supported by {{es}} APIs:

`::data`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`::data`
`data`

I think the :: should be removed from here and the failures entry below. Without them it renders nicely:

screen

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One concern I have is that we need the :: because it is part of the selector itself. For example: my-data-stream::data. If we remove the :: we risk a user not understanding they are delimited by the double-colon.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry! I was reading those as a docs formatting thing, rather than the parameter.
This is why I shouldn't review PRs first thing in the AM. :-)


Some APIs that accept a `<data-stream>`, `<index>`, or `<target>` request path parameter also support *selector syntax*.

Some indices and data streams are made of multiple components besides their default data. These other components store additional data which can be queried separately like a regular index. Some APIs may operate on subsets of these index components which can be specified by using a selector suffix.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Some indices and data streams are made of multiple components besides their default data. These other components store additional data which can be queried separately like a regular index. Some APIs may operate on subsets of these index components which can be specified by using a selector suffix.
Some indices and data streams are made of multiple components besides their default data. These other components store additional data which can be queried separately like regular index data. Some APIs may operate on subsets of these index components which can be specified by using a selector suffix.

Maybe this is a bit more clear.

```console
# Search for regular data on a data stream
GET my-data-stream/_search
# Search for failure data on a data stream
Copy link
Contributor

@kilfoyle kilfoyle Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Search for failure data on a data stream
# Search for failure data in a data stream

I'd use "in" when talking about the contents of the index / data stream (as opposed to performing operations on an index).

Copy link
Contributor

@kilfoyle kilfoyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀
Added just a few small suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Data streams Data streams and their lifecycles >docs General docs changes Team:Data Management Meta label for data/management team Team:Docs Meta label for docs team v8.19.0 v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants