Extend Canvas TextMetrics for editing and text styling

[AndresRPerez12]

This change adds the following functionalities to TextMetrics objects:

• Get the set of rectangles that the user agent would render as the selection background when a particular character range of the measured text is selected.
• Get the rectangle corresponding to the actual bounding box for a character range of the measured text. This is the equivalent to TextMetrics.actualBoundingBox restricted to the given range.
• Get the string index for the character at a given offset distance from the start position of the measured text.
• Split a character range of the measured text into grapheme clusters. That is, for the given range, it returns the minimal logical units of text, each of which can be rendered, along with their corresponding positional data.

Additionally, a new method is added to the Canvas 2D rendering context to render these clusters. When rendering, all relevant CanvasTextDrawingStyles are used as they were when the text was measured.

See the discussion in the issue: #10677

• At least two implementers are interested (and none opposed):
  • Chromium: The new methods have been implemented (https://issues.chromium.org/u/1/issues/341213359)
  • Gecko: Position has been requested ([HTML Canvas] TextMetrics Additions mozilla/standards-positions#1144)
  • WebKit: Position has been requested, and some portions have already started landing ([HTML Canvas] TextMetrics Additions for Editing and Graphic Design WebKit/standards-positions#436)
• Tests are written and can be reviewed and commented upon at:
  • https://github.com/web-platform-tests/wpt/tree/master/html/canvas/element/text
  • https://github.com/web-platform-tests/wpt/tree/master/html/canvas/offscreen/text
  • Relevant tests: 2d.text.measure.selection-rects*, 2d.text.measure.getActualBoundingBox*, 2d.text.measure.index-from-offset*, and 2d.text.measure.text-clusters*.
• Implementation bugs are filed:
  • Chromium: https://crbug.com/341213359
  • Gecko: …
  • WebKit: https://bugs.webkit.org/show_bug.cgi?id=284464
• Corresponding HTML AAM & ARIA in HTML issues & PRs:
• MDN issue is filed: …
• The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

---

/acknowledgements.html ( diff )
/canvas.html ( diff )
/infrastructure.html ( diff )

[AndresRPerez12]

Tagging: @whatwg/canvas

[schenney-chromium]

Sorry for leaving some comments out of the review.

[schenney-chromium]

Is this the index one past the end? Range APIs seem to behave that way but I'm not sure how conventional it is.

[AndresRPerez12]

Yes, it is one past the end. I hadn't realized I didn't clarify it in the spec. I'll add it explicitly as I don't know how conventional it is either.

[schenney-chromium]

Is it a requirement that the drawing command uses the same parameters as the measurement? If so I would say that directly here, because right now it seems suggestive but not prescribed.

[AndresRPerez12]

Yes, that is specified in the section for fillTextCluster(). The idea when I first wrote it was to just mention the behavior expected, even if the implementation would most likely be to store the parameters in the cluster (as is implemented right now in Chrome).

But, following a suggestion by @Kaiido in another comment, I changed the phrasing of that requirement by using TextCluster as an opaque object that stores those parameters in cluster when they are created. Hopefully this is more clear. Let me know what you think!

[Kaiido]

Thanks for doing this!

A couple of questions that came to mind:

• Should there be a strokeTextCluster() method akin to strokeText()?
• Should the TextCluster be serializable? (so that we can send it to/from a Worker).

Also, I didn't receive the @whatwg/canvas notification from your previous comment. Hopefully this one will work better.

[Kaiido]

Disclaimer: I'm not an editor, so take this with a grain of salt.

I feel this would be a lot clearer if the TextMetrics interface had an internal value to store the original text for each instance and reuse it where needed, instead of this tedious "the text that was passed to measureText() to obtain the cluster".

See for instance how CanvasPattern objects have a transformation matrix that isn't exposed:

Patterns have a transformation matrix, which controls how the pattern is used when it is painted. Initially, a pattern's transformation matrix must be the identity matrix.

It might even be needed to have a clear relationship established between the TextCluster and the TextMetrics from where it's been created, so that you can do something like the cluster's textmetrics's text, or, given that when the TextCluster is created text is available, and TextMetrics may not need it otherwise, you could maybe store it directly along the TextCluster (still in an internal value).

[AndresRPerez12]

I hadn't noticed that objects like CanvasPattern were defined in this way as opaque. I like that concept and I feel it applies well to this use case. Thank you so much for the suggestion!

I have updated it to reflect this. For now I went with the second approach you mentioned (storing the text and the CanvasTextDrawingStyles directly in the cluster as internal values). Hopefully this provides more clarity. Let me know what you think!

[Kaiido]

This object should probably be created explicitly sooner, in the getTextClusters() method. However I'm not sure how this should be done since we want the computed values at that time.

[AndresRPerez12]

With the use of the internal values of the opaque object, hopefully the relationship is clearer now. Passing the exact object from the cluster won't always be possible due to the possibility of passing align and baseline to to fillTextCluster() when rendering, but most of the values now come from the internally stored value.

[AndresRPerez12]

Thank you both for your comments! I have addressed them and uploaded the changes. Please let me know what you think.

Answering some of the open questions:

• Should there be a strokeTextCluster() method akin to strokeText()?

I have added it to the PR. I agree that it makes sense to have both.

• Should the TextCluster be serializable? (so that we can send it to/from a Worker).

We feel that for now, it's simpler to not make it serializable. Once we merge this and we know exactly what clusters look like and what values they are storing internally that would require serialization, we can look into it.

Also, I didn't receive the @whatwg/canvas notification from your previous comment. Hopefully this one will work better.

Thanks! I'm not fully sure how the tags work, so I really appreciate it!