[wg/media] Media Working Group Charter

[w3cbot]

This issue was created because the 'horizontal review requested' label was added to
§ w3c/strategy#504

This review is requested prior to the Advisory Committee Review.

New charter proposal, reviewers please take note.

Charter Review

• Charter
• Diff from charter template
• Diff from previous charter
• Chair dashboard

This is an existing WG recharter

Communities suggested for outreach

None in particular. Through group participation, agenda topics, and joint meetings, the Media WG has regular exchanges with the WebRTC Working Group, the Timed Text Working Group and the Media & Entertainment Interest Group (which, in turn, exchanges on media topics with a number of external organizations).

Known or potential areas of concern

Where would charter proponents like to see issues raised? On the w3c/charter-media-wg issue tracker

Anything else we should think about as we review?

The draft charter mentions two additional issues in the description of the work planned on the Encrypted Media Extensions deliverable. These are more "maintenance" than "new features" but it seemed worth expliciting given that the scope of work on EME remains restricted.

The WebCodecs section also describes planned work to give applications the ability to use an HTMLMediaElement and buffering mechanisms defined in Media Source Extensions (MSE) to achieve adaptive playback of media that originates from WebCodecs structures, including creating a way to represent encrypted audio/video chunks within WebCodecs so that playback can leverage EME.

By itself, the possibility to represent encrypted audio/video chunks in WebCodecs does not extend EME in any way (and EME will not need to be updated at all for that). In a typical scenario, an encrypted encoded media chunk (EncodedAudioChunk or EncodedVideoChunk) will be passed to MSE and connected to a <video> element. Playback of the encrypted content will then handled by EME, as for any other encrypted content that reaches a <video> element.

By definition, encrypted media chunks cannot be decoded as-is into a raw VideoFrame, although the spec does not preclude creating an "opaque" VideoFrame. If implementations supported this, such a VideoFrame could perhaps in turn perhaps be used in other scenarios. Here is an analysis of these other scenarios to help reviewers grasp the ins and outs of the proposal:

1. A VideoFrame can be injected into a <video> element through a VideoTrackGenerator. With encrypted chunks, this would achieve the same thing as connecting it to a <video> through MSE+EME (apps would have to handle the buffering themselves, but that's orthogonal to encryption).
2. A VideoFrame can be directly injected into a <canvas> element. Injecting an encrypted VideoFrame into a <canvas> will not work out of the box since that would de facto reveal the bytes. The concept of a "tainted canvas", used for cross-origin content, could perhaps be extended for encrypted VideoFrames. For that to work, EME would also need to be integrated to <canvas>. If all that happens, this would make it easier to apply content protection to still images, but this is actually already doable in practice, see discussion in Support for content protection w3c/webcodecs#41 (comment)
3. A VideoFrame can be imported as external texture into WebGL and WebGPU pipelines. As with <canvas>, injecting an encrypted VideoFrame will simply not work today because these pipelines do not have provisions for encrypted content, and would require significant work on the APIs and implementations. Should WebGL and WebGPU decide to add support for content protected textures, they would likely want to leverage encrypted structures that can more directly be used as textures than VideoFrame in any case.

Cc @chrisn, @marcoscaceres.

Charter facilitator(s)

cc @tidoust

[jyasskin]

Thank you for sending this to us. We don't have any concerns about the charter update, but given the ideas about extending EME, we wanted to give a sense of how the TAG will think about any such extensions.

We see EME as a bargain struck between the web's users and content owners, with web browsers negotiating on their users' behalf. The content of this bargain is still controversial, even 8 years later. We're not taking a position on whether that bargain was justified at the time, but we do think that the web's users deserve their representatives to keep negotiating hard for better bargains in the future. We shouldn't just give away new ways to use EME without ensuring that end users are getting something valuable in return. We urge you to get broad agreement with browser vendors before putting this sort of work on a charter.

When considering new capabilities, there should be some clear body of content that would stay off the web without that capability, and that content should be valuable enough to offset the downsides of EME. If the content itself would likely just be offered in a way that's less convenient for its owners, those owners could instead offer something else of value to the web's users.

Thanks again for continuing to improve media on the web.