Updated review of Web Neural Network API

[anssiko]

Hello TAG!

I'm requesting an updated TAG review of WebNN API - previous TAG review: #933

• Explainer¹: https://github.com/webmachinelearning/webnn/blob/main/explainer.md
• Specification: https://www.w3.org/TR/webnn/
• WPT Tests: https://github.com/web-platform-tests/wpt/tree/master/webnn
• User research: N/A
• Security and Privacy self-review²: https://github.com/webmachinelearning/webnn/blob/main/security-privacy.md
• GitHub repo: https://github.com/webmachinelearning/webnn/
• Primary contacts:
  • Anssi Kostiainen, @anssiko, Intel (chair)
  • Dominique Hazael-Massieux, @dontcallmedom, W3C (staff contact)
  • Ningxin Hu, @huningxin, Intel (editor)
  • Dwayne Robinson, @fdwr, Microsoft (editor)
• Organization/project driving the specification: W3C Web Machine Learning Working Group
• Multi-stakeholder support³: Participation by all major browser vendors. Implementation experience across major OSes, platform APIs and key JavaScript ML frameworks.
  • Chromium comments: High-level feedback addressed. Informed by this feedback, the group has initiated work on core operator set.
  • Mozilla comments: N/A, Mozilla recently joined the WG
  • WebKit comments: N/A, Apple actively participates and contributes in the WG
• Status/issue trackers for implementations⁴: Chrome Platform Status, Edge WebNN Developer Preview

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• Previous early design review, if any: 2000, 2022, 2024
• Relevant time constraints or deadlines: We plan to publish a new CRS during Q2 2025
• The group where the work on this specification is currently being done: W3C Web Machine Learning Working Group
• Major unresolved issues with or opposition to this specification: none
• This work is being funded by:

You should also know that...

Below is a summary of substantive changes since the previous TAG review in 2024 to assist you in this delta review.

Significant new features:

• Add 12 new operators ("Wave 3") to support well-known transformer models based on an analysis of top 20 downloaded Hugging Face models. The models comprise of a diverse set of tasks, including Small Language Models and Large Language Models, object detection, sentence similarity, feature extraction, speech recognition, image segmentation, text classification, and text generation. See issue, PR, presentation for details on these operators and motivation.
• Add MLTensor interface that represents a tensor, an input or output to an MLGraph. Update MLContext interface with new createTensor(), readTensor(), writeTensor() methods, change compute() to dispatch() for scheduling the workload, add new destroy() method for MLContext, MLGraph, MLTensor. These changes allow buffer reuse, chained inference, better resource management with explicit destruction, and the opportunity to share memory with WebGPU, minimizing copies. See MLTensor Explainer for details.

Other substantial changes:

• Remove MLDeviceType abstraction based on implementation feedback, see Device Selection Explainer for details
• Expose the API to shared workers and service workers to allow shared logic in a worker and use in web extensions
• Add MLContext.lost attribute to signal context lost error at the earliest opportunity for improved debuggability
• Add opSupportLimits() to allow checking whether operators/types are supported before creating a graph
• Enable Float16Array by default, note Uint16Array workaround informatively
• Convert MLOperand methods into readonly attributes (dataType, shape) per Web Platform Design Principles
• Add optional MLOperatorOptions.label for more diagnosable error messages
• Add MLNumber for specifying numeric inputs of any type
• Add resource contention and denial of service considerations to security considerations

The group would like to seek TAG's explicit advise on the following:

• Operator naming 2D vs 2d Operator naming 2D vs 2d webmachinelearning/webnn#821