Summary
Xgrammar includes a cache for compiled grammars to increase performance with repeated use of the same grammar. This cache is held in memory. Since the cache is unbounded, a system making use of xgrammar can be abused to fill up a host's memory and case a denial of service. For example, sending many small requests to an LLM inference server with unique JSON schemas would eventually cause this denial of service to occur.
Details
The fix is to add a limit to the cache size. This was done in mlc-ai/xgrammar#243
An example of making use of the new cache size limit can be found in vLLM here: vllm-project/vllm#16283
Impact
Any system making use of Xgrammar and taking requests as input from potentially untrusted parties would be vulnerable to this denial of service issue.
References
russellb Reporter
Ubospica Remediation reviewer
DarkSharpness Remediation developer
Summary
Xgrammar includes a cache for compiled grammars to increase performance with repeated use of the same grammar. This cache is held in memory. Since the cache is unbounded, a system making use of xgrammar can be abused to fill up a host's memory and case a denial of service. For example, sending many small requests to an LLM inference server with unique JSON schemas would eventually cause this denial of service to occur.
Details
The fix is to add a limit to the cache size. This was done in mlc-ai/xgrammar#243
An example of making use of the new cache size limit can be found in vLLM here: vllm-project/vllm#16283
Impact
Any system making use of Xgrammar and taking requests as input from potentially untrusted parties would be vulnerable to this denial of service issue.
References