v0.1.19

What's Changed

• Support minItems, maxItems for array by @Ubospica in #296
• [fix]fix the handling for '?'. by @Seven-Streams in #305
• [Ci] Add a check in benchmark.yaml. by @Seven-Streams in #301
• [CI Fix] Fix the check for benchmark. by @Seven-Streams in #309
• [Feature] Rewrite the FSM.h to support some regex grammar. by @Seven-Streams in #302
• [Fix] matching error when encountering multi-byte UTF-8 encoded characters by @Irfnfnkemed in #312
• Bump to v0.1.19 by @Ubospica in #313

Full Changelog: v0.1.18...v0.1.19

v0.1.18

Highlights

• Provides a MLX kernel to support mac devices
• Fix bugs in json schema converter
• Fix the compilation failure issue of torch kernels
• Supports float range, format in json schema
• Adds a function testing._is_single_token_bitmask for spec decoding
• Releases GIL for most of the methods

What's Changed

• Bump to v0.1.17 by @Ubospica in #271
• [Feature] LRU cache for grammar compiler by @DarkSharpness in #243
• Setup tmate by @Ubospica in #281
• [Fix] Fix test_json_schema_converter::test_non_strict and nanobind version issue by @Ubospica in #282
• [Fix] Switch release compilation flag from -Ofast to -O3 by @tboerstad in #268
• [Minor] Fix missing mark hf_token_required by @DarkSharpness in #284
• Add module xgrammar.contrib.mlxlm by @wangkuiyi in #273
• Float range support for xgrammar by @zanderjiang in #289
• Disable slow C++ test cases by @Ubospica in #290
• [Feature] Cover JSON schema string format by @Ubospica in #266
• HuggingFace LogitsProcessor to accept also list of compiled grammars by @lukaszkolodziejczyk in #275
• update ci for hf token by @zanderjiang in #267
• Release GIL for key methods by @Ubospica in #291
• Add MLX kernel to support Apple Silicon by @wangkuiyi in #287
• [Bugfix] Fix path replacement issue in apply_token_bitmask_inplace_cuda.py by @nFunctor in #274
• Refactor kernels by @Ubospica in #293
• Add util function testing._is_single_token_bitmask by @Ubospica in #294
• Bump to v0.1.18 by @Ubospica in #295

New Contributors

• @tboerstad made their first contribution in #268
• @lukaszkolodziejczyk made their first contribution in #275
• @nFunctor made their first contribution in #274

Full Changelog: v0.1.17...v0.1.18

v0.1.17

What's Changed

• Update README.md by @Ubospica in #246
• [Feature] Add CI to close inactive issues and check formats. by @Seven-Streams in #241
• Enable editable installation by @Ubospica in #247
• Cleanup useless build scripts by @Ubospica in #248
• fix(python): use correct name for pybind11 FindPython mode by @henryiii in #249
• Update logics for stale issues by @Ubospica in #251
• [Fix] Add support for kDLCUDAHost in token bitmask device type check by @Irfnfnkemed in #242
• Convert from pybind11 to nanobind by @Ahajha in #230
• [Test]Modify the tests to support python3.8. by @Seven-Streams in #252
• [Feature] minLength, maxLength, array in type by @Ubospica in #254
• [Test] Add more test environments. by @Seven-Streams in #255
• ci benchmark with json-mode-eval and bfcl dataset by @zanderjiang in #236
• Remove a duplicated line in apply_token_bitmask_inplace_triton.py by @wangkuiyi in #263
• [Fix] Fix primitives for json schema converter by @Ubospica in #264
• test for new integer range fill_next_token_bitmask time by @zanderjiang in #209
• Fix CI by @Ubospica in #262

New Contributors

• @Seven-Streams made their first contribution in #241
• @henryiii made their first contribution in #249
• @Irfnfnkemed made their first contribution in #242
• @Ahajha made their first contribution in #230
• @wangkuiyi made their first contribution in #263

Full Changelog: v0.1.16...v0.1.17

v0.1.16

What's Changed

• [Fix] Postpone cuda import to the calling site by @Ubospica in #231
• [Feature] Support model vocab size being less than tokenizer by @Ubospica in #237
• [Style] Remove unused headers by @DarkSharpness in #219
• Fallback to triton if we fail to compile for CUDA by @zbowling in #223
• [Feature] Build and run C++ Python tests by @DarkSharpness in #218
• [Fix] Fix missing dependency in ci by @DarkSharpness in #239
• Add dependency for ninja by @Ubospica in #244
• Bump to 0.1.16 by @Ubospica in #245

Full Changelog: v0.1.15...v0.1.16

v0.1.15

What's Changed

• Update README.md by @Ubospica in #225
• [Feature] Expose Grammar.union by @Ubospica in #227
• [Feature] Support Deepseek R series models by @Ubospica in #226
• Bump to v0.1.15 by @Ubospica in #228

Full Changelog: v0.1.14...v0.1.15

v0.1.14

Highlights

This version supports XGrammar on Linux Arm64, removes the restriction of glibc 2.28, and adds a source distribution.

It fixes the self-recursion error, extends apply_token_mask_inplace to support some corner cases.

It changes the API of StructuralTag, renaming field "start" to "begin".

What's Changed

• update GenerateRangeRegex by @zanderjiang in #182
• [Fix] Fix compatibility for apply_token_bitmask_inplace by @Ubospica in #206
• [Feature] Support self-recursion in json schema reference by @Ubospica in #210
• Modernize build to support sdists by @zbowling in #190
• Update README to announce that Modular MAX uses XGrammar by @zbowling in #217
• [Feature] Support padding for logits and unequal batch size for logits and bitmask by @Ubospica in #220
• [Refactor][Breaking] Rename start->begin in StructuralTag by @Ubospica in #221
• Bump to v0.1.14 by @Ubospica in #222

New Contributors

• @zbowling made their first contribution in #190

Full Changelog: v0.1.13...v0.1.14

v0.1.13

Highlight

This version enhances the compatibility of XGrammar on various platforms. It also provides full support for regex. Now most features are supported. It also enhances the efficiency of the token bitmask application kernel.

What's Changed

• [Fix] Fix popcount for windows by @Ubospica in #167
• [Fix] Rollback safely when token acceptance fails by @benchislett in #164
• fix(fsm): fix error of dangling-reference in gcc-13 by @DarkSharpness in #171
• fix(test): fix the concurrency issue in #168 by @DarkSharpness in #170
• [Debug] Support cpptrace by @Ubospica in #176
• [Feature] Support Regex for GrammarCompiler by @Ubospica in #185
• [Docs] Add news section by @Ubospica in #191
• [Fix] Remove redundant grammar_functor.cc by @Ubospica in #195
• [Maintain] Add toml formatter by @Ubospica in #196
• [Maintain] Add no-sort to toml fmt by @Ubospica in #198
• [Maintain] Update toml formatter by @Ubospica in #199
• [Feature] Support Optimized CUDA Token Bitmask Kernel by @syuoni in #186
• [Maintain] Support Python 3.13 for Manylinux by @Ubospica in #200

New Contributors

• @syuoni made their first contribution in #186

Full Changelog: v0.1.11...v0.1.12

v0.1.11

Highlight

In this PR we supported the structural tag. This is a new feature that can support strict function calling (and many more flexible patterns).

What's Changed

• [Feature] Structural tag by @Ubospica in #162
• [Fix] Fix #162 by @Ubospica in #163
• [Fix] Fix broken rollback char count after accepting stop token by @benchislett in #161
• [Feature] Optional Token Bitmask by @Ubospica in #165

Full Changelog: v0.1.10...v0.1.11

v0.1.10

Highlight

In this version we enhanced the ability of json schema, ebnf, and provided APIs for grammar concat and union.

What's Changed

• [Feature] Support regex and repetition range by @Ubospica in #144
• [Refactor] Rename internal classes for better structure by @Ubospica in #145
• [FunctionCalling] Support TagDispatch by @Ubospica in #146
• [Fix] Fix doc dependencies by @Ubospica in #147
• [Fix] Fix doc dependencies by @Ubospica in #148
• [Feature] Grammar concat and union by @Ubospica in #149
• [Fix] Fix compilation error with virtual overloaded function by @Ubospica in #150
• [Refactor] Refactor GrammarMatcherBase by @Ubospica in #151
• [3rdparty] Avoid macro redefinition in picojson by @MasterJH5574 in #153

Full Changelog: v0.1.9...v0.1.10

Release v0.1.8

Features

• Enhance JSON Schema converter by @Ubospica in #134
• Enhance EBNF Parser by @observerw in #125
• Support sentencepiece tokenizer by @zanderjiang in #120
• Enhance ApplyMask kernels to provide better support in mix-structured-and-unstructured cases by @Ubospica in #128