v2.19.0

Bug Fixes and Other Changes

• Add docker build scripts and enable aarch64 pip wheels.
• Regenerate stubs with Mypy 1.13.0
• Handle the punctuation definition mismatch between different Unicode versions.
• Replace outdated select() on --cpu in core/kernels/sentencepiece/BUILD with platform API equivalent.
• cleanup of deprecated test methods
• Update tensorflow version.
• Limit dm-tree to 0.1.8.
• Remove srcs_version and python_version attributes, as they already default to "PY3"
• Remove invalid public_names_test
• Fix protobuf dependency.
• Update test files for new ICU version
• Init python_wheel_version_suffix_repository for TF dependency in tf-text project after tensorflow/tensorflow@805775f.
• Add numpy py dep and fix python dependencies.
• Add explicit tf-keras dependency.
• Update Version for 2.19.0-rc0
• Revert "Init python_wheel_version_suffix_repository for TF dependency in tf-text project after tensorflow/tensorflow@805775f."
• Refresh requirement lockfiles
• add @absl_py//absl:app dep to tensorflow_build_info
• Update Version for 2.19.0

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

C. Antonio Sánchez

v2.19.0-rc0

Bug Fixes and Other Changes

• Add docker build scripts and enable aarch64 pip wheels.
• Regenerate stubs with Mypy 1.13.0
• Handle the punctuation definition mismatch between different Unicode versions.
• Replace outdated select() on --cpu in core/kernels/sentencepiece/BUILD with platform API equivalent.
• cleanup of deprecated test methods
• Update tensorflow version.
• Limit dm-tree to 0.1.8.
• Remove srcs_version and python_version attributes, as they already default to "PY3"
• Remove invalid public_names_test
• Fix protobuf dependency.
• Update test files for new ICU version
• Init python_wheel_version_suffix_repository for TF dependency in tf-text project after tensorflow/tensorflow@805775f.
• Add numpy py dep and fix python dependencies.
• Add explicit tf-keras dependency.
• Update Version for 2.19.0-rc0
• Revert "Init python_wheel_version_suffix_repository for TF dependency in tf-text project after tensorflow/tensorflow@805775f."
• Refresh requirement lockfiles
• add @absl_py//absl:app dep to tensorflow_build_info

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

C. Antonio Sánchez

v2.18.1

Major Features and Improvements

• Add Python 3.12 and aarch64 support

Bug Fixes and Other Changes

• Add docker build scripts and enable aarch64 pip wheels.
• Replace std::string_view with absl::string_view
• Protect the fast wordpiece tokenizer from infinite looping.
• Update lock files
• Update version for 2.18.1
• Set HERMETIC_PYTHON_VERSION correctly

v2.18.0

Release 2.18.0

Bug Fixes and Other Changes

• Fix out-of-bounds read in whitespace tokenizer
• Add unit test for fixed bounds check in IsWhitespace
• Add Hermetic CUDA rules.
• Remove tf/lite/kernels/shim:tf_headers from tf/core:framework
• Update version
• Update configure.sh

v2.18.0-rc0

Release 2.18.0-rc0

Bug Fixes and Other Changes

• Fix out-of-bounds read in whitespace tokenizer
• Add unit test for fixed bounds check in IsWhitespace
• Add Hermetic CUDA rules.
• Remove tf/lite/kernels/shim:tf_headers from tf/core:framework
• Update version
• Update configure.sh

v2.17.0

Release 2.17.0

Bug Fixes and Other Changes

• negative sampling excludes positive class
• revert html encoding
• much faster set-intersection based version
• Fix notebook failure with Keras 3.
• Remove tensorflow-macos from setup.py
• Update tensorflow-macos to 2.16.1
• Update version

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Alex Shroyer, C. Antonio Sánchez, Maggie Zhang

v2.17.0-rc0

Release 2.17.0-rc0

Bug Fixes and Other Changes

• negative sampling excludes positive class
• revert html encoding
• much faster set-intersection based version
• Fix notebook failure with Keras 3.
• Remove tensorflow-macos from setup.py
• Update tensorflow-macos to 2.16.1
• Update version

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Alex Shroyer, C. Antonio Sánchez, Maggie Zhang

v2.16.1

Release 2.16.1

Major Features and Improvements

Breaking Changes

Bug Fixes and Other Changes

• Update tf-text setup scripts.
• Support resource manager scoped Sentencepiece resources.
• Remove use_unique_shared_resource_name.
• Remove tensorflow_text dependency on tf_hub library.
• Fix TF patch, bump TF commit
• Update version

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Raviteja Gorijala

v2.16.0-rc0

Release 2.16.0-rc0

Major Features and Improvements

Breaking Changes

Bug Fixes and Other Changes

• Update tf-text setup scripts.
• Support resource manager scoped Sentencepiece resources.
• Remove use_unique_shared_resource_name.
• Remove tensorflow_text dependency on tf_hub library.
• Fix TF patch, bump TF commit
• Update version

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Raviteja Gorijala

v2.15.0

Release 2.15.0

Bug Fixes and Other Changes

• Update TF versions and scripts to allow consistently building against tf-nightly.
• No public description
• Update phrase tokenizer to handle end-punctuation.
• Remove private Keras imports.
• Update tensorflow_hub dependency.
• Sprawling .pyi updates related to pybind11 PRs #4831, #4833.
• Report unsupported tensor type in RaggedTensorToTensor in Prepare.
• Check in generated pyi files for some py_extension targets.
• Update version
• Update WORKSPACE