[Bugfix] Fix pydantic.errors.PydanticUserError

[Potabk]

FIX pydantic.errors.PydanticUserError, when we are using python<3.12, pydantic needs the TypedDict from typing_extensions

FIX #17966

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[Yikun]

Looks reasonable, pydantic will do real check on typing imported typeddict. Then, it raised error in python 3.10.

https://github.com/vllm-project/vllm-ascend/actions/runs/14954620271/job/42009151862?pr=806

Consider, vllm support python > 3.9 currently, we need this.

[Yikun]

@frieda-huang @DarkLight1337 Would you mind taking a look on this?

[Potabk]

I have tested the latest code in Python's interactive terminal with vllm latest code on A100:

Python 3.10.16 (main, Dec 11 2024, 16:24:50) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from vllm.entrypoints.openai.serving_engine import OpenAIServing

the result is:

INFO 05-11 23:19:22 [__init__.py:248] Automatically detected platform cuda.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/autodl-tmp/wl/vllm/vllm/entrypoints/openai/serving_engine.py", line 102, in <module>
    class RequestProcessingMixin(BaseModel):
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py", line 237, in __new__
    complete_model_class(
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py", line 597, in complete_model_class
    schema = gen_schema.generate_schema(cls)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 706, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 999, in _generate_schema_inner
    return self._model_schema(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 832, in _model_schema
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 832, in <dictcomp>
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1201, in _generate_md_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1367, in _common_field_schema
    schema = self._apply_annotations(
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 2279, in _apply_annotations
    schema = get_inner_schema(source_type)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 2261, in inner_handler
    schema = self._generate_schema_inner(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1004, in _generate_schema_inner
    return self.match_type(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1118, in match_type
    return self._match_generic_type(obj, origin)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1141, in _match_generic_type
    return self._union_schema(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1429, in _union_schema
    choices.append(self.generate_schema(arg))
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 706, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1004, in _generate_schema_inner
    return self.match_type(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1118, in match_type
    return self._match_generic_type(obj, origin)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1165, in _match_generic_type
    return self._sequence_schema(self._get_first_arg_or_any(obj))
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1813, in _sequence_schema
    item_type_schema = self.generate_schema(items_type)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 706, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1004, in _generate_schema_inner
    return self.match_type(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1118, in match_type
    return self._match_generic_type(obj, origin)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1141, in _match_generic_type
    return self._union_schema(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1429, in _union_schema
    choices.append(self.generate_schema(arg))
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 706, in generate_schema
    schema = self._generate_schema_inner(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1004, in _generate_schema_inner
    return self.match_type(obj)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1086, in match_type
    return self._typed_dict_schema(obj, None)
  File "/root/autodl-tmp/miniconda/envs/vllm_test/lib/python3.10/site-packages/pydantic/_internal/_generate_schema.py", line 1506, in _typed_dict_schema
    raise PydanticUserError(
pydantic.errors.PydanticUserError: Please use `typing_extensions.TypedDict` instead of `typing.TypedDict` on Python < 3.12.

and I test with this patch:

Python 3.10.16 (main, Dec 11 2024, 16:24:50) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from vllm.entrypoints.openai.serving_engine import OpenAIServing
INFO 05-11 23:25:19 [__init__.py:248] Automatically detected platform cuda.
>>> 

[frieda-huang]

@frieda-huang @DarkLight1337 Would you mind taking a look on this?

Ok. I'm on it!

[frieda-huang]

@frieda-huang @DarkLight1337 Would you mind taking a look on this?

Looks reasonable to me as well! Will take a note on that for future contribution.

[DarkLight1337]

Thanks for fixing!

[Potabk]

Some error occurred in entrypoints/openai/test_vision.py::test_single_chat_session_image, maybe the network causes