[Usage]: Internal Server Error

[Schweizliu]

Your current environment

1、

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "qwen2.5-vl-72b/",
    "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://modelscope.oss-cn-beijing.aliyuncs.com/resource/qwen.png"}},
        {"type": "text", "text": "What is the text in the illustrate?"}
    ]}
    ]
    }'

发送请求时，经常出现Internal Server Error情况
vllm日志输出File "/usr/local/python3.10.17/lib/python3.10/site-packages/aiohttp/helpers.py", line 685, in exit
raise asyncio.TimeoutError from exc_val
asyncio.exceptions.TimeoutError
是否有其他方式进行本地图片请求

2、多次进行单图片请求，vllm日志显示的吞吐差别较大，分别有14.3token和8token，请问原因是？

How would you like to use vllm on ascend

我想有多并发或多请求的方式

[Schweizliu]

看错了，qwen2.5-vl-72b的生成速度是2token，是否达到内部性能标准呢

[wangxiyuan]

thanks for the issue, we'll take a look now.