[Benchmarks] Add qwen2.5-7b test (#763)

### What this PR does / why we need it?
- Add qwen2.5-7b test
- Optimize the documentation to be more developer-friendly 

Signed-off-by: xuedinge233 <damow890@gmail.com>
Co-authored-by: xuedinge233 <damow890@gmail.com>

Author: Potabk

benchmarks/README.md

@@ -1,41 +1,44 @@
 # Introduction
-This document outlines the benchmarking process for vllm-ascend, designed to evaluate its performance under various workloads. The primary goal is to help developers assess whether their pull requests improve or degrade vllm-ascend's performance.To maintain consistency with the vllm community, we have reused the vllm community [benchmark](https://github.com/vllm-project/vllm/tree/main/benchmarks) script.
+This document outlines the benchmarking methodology for vllm-ascend, aimed at evaluating the performance under a variety of workloads. The primary goal is to help developers assess whether their pull requests improve or degrade vllm-ascend's performance. To maintain alignment with vLLM, we use the [benchmark](https://github.com/vllm-project/vllm/tree/main/benchmarks) script provided by the vllm project.
+
 # Overview
 **Benchmarking Coverage**: We measure latency, throughput, and fixed-QPS serving on the Atlas800I A2 (see [quick_start](../docs/source/quick_start.md) to learn more supported devices list), with different models(coming soon).
 - Latency tests
     - Input length: 32 tokens.
     - Output length: 128 tokens.
     - Batch size: fixed (8).
-    - Models: llama-3.1 8B.
+    - Models: Meta-Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct.
     - Evaluation metrics: end-to-end latency (mean, median, p99).
 
 - Throughput tests
     - Input length: randomly sample 200 prompts from ShareGPT dataset (with fixed random seed).
     - Output length: the corresponding output length of these 200 prompts.
     - Batch size: dynamically determined by vllm to achieve maximum throughput.
-    - Models: llama-3.1 8B .
+    - Models: Meta-Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct.
     - Evaluation metrics: throughput.
 - Serving tests
     - Input length: randomly sample 200 prompts from ShareGPT dataset (with fixed random seed).
     - Output length: the corresponding output length of these 200 prompts.
     - Batch size: dynamically determined by vllm and the arrival pattern of the requests.
     - **Average QPS (query per second)**: 1, 4, 16 and inf. QPS = inf means all requests come at once. For other QPS values, the arrival time of each query is determined using a random Poisson process (with fixed random seed).
-    - Models: llama-3.1 8B.
+    - Models: Meta-Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct.
     - Evaluation metrics: throughput, TTFT (time to the first token, with mean, median and p99), ITL (inter-token latency, with mean, median and p99).
 
-**Benchmarking Duration**: about 800senond for single model.
+**Benchmarking Duration**: about 800 senond for single model.
 
 
 # Quick Use
 ## Prerequisites
 Before running the benchmarks, ensure the following:
+
 - vllm and vllm-ascend are installed and properly set up in an NPU environment, as these scripts are specifically designed for NPU devices.
+
 - Install necessary dependencies for benchmarks:
     ```
     pip install -r benchmarks/requirements-bench.txt
     ```
     
-- Models and datasets are cached locally to accelerate execution. Modify the paths in the JSON files located in benchmarks/tests accordingly. feel free to add your own models and parameters in the JSON to run your customized benchmarks.
+- For performance benchmark, it is recommended to set the [load-format](https://github.com/vllm-project/vllm-ascend/blob/5897dc5bbe321ca90c26225d0d70bff24061d04b/benchmarks/tests/latency-tests.json#L7) as `dummy`, It will construct random weights based on the passed model without downloading the weights from internet, which can greatly reduce the benchmark time. feel free to add your own models and parameters in the JSON to run your customized benchmarks.
 
 ## Run benchmarks
 The provided scripts automatically execute performance tests for serving, throughput, and latency. To start the benchmarking process, run command in the vllm-ascend root directory:

benchmarks/tests/latency-tests.json

@@ -8,5 +8,15 @@
       "num_iters_warmup": 5,
       "num_iters": 15
     }
+  },
+  {
+    "test_name": "latency_qwen2_5_7B_tp1",
+    "parameters": {
+      "model": "Qwen/Qwen2.5-7B-Instruct",
+      "tensor_parallel_size": 1,
+      "load_format": "dummy",
+      "num_iters_warmup": 5,
+      "num_iters": 15
+    }
   }
 ]

benchmarks/tests/serving-tests.json

@@ -22,5 +22,29 @@
       "dataset_path": "./ShareGPT_V3_unfiltered_cleaned_split.json",
       "num_prompts": 200
     }
+  },
+  {
+    "test_name": "serving_qwen2_5_7B_tp1",
+    "qps_list": [
+      1,
+      4,
+      16,
+      "inf"
+    ],
+    "server_parameters": {
+      "model": "Qwen/Qwen2.5-7B-Instruct",
+      "tensor_parallel_size": 1,
+      "swap_space": 16,
+      "disable_log_stats": "",
+      "disable_log_requests": "",
+      "load_format": "dummy"
+    },
+    "client_parameters": {
+      "model": "Qwen/Qwen2.5-7B-Instruct",
+      "backend": "vllm",
+      "dataset_name": "sharegpt",
+      "dataset_path": "./ShareGPT_V3_unfiltered_cleaned_split.json",
+      "num_prompts": 200
+    }
   }
 ]

benchmarks/tests/throughput-tests.json

@@ -9,6 +9,17 @@
       "num_prompts": 200,
       "backend": "vllm"
     }
+  },
+  {
+    "test_name": "throughput_qwen2_5_7B_tp1",
+    "parameters": {
+      "model": "Qwen/Qwen2.5-7B-Instruct",
+      "tensor_parallel_size": 1,
+      "load_format": "dummy",
+      "dataset_path": "./ShareGPT_V3_unfiltered_cleaned_split.json",
+      "num_prompts": 200,
+      "backend": "vllm"
+    }
   }
 ]