Skip to content

Issues: triton-inference-server/client

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

45 Open 14 Closed
45 Open 14 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

DLPack tensor is not contiguous. Only contiguous DLPack tensors that are stored in C-Order are supported.
#834 opened Apr 20, 2025 by 631068264
1
Cannot obtain server metadata (C++ API)
#832 opened Apr 17, 2025 by mpmischitelli
grpcio version
#831 opened Mar 31, 2025 by sewerynkalisz
[QUESTION] Difference in Streaming Behavior When Using Torch Tensor Conversion vs. NumPy in Iterative sequential batcher python client
#830 opened Mar 29, 2025 by chinmay-choudhary
Get model config without loading
#828 opened Mar 25, 2025 by scotgopal
Windows C++ Build: vcpkg's rapidjson not properly linked
#825 opened Feb 27, 2025 by Haladesta
How do I pass control input to python-stateful backend?
#824 opened Feb 21, 2025 by lionsheep0724
[Feature Request] Official Rust Client Library for Triton Inference Server
#823 opened Feb 21, 2025 by franklucky001
Sticky Load Balancing during inference not hitting all the Triton Servers sitting behind the Load Balancer
#820 opened Feb 13, 2025 by animikhaich
[Feature Request] Support using system versions of gRPC, Protobuf, Abseil in CMake
#819 opened Jan 19, 2025 by prm-james-hill
[Question] Help with Client-Side Batching for Large Requests in Triton
#818 opened Jan 17, 2025 by harsh-boloai
Read back data written in cuda shared memory in python
#816 opened Dec 16, 2024 by MaximeDebarbat
Performance Discrepancy Between Triton Client SDK and perf_analyzer
#815 opened Dec 10, 2024 by wensimin
2
InferenceServerHttpClient::Create() failed with error: std::bad_alloc
#806 opened Nov 13, 2024 by megadl
Incorrect grpcio version Python client
#804 opened Nov 8, 2024 by tigersoldier
[GRPC_CLIENT] LoadModel with serialized model raise an error: "Poll failed for model directory" on native windows triton.
#803 opened Nov 6, 2024 by FilipLeja
The dependency information of the Python package needs to be updated
#796 opened Oct 16, 2024 by penguin-wwy
[Question] numpy 2 support
#792 opened Oct 10, 2024 by m3at
7
Potential memory leak
#791 opened Oct 8, 2024 by prasannagn16
2
Tritonclient CUDA shared memory set_shared_memory_region_from_dlpack fails with non-set stream for Torch tensor.
#789 opened Oct 5, 2024 by piotrm-nvidia
genai-perf exception offline-mode
#785 opened Sep 24, 2024 by sochiseung
原始是byte[]的数据，没法传。会报triton.client.InferenceException: [request id: <id_unknown>] inference input data-type is 'INT8', model expects 'BYTES' for 'xxxxx'
#784 opened Sep 24, 2024 by freeleeq
3
Re-use client or create new one each time?
#783 opened Sep 19, 2024 by datdo-msft
KV-cache of Tensorrt_llm
#782 opened Sep 16, 2024 by huynguyen-bku
Use try-with-resources for InferenceServerClient to handle resource management and potential TimeoutException
#780 opened Sep 7, 2024 by 1ComputerMaster
ProTip! Add no:assignee to see everything that’s not assigned.