aiobatch

https://github.com/hoveychen/aiobatch

aiobatch is a lightweight dynamic batching library for PyTorch and ONNX model inferencing.

Tokens:3,430
Snippets:29
Trust Score:7.5
License:Apache-2.0
Update:2 months ago
Tokens:
Raw