Hugging Face's logo

qualcomm
/
WeDetect

Object Detection
PyTorch
real_time
android
Model card Files
xet
 Community

WeDetect: Optimized for Qualcomm Devices

WeDetect is a text-conditioned object detection model that uses a prompt-then-detect strategy. Text class names are reparameterized into the model weights before export, producing a fixed-vocabulary detector that runs efficiently on-device. The model uses LTRB (Left-Top-Right-Bottom) regression with multi-scale feature maps at strides 8, 16, and 32.

This is based on the implementation of WeDetect found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

Due to licensing restrictions, we cannot distribute pre-exported model assets for this model. Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

See our repository for WeDetect on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.object_detection

Model Stats:

  • Input resolution: 640x640
  • Model checkpoint: wedetect_tiny
  • Model size (detector) (float): 143 MB
  • Model size (detector) (mixed_with_float): 35.9 MB
  • Model size (text_encoder) (float): 592 B
  • Model size (text_encoder) (mixed_with_float): 13.3 KB
  • Number of parameters (detector): 37.3M
  • Number of parameters (text_encoder): 3.00

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
detector ONNX float Snapdragon® X2 Elite 10.367 ms 5 - 5 MB NPU
detector ONNX float Snapdragon® X Elite 21.464 ms 71 - 71 MB NPU
detector ONNX float Snapdragon® 8 Gen 3 Mobile 15.79 ms 0 - 330 MB NPU
detector ONNX float Snapdragon® 8 Gen 1 Mobile 82.002 ms 2 - 328 MB NPU
detector ONNX float Qualcomm® Dragonwing™ IQ-8275 35.024 ms 5 - 13 MB NPU
detector ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 20.741 ms 0 - 77 MB NPU
detector ONNX float Qualcomm® QCS8450 82.002 ms 2 - 328 MB NPU
detector ONNX float Qualcomm® Dragonwing™ IQ-9075 31.729 ms 5 - 12 MB NPU
detector ONNX float Qualcomm® Dragonwing™ IQ-X7181 21.464 ms 71 - 71 MB NPU
detector ONNX float Qualcomm® Dragonwing™ Q-8750 11.277 ms 1 - 257 MB NPU
detector ONNX float Snapdragon® 8 Elite Mobile 11.277 ms 1 - 257 MB NPU
detector ONNX float Snapdragon® 8 Elite Gen 5 Mobile 8.205 ms 3 - 262 MB NPU
detector ONNX mixed_with_float Snapdragon® X2 Elite 10.675 ms 9 - 9 MB NPU
detector ONNX mixed_with_float Snapdragon® X Elite 16.604 ms 35 - 35 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Gen 3 Mobile 11.325 ms 2 - 362 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Gen 1 Mobile 25.246 ms 1 - 363 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-8275 15.636 ms 6 - 14 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 16.036 ms 0 - 44 MB NPU
detector ONNX mixed_with_float Qualcomm® QCS8450 25.246 ms 1 - 363 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-9075 15.612 ms 8 - 15 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-X7181 16.604 ms 35 - 35 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ Q-8750 11.259 ms 7 - 315 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Elite Mobile 11.259 ms 7 - 315 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Elite Gen 5 Mobile 9.54 ms 5 - 316 MB NPU
detector QNN_DLC float Snapdragon® 8 Gen 3 Mobile 15.312 ms 0 - 360 MB NPU
detector QNN_DLC float Snapdragon® 8 Gen 1 Mobile 81.626 ms 5 - 375 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 34.712 ms 5 - 12 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 20.06 ms 5 - 8 MB NPU
detector QNN_DLC float Qualcomm® SA8775P 30.339 ms 3 - 297 MB NPU
detector QNN_DLC float Qualcomm® SA8650P 30.339 ms 3 - 297 MB NPU
detector QNN_DLC float Qualcomm® SA8255P 30.339 ms 3 - 297 MB NPU
detector QNN_DLC float Qualcomm® QCS8450 81.626 ms 5 - 375 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 30.823 ms 5 - 11 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ Q-8750 10.974 ms 5 - 275 MB NPU
detector QNN_DLC float Qualcomm® SA7255P 116.277 ms 1 - 292 MB NPU
detector QNN_DLC float Qualcomm® SA8295P 86.746 ms 0 - 284 MB NPU
detector QNN_DLC float Snapdragon® 8 Elite Mobile 10.974 ms 5 - 275 MB NPU
detector QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 8.0 ms 5 - 286 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® 8 Gen 3 Mobile 13.038 ms 0 - 358 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-8275 18.202 ms 5 - 12 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 18.075 ms 5 - 38 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA8775P 18.211 ms 1 - 307 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA8650P 18.211 ms 1 - 307 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA8255P 18.211 ms 1 - 307 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-9075 18.709 ms 7 - 13 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ Q-8750 10.951 ms 0 - 304 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA7255P 43.579 ms 1 - 306 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® 8 Elite Mobile 10.951 ms 0 - 304 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® 8 Elite Gen 5 Mobile 8.061 ms 5 - 325 MB NPU
detector TFLITE float Snapdragon® 8 Gen 3 Mobile 16.101 ms 0 - 345 MB NPU
detector TFLITE float Snapdragon® 8 Gen 1 Mobile 95.008 ms 0 - 335 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ IQ-8275 35.407 ms 0 - 83 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 21.346 ms 0 - 7 MB NPU
detector TFLITE float Qualcomm® SA8775P 31.601 ms 0 - 267 MB NPU
detector TFLITE float Qualcomm® SA8650P 31.601 ms 0 - 267 MB NPU
detector TFLITE float Qualcomm® SA8255P 31.601 ms 0 - 267 MB NPU
detector TFLITE float Qualcomm® QCS8450 95.008 ms 0 - 335 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ IQ-9075 31.973 ms 0 - 83 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ Q-8750 11.789 ms 0 - 267 MB NPU
detector TFLITE float Qualcomm® SA7255P 117.88 ms 1 - 266 MB NPU
detector TFLITE float Qualcomm® SA8295P 83.175 ms 0 - 267 MB NPU
detector TFLITE float Snapdragon® 8 Elite Mobile 11.789 ms 0 - 267 MB NPU
detector TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 9.865 ms 0 - 274 MB NPU
text_encoder ONNX float Snapdragon® X2 Elite 42.429 ms 1 - 1 MB NPU
text_encoder ONNX float Snapdragon® X Elite 97.256 ms 550 - 550 MB NPU
text_encoder ONNX float Snapdragon® 8 Gen 3 Mobile 65.073 ms 1 - 2267 MB NPU
text_encoder ONNX float Snapdragon® 8 Gen 1 Mobile 147.311 ms 1 - 1860 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ IQ-8275 102.652 ms 1 - 4 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 93.386 ms 0 - 644 MB NPU
text_encoder ONNX float Qualcomm® QCS8450 147.311 ms 1 - 1860 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ IQ-9075 99.381 ms 1 - 3 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ IQ-X7181 97.256 ms 550 - 550 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ Q-8750 46.947 ms 1 - 2019 MB NPU
text_encoder ONNX float Snapdragon® 8 Elite Mobile 46.947 ms 1 - 2019 MB NPU
text_encoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 39.926 ms 1 - 2263 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® X2 Elite 42.58 ms 1 - 1 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® X Elite 97.108 ms 551 - 551 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Gen 3 Mobile 65.184 ms 1 - 2267 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Gen 1 Mobile 146.093 ms 2 - 1862 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-8275 102.648 ms 1 - 4 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 97.242 ms 0 - 643 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® QCS8450 146.093 ms 2 - 1862 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-9075 99.413 ms 1 - 3 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-X7181 97.108 ms 551 - 551 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ Q-8750 47.98 ms 1 - 2019 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Elite Mobile 47.98 ms 1 - 2019 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Elite Gen 5 Mobile 40.545 ms 1 - 2263 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 58.729 ms 0 - 2446 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 139.283 ms 0 - 3011 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 95.311 ms 0 - 3 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 82.142 ms 0 - 2 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8775P 95.659 ms 1 - 2019 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8650P 95.659 ms 1 - 2019 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8255P 95.659 ms 1 - 2019 MB NPU
text_encoder QNN_DLC float Qualcomm® QCS8450 139.283 ms 0 - 3011 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 93.468 ms 0 - 2 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 45.558 ms 0 - 1973 MB NPU
text_encoder QNN_DLC float Qualcomm® SA7255P 260.988 ms 1 - 2035 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8295P 107.374 ms 1 - 1903 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Elite Mobile 45.558 ms 0 - 1973 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 34.533 ms 0 - 1983 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® 8 Gen 3 Mobile 62.166 ms 0 - 2447 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-8275 95.04 ms 0 - 3 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 85.167 ms 0 - 520 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA8775P 95.579 ms 1 - 2019 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA8650P 95.579 ms 1 - 2019 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA8255P 95.579 ms 1 - 2019 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-9075 93.403 ms 0 - 2 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ Q-8750 46.559 ms 0 - 1974 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA7255P 261.126 ms 1 - 2036 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® 8 Elite Mobile 46.559 ms 0 - 1974 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® 8 Elite Gen 5 Mobile 34.501 ms 0 - 1982 MB NPU
text_encoder TFLITE float Snapdragon® 8 Gen 3 Mobile 73.275 ms 1 - 3679 MB NPU
text_encoder TFLITE float Snapdragon® 8 Gen 1 Mobile 164.282 ms 1 - 3564 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ IQ-8275 109.282 ms 0 - 553 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 99.84 ms 1 - 4 MB NPU
text_encoder TFLITE float Qualcomm® SA8775P 111.23 ms 1 - 2145 MB NPU
text_encoder TFLITE float Qualcomm® SA8650P 111.23 ms 1 - 2145 MB NPU
text_encoder TFLITE float Qualcomm® SA8255P 111.23 ms 1 - 2145 MB NPU
text_encoder TFLITE float Qualcomm® QCS8450 164.282 ms 1 - 3564 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ IQ-9075 109.035 ms 0 - 552 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ Q-8750 55.152 ms 0 - 2074 MB NPU
text_encoder TFLITE float Qualcomm® SA7255P 289.63 ms 1 - 2145 MB NPU
text_encoder TFLITE float Qualcomm® SA8295P 112.884 ms 1 - 2099 MB NPU
text_encoder TFLITE float Snapdragon® 8 Elite Mobile 55.152 ms 0 - 2074 MB NPU
text_encoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 41.404 ms 0 - 2220 MB NPU

License

  • The license for the original implementation of WeDetect can be found here.

References

Community

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Object Detection
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for qualcomm/WeDetect