Edge inference / latency
Which processor on one laptop chip should run live instance segmentation inside a 33.3 ms frame budget: the NPU, the Arc iGPU or the CPU?
YOLOv8n-seg through OpenVINO on an Intel Core Ultra 7 155H, across three processors and FP32, FP16 and two INT8 builds, with 1,000 timed frames per configuration. For INT8 on the iGPU and the NPU, a flat-out benchmark and a live webcam loop reversed the p99 order, although the NPU missed the 33.3 ms budget less often in both; the CPU was not run live. The iGPU's flat-out tail came from frames where its CPU-side stages stalled, not from inference.
- Python
- OpenVINO
- NNCF INT8
- NumPy
- pandas