Video Analytics Hardware 101
Video analytics can be supported on different hardware types and programming frameworks, impacting cost, efficiency, and accuracy.
The primary factors reviewed in this report are:
- CPU including Intel
- GPU including NVIDIA, CUDA and Tensor Cores
- VPU including Myriad / Movidius
- SoCs including Ambarella and Huawei
- Tradeoffs between using CPU, GPU, VPU and SoCs
This is part of our new Video Analytics Course starting in March.
4 ********** *******
*** ** *** ******* *********** ** increasing ***** ********* ******** ** ********* processing *********. ***** ********* *** **** commonly ********* ** ***, ***, ***, or ***.
Executive *******
*** ********* ********* *** ********* ** the * *******:
- **** ** ******* ** ********* *** less ********** **** *** ***** ********* because ** ********** *********** *** ***** has ******** ******** ** **** ** its ******** **** ** ******** ***** analytic **********.
- ****, **** ******** ******, *** ******** used ** ******* ********** ***** ********* for ****** ****** ******* ** ******* or ******* *********.
- ****, **** ******* ***** ******, *** been ******** *** ***** ********* ** edge ******* **** ** ** ******* but **** ****** ******* ******** ** date ****** ****.
- ****, **** ******* ********* *** ****** Hisilicon, **** ****** *** **** ****** choice *** ***** ********* ** ** cameras, ***** *** ******** ***** *** simplicity ** ******* ***** ******** *** analytics. *******, ******'* ********* ********** *** been **** ****, ** ***** *** now, *** ** ** *********.
Processing ** ***
**** *** *** **** ****** **** computing ********, *********, *** **** ********* offer ******** ***** ******** *********** **** GPUs.
***** **** *** ** **** ***-***-***** to *** ***** *********, **** *** only ******* *** ***** ***** *** limited ******* ** *******. ****** **** depends ** *** **** ** *********, the ********** ** *** **********, *** the *** ****, ** ** ********* not ****-********* ** *** ***** ************ analytics ** ****.
*******,*****'* ************ ** ******** *** *********** ** video ********* ** ******** ***** **** (e.g. **** **/** *** **** **********). Since ***** ************ ********* ****** ******* Intel ****, **** ******* **** *** time ******** ** ********** ******* *** other *******'* ****. *******, ***** **** analytic ********* *** ***** ******** **** CPUs, ** ** *** ****** *******.
Processing ** ***
**** *** ******** ****** *** ********* in ******* ******* **** *** ******** to ******* ******** ***** ********** **** hundreds ** ********* ** ***** *** GPU ****. **** ******* * ******** for ******** ********* ** ****** ****** networks *** ***** ***** *** ******* analytics.
************, ***** **** *** ********* ****** for ****** ****** ***** ******* ******** cards *** ** ***** ** * single ****** ** ******* ******** ** cameras, ********* ** *** ********** ** the ********.
*******, **** **** **** *** ********** video ************ **** ********* ** ******* per ****, ****** **** ** ***** analytics *** ***** **** **** ** only ******-***** **** **** ****.
NVIDIA **** *** ****** *****
****** ** *** **** ******** **** GPU ******** *** ***** *********. ***** NVIDIA ****** ******** *** ********, **** video ******** ********* ******* ******* *** and ****** ****. ******** ***** ** these ******** ********* ***** ********* ** CUDA ***** *** *** ********** *** cards ***** ******** ** ****** *****, at ***** ******* **** $*** - $2,000 *** ****.
****** ****** *** ***** **** ************* higher **** *** ****** **** ****** (***** *,***** ****) **** ******* ** ******, *** many ************ ************ ***** *** *** the ********** ********** ***** *** *** high **** ($*,*** - $**,*** *** card) ** * *********** ********.
***** ******** ****, ******'******* ************** ********** *** **** ** ********* processing. ***** **** *** *** ******** used ** ***** ************ ******** *********, their *** **** ($*** - $***) for ******** ** **** ***** ** highlighted ** ** ******** *** ********** per-camera ***** *** ***-****** *****, ******** systems.
****** **** *** ****** **** ****** IP ******* ******* ** *********** ** relative ****, ****, *** ***** ***********. NVIDIA *** ****** *** ** ******* for **** ****** ** ** *******. By ********, **** *** **** **** targeted **** *******.
Processing ** ***
***** **** *** *****-**** **** **** and *** ** **** *** ********** video *********, **** ** *** ******* video ********, ******* **** ** * disadvantage ** ****. *******, **** **** designed *** ** **** (*****/******) ********, offering ******** ********** *** ****** *********** than ****.
***** ****** (******** **** ********) ** the **** ****** **** ******** *** video ************, ******* ** ******-***** ***** and **** ** ******* ********** *** cameras. ******* ***** **** ******* *** same ********* ** ***** ****, ******** providers *** *** ******* *** *** surveillance ******* **** ******* *********** ******** to ***** ****.
*******, **** *** *** ******* *********** traction *** ********* **********. ****** ************* have **** ********* **** ****, ****** VPUs, ******* **** ***** ******** ***, more ********, ***** ********* **********. ** such, ** ** ********* **** *********, both ** ***** ** ******** *********** costs *** ****** *** ******* *** for ******** ***** ******** **********.
Processing ** ****** *** (****** ** * ****)
*** ** *******, **** *** ******** chosen **** **** ** **** ***** they ******* ***** ******** **** * single ****** **** ********** *** ******* the **** ** **** ******** *** analyzing *** *****.
*** ********** ***** ** ******** **********, and **** ****** ** ****** **** are ***** **** *********** ***** ********** engines ***** **** ****** ******** ********* possible, *** *********** ****** ****** (*** also**** ****** ********* ********).
Ambarella
**-************************ * ****** **** *** ************ for ** ******* ******** ***** *********. Ambarella *** * **** ******* ******** for ***** ************; ** *** *. The ** ****** ** *** "******** vision" ** ***** *********-******* ********. *** S ****** ** ******** *** ******** and ********** ** *******, ******** ** low-cost ********* ***********, *** ******* *********.
*******, ***** ** ****** *** ** Ambarella *** ********* **** ********'* ** ban,*** ******* *** ********** ***** *** ****** ****** **** could *********** ****** ***** ************ *********, especially ***** *** **** *** ******** to ***** ********.
Manufacturer ***** - ****** *** ****
** ******** ** *** ******* **** SoCs **** ********* *** ******, *** video ************ ************* ******* ***** *** chips - **** (******) *** ****** (*******). ***** ************* *** *********** ***** chips ** ******* ***** *********. *******, even *** ***** *************, **** *** a ******* ** ***** *** ********* chips *** **** *** ******* (*.*., Ambarella).
Huawei *********
************ *** **** ****** *** **** in ** *******,********* ** *** ********* ************* ****** * ****** ** ** ***** sanctions ******* *** ****** ******.
*********'* ****** **** **** **** ******** used, ******** * ***** ** ***-****, limited ********* ****** (******* *********), ******** **, *** ******** ****** networks ********** ******** (****** *********). ********* *********** ****** ************* ******* models, *** ********* **** *** ******** with *** **** ******** **** *****-**** chips.
Challenge ** ***** ****-**** ** ****
**** ********* ****** ** ***** **** to *** ***** ********* ** ** cameras *** * ****** *** *************. While ***-** **** **** **** **** $10, **-******* **** ***** ******** *** price ** * ** **, ********* on *** ** ************ *** ****** purchasing ** *** ************.
** ******** ** $** *** *** cost ** * ****** ***** *** seem *********** ***, **** ****** ** by *** ************, *** ***********, *** the ******, *** ****** ** * total ***** ******** ** ~$*** *** camera.
Processing ** ******** ***
************ ********* **** *** **** ** decrease **** *** ******** *********** ******** to ****/****/****. *******, ******* ********* *** analyzing ***** **** *-** ******* **************, the ********** ***** ** ******, ***** typically ******* ******** ** ************ ********* to ******** ********** **** *** *** processing *********** ** *** ********.
************, ******* ********* *** ********* **** in *****-********* *******, ************* ** *** want ** ****-******* *** ******** ** keep *** ********'* ***** ***, ***** generally ***** ********-***** ******** ** ***** than **** ***** *******.
*******
** ****** **** ********* **** **** to *** *** ******* **** ** the ******* ***** ************ ************* **** cameras *** ********* ********* ****** *** camera ****** ********* *************** *** ********* premium *******.
************, *** ************* *** ********** ********** power *** ****** ******* *******, ****** by ******** ****** ************ ******* ** video ************ (*.*. **********, *********, **********, medicine), ***** **** ******* ** ****** SoCs ** ****.
* ***** *** ********** ** "***********" needs *******. *'* ****** **** *** Avigilon, "***********" ***** *** ***'* *** help ** ** ***** *** ***** tech ******* **** ** ***** ********* and ******* ****. ** *****'* **** it ***'* ****, ** **** ***** that *** *** ** **** *** to **** ** ****. * ***'* think *** *** ************ **** "*******" a ***** ***** ***** ** ******** if *** ****'* *** ** **** them. ************* **** ********* "*******" *** hardware **** **** *** *** **** more.
** ** ********** * *** ******** machine *** *** ***** ********* ******** for ********** ******, *** **** ** go *** ** ***** *** ** make ** **** ********* ** *** haven't ********* ***** ******** ******* **** have ******** ****** ******** **** *** can **** *** **** *** *** the ******* **** ****. ********* *** are ******* *** ******* ******** ** make *** *** **** *** **** ACC *** *** ******** ** **** as ***** ********* *******.
* ***** *** *** **** ** common ******** *** ** ** * little ***** ** ** *******, *** there *** **** ****** **, **** are **** ****** ** **** ********** so ** **** ********* ** ******** their ********.
*** ***** ******* **** ***** ** is ***** ** ***'* ***** ****** to **** **** ******* ** * rack/server **** ****** ** **** **** have ******* ****** *** ********** *** solution **** **** ** ****** *** won't ** ******* *** ******** ******** issues, *** * ***** *** *** how **** ****** **** ******** *******.
* ***** ****** *** ***** **, it's ** *** ****.
* *** *** ****** *** ******* on *** ***** ******* ********.
***** * *** ** ******** ** edge ***** ********** **** **** *********** and ********** *****. ** *** ** requirements ****** ****** **** **** ***’* these **** ******* *** *** ** processing ***** ********* **** * **** investment ** ***** ************* ** *** them ** ***** ******* ********? ** it *** **** ****** ** ******* in *** *****?
***** ******!
* ** ***** **** **** ********** is **** ********** *** **** ******* you ******* *** * ***** **** manufacturers **** **** **** ***** ******* it ***** ***** **** ******* **** relevant, *** * ***** *** ****** will **** **** *** ******** ** edge ********** ***'* ******** *** **** term ******** ** **** *** ****** when ********* ** *** **********.
**** ***** ****, * ***'* ***** cloud ** *** ****** ****** *** because ***** *** ***** *** **** infrastructure *********** **** ***** **** ** stream ***** **** ****** * *** which **** ******* ****** **** *********** of ******** ******* *** ****** ****.
* ***** ***** *** **** ******** approach ** * "******" ***** ***** the ********* ********** ******* ** ****, but *** ****** ** ******** ******* replacing *** ** **** *******/*******. **** the ***** *** ** ******** *** it's ******* ********** **** ***** **** management ** **** *********/*******.
**** ** *** * *** ** from ** **********
*** *** ***** **** ****** ** your ************ *** ******** *****?
*** **** ********** ** *** *** forward...do *** **** *** ****** ****, space ********, ***** *********** *** ***** requirements ** ***** **** **** ** equipment? *** ***** ****** *** ************...** the ******.... * **** **** ****** that ****** **** **** ******* *** this****** *** ****
** ***** ** ****** *** ****** which *** ********* ** *** ****** are ***** *********, ** **** ***** time ** *** ****** *********
"** *** ** ************ ****** ****** over **** ***’* ***** **** ******* run *** ** ********** ***** ********* them * **** ********** ** ***** manufacturers ** *** **** ** ***** upgrade ********? ** ** *** **** easier ** ******* ** *** *****?"**** ** ******* *********... ***** * years *** ************ **** *** ******** patch **** ******* *** ** ***** end ** **** *** ********* ******* a ************* ** *** *******, *** need ** ******* *** *******.
** ** *** **** ****** ** process ** *** *****?
***, '******' ******** ********* (***** ** eliminates *** **-***** ********** ****) *** it's *** **** ********* *********. ** discuss **** **** -***** ********* ************ ********* ******* ** ****** *** ** From ***** ** ****
******* **** ****** *** **** *** that ****** ****** ****...
*** ***** **** * ****** ***** device, **** ** *** **** *** basic ****, *** ** **** * multi ********** ******** ** *** ***** (iPhone, ******* ***)
** **** ******* - ***** **** the ***** ****** **** **** ***** and ****, **** ***** ***** **** I/O ***, **** ******* ******* (**), next ** **** ** * **** network ******* ****** **** *** ******* to ******* ******** **** ** **** to **** *** ******** ***********.
* ** **** **** ******* ******* to ****** ****’* ****** ** **** space...
** *** ** ***** ******* ** actual ******** ** ***** **’* ***** deployment? ** *** **** **** ********* a ****** ** ********* ** ***-****?
** ***
** *** *** ******, ** **** are **-********** *** ****** ******* ** the ******, ** (****** * ***** about *** **** '***' ***** ***** there ** ******* ** *** ***** but ** **** ****, **'* **** rare). ******** ********* ***** *** **** processing ***** **** ******* *********. *** alternative ** ** ******* *** ********* centrally *** **** ***** * ******** update **** * ***** ******* *****.
******** ********* ***** *** **** ********** power **** ******* *********.
***, ****’* ****. ****, *** **** few ***** **** ******* ** ********** expansion ** *** *****/***********, **** ***** make ***** *****.
************, ******** ***** **** *** ***-**** series *** ********* *** * *** hundred *******. ******** * **** ** industry ********** ** ******* ******** (****-*** video) *** ** ****** ******** *** came **** *********. ******* **** ************ are *** *** **** **** ***** cost ********* *** ******** *****.
*** *********** ** ** ******* *** analytics ********* *** **** ***** * firmware ****** **** * ***** ******* model.
******. ***** ******* * ***** **** the ********** ****** **** ***** ******** learn ** *** ***, ******** ** the ***** ******, **** **** * human *****, ***** ***** ******** ******** of ***** ******* *******. ***** **’* apparently *** *** *** ****, ** may ** ***** **********.
***** ***** ** ******* ** *** world *** ****...
** ***...
**** ** "********"? **** **** * am ***** **. ** ** ***** far *** *************** *********, *** *********** for **** ** ***** ******* (************ large ******, **********/******* ****, ***.).
**** ** *** ********* **** ********* a ******** ******** *** ******** ****** to ** ****-*********** **** ****** ********.
***** ** * **** ****** ***** towards ***** **** **** ******** ****** in ******** *******, ***** *****-********* *** identified (******* *** **** *****) *** used ** ****** * **** ******** set ** **** ****** ****** ***** positives. ** * ***** **** ***** be ******* ** ** ****-********** ********, as *** ******** ******* ********* ******* an ****** ***** **** ** ******** objects ** ** *******. **** ***** object *******, ** ***** ******** ******** examples **** * *** ** **********, it *** ** *********, ** **** are *** ****** *********, ** ***** not **** ****.
******, *****, *** **** ***** *********** as ******.
******** *** ******** *** **** *****, but **** ***** **** ****. ***** is * **** **** ********* ********** as ***** ****** *** **** **** similarity ** ********** ******. **, ** choose ** ******* **** *** ******** domain, ** *** ***** * ************* neural ******* ** **** ***** ** a *****'* ***** *******, **** *** conduct ******** ******** ** ******** ***** alarms, *** **** ** ********** ******* that *** ** ***** ****** ********** for ******** ******** ****** ** *** have ***** ****** ****** ****. **** is ************ ***** *** **** *****, where *** ******* *** ****** **** like ******* *** * ***** ***** puff ** *** ******** *** *** recognized ** * **** ****.
****** *****, ****, *** **** ***** information *******!
******* ************* ****** **** ***** *** become **** ********. *'** **** ********* appearance ********* ** ******** ********* ***** alarms ****** ** *********** ********** *** stray ********. *** *** ******** ********* examples ****, **** ********, *** ** pieced ******** ** **** **** ** lines ** *****.
****** * ********** ****** ** ********, which *** ******** ********* ****** ******** frames. *** *** ***** **** ** analysis ** ******** * *** ** not **** ********* **** **** ********* video ** ******* *** ** *** analysis ******.
*******, ****** **** *****, *'* * know-nothing. * **** **** ****** ***** video ************, *** ***'* **** ****** the ******** ** ************ ********. * only ******* ****** ** **** ** 14-year-old **** *** ****** ********. **** profound **** ** ********* ** * serious **********. * **** * ******** (classification ** ******* ****** ************ ***** at ******* * ****** * ******) that *'* ********* ******* ** * time-delimited **** ** * **** ****. I **** ** ******** ***** *** list *** ******* ** ********, **** zoom ** **** **** ****** ****** video. ** ******** ******* ******* **** metadata ****** *** ***** ******, ******* to *** ******** *** ** ****** in *** ***** ******? ** ** built **** *********** ********** ****** ****? Has ***** ***** ********** **** ****? Are **** **** ********* ********* ** emerge? **** ** ** **** **** would ** *********** ** **** **** about.
****** *** * ***** *** **** that ** ********** ** ** ********!
*********! * ***'* **** ** **** that *****, *** * *** **** you **** ******* ********* ** ******** at ***** ********* ************ ***, *** ONVIF ******* *:
*** ***** *** ** *** *** with ** ** ** ******* **** interesting "******* *"
*** *****
******* * - ******* ********* - ONVIF
******* ** **** *** *** *********** to ********* **** ** *** ***** "smart" ****** ** **** **** **** more ***** *********.
*** ******** ** **** **** **** time ****** ** **** ***'* *** cameras *** ** *** **** *** rapid ********, ** *** *** **** "independent" ********* ********** 😁
** ****?
*** ** ***** **** *** ***** surveillance ********* *********? *** ***** ** not ******* ** ***** ** **** but ******* ** ** ***** ****. FPGAs ** *** **** ** ** a ****** ****** *** ***** ************ analytics.
****'* **** **** ********* ******** **** CPU/GPU, ************ ** **** ************ (******* power ***********), *** **** **** * steep ******** ***** ** *** *************** side **** *****. *********** *** **** analytics ********* ***** **** **** **** of ****-****** ********* **. * ******-** development, * ***'* ***** **'** *** many ****-***** ***** ********* ** *** security ******** *** **** **** (** maybe ****).
***** *** * *** ********* ********* FPGAs *** ******* ******, **** ** Xilinx (***********). *'* **** *** **** it's ***** ******** *** ******** ** the ***** ****. ***** ** ********* trying ** ***** *** ****** ******.
** ***** **** **** *** **** (Myriad * *****) ***** **** ***** and *** *** ****** ******** *** performance *** ****** *** ****. ** we ******* *** ** ******** *********** with **** ******* *** ***** ** much ****** **** *** ****.
* ***** **'* ***** ********** **** many ******* ***** **** ********** **** a *** ******** ** ****, ********* called ********* **** "** ******** ***" or "**** ******** ***". ****'* **** OpenVino ****** *** ** **** ********* of. ** ** * **** *** in *** **** ***** **** ****** GPUs *** ****, *** ********* **** powerful **** ** ******** ***, ** course. **** ** **** ** **** some ****** **** ****** **** ** not ******* *** *** *** *** thus ** * ************ ****** *** add ** ******** ***.
******, **** ******** ***** ********** *** AI ************, *.*. *****.**.
** *** **** ** ******, *** product ** **** ** *** **** results ***** * ***-*** ***** *** (Celeron *****) **** *** * ************* capable *** ** **. ** **** allow *** *** ******** ******** ** a *** ***** ** *********** ***** increases *** ****** ** ******** **** can ** ********* ******* ******* ****** when **** ******* *** ************** ****** activity.
** *** **** ******** ** ** coral.ai? ** **** ** ***** ******* great. * ** ******** *********** ***** still ** ******* *** *** ******** app **** ****** ** ****** ******.
***** ** "***" ****** ******* ***'** *** * **** ** places **** ***** *** *** *** the *** ***********.
*******, **** **** ** **** ******* keeping ** ** ***** ... ***'** probably **** ** ***** ******* ** the *******, *** ***** **** ***** back ************ ** ****'** *** ***, to *** ***.
* **** ** ******* **** *** performance ** ** ** *********** ** not *** **** ********* ******* *** real ***********. ******* ****** *** ******** we **** ** ********** ***** ******* and **** ************* *** **** ***** total ****** ** ***** ******** ** be ********* ** * ******.
*** ** *********** *** ******* **** of ***** ********. ******* ** **** that **'* ******** ** ****** ******* such ***** **** ****** ****** *** total ****** ** ********* ***** ******* may ** ***** *** (** ********** and ****** * ****** **** ******** AI ********* *** *** **** ****** video ******** ** * ** ************). AFAIK ** *** *** *** ********** such ****** ** ***** ********.
***** ********* ******'* **** ******** ******* for ***** ************* ** **** *** HDDL *****. ** ***** ************* ***** becomes * ***** *** ******* ****** of ********* ***** ******** ** * single ******.
****** *** ******** ***** ************* *** amount ** ************* ***** (*****) *** their *********** ****** ** **** *****, video ********** *** *****. ** ***** that ********* *** ***'* **** *** most ******** ****** ****, ******* ** cannot ******* ****** ************ ***** ******** to ***** ******* ** ************. **** (surprisingly *** **) ****** ****** ***** can ********** ***** ** **** **** video ******** **** **** ******** (**-** vs **-**). ** **** ***** ** taken **** *******.
*’* ******* ******* ** ******** *** supporting *** ***** ***** ******* **** the ***** **** **** ** ****, that **** ********** *** *****. ** this ****** ******** *** *** ********* that **** **** ******** ** ***** integrators **** ****** ******** **** *** parties? * ******* **** ************ *** just ****?