How ViTs/ChatGPT Can Automatically Alert On Protests Tested
While traditional video analytics are improving for identifying people and vehicles (see IPVM's 2024 ranking / test results), accurately identifying complex events such as fighting or protests is far more difficult.
Now, vision transformers that provide context on images significantly improve such capabilities.
Based on IPVM research and testing, we detail how ViTs can automatically alert specific events or objects and provide a proof-of-concept application using the ChatGPT API we implemented.
For more:
- The Future of Video Analytics - CNNs Vs. VITs (Visual Inference Transformers)
- The Impact of Generative AI On Video Surveillance
- Generative AI For Video Surveillance 2024 - IronYun, NVIDIA, Verkada
Executive *******
****** ************ *** ***** ********* ******* context-aware *****/***** ********** ************, ********* ** higher ******** *** *********** *** ********* custom ******* *** ****** **** *********** video *********. **** ******** ******** ******* and ****** ******* ****** ******** ** specialized *********.
**** ******* ***** **** **** *** classify ********* **********, ********** ************** ******* a ******* *** ***** ******. **** can **** ******** ******* ****** *** scene, ******** *** ******* **** ***** pose * ******. **** ***** ** concept ********** ****' ***** ******** ************, showcasing **** ** ** **** *** video ************.
** ********, *********** ***** ********* *** limited ** ********* * ****** *** of ******* **** ******, ********, *** specific ***** **** ** ********* ** weapons. ***** ******* ********* **** **** attempted ** ******* ***** ********* ** alert ** ******** ** ********, *** performance ** ***** ********** *** ******* (i.e.,***** ******** ********* *** ****** *********** Tested*** ***:********* ******** *** ****** ** ***** Gong, **********, ********,***** ******* ********* / ****** ******, Deletes ********).
*******, **** **** ******* ****** **** computational ********* **** *********** ***** *********. Currently, **** *** ********* **** ** API ***** ** * ***** ******** sending ******/***** ******** *** **********. ** these ****** *** ****** ********* *** "tiny" ******** ****** *********, **** **** be **** **** **-******* *******.
IPVM ***** ** ******* *** ******* ******
*********, ******* **** *** ******* ****** or ******* ******* ****** ****. *********** ****** ***** ******* ********** ******, ** ********* a ****** ****** *** ***** ***** clips (*** ** **** *******) *** used *** ********* ****** *** ******* alerts:
***** *** ****** **** * ***** that * **** ** ******. ****** reply **** '***** ** * *******' or '***** ** ** *******' ***** on *** *****.
**** * ***** **** * ******* was *********, *** ****** ************ ********* identified *** ********* ****, '***** ** a *******.'
**** ** **** * ***** **** a ****** *******, **** * ***** of ****** ** *** ****** ** motion, *** ****** *********** ********* ****, 'There ** ** *******.'
Higher ******** **** ****** **** ****** ******
**** ******* ***** **** ********* ********** frames **** * ***** ********* ******** in *********** ******/******. ******* **** ********* a ******* ***** **** * ****** frame ***** ******* "******* ** * situation," ********* ** ****** ********** ******:
** *********** *** ***** ** ****** cues ******* ** *** ***** ****** provided, *** ** * ******* ********** of * *****. *********** *** *********** of * ******-***** ******** *** ******* the ******* ** ****** ** ******** over ****, * ***** ******** * confidence ***** ** ************* **-**% *** each **********. **** ** *** ** the **** **** * ****** ***** might *** ******* *** *** ******* of * *********, *** **** **** indicating * ***** ** *** ******* of *** *** *** ** ******* or *** ** ************** ******* ********** context.
*******, ****** ****** *** ****** ***** to **,*** ******, ************* ~** ****** with *** ******, **** ***** ****** are ******* ** *** ******. *********, in *** **************, ** ******* ****** to * *** ******* ** ****** at ~* ** ~* ***.
FoV ***** ***********
*** ********* ***** **** **** ***** of ******* ** **** ******* ***** surveillance ******* ***** ***** **** ***** (100' **** ** *******) ***, ***** the ****** ** *** ****** *** we **** ****, ** ***** **** important ******* ******* ** *** ***** density *** ************ ** ***** ***** and ***** **** **********.
**** **** ******* **** **** *** with **** ****** *************** ******** *** video ************ ************.
Analyzing ******** ********
***** ******* ********* ********* **** ******* fighting ********* (***** ******** ********* *** ****** *********** Tested,******* ********* ******** ********** *********** *** Public ******, ***.), ****** ************ ******* ****** and ********** ** ****** *** ******** without ****** ********.
*** ****** ****** *** ********* *** following *****, ********* *** "***** ** a *****" *******, ********** *********** *** scene:
*******, **** ** ********* * ***** where ********* *** ****** * "***** battle," *** *** ********** ******** *** "There ** ** *****" *******:
Can ********* ***** ** ******* *** ** ***** ** ******
*** *** **** ********** ********** *** types ** ******* *** ******* ***** animals **** * ******. *******, ******** conditions, ***** ***********, ****** ****, *** other ******* ** ***-***** ****** ******** their *********** ** *********** *** ****** in * *****. ***** *** *** results:
"**** - ******":
"*** - ******":
"**** - ** ******":
"** ****** - ** ******":
OpenAI *** **************
** **** ****** ** ******* ****** and ******* **** ** * ****** image (****** ************** ** *** *****) for *** **********.
** **** ******** *** ****** **** the ***** ** *** ****** ** images ** **** ******, ******** *** frames *** ~* ***. *** ****** and ****** *** **** ** ** API *******, *** *** ****** ** printed ** *** ********.
*************/********* ********* *** ******* ******** **** process ** **************** ********* ****** *** sending **** ** *** ******** ** certain *********. *** ****** *** **** be ********* ** ***** ********* ****** from * ****, **** *** *** returning ****** (**** ** *****) ****** for **** *****.
*******
***** ********** ****** ** ******* ********* through *** ***** ** ****** (~$** an **** ** ****), *** ***** implementation ** ****** ************ ***** *** potential *** ************ ****** ******. *** those *** **** ****** ******, * smaller *********** ********* *** ** **** to ****** ** ** **** ******, constraining ***** ***** ********* ****** ***** analytics.
** ****** *** ********'* ******** ** these ********** ** *** **** ******* years, **** ********* ******** ****** ****** on ***** *****.
***** **** *******, ** ********** *********** the *** ***** ** ******* *** be **** ** ****** *** ******** of:
"**** ** * *****" ** "**** is *** * *****".