AI Video Tester Released

IPVM has released the world's first AI video tester that lets you see how various AI models (including from Amazon, Google, Microsoft and YOLO) work on your own video.

Why?

While there is lots of hype about 'AI', 'Deep Learning', Neural Networks, CNNs, etc., it is hard to know how well they work. Worse, it is difficult to tell how they will work on your own video rather than marketing demos or generic photos.

How It Works

You can try the AI video tester here by choosing from one of our sample videos or (members can) upload their own video. Below is a sample video from a moderately challenging video surveillance scene analyzed by Amazon Rekognition. Notice it does well generally but periodically thinks it sees a bathtub:

And our Tester maps out where and when each object is seen plus lets you scan the timeline below to see frame by frame which objects are detected, as shown in the gif below:

Use Cases For It

We do not expect everyone to make use of this but here are the main use cases we see:

Research

If you, like us, are doing research into computer vision, this tool is a unique means to quickly and easily gauge the performance of various models. That was the first reason we built it, as we build up our computer vision testing, we wanted a way to do faster and better testing. You will see us using this in upcoming reports.

Education

Most video surveillance professionals know little about AI beyond buzz words (e.g., 'neural nets' and 'layers', etc.) but they have little idea how well they actually perform. This tool makes it easy, instead of spending days and having to know how to set up each of these models, you simply add whatever videos you want and let the tester run it for you.

For example, many of our beta testers were surprised about the results they saw, including how poorly many models worked in many challenging surveillance conditions.

We have an upcoming AI Video Analytics course and this tool will be a core component of the exercises and training.

Comparison

For those looking to use these models in production, this will help directly in making product comparisons. Of course, there is a major limitation, the tester does not include any surveillance manufacturers products yet.

Future Improvements

For now, we are releasing the foundations of the Tester. 

The most obvious improvement is more models / systems, from OCR and LPR offerings to various manufacturer's AI systems. While we will add more, we will not ever be able to make it all inclusive since many video surveillance system either do the analytics inside the camera or do not provide sufficient APIs.

Another improvement we are exploring is to make a simple video analysis tool for exported video, i.e., using one or more of these models to help integrators find people or vehicles from long recorded video clips.

We can also add facial recognition to this by using our Tester and adding on a face database component so members can experiment with different systems on their own video before deploying on site.

And, of course, we are definitely open to suggestions from members for improvements.

Try It Out - Give Us Feedback

Try it out, let us know what you think, questions you have and improvements you want.