Deep Learning Analytics For Unattended Baggage Detection?

Great question. Lots of companies for many years have marketed baggage detection. What's more recent, of course, is using deep learning. I have not heard anyone prominently marketing this recently and we have not tested that type recently.

Looks like Convergint does (www.convergint.com). If you dig deeper into Bag Left Behind you get this, "Ultinous’s technology can filter and process data, trace moving people, detect abnormalities, and generate predictive alarms so that security teams can take appropriate action preventing unwanted incidents. Utilize the potential of the latest AI technologies in your products using their easy to integrate API to create a thus far unprecedented user experience for your customers."

I would take any claims of AI or Deep Learning with a grain of salt. It's too easy to claim it's being done, too hard to do well and very rare to see any real test data to help assess efficacy. 

I would clarify the needs of the client with a very clear expectation based on how they plan to proceed with an “object left behind” event.

Detecting an object left behind in a closed area with little movement is pretty easy.  However, how fast do you need to alert?  How long will it be left?  What action will be required?  Will this increase liability or decrease it?  What are you trying to prevent?

Detecting an object left behind in a busy scene is far more difficult.  You have all of the above and must add in factors of temporary blockage by passerby’s and such.  This adds assumptions and that adds risk of missing it.

Okay, you have an object left behind and it meets all the set criteria, no false positives, no missed bags.  Now what?

If it’s actually a harmless bag and they clear out a train station or stadium to investigate and it’s actually someone’s luggage.....will they do that again?

If it’s actually an IED, and you review video and determine it was just a guy in a suit that got rushed into a cab and it detonates.  Would there be more or less liability?

NY....”See something, say something”

Just my 2 cents worth. 

If this existed, and worked, you would know about it.

There are a handful of VA companies that claim to support left-object, but you need to be careful to ensure your scene, and object sizes, will work with the product you select. You may find that you need more cameras and licenses than what is within their budget.

If I were an integrator, I would be very very cautious about offering left-object analytics without tightly defining the application, and the scope of ongoing support you would be expected to provide.

1) Deep Learning can be used for detection and/or for classification.. It detects in the image specific classes of targets which have been trained in specific conditions. So “what is object?”, first of all? A bag, a trolley, a box, a plant, a cleaner machine, a chair, a shadow, a sudden light reflection, the feet of a person partially hidden by a bench, ...?..

2) Deep Learning then is not able to tell if that detected object is unattended or moving and for how long.. In few words, it’s not able to tell that this specific object is the same target of some frames ago.. That’s anyway given by a tracker, which it’s totally another story..

3) Deep Learning has a great wow effect when lots of targets are detected and classified in complex areas; and of course, that’s great.. But none in this “wow” ever notes also the huge amount of missings and falses.. If you counted each object detected as an alarm, you would cry after 40 seconds.... Could you imagine in the long term and with dynamic scenarios..

4) Deep Learning works well with high resoluted targets, for having reliable features to work with. For detecting a bag in a panoramic image you may need to process a huge resolution; thus a huge amount and cost of computational power could be needed.

5) I agree with Undisc.#2 and #4 when they write about how much it’s relative the real need of a “left object detection” in real environments.. Is your customer really that sure a left object detection has a real sense and a practical value in its case? Because for my experience the 90% of times it’s requested just for “inertial fashion” and at last it would have really few sense even if it worked... If instead in this case it had effectively a sense, anyway which case/environment are you talking about? Because Brian is absolutely right as well: marketing fairy tales, hyper claims and YouTube’s nice movies asides, anyway to make work well an “unattended object detection” in real life it’s still very hot stuff in dynamically complex or crowded areas....

So, resuming, technically and theoretically it would have a sense to use deep learning as base for detection and classification, integrated with other levels of “classical” video analysis (funny for me to call it “classical”....🙄😅). But at today this would be still severely too demanding, so basically unapproachable yet in most practical applications, for too high cost/benefits ratio.

Maybe in future, but not yet now..