Member Discussion

What Are The Most Cutting Edge Automated Surveillance Technologies?

New subscriber here... I'm interested in knowing what are the most cutting edge automation technologies available on commercial surveillance systems.

What initially drew me to IPVM was an article about AISight by BRS Labs.

I also know about a company called Persistent Surveillance which offers to collect and forensically analyze aerial imagery over a wide area with DVR capability.

Are there other really high-end systems I could learn more about, and what they offer?

For example, I have read in computer vision journals about some capabilities under development, and am wondering if any of the following is now found in commercial systems:

  • Stitching together camera views with overlapping fields of view, so you are looking at a composite picture rather than separate views from each camera.
  • Predicting future target location and timing
  • Directing moveable cameras to steer in the anticipated direction of target trajectories
  • DVR capabilities that allow one to simultaneously watch the time-synchronized replay of video over several cameras

Hi. The things you mention are real. Mostly about video analytics. That last bit you mention about synchronized playback is actually a pretty common feature in many VMS systems. Some of the former things related to object tracking are bit more exotic.

What's less real are practical applications of the "advanced" technologies that deliver real value to real paying customers with real problems that said solutions solve.

What's your interest? Are you simply looking for interesting technology to investigate, or do you want to learn more about these technologies to productize, consume, or otherwise?

The reason I ask is because knowing your interest will help guide any discussion in a productive direction.

Steve, can you point me to some existing DVR systems which allow one to replay multiple video streams that are time synchronized?

Hi, look for a DVR or VMS software with a "sychronized playback" feature. Here's a random example from a google search for "VMS sychronized playback."

Using the Synchronized Player in the Sentry VMS Client - YouTube

Synchronized playback is the standard approach for video playback.

There are some consumer recorders that only let you watch one video at a time and there are some enterprise VMSes that default to unsynchronized playback but pretty much every professional recorder supports time synchronrized playback.

First, I want to disagree with Steve and emphasize that the 'real'ness is questionable here.

It is a fact that these things can be done in computer vision labs. No doubt.

The problem is getting the robustness and accuracy of delivering this in production. If BRS Labs could reliably do what they market, they would have been bought out for the 1 billion dollars they claimed 5 years ago.

As for companies to recommend, I can't think of anyone that deliver BRS Labs marketing claims or even, for that matter, companies that claim to do so.

First, I want to disagree with Steve and emphasize that the 'real'ness is questionable here.

I don't think you and Steve disagree though. Steve's first use of 'real' was only a tie-back to the OP's 'do these systems really exist?' After seperating the banal multi-view playback from the list, Steve returns to the 'real' issue with a vengence and calls into question the 'real'ness of the analytics even more derisively than yourself with his four-fold 'real' repetition and scare-quoted "advanced", shown here:

What's less real are practical applications of the "advanced" technologies that deliver real value to real paying customers with real problems that said solutions solve.

Rukmini, correct, John doesn't know we don't disagree. :)

The technologies are quiet real. What's (often) unreal are the claims regarding applicability and value that some of these technologies have to customers -- especially in surveillance.

There's a lot that happens between the lab and the customer's site. Some of it has to do with the unexpectedly narrow operating conditions for said technology. Some has to do with the business model(s) whereby manufactures tend to sell components not solutions. Some has to do with what are often mismatched or unrealistic expectations and requirements.

Anecdote: Foreign defense department wants video based analytics for monitoring base perimeters. Customer agrees to set up the camera angles correctly, points it at an open area between fences so object tracking/tripwire algorithms (that do in-fact work), can do the job for them. Everything's awesome until customer realizes that the system doesn't work as well if the bad guys wear camouflage clothing. Audible gasp. "Hey, where'd the integrator go?" Is object tracking/tripwire analytics real? Yes. Does it solve this customer's problem? No.

Hmm, from the technology side, is this an accurate summary of your comments?

Higher-end algorithms are not sufficiently robust out-of-the-box for real-world applications. To be successful they require active tuning by an integrator over time and situations.

"To be successful they require active tuning by an integrator over time and situations."

Unfortunately, it's typically more than this, especially if you are doing something more than simple motion in a region.

There are many scenarios where the environment is just too complex for the sophistication of analytics. This could be because of lighting challenges, number of people / objects in the scene, phenomenon that look too close to the targeted tracking item (like gun detection analytics).

Certainly, though, having someone to actively and regularly tune a system helps. In my experience, even many government organizations eventually give up on analytics if they require that much 'massaging' to work.

A lot depends on how you define successful. If a video analytic correctly identifies an event 95% of the time is that successful? It's often not in mission critical or safety critical applications. Unfortunately that can thwart use in security or military.

What's more, a technology that is 99% successful in ideal circumstances (say, clear sunny day), can become 50% successful in the early morning when the shadows from a tall tree are long across the scene and the wind is blowing the leaves and there was rain last night so there's a glimmering reflection off a puddle that is causing false positives in the scene. But only in winter, not in summer when the sun rises at different point on the horizon which also happens to be when the system was commissioned. Active tuning by an integrator indeed.

Given the complexity involved with real-world visual imagery (especially outdoors!) no video analytic is 100% accurate all the time. So what can be done with analytics that are (say) 90% or less accurate? If the risk of false positives or false negatives are too high (like, people might die), then these algorithms are never a sufficient replacement for human operation of security tasks. They may be sufficient to augment human tasks. Like present a bounding box so a human operator can more quickly take their eye to a target. Or they may be sufficient in ancillary applications such as forensic analysis of video. Unfortunately in those applications the benefits often do not justify the costs involved.

Personally I believe there is a lot of application of video analytics outside of security still yet to be commercialized. We already have massive advances in technology such as face detection for social networking. I believe applications like people counting estimation, demographic estimation, traffic flow analysis, etc., will become more common and more valuable in other commercial segments than security.

So what can be done with analytics that are (say) 90% or less accurate?

If humans are less accurate/slower at the same task, then there is still value, no?

Rukmini, maybe you have an example? Not that I doubt they exist, I'd like to understand what you're thinking of when you ask this.

To wax philosophically for a sec.. It's all about trust and confidence. What makes you feel more safe? An algorithm you know is maybe 90% accurate, or a human operator whose performance capabilities for the task you can't predict? I think most people trust the human more than the machine. Even if we can prove that the human is less accurate, less attentive, etc., that 10% miss rate from the automation is a serious hang-up for people.

Many times "value" in analytics is pitched from the point of view of labor savings. And that's where it gets dicey. As soon as you take cops off the beat, and your automation replacement misses a crime, people declare the system a failure--even if that same cop might have missed the same crime..

Like I say, I think there is potential value to augment human operations with video analytics. But without those hard dollar savings and ROI numbers, it's a tougher sale. Traditionally the price of video analytics has been relatively high--especially if there's a lot of integration, configuration and tuning required. So the value is not perceived to be there if all it does is makes a human operator a little more effecient or effective in some way that's difficult to quantify.

The good news is the price (at least component-wise) continues to come down. There are $10 SoCs out there that can do a ton of interesting analytics. There's also mountains of research going on right now by the likes of Google, Facebook, et. al, in object recognition/classification, etc., that will likely improve performance and applicability. So when these features can be simply added to existing security systems to augment human operations (or replace them if you want to take that risk) at a price point that's trivial, then there's some marginal value as a new feature--but not necessarily THE value that defines the system and its capabilities.

Rukmini, maybe you have an example? Not that I doubt they exist, I'd like to understand what you're thinking of when you ask this.

I'm sure you know more examples than me, but here's a couple off the top of my head...

  • analytics that estimate object height and distance in real-time
  • object tracking and aging of > 100 items in an airport
  • facial recognition from a database of 100,000 faces

Interesting….But I imagine too expensive for the average customer. I was impressed; have to watch this one for a while.

I am a DoD human factors specialist researching human interaction with autonomous surveillance systems. My organization has a background in UAV sensors, but I am now leading a group looking into multiple, ground-based cameras/sensors. I am doing background digging on academic research (ie, computer vision mostly) but would also like to know what commercially available systems already have to offer. No sense improving something that is already good.

Below is a somewhat vague abstract from a conference presentation:

Urban security, reconnaissance, surveillance, target acquisition, and event response requires intelligent data gathering, storage, integration, and accessibility. The expansive layout of the terrain, as well as such confounding issues as limited fields of view and the inherent ability of the infrastructure to mask criminal activity and to conceal people of interest, limits the physical measures that can be taken to adequately respond to and prevent the occurrence of such threatening incidents. A popular method by which to increase presence over such a large area is to employ an electro-optical (EO) camera network. Such a robust sensor system alleviates the manpower required to be physically vigilant, and consolidates such complex tasks into those in which a single operator could perform. The notable drawback of this remote presence, or tele-presence, is the lack of situation awareness that could arise due to the limitations placed on the operator to naturally (using more than just sight alone) obtain discerning information about the disparate, complex environment. In response to this deficiency, **** is conducting multi-modal research to develop an interface aimed to alleviate these challenges and to allow the operator to move fluidly throughout and conduct operations within such an urban environment. The focus of this presentation will be to describe the architecture of this interface, to discuss the results of an end user evaluation of the interface, and finally, to express the team’s vision for this tele-present operator and the role this interface will play in realizing this vision.

Ah, so that would explain your interest in object tracking "targets." :)

We've had quite a few object tracking related projects in our labs. I'll see if there is some academic work that was based on that I can point you towards..

As far as what is "commercially available," there is a huge gulf between the technology's capabilties and the practical, commercial applications. As such many of these technologies remain on the shelf due to lack of sufficient business case for productization.

Thanks that would be very helpful

Check out Fast Protect AG. Their Terra 4D Geospatial Situation Management hints at the future for at least some applications.

Sorry, Carl, my stupid finger accidently disagreed with you when going for the link.. I have agreed with you and noted your insightfulness as well in compensation. Fast protect looks cool, have you had a real demo or any contact with them?

No, but I have a funny story about them. The manufacturer of our first VMS was a company located in Hunenberg, Switzerland named Fast Video Security, AG. They also licensed the system to Honeywell, who still sells it as the Honeywell Enterprise system. In 2006, the investors sold the company to Nice Systems and the CEO and others from Fast Video Security formed Fast Protect, AG after Nice had no need of their services.

Fast Video Security's product, Alpha Silver, was way ahead of its time but competition in the VMS space along with a miscue when they tried to release an all-in-one product they called "Alpha Blue" caused them to have financial trouble, hence the sale. However, I really liked the CEO, Beat Meier, and have followed him on and off over the years.

No, but I have a funny story about them.

You sir, have an extremely dry sense of humor. :)

LOL. I think nearly anything which includes Nice Systems and/or Honeywell is funny. Perhaps not "Funny: haha" but at least "Funny: peculiar".

Obviously no love lost there....