Avigilon Appearance Search Aims to Transform Video Surveillance

Strange Timing?

Probably to have something new for IFSEC....

Reactions:

John Honovich

In reply to Itamar Kerbel

IPVM

It is definitely tied to IFSEC.

But the core IFSEC attendance base is weak for Avigilon. For example, the UK accounts for ~7% of Avigilon's revenue. All of EMEA and the UK combined, which is almost everyone who goes to IFSEC, is ~25% of Avigilon's revenue. By comparison, North America represents ~66% of Avigilon's revenue.

And then there's my other points - it's not scheduled for release until the end of the year, the summer means many people will quickly forget, etc.

Even huge vendors like Axis usually only schedule B level / secondary items for a show like IFSEC, leaving the biggest announcements for ISC West and ASIS / Essen.

We'll see. If Avigilon has something bigger than this in 2.5 months to announce at ASIS, than this move makes sense.

Reactions:

Undisclosed Manufacturer #9

EU make be a weak market for Avigilon, but I've heard a few comments from my EU contacts, manufacturers and integrators , about Avigilon being one of their few non-EU competitors there. Makes sense to me as it seems in the EU they still seem to value quality over price over there to a certain extent.

Reactions:

Undisclosed Manufacturer #1

http://www-03.ibm.com/software/products/en/intelligent-video-analytics

IBM demo'd a similar capability 10 years ago at ISC West and still offers the solution today with Milestone and Genetec.

If I remember correctly IBM held a few patents covering these capabilities.

Indexing and Searching According to Attributes of a Person(US20100106707, 2008)
Calibration of Video Object Classification (US20100054540, 2008)
Video Object Classification(US20100054535, 2008)
Detection of Abandoned and Removed Objects in a Video Stream (US20090238462, 2008)
Real Time Processing of Video Frames for Triggering an Alert (US20090244390, 2008)
Object Detection System Based on a Pool of Adaptive Features (US20080232681, 2007)
Rule-Based Combination of a Hierarchy of Classifiers for Occlusion Detection (US20080247609, 2007)

IBM Smart Surveillance Research

Reactions:

(1)

(2)

John Honovich

In reply to Undisclosed Manufacturer #1

IPVM

We spoke with IBM not too long ago. It's still around, like you say, but it remains a very uncommonly utilized niche / expensive product. Combine that with IBM's continued cut back and layoffs and it is just not much of a factor.

Reactions:

(1)

Undisclosed Manufacturer #1

Wouldn't be terrible if someone notified IBM and they went after Avigilon for patent infringement?

Reactions:

(1)

John Honovich

In reply to Undisclosed Manufacturer #1

IPVM

Maybe Avigilon will read your comment and go after IBM for patent infringement? ;)

Reactions:

(2)

Michael Miller

We'll see. If Avigilon has something bigger than this in 2.5 months to announce at ASIS, than this move makes sense.

ACC 6 should be a big deal and I am just guessing here but I would expect to see this at ASIS.

Reactions:

(1)

Undisclosed Integrator #6

In reply to Michael Miller

I'm hopeful. My wish list for ASIS and ACC6:

Smart codecs would be nice.
More impressive analytics?
Support for NAS external storage.
Less expensive failover
4K PTZ
Better WDR
Cameras with integrated audio
Integrated EoC or EoUTP extenders

At the very least 1, 3, and 5 should happen. At this point it seems like Avigilon has been slow to the plate with Smart Codecs - Axis is on their 2nd gen, Panasonic has it, and Hikvision/everyone they OEM to has it. Perhaps it interferes with the "secret sauce" of HDSM.

Reactions:

Michael Miller

In reply to Undisclosed Integrator #6

@Undisclosed 6 Integrator

I have a feeling a "smart codec" is right around the corner with the H4 camera line. Looks like all the pieces are in place with "idle scene mode" Avigilon isn't normally first out of the gate with new "stuff" but when they do release new "stuff" is works well and is reliable.

Reactions:

Undisclosed Integrator #2

As an user/investigator, this seems to be a nice tool to have. Often subjects may disappear from one camera, and potentially show up on one of many cameras at a later time. And often subjects may stay inside the location for a long time, where you know the time of the incident, but are more interested in where and when the person entered the location.

I do what this tool does manually today. I just memorize a couple of garment colors and the shape of the person, and then start skimming video at 4-8 times regular play speed, one camera at a time. I feel that I get good it for time value by doing this. But it does take time. If I still don't get a hit when I'm done, I decide if I have more time to spend on the case, and maybe start a more rigorous search.

With a tool like this, the "skimming stage" will be done by, as I understand, searching trough meta information saved by the camera alongside the video. My guess is that the search will be fast, like the motion area search. If I got a hit with this tool, I saved loads of time. If I got no hit, I just do what I have done up to now, pretty much without a waste of time. I would not rely on this tool as a substitute for the real work. More like a shortcut that sometimes saves me time.

It's a perk, but I would not choose it over good image, unless image quality isn't very important. Like in an overview, or somewhere I would get a bad image anyway. So being locked to Avigilon just to get this feature on all cameras, would not be a priority.

Reactions:

(1)

John Honovich

In reply to Undisclosed Integrator #2

IPVM

2, that's good feedback!

If it's fast, it's certainly worth trying. Given how slow VideoIQ's version was, I am curious how fast the final developed Avigilon product will be. Surely it will be faster but it will be interesting to see how fast.

Reactions:

(1)

Undisclosed Integrator #6

Was the VideoIQ version processed on a server, the client, or on their various devices (Rialto/ICVR)?

Reactions:

Brian Karas

In reply to Undisclosed Integrator #6

IPVM

VideoIQ didn't really have a "server", it was cameras/encoders communicating directly with software on a client PC.

The object matching in VideoIQ was done by the client PC. In short, the PC would download the meta-data description info from each device you wanted to search on, and perform the search/analysis locally on the client, find the matching segments of the video, and then request those video clips from each device.

The search process itself was relatively fast, but pulling many clips from many devices could be somewhat slow, depending on the number of hits on the network connectivity between the client and the device.

Reactions:

(2)

Octavio Villa R.

IPVMU Certified

I believe Nice was doing something similar in the las ISCWest 2015 or 2014. Did you know / have tested it?

Reactions:

John Honovich

In reply to Octavio Villa R.

IPVM

Octavio, you are right, I forgot to add them in. It's been updated.

Our original coverage of NICE Suspect Search Examined (2014)

Note: NICE was spun off as Qognify (2015)

As for testing it, testing NICE historically has been a low priority because of their very limited business model / market share (primarily going after to mega projects but very few others). I did reach out to Qognify to see what, if any changes, they had made since the spin off.

Thanks again for mentioning this.

Reactions:

Undisclosed End User #3

How do Ipsotek and Briefcam compare?

(Briefcam is more of a forensic tool, rather than live tag & trace, but it does offer support search for simiilar tagged obects)

Reactions:

John Honovich

In reply to Undisclosed End User #3

IPVM

3, good question / suggestions on Ipsotek and Briefcam.

I do think Briefcam will be helped by Avigilon's move and its probably the best overall counter that Milestone and Genetec can embrace if Avigilon makes headway marketing this.

Though Briefcam definitely specializes in advanced search, it's different from Avigilon. On the plus, it provides synopses that summarize long time periods on a camera in short clips. And it has filters to customize those synopses based on color, size, direction, etc.

However, I do not believe they are doing the type of appearance or suspect search that Avigilon or NICE is claiming. I see no claims from Briefcam that you can click on an object and show me objects that look like that across all cameras.

That said, I've reached out to Briefcam to do an update since I do think Avigilon's move will help promote the overall advanced search space.

Btw, this is Briefcam's most recent product / tech demo from a month ago:

As for Ipsotek and/or Tag and Track, I am skeptical that any of that works regularly even after doing a lot of custom setup. Small players, including Viseum, have been claiming things like that for years but with very limited progress / acceptance.

Reactions:

(1)

Itamar Kerbel

Please correct me if I'm wrong but while Briefcam does what it does very well, it is not a real time tool.

You cannot just sit and say I want to have a clip of the last 3 hours and after a few seconds it is ready. You have to set your time frame and wait a very long time. Only after the long processing period you can do your own search.

I think what Avigilon is offering (will offer) is a little different.

Reactions:

(2)

(1)

Brian Karas

In reply to Itamar Kerbel

IPVM

According to Briefcam, a 1hour worth of video can be processed in ~5 minutes (possibly less, depending on amount of activity) on a Xeon processor.

The software will start showing results as soon as the initial portion of the clip has been processed. A 3 hours clip therefore might take 15 minutes in total to process, but a segment of processed video would be available to the operator within the first minute or so.

If you want to narrow down the search using Briefcams filters you would need to wait for the entire video to be processed to get the full results of course.

Reactions:

(2)

Undisclosed Manufacturer #4

Actually, both VidSys and Proximex have something similar.

VidSys has this feature called "VirtualTracker", which tracks an object (person, asset, vehcile etc.) in real time as the object moves across cameras in a networked security system. The video streams from multiple cameras are merged into a single screen view with on-screen controls.

Proximex has a tracking feature called "EZ Track", which provides very similar function as the "VirtualTracker" by VidSys.

What I am not sure about is how much video analytic is involved in both tracking features. And both of them seem to be server-based, embedded VMS functions in their PSIM platforms??

Anyone knows? thanks

Reactions:

John Honovich

In reply to Undisclosed Manufacturer #4

IPVM

According to Proximex:

The EZ Track solution is based on camera topology, which is easier to deploy, setup and maintain and does not require the use of video analytics.

In general, while there are some offerings that claim to do this, it's typically based on a manual setup of the physical relationship of cameras rather than video analytics.

Reactions:

John Honovich

In reply to Undisclosed Manufacturer #4

IPVM

Also, there is Snap Surveillance, which does use video analytics but the operator still needs to click to choose the next camera, not automagically following. From our 2015 roundup:

Snap Surveillance offers an analytic which automatically builds an association between surveillance cameras by analyzing where objects disappear from one and most commonly appear on another. Users click on a portion of the live or archived video where a tracked subject exits, which brings up the associated adjacent camera, similar to Genetec's Visual Tracking feature, though created without manual configuration. A clip may also be recorded of operator tracking, which records the main camera selected by an operator as he or she tracks a subject through the system, for easy export.

Snap MSRP is $75 per camera channel and integrated with Milestone XProtect and Genetec Security Center. This video shows an overview of Snap's offering:

Reactions:

(2)

(1)

Undisclosed Manufacturer #4

Agreed. It seems that many VMS can track people and objects across cameras. Salient also has a tracking feature, which is similar to what you just described.

Maybe IPVM could make a comparison of these tracking functions. Here is a list to start with:

Avigilon:

NICE/Qognify:

VidSys:

Proximex:

Salient:

Snap Sur.:

Genetec:

something like that.

Reactions:

John Honovich

In reply to Undisclosed Manufacturer #4

IPVM

I removed the contents of your list. Some of it is clearly wrong or misleading and I don't want people to take it for fact.

As for comparing them, what Avigilon and NICE are claiming to do is completely different than, e.g., Snap and Genetec. Calling them both 'tracking' functions is confusing and mixes two types of function.

Avigilon and NICE are providing a fuzzy search tool that hopefully returns one or a few good matches out of a pool of images but does not claim to know the physical relationship or path between objects across cameras.

Genetec and Snap 'tracking' is an assistance function that makes it easier for operators to select what camera to go to next to physically track a suspect along a path. Neither Genetec nor Snap are claiming to track like Avigilon or Nice Appearance / Suspect search.

Reactions:

(1)

Undisclosed #5

http://researcher.watson.ibm.com/researcher/view_person_pubs.php?person=us-lisabr&t=1

Over the last couple of months I have been looking at the academic/industry research side. There is a large amount on this subject so it makes me wonder what can be claimed in Avigilon's patents that has not been done already in some way or another.

Someone mentioned IBM already. Look at the 2012 papers and before published by this IBM researcher...pretty similar objectives.

Lisa Brown was the chair for the 3rd Workshop on Vehicle Retrieval in Surveillance (VRS 2015)

https://avss2015.org/workshop-track-1-traffic-and-vehicle-surveillance/

Some papers related to appearance search in this conference:

Invited Talk: Object-based Retrieval in Surveillance Video using Behavior, Events and Appearance
Anthony Hoogs, Kitware Inc.

Vehicle Identification Using Distance-based Appearance Model
Huang-Chia Shih and Hao-You Wang (Innovation Center for Big Data and Digital Convergence, Taiwan)

Appearance or patter matching has been around forever in compute vision...so I am wondering if Avigilon is patent touting, just to scare away others from implementing their own appearance-based analytics.

Would like to see IBM shaking things a little. I do remember reading IBM claim that they are near real-time.

Reactions:

(1)

John Honovich

IPVM

New Avigilon IFSEC marketing video showing appearance search in action with the same man in red:

Reactions:

(1)

Undisclosed End User #3

Depending on your requirements - should you be wary of demos that track individuals in largely uncrowded scenes?

Reactions:

John Honovich

In reply to Undisclosed End User #3

IPVM

should you be wary of demos that track individuals in largely uncrowded scenes?

Yes, unless your environment is always uncrowded. Crowds makes analytics accuracy a lot more challenging.

Reactions:

(3)

Murat Altu

AxxonSoft

Great to see that Avigilon is promoting forensic search analytics! I also believe, that it will 'transform video surveillance'. All we need is to inform customers about efficiency of this technology, create demand and prove it works! Next big thing is finally coming :)

Reactions:

(4)

Undisclosed Integrator #7

I just shared this with a coworker; he responded with a YouTube video shared 9 days ago by Samsung SDS America. The video is sort of showcases their "complex event" and "visual" search functionalities using computer animation. I'm wondering if anyone has any actual hands-on experience with this platform, and how it compares to this latest Avigilon offering.

Reactions:

Brian Karas

In reply to Undisclosed Integrator #7

IPVM

Is this the video you're referring to?

Reactions:

(1)

Undisclosed Integrator #7

In reply to Brian Karas

No, the video you shared is moderately more informative than the one I viewed. This is the one I viewed:

Reactions:

(1)

John Honovich

In reply to Undisclosed Integrator #7

IPVM

Feedback from Hanwha Techwin on that video / Samsung SDS:

This offering is an analytic package being promoted by Samsung Data Systems a company within Samsung holding company

So different group / company. We'll try to get in touch with someone at SDS and learn more.

Reactions:

Murat Altu

AxxonSoft

And, please, ask if it is truth about Moscow SafeCity project. I believe, not :)

Reactions:

Undisclosed End User #8

This application is very common in China, and several Chinese companies own similar products. It can search the exact same target ranked by comparison similarity, in different lighting condition and pose.

Reactions:

(1)

Murat Altu

In reply to Undisclosed End User #8

AxxonSoft

Yes, this kind of technologies are now common in SafeCity installations in China and Russia. Here is one of the usage scenario from AxxonSoft installations in one of many SafeCities around Russia:

AxxonNext 4 Search Opportunities

Reactions:

(1)

John Honovich

IPVM

Murat,

That video shows (1) directional motion, (2) filter by color, (3) LPR and (4) facial recognition.

It does not show appearance or suspect search.

I am not saying its better or worse than what Avigilon and NICE are doing but we need to be clear about what's being claimed.

As for your facial recognition demo, since you are here to promote your offering, how well is in the wild / across the city facial recognition working in production?

For example, is this a real criminal you matched?

Reactions:

Murat Altu

AxxonSoft

Axxon Next 4 Face Search Demo

Yes, appearance search is a little bit different. We need to store more than one color of object in database together with shape information. We store only object type (car, human, group), dimensions and single color.

Regarding face search, the results of search in our promo video are real screenshots from working system in one of the SafeCities, but this guy is our staff. He was moving with his car from that shop and entered some places in different times and we found all cases with search tools, so all screenshots are real.

In this SafeCity project, cameras are integrated with door phones in every entrance of every apartment building, so pictures are quite good.

Face search works really great, because we are just sorting all previously captured faces by similarity with the one you are looking for.

Here is the video where we search with absolutely different pictures from passport and some Facebook selfy in 5 days real SafeCity camera archive and no any false results:

Everyone can test it by himself: download AxxonNext 4, install and in demo mode (from 8am to 6pm) it works with all features enabled. There is also a possibility to import AVI files for analysis. Some police departments use system in this way when video exported from other NVR/DVR. And AxxonNext is first VMS officially Profile G compliant to get access to external storages online and this is also the way to import online without need to export to AVI.

BTW, important to say, that face search is included to the enterprise license with all other types of search (direction, loitering, color, number of objects, and so on)! It makes this technology available for every camera in the system. The only limit is performance of the server doing online analysis to generate scene description and face vectors to fill database.

Next challenge is to dramatically increase performance by using GPU acceleration on server side, because number of channels per system grows incredibly fast, while CPU performance not.

Reactions:

John Honovich

IPVM

Murat, your 'no any false results' demo proves nothing.

You (1) search for 2 people only, (2) those people stare right into the camera in a very narrow field of view, (3) there's no indication over what time frame or how many other people have been recorded. Net / net: it's simple to demo something like that but it's inconclusive at best.

Here it is embedded so everyone can see if for themselves:

Btw, I do like that your VMS is profile G complaint. Have you tried connecting it to Milestone using their new Bridge so you can takeover their systems?

Reactions:

Murat Altu

AxxonSoft

You can see 4 days and 4 nights on right vertical timeline and this is entrance to the appartment building with couple dozen of flats, so traffic there is quite busy.

BTW, we have also nice option to combine face capture with TimeCompressor and quickly review all faces in motion. There is not much benefit from this, because you can always see them as thumbnails, but looks impressive :)

Regarding Milestone, I can't find them listed as Profile G compliant at onvif.org, while 1263 different devices are in and only 2 clients including us. Once they do it, for sure, we can import archives from Milestone also and offer all search possibilies of AxxonNext. It is easy as exporting - just select the range and click button and 1 hour of video takes few minutes to analyse and generate all meta-data.

Reactions:

Undisclosed End User #3

Using Facial Recognition to tag and track suspects, would seem to be a step up from tracking say human objects wearing red. Wouldn't it?

e.g.

Reactions:

John Honovich

In reply to Undisclosed End User #3

IPVM

Using Facial Recognition to tag and track suspects, would seem to be a step up from tracking say human objects wearing red. Wouldn't it?

Sure, if it works.

Main issues with facial recognition done on surveillance cameras / systems:

Relatively narrow FoVs needed.
Relatively constrained down and side tilts required.
People need to be facing the camera head on, etc.

Even with all that, it has issues, especially if you are searching over large numbers of cameras, time frames and crowded areas.

Reactions:

Murat Altu

AxxonSoft

Analytics is progressing together with CPU power. Main constrain is not FoV or side tilts, but CPU power. During next ASIS and ISC West we are going to present face search live in exhibition crowded environment. Hope you'll stop by to evaluate.

Reactions:

(1)

John Honovich

IPVM

Main constrain is not FoV or side tilts, but CPU power.

Then why do you have your subjects in that video stare right into the camera? ;)

Related, that guy staring into your camera would be the worst criminal!

Reactions:

(1)

Murat Altu

AxxonSoft

Ok, here is another example from metro: Man found in the left corner and source face is very much tilted.

John, please, believe me, it works really good. Hope you visit out booth or play by yourself as it is available without even need to ask us for the key or something.

Reactions:

(1)

Undisclosed Integrator #6

Reactions:

(1)

(3)

Undisclosed Manufacturer #9

For Murat, or any manufacturer demonstrating these "intelligent searches" or anything similar, I would suggest demonstrating more realistic situations and environments than idealized ones. As John alludes to, no one is going to be staring straight into a camera unless (1) they intend running face first into one or (2) they are asked to and decide to comply.

I have learned when I put on my salesman hat I make more of an impression to clients debunking idealized situations that are not that much real world, and at the same time I am running into more clients lately who are are able to discern it more for themselves and being more critical of such claims.

Your product may actually work great and be easy to sell to someone who cannot tell the difference, but you take a risk if it's an informed person who knows how to spot a well "staged" demo, or someone like me who comes along before the sale and debunks it, if you don't have a back up demo that looks more like real world.

Reactions:

Undisclosed #10

In reply to Undisclosed Manufacturer #9

IPVMU Certified

I would suggest demonstrating more realistic situations and environments than idealized ones.

To be fair, it's an idealized test of a realistic situation. As opposed to an ideal test of an ideal situation, e.g. if there was no head tilt and the angle of all the shots were similar. Which we used to see in gen 1 demos.

Moreover, do you think the four frames taken from the pinch point are unlikely to actually occur? Note that only the distant ones are facing the cameras, as would be expected as one approached the narrowing.

Its easy ask for more realistic and challenging situations until the system fails and then say ITYS, but the question is whether the capability as shown could provide necessary ROI.

I think, depending on the use case, absolutely.

Reactions:

Murat Altu

In reply to Undisclosed #10

AxxonSoft

ROI is the thing...but not because of quality of analytics.

Typically you can connect few hundreds cameras to one server without analytics. But with face recognition engine - few dozen. So, even if license is free, you need to buy a lot more processing power - 10 times more! And than ok, I better search manually :)

Reactions:

Murat Altu