Video Analytics In A Mine - Best Option?

I saw a specification today were a mine wants a Flir IP Thermal cameras to connect directly via analogue cable to an IOImage Analytics encoder in the field, then record this on Milestone via the network. It doesn't seem like a good design to have 2 hardware device in the field in a remote location.

I would think it would be a better design to use a thermal camera that could handle onboard anlystics such as Axis with Agent VI(or equivalent), rather than have 2 hardware devices in the field. Another solution would be to have the analytics at the server end as a plugin for the Milestone or a hardward device that can take onvif video feed such as VideoIQ Rialto R-Series and intergrate back into Milestone.

Any input on best practice, adavantages and disadvantages, suggestions?


Hello!

Well, according to me actually there may be 3 reasons for this kind of requirement.

The first possible one is that the Customer simply "likes" and trusts that camera with that encoder, maybe timely "pushed"...;))

The second one, more technical, may be for example that if the camera should need to go very deep and far (several hundreds of meters for example) inside the mine, probably it's more expensive to use IP cables (not for that long ranges) and optical fibers than long analogue cables (still more efficient for longer ranges) and encoders out of it. But this if we have the camera inside the mine and the encoders at the control room outside. But if it is specified camera+encoder in a single "box" inside the mine, yes it looks quite a weird requirement actually: because anyway then you would need to carry IP signal for long range..

The third one, again technical, is if the requirement is for a radiometric information; thus, for a camera measuring the absolute temperature. For example, I can immagine that inside a mine it could be very interesting to see where there is a peak of heat to prevent digging in critical dangerous spots where, for example, a dangerous hot gas may explode or whatever else.. And to do this, it's definitely needed a radiometric camera, not a differential one like the one you mention: in that case you need to know "80°", not simply "there is something with a different temperature out there..".. Is the specified model of Flir camera a radiometric one or a differential one (which kind of microbolometer, in few words..)?

May it be one of these 3 reasons?..

In case of "none of the above ones", I fully agree with you it would be more convenient to use an Axis Q1931 camera with analytics onboard. Of course I can't confirm and suggest to you to use AgentVI, for obvious reasons.... (I'm from TechnoAware....;)))))....).. But of course that's not related with the problem...))))

Cheers,

Simone

(TechnoAware)

What about keeping it simple. Since thermal cameras are low resolution, use the analog output and connect straight to a Video IQ encoder located at the headend. It's much less expensive than a Rialto Blade. It could then interface into an ExacQ platform . Is there any need to monitor data from the Flir cameras?

The solution is simply to detect intruders on the perimeter. Really trying to get an idea what the best design would be for implementing and what the rest of the industry is doing.

I see it as a negative to take a thermal camera and convert it to analogue in the fieled and not have it on the network for remote configuration. Kind of seems counter productive to take a digital signal convert it to analogue and then encode it back to digital. Plus the added points of failure on the perimeter by haveing 2 hardware devices.

I can understand that this may be a viable solution if it is cheaper, but is it really cheaper? It seems that most of the video analytic "encoder" hardware only caters for analogue feeds and not IP. Also I get the impression that analytic encoders are the most popular way of implementing video analytics for intrusion detection.

Well, in this case my feeling is to confirm definitely to you yours.. IP thermal camera + analytics is available and by now much cheaper. I made some tests with the new Axis Q1931, and I definitely feel to suggest it to you! In Europe it costs less than 3.000 euros (end-user price); add 450-500 euros for the analytic module and they may reach 3.500 euro (around $4.700,00) per camera..

Here below some best practice, imho of course:

1) Analytics edge-based or server-based?

You can have both in the market (Axis has many ADP with edge solutions for Intrusion detection). I generally suggest the edge-based solution in case of effective constrains about having or not the timely bandwidth to carry a good quality video flow (CIF resolution @8-10fps is already enough, but it must be stable) from all cameras to the control room, or about not having a server in the control room for the server-based processing (because of budget, or no place for it, whatever reason..). When there are not constrains, I generally suggest the server-based solution: moreover, if they already have a server with Milestone, if there are not too many cameras probably it may be used also for charging the analytics on it.

2) How many cameras and how?...

Eh, this is another critical point in the market when we go to perimeter protection.. Of course the Customer would like to spend the less possible and someone is still convinced that we can use high MP cameras or thermal cameras to look at 1km..... Well, sorry, forget it!.. There are actually 3 problems:

First, unfortunately there is a phisics of the propagation of light and heat: and this phisics teaches that it is completely different the attenuation in a dry temperate condition or through for example rain, heavy humidity, snow, very warm or very cold air, fog, .. So, in few words, you can even have a 200MP camera, but if there is fog with 50 metres of visibility or heavy rain, you will not see nothing by your naked eyes at longer than 50 metres with an optical camera and maybe 80-90 with a differential thermal one!!.. And this is definitely NOT a problem of analytics, I am writing about naked eyes... We made for example an experiment, making correct detection with a thermal camera at 570 metres with our analytics; yes, in a very dry and temperate day.. The same camera, same environment, same position one month ago in a snowing day with -2°, you could see nothing at all over 80 metres by naked eyes!....

Second problem: to see far of course they use very long range lenses, such as 60mm... But, are we making a perimeter or a monitoring at a range?.... Because if we want to see a perimeter, I would like to see "from here to there", and "from there to there", linearly covering all the perimeter.. With a 60mm lens, I see "there" but I don't see maybe 50 metres of feet!.... Absurd, isn't it?.. 60mm lenses are correct if I am in a point and I want to see at a range (climatic conditions permitting!...) over there and I don't care about what I have near..

Third problem (for optical cameras) is about night in places with no light at all. How many cheap IR illuminators do you know illuminating effectively over 80-100 metres?...................... And high MP cameras looks better or worse than lower resoluted ones in the dark?.......

So, our suggestion, if the Customer wants to be assured of the detection 24h/365d, is to never go over 40 metres with optical cameras and never go over 80-90 metres with differential thermal cameras, with 30°-40° of HFOV. Of course a waste for good conditions, but definitely safe for ugly ones..

But if instead the Customer wants less cameras with longer range to pay less it's not a problem: he can just make sign to its thieves a deal where they promise to come to steal only in good weather condition, preferably by day, and if possible not passing too much close to poles....;))

This of course unless you are not in a place where it never rains/snows with the same temperature all the year.

Cheers,

Simone

(TechnoAware)

First, unfortunately there is a phisics of the propagation of light and heat: and this phisics teaches that it is completely different the attenuation in a dry temperate condition or through for example rain, heavy humidity, snow...

Testing Thermal Performance When Snowing has some good video of this...

Hi Simon.

So as I understand it you are saying do away with the anlytics encoder it is cheaper and better design to load an analytics app in the camera. Add benefit is less load on server and less bandwidth.

Secondly you are saying detction distance of thermal cameras need to be considered carefully in the worst case senario.

I find it interesting that you got 570m detection with a 60mm thermal, when the brochure says 1200m detction of a person, your example is half of what the spec sheet says. Further more you are saying that this specification is best case senario. So 300m detection would be more accurate (25% of stated detection range of manufacturer). Ontop of this you are saying the first 50m is actually a dead zone, so you actually end up with a usauble detction zone/area of under 250m with a 60mm thermal?

Is there a formulae to work out max detction range for thermal cameras when combined with video analytics? and what percentage should you subtract for moderately bad conditions such as rain when designing these perimeter systems? 30%

Yes, exactely.

Actually you focused the problem. In the brochures you read 1200mt for a persons, but there are 3 points that need to be focused very carefully.

First. If you take the Axis brochures (http://www.axis.com/it/files/brochure/bc_thermal_46702_en_1203_lo.pdf) and you read carefully at page 10, you will find out an "*" linking to a very important sentence written under the table: "According to Johnson’s criteria. The ranges vary in different weather conditions.". And Axis it's the only one, as I know, to write it honestly. And it's correct. The Johson's criteria it's just a theorical/geometrical one, considering only the angle, the resolution and the trigonometric calculation, without considering at all the propagation of the waves in the air and the visibility/constrast of the target. Do you remember the exercises of phisics we were making at school where it was stated at the beginning "assuming to be in vacuum, ...". Eh, this is the same.. 1200mt, 3 pixel detected, assuming to be in vacuum.. But unfortunately (or well, actually luckly for our life...))...), we are not in vacuum.. In the real life I think those 3 pixels are definitely invisible.......

Second. 1200mt, by Johnson criteria, but assuming a person 3 pixels.. Even if they were visible, would you be able to be sure that you are seeing a person looking at 3 pixels in the image?.........

Third. This specification is anyway though for the eyes, not for analytics. Of course, correctly, the cameras first of all are born to make the cameras, not necessarily for making do the video analysis to us. Thus, they assume 3 pixels are theorically enough to see a person (0,5x1,80mt). But for video analysis 3 pixels are definitely not enough!.. Or better, we can even detect just 1 pixel but with how many false alarms?? Everybody always speaks about detection, but they always "forget" to talk about related false alarms...;)) The video analysis is analyzing not only "what's changed", but also of course the dynamic information of how groups of detected pixels are coherent one to eachothers: this means that if I have 1-3 pixels you have a certain information; if you have 100 pixels of course you have another certain amount of information, thus a completely different level of confidence of the detection, thus much better performances (= detection with much less false alarms).

Cheers,

Simone

About 60mm. What I end up is that it's not correct to use 60mm for a perimeter. Anyway it's not suggested to put poles further than 80mt (if you want to work 24h/365d in any conditions). This means that, if you can't see before 50mt, the camera before must see since 50mt to 130mt (80+50).. Much better to use a 10mm or 19mm lens, reducing the dead point and anyway seeing until where it is possible to see in ugly conditions..

If, instead, you are not making a perimeter but a range (for example you are uphill with your camera and your goal is to see the entrance of a building downhill.. you care only about the entrance, you don't care at all what it happens along the hill..), so 60mm and more are correct to be used of course. But beware, in ugly weather conditions anyway you may happen to hardly see something over 80-90mt anyway!....

Unfortunately not, then, there are not practical formulas. Or better, yes actually there are and very clear: the phisics formulas of propagation of an electomagnetic wave through a space. But well, they are not that easy to manage maybe, because you would need to know the phisical attenuation coefficient of the space in that moment and the variables to calculate it are incredible a lot: if raining few, a lot, which size of drops, humidity, frequency of the wave, density of air, ...........). So you can rely only on our "empirical" experience of hundreds and hundreds of tests made on field in years of deployments anywhere..;)

Re-cheers,

Simone

Hi Simone

I have now looked at your company website and see your recommendation in perfect conditions for detction is 100 pixels or 10 pixels per meter at the furthest point, as well as 10 consectuvive frames before detction at 8fps.

This is the formulae I was looking for.

Would be nice to have a reference chart for each popular model of thermal with minimum and maximum recommended detection ranges. Thanks

We are friends with anyone...;) But surely we happen to work much more with Axis. But we worked and we do work also with Flir of course, with Samsung, with DRS, with SightLogics, .. What I can tell to you is that maybe a brand can have better or worse models, according also to the pre-processing filter they apply to "clean" and enhance the quality of the thermal image giving it to the video analysis.

Nevertheless, I can tell to you that most of all it's the basic sensor (microbolometer) anyway that leads in these kind of performances. And the basic sensor is more or less very siilar when we go to these kind of camera models thought and positioned for video surveillance. So in this kind of classification of cameras, I can tell to you that more or less the ranges that you should respect for safer detection in all conditions are anyway pretty similar brand by brand, some more some less but not that much.

Of course, correctly, to enter the market of video surveillance with timely competitive price, the manufacturers can't use the same microbolometer used for military purpouses..;) I mean, the sensor is exactely like an antenna: there are more and less sensitive antennas, able to percect signals from more or less far, and of course more or less expensive. When they tell to me about those cameras used for military coastal protection, able to detect in the fog at 2km and much more....... of course, but how much do they cost??...... You well understand that there is a difference if I try to see Saturn by a binoculars bought in Amazon for $50 and a high performing telescope bought for $5,000...;))

Surely, year by year, thanks also to the push of the market, we will see more and more sensitive sensors costing less and less. So I am sure that in the future these ranges will definitely increase (but then you will have to watch out again to resolution and lenses....;))...). But today, we are "here". Tomorrow we will see...

Simone

A lot of good (and curious) comments in this thread.

First off, IMO, the fundamental idea of "less is more" is certainly true here. Less devices in the field, less moving parts, less things to break and maintain is certainly the ideal approach. You have to weigh that against what is the overall cost of the various solutions. I think when you look at all the factors and the state of technology today, you'll find that there aren't very many acceptable options for edge analytics where the analytic comes from a 3rd party running on some other camera hardware.

Axis has some native on-board edge analytics, tripwire and coarse motion detection. This isn't a terrible choice for when you're looking for a simple solution covering a small area in a low activity outdoor environment. By low activity I mean minimal motion at all, not just low people activity. It's fairly cheap and serves a certain set of use cases OK. I don't think even Axis is going to tell you this is a head-to-head competitive option to more advanced products though.

You can also use some analytics software that is designed to be hosted on a general purpose camera. This option tends to be a little better, but the cost increases significantly, as does the setup and maintenance complexity. If you have full time operators that are highly trained in both systems they might be able to get by with this because they can continuously "tweak" stuff to slowly filter out false alarm objects over time. It can be high-maintenance, but workable.

Server-based systems are pretty rare these days, but there are a few that exist. They tend to fall between the "free" analytics, and the good edge-based stuff. Giving you decent performance in less demanding environments where the overall performance of the analytics isn't one of the top-3 measured criteria for the deployed system.

If you test all these options and look at total system cost, ongoing setup and maintenance costs, and false alarm rates you'll likely come to the conclusion that your best option is most likely to find a thermal camera optimally suited for the viewing task, and an analytics option optimally suited for the analyzing task.

There are lots of thermal camera options out there, one thing I always suggest people look at when evaluating cameras is the overall exposure control options on the camera. IMO, FLIR has some of the best knobs and dials in this capacity. Ideally, you can go with out-of-the-box settings, but like optical cameras, you may need to tune things for the specific environment to get the highest contrast image.

We (VideoIQ) can take an analog or IP feed from a thermal camera and add our edge storage and analytics with our Rialto A4 or I4 respectively. I think you'd find that relative to the other options discussed (and not discussed) so far, we'll end up giving you the best overall performance and flexibility in this scenario. Of course, I'm paid to say that, but I only cash those paychecks because I firmly believe in the product.

Simone also makes some good points in his comments, you need to factor your design according to what the customer really expects or is willing to tolerate. His numbers are a little more conservative than what I usually tell people, but it gets to the same point. You are essentially building a sensor network, if you try to push those sensors to the limit, then you are likely to suffer from decreased performance, just like any other sensor.

For thermal cameras we've been working pretty closely with FLIR over the years. I've come up with the following chart that shows typical semi-conservative coverage ranges/areas for various FLIR cameras coupled to our Rialto analytics appliance:

FLIR/VideoIQ Coverage Chart

This is neither a "best case" or "worst case" chart, it's designed to show typical deployed coverage ranges. This would mean you have decent object contrast, clear shot (you're seeing most of the person, they're not heavily obscured by foliage for example) and so forth. It's designed to have enough pixels on target to analyze things properly to intelligently IGNORE as well as DETECT. Also, this isn't showing all the FLIR models, it was focused heavily on the newer FC cameras, as those have been tremendously popular lately, but you can use the same basic HFOV data for other F-Series cameras not listed here.

Obviously, as you try to opt for longer ranges you run greater risk of various things in the environment preventing you from getting good clear shots over the entire distance. I don't recommend you try to get detection beyond about 1000' unless you fully understand the environment and all of the components of the solution.

The next question people usually ask is "what rule works best". In most of our deployments it's pretty simple: you draw an ROI (Region of Interest), and tell the system to alert you when a person is active in that ROI during a specified time period. It can be 24/7, or only at certain times or on certain days. Many people initially think that some kind of tripwire rule is best, but in many cases tripwire is a mask for a deficient product. It's used to reduce false alarms because the system is only looking at activity in a very narrow area, and then just looks for blobs of pixels crossing a line in a particular direction. This reduces false alarms by ignoring most of the image, but if something prevents you from seeing the person just as they are crossing the tripwire, they are "home free" in the scene relatively quickly. We are able to analyze the entire FOV continuously and accurately, so we don't need to do tricks with the rules to filter out nuisance objects. Ideally, we see the person as they enter the ROI and trigger an alarm, essentially acting as a tripwire arond the edges of the ROI. But, if there is some rain that night, or the person has snuck in next to an overgrown bush, and we don't get to "see" them until they are 20' into the perimeter (and past the point where any tripwire would have been) we will still generate an alarm. By making the analytics smarter, we reduce the need to overthink the rules.

There are also "behavior" analytics options, but that has been more marketing gimmick than reality. Customers know what they want... "Tell me when a person is entering my property". We don't need to develop a pattern of behavior in the area, we know what we want to catch, where we want to catch it, and when we want to catch it. Also, if there is lots of intrusions, you run the risk of those behaviors becoming "normal" over time. Similarly if the customer wants to run regular intrusion tests of their own on the system you want to ensure those activities don't contribute negatively to the system's profile of the scene.

Sorry for the long reply, I wasn't intending to write that much, but hopefully it gives you some additional things to consider. As always, if you'd like to challenge my recommendations you're welcome to get an eval unit as test it for yourself :)

Great information, Brian!

But one can't help but wonder

Of course, I'm paid to say that, but I only cash those paychecks because I firmly believe in the product.

Was that tounge-in-cheek, or do you actually have some uncashed checks in a picture frame?

Do you think then that perhaps the moral decline of western society can be traced inversly to the rise of ...direct deposit? ;)

I used to have a couple of uncashed checks, but they were souveniers of some stuff I did in the music industry back in the 90's.

I promptly cash, and my wife promptly spends, my VideoIQ checks :)

The direct deposit question poses an interesting thought exercise. I'll ponder it the next time I'm killing time on an airport layover and get back to you.

Thanks Brian for the insight. Your recommended ranges back up what Simone has said and is very usefull for design guidelines, still can't get over the difference opposed to the sales hype about the ranges. I stronly believe the thermal camera manaufactures need a standard for detction in their brochures that matches what is possible in real life scenarios as used by analytics.

Interested in your comment regarding monitoring a region of interest opposed to a tripwire, this certainly has set some alarm bells off with regards to some of the video analytics I have seen.

Additionally it appears both of you are indicating that not all thermals are the same and that certain thermals produce betetr contrast and lower noise than others and that this will also play a small role in the detction range.

Another important point seems to be that detction is one thing, but the number of false alarms a system produces is another.

Sounds like you've distilled most of the information pretty well.

Regarding the thermal cameras giving you detection information... I think that's very hard for them to do, since half (or more) of that depends on WHICH analytics product you're using.

It also depends on how you define "detection". I often tell people in our sales and tech training classes that I'd rather make a BETTER detection LATER (where "late" is a factor of a few seconds) than a BAD detection FAST. Many products seem to promote their instantaneous response to anything that moves as a benefit. And, when you hand select some demo videos for marketing purposes (which, let's face it, we ALL do), this can be seen as making a product seem like a clearly intelligent unit.

We're usually monitoring fairly large areas in these outdoor environments. We have, relatively speaking, ample time to analyze an object to get a clear opinion on it. I'd rather our product spend an extra 3 seconds when neccessary to give you a better classification instead of just calling every "tall" object that moves a person. It all comes down to your requirements and the approach of the product.

While the analytics industry hasn't seen tremendous technological advancement in the last few years, I think there are at least enough decent vendors that you can do some comparisons and make your own choices about tradeoffs, performance benefits, etc.

You're correct about the false alarms as well. NO system is immune to false alarms, the only way to give you zero false alarms is to also end up missing some events as well. I'll be long retired before anyone stands a chance of disproving me on that statement. What we try to do is reduce false alarms to the absolute minimum before running the risk of missing valid events. This usually gives you a valid:false ratio of 100:1 or more. But, it's also very situation dependant. If you have a site that sees 500 critters every day, and 2 people per week, it's going to be harder to keep that ratio in check, but you can probably still keep it to a couple of false events per day at the most.

Basically, if you come in to this with reasonable expectations and a clear understanding of things you can create a system that the customer will be highly satisfied with at a practical price point.

The important value in false alarms rating is not only how many they may be, but how much do they cost, their verification and management.

For example, a very good barrier system may give 1 false alarm each 6 month (if installed in very stable ground!!.. try on herb or grass and you will maybe cry...)? Ok, let's even assume this for example. But if there is no videosurveillance and remote connection, how can the operator verify the alarm? It receives a "call" or a contact, but then he needs to stand up, maybe to make some kilometer and go to see. And searching where? Searching what? Maybe it was a dog, maybe 4 criminals who as he comes they hit him... So, 1 false alarm per 6 months; but a very expensive one.......

If you instead have a control room, with operators and a good VMS installed (you wrote they have Milestone, perfect) receiving an automatic real time alert from a good video analytic, maybe you can more than 1 false each 6 months, but their cost is to raise the head, to see a picture in a screen and to say "oh, just a dog.. farewell"... So maybe more false alarms (maybe..), but the total amount of the whole cost is much much much cheaper...

Then, of course the problem is when you don't use real video analysis, but just simple motion-detection-based products. In that case, even with thermal cameras, in outdoor you may have several falses per hour and of course even managing them by Milestone it's going to be too much!.. But if you use real video analysis, with thermal cameras you can reach with no problems performances such as much less than 1 false per month (I always mean per camera of course)..

Cheers,

Simone