Video Analytics For Cross Line Detection In A Marine Environment

I’m working on a Genetec installation where the customer requires alarming/notification when vessels (sailboats, tour boats, kayakers, etc) enter a private cove off of an inlet. An Axis Q1932-E (35mm) thermal camera has been installed at the cove entrance aimed across the water. The distance is approx. 360ft from shoreline to shoreline and depending on the tide the camera is 18-25ft above the water. Depending on the weather, the Axis on-board crossline detection does not perform well with the waves in the field of view and we are looking for an alternate solution.

With the motion from the water in the field of view being the challenge here, does anyone have any experience with a similar application implementing analytics for crossline or object detection over water? Is there an analytics suite that worked well? Or equally as helpful, what didn’t work? Any suggestions and feedback would be appreciated.


That's a tough one since, as you probably know, water applications have greater accuracy challenges given the movement of the water. We haven't tested analytics in a marine environment so we cannot recommend.

Since you are using Axis cameras, have you scanned the Axis camera applications list? That certainly does not tell which ones will work well in a marine environment but those are possibilities for your cameras.

Thanks John, I've reached out to a few companies on this list (at least the ones I'm familiar with), inquiring about marine environment performance.

There was this project a long while back using Aimetis video analytics with thermal cameras in a marine environment, though I think we would have heard by now if it was actually successful.

Maybe at least they learned something valuable about marine detection that they incorporated in their present offering. :)

You may have a tough time with kayaks on the water and a thermal camera, it's unlikely you'll get good enough contrast many times.

I'd probably try to do this with 2 optical cameras if I were you.

The image quality is surprisingly good. Unless the kayaker is at the very far side of the cove, there is (imo) sufficient detail/contrast to detect. We thought about using two cameras with an 'AND' condition for alarming; however in poor weather I don't see why both wouldn't be in a constant state of alarm.

We also tried reducing the contrast and gain on the camera to effectively "smooth" out the waves, this works until the weather changes.

Sounds just like a project Sagy Amit did with FLIR thermal and IOImage analytics for a power plant. If someone can get him off a constant sabbatical maybe he can discuss.

My ears were ringing Greg Cortina...All the way in Fiji :P

"All the way in Fiji"

Clearly doing analytic testing in marine environments ;)

Any advice here?

Under 400ft distance, could you do a beam break sensor of some kind?

We considered this too, however with the changing tide and the less than desirable shoreline, the device would have to mounted quite high above the waterline (close to the camera height at 18-25ft). As there are many small vessels (lower than 18ft) that enter the cove regularly, this wouldn't provide consistent alarm notification.

I have a couple of minutes so I'll lend some experience with thermal and video analytics. I am with FLIR and I have a significant background in video analytics.

Thermal was the right way to go. I don't know the scene so I don't know if you are looking at waves coming in, waves across, small waves (tides) or crashing waves (Hawaiian North Shore). All of that would matter for where you would want to set up your detection.

A traditional trip wire is not what I would suggest for this application. You should have an analytic that uses detection areas, has sufficient time to classify the object before entering the area and if possible, a length of time in the area before alarming. This would make the detection more reliable. Not perfect, more reliable. I didn't see what the acceptable false alarm rate is or missed detection requirement. Those matter if you want to meet the customers expectations.

Adjusting contrast and such on a thermal camera is nothing like what you do with a visible camera. As you noticed, the weather significantly impacts the image. If you are going to adjust anything beyond the Region of Interest you should have someone with thermal knowledge help.

Greg (just a FLIR sales guy with some background.)

Thanks for your input Greg. With regards to the waves, this is a small cove off of a protected inlet (PNW) so small waves/ripples and in rough weather some chop/whitecaps. No crashing waves. I agree with your suggestion for detection method, it sounds like that would improve reliability... is there a specific analytics product you'd recommend for this?

Dear "Undisclosed1",

As Greg mentioned above, we had a similar challenge. Luckily for you, I am currently a beach bum and I can be unbiased.

As with any Video Based technology I suggest setting the expectations first:

1- Especially frequent in marine environment in CA, the marine layer, prevalent throughout the spring (May grey, June gloom..) will make any camera system useless.

2- In order to successfully establish virtual rules, the physical rules have to be defined. In my specific case, the exact azimuth/point at which a kayaks became a threat had to be defined since the lagoon was partially public and was used as an oyster farm. Thus, it is easier to establish low False Alarm Rate and High Probability of detection when the main asset is defined. this could be, the shore line, a pump, an intake, etc.

3- Video Analytics works best with X axis motion Vs. Y axis. in other words, speed and accuracy is better with motion across the Field of View and not toward or away from the camera. In my case there was no physical way to mount the cameras that way, so we mounted the cameras very high, at 90 Feet!

4- As the Video Monitoring outfits out there can attest, Video Analytics can NOT replace the human guard. The final decision on a possible threat has to be made by human. In my project I used a dual, fixed to PTZ rules where the PTZ would automatically "slew to cue" to minimize the team workload.

If this did not deter you from the VA solution, I suggest the following:

Contact a few of the leading companies that specialize in VA. You can contact me directly for suggestions.

let them compete on the project. It is their job to provide you with a free pilot demo product for a set time. If they are serious, they will also offer support in setting up the rules.. In my 10 years of specifying Video Analytics I never shied away from an opportunity to establish my product superiority ;)

In return you can offer them (beyond the sale $$$) a write-up, case study or any sort of public recognition.

Minimizing the false alarms is a process that will take some time. The rules will need to be tweaked a few times.

Hope this helps.

Sagy

...let them compete on the project. It is their job to provide you with a free pilot demo product for a set time...

We were just talking about this... RFPs: Say No To Specs

You all are too much....

Free or not, I agree with Sagy's more basic point that it has to be proven on site as the class of technology is nowhere near mature enough for this application to simply turn on a box and have it work.

Do you think a manufacturer is likely to take on the risk themselves for a project of this size?

"if you have a winning product, give it away for free..."

Every dot com giant started as a freeware. LinkedIn, Facebook, IPVM :P

When the product proves its value, people are happy to pay for it.

Lol, but none of those companies ever sent engineers on a customer site to do free work:)

Based on U1's initial description, unless there's potential for much larger future business, I am not sure how much manufacturers could give away for free.

Hmm. I like that video! I think there will be peace in the middle east before that becomes reality in our industry ;)

I won't recommend a specific analytic but I will say most manufacturers will work on an evaluation system. They don't like it, but they will. Especially if they get to use the site as a reference. I avoid recommendations not because I don't believe there is a product that will do a good job, but it's all the support in general and that can be location sensitive. Ask them all with a good understanding of what you are doing and see who bites.

I have seen a few Marine examples from Aimetis using thermal cameras, if you ask the local region manager I'm sure they will share some of the video examples that I have seen work well.

I know the beam scenario has been raised, but why not do this given the height / tide / vessel size is an issue?

Stack 2 sets of point-to-point beams (2 or 3 perhaps) - this will be the vertical dimension. Place another set at some distance away (about the length of a kayak perhaps) - this will be your horizontal dimension. Add all into an intruder alarm panel. Programme it so that 2 of the 3 detectors in a stack (vertical dimension) AND both the beams on the horizontal dimension have the be tripped for an alarm to be generated.

Alternatively, do a beam array like suggested above AND link in an output from the analytic so both need to be tripped in order to generate an alarm - best of both worlds!

Depending on the budget, there are radar and sonar specialty systems just for this application:

L-3 Maritime Surveillance & Security Solutions

As follow up to this post, I wanted to share the results of our testing. We sent sample video clips to a number of analytics companies to process and then reviewed the results. The results we received varied. It was clear that some products were not suitable for this application right off the bat (based on the results of the processed video or Genetec integration limitations). We decided to setup a proof of concept with a couple of the more promising results. Analytics products considered included AgentVI, ACIC, Aimetis, ASV, Kiwi, and UDP/VCA.

Based on the proof of concept testing, we selected the Kiwi solution for this application. We worked closely with their system engineer and after several iterations of configuration adjustments were able to achieve “reliable” alarming when a vessel enters the cove. I use “reliable” because the solution is not 100% accurate. I don’t have a percentage of false/missed detection vs. accurate detection, but given the variables within the scene (changing weather, tides, chop, etc) 100% accuracy was never the goal. The Kiwi solution provides consistent notification to Control without inundating Control with false alarms. The solution has been running for about 4 months now and overall the client is very pleased with the performance. Finally, the cost of the Kiwi solution was competitive with other solutions considered.

1, thanks, very helpful follow! Good to hear that Kiwi worked well for you.

How much better was Kiwi than the others? In what ways?

It’s difficult to quantify “how much better”, as we weren’t able to achieve anywhere near “reliable” detection with any of the other solutions. The other solutions seemed to either a) fail to detect altogether, or b) inundate Control with false alarms to the point it had to be disabled as it was interrupting operations. There really wasn’t a middle ground, each indicated the water in the field of view as the challenge (as we suspected).