Casino Surveillance Pro Interview: James Lathrop

By Dan Gelinas, Published Feb 15, 2019, 09:16am EST

James Lathrop [link no longer available] has been working in casinos for almost 25 years. During that time, he says he has held "just about every job you can do in the casino," including Surveillance Technician, Surveillance Observer, Surveillance Manager, Security Shift Manager, and Casino Shift Manager (he's currently Shift Manager at Hobey's outside Reno) throughout various casinos in Nevada, Oregon, and Washington. He has seen a lot of change. One thing that has remained consistent, however, is the need for good security and surveillance to create an audit trail and to protect against loss from theft and cheating.

In this note, we interview Lathrop on his experiences working security and surveillance in the high-stakes world of casinos and cover the following topics:

  • Lathrop background
  • Early Pivot3 fail
  • Catching the casino bad guys (Operations)
  • Technological influencers
  • Facial recognition on the rise
  • Prevalent brands in casinos
  • Finding the right security partners
  • Integration: The Holy Grail

**** ***** **** *******, who ****** * ******-************-******* blog****** ************ ****** ***** *** ***** his ****** ************ *********** since *** ***** '***.

**********

****: *** **** **** you **** ******** **** surveillance ** *******?

***** ****. * ********** moved ** **** ** 1985 **** ******** **********, just ******* * ****** to **** ** *** casino ********. ***** **** I've *** **** ***** every *** *** *** do ** *** ******. I ***** ** ** surveillance ***** ******* ** a ******** ***** ******* for ******* *****. **** I ***** ** ****** and ******* *** ************ department ** * ***** tribal ****** **** ******* Falls. ***** ****, * moved ** * ***** tribal ********* ** *** state ** ********** ***** I *** * **********, installer, ********* ******* ******* and ****** ************* ** what ********** ****** * very ******* ******* ******. Counting *** ****** **** I'm ********* ******* **, where * ** *** Casino ***** *******, * have ****** ** ** different ********** ** ******, Oregon *** ********** *****.

****: **** *** *** brand ** **** ******* digital ****** *** **** was ** **** ** manage **?

**** * *** ******* in ************ ** *** all **** *** *********, but **** *** **** the *** ** *** dodo **** ** *** early *****. * *** been *** **** **** at **** ****** ****** called ********** ******* ******. ** **** an ************ ****** *** the ******* ********* *** for *** *****. ********** else *** * **********. We ****** *** ******* and ******* ****** **** Dell. *** ** *** cameras **** ***** ******. It *** ****. ** had ** **** ******* cards. ***** **** **** IndigoVision. **** *** **** of *** ***** ** the ****. ** *** an ********* ****. ** course **** *** ** years ***—******** *** ***. We *** * ***** of ********* ******* *********. We **** * **** a ******* ****** ****** who *** **** ** a ******* **** **** 6 ******. ***** ******* and ****** * *** because **** *****’* ******** for *** **** ** application. **** *****’* ******** to ** *** **/*/***. So ** *** * lot ** ******* **** that. ** ********** ****** those. *** *******, *** of *** **** **** Pelco ******* **. ** bought ******** *** ******** for ***** *******. ** had **** *****, *****, Honeywell. ** *** **** a **********. * **** when ** ******* *********, 'I ***’* **** * science **********, * **** something ***** *** ******.' But ** ******, ********** then *** * ******* experiment. ********** ** *** in *** ************ **** was *******-********. ** ********* went ***** *****, ** literally *** ** **** down *** ****** *** ask ****** ** *****.

Catching *** ****** *** **** (**********)

****: *** ***** ** your ********** *** ***** surveillance ****** ***** ******* cheating ** * ******?

*** *** ***** ******* almost ** ***** ** you **** ** ******* to ****. *** *'* not ******* **** *********, like ******** ** ********* players. *****'* **** * lot ** ******** ***** and **********. **** ********** for *******. *****'* ** old ******: ***** *** two ***** ** **********--***** that *** ******** * little, *** ***** **** are ******** * ***.

****: *** ***** ********* types ** ********* **** fit ********* ***** ****** the ******?

***** ** * ************ solution *** ***** *** almost **********, *** ** takes ** *****, *** a *********** ** **** upper ********** ** ***** it. ***** *** ** a *******, ***** ************ is *** * ****** center. ** *****'* ******** any ****** *** * casino. **** *** **** to **** ********* ********** is **** ************ *** be **** ** * tool ** ***** **********, to ***** ***** *** enhance *******. ***** *********. Food *** ********. ******. Security *** ******, *********. Not **** ***** ***** and *****. ** *** do ** ***** *** can **** **** ************ into * ********* **** keeps *** ******, ******* they ** ********** *********, or ******** *********, **** ever **** ****** ** your ******** ** ***** with. *** **** ********* has * ******** ****** on *** ****** **** of * ***********.

****: *** *** ****** most ******* **** * security ******** **** * security **********? ** *** they ********* ** ** or ********** ** *** security?

*****, ** ******* ** the **** ** *** facility. *'* * **** believer *** ******* *** IT ********** ** *** away **** ******** *** surveillance ** ********, ****** because ** *** ********* conflict ** ********. * would ********* ****** ** in-house ****, ** **** of *********** *** **** just ** **** ***** IT ** **** ** about *******.

****: **** ** ******** among ************ ********* ****?

* *** ******* ** the ******** *** *** state ***’** ********* **. Washington ***** ***** * worked *** ***** ** the ******* *****, ***** was *** * *** of ******** ******* **** paid **** ****. **** were ****** ** *** experienced ****** ********* ** move ***** **** ******. It’s ********* **** ** Nevada ***** ** *** can *** ** ******** past ***** ** *** probationary ****** **’* **** “Woo-hoo!” *’** ***** ****** managers ** ****** *** have **** **** ** being * *** ****** because *** ***** ** actually ****** *** **’* less ******.

****: **** ********** ** your ******* *** ********* at ******* **** ***** (i.e. ***** ** *** fps)? *** ** **** necessary, ** ***** ***** been ******?

** ****** *** ****** is ****** ****** **********. If *** **** * better ******* *** *** always ******** *** **********. The ********* ***** *** ones **** ******* *** table ***** *** *** money. **** ** ***** cameras ******* *** ** recorded ** ** *** and ** * ***** resolution ** **** ******* space ** **** **** drive *****.

****: *** **** ** need ** ** **** high? **** ** **** need ** ** ****** for *** ******?

****** **’* ** * situation ***** *** **** to ******** *** *** cards ***** *****, * can’t ******* * ****** for ** ** ** higher. * ***** *** human *** **** **** at ****** ** ****** per ******. ** *** had * ***** **** or *********, ** ***** help ** *** *** a ****** ***** ****. I ***** *** ****** we ******* ** ** FPS, ** ** *** a **** **** **** the ****. **’* **** TV *** ********* **. As *** ** ****-**-***-***** stuff ****, **** ****** drop **** ** *.* FPS ** ****. **’** drop ** * ** .5 *** ** *****’* no ****** ** *** scene. ** *** ****** mandates *** **** ** the *****. *** **** your ****** ***** ***** eating ** **** ********* and ******* *** **** money ***** ***** *****’* money ***** **********.

****: *** **** ****** of ********** *** ***** into **** ***/******/*******/***** ************?

*** ****** **** ** much ********** ** *** can ******. *** ** takes ** *****, *** a *********** ** ***** it.

****: *** **** ******* is * ****** ******** responsible *** **********? ** that *********? **** ** the ******** ****?

******* ** *** ****** of ******* *** *** size ** *** ********. Most *** ********** ***** up ***** ****** ******** between ******** *** ************. surveillance **** ****** *** the ****** *** ******** to ** **** *****, a ******** ******* ***** handle ********** **** ********* back ** *****, ******** doors, ******* ****, ***.

****: *** *** ****** surveillance ********* ****** *** advent ** ********/** ************? What *** *******?

*** **** ***** **** has ******* ** ***** quality, ** ******, ***** with *** ** ******* and ****** *******. **** of *** '********' ** still *** ****.

****: **** **** **** footwork ******?

*** ******** ** **** I ***** ** ** knowing *** ** ***** cards *** *** ** spot **** ********, *** to **** ********. * surveillance ******** ***** ** have ***** *****. *** have ** ** ***** of *** *** ******* on *** ******. *** players **** ****** ****** at ********** ** **** what **** ***** ** doing.

Technological ***********

****: **** ************* *********** in ************ ***** *** say **** **** *** biggest *********** ** *** vertical?

*********, ****** *********** ******** is *** ******** ** the ****, **** ** a ********* ** ********** crime *** **** ** identify ***** *** **** already ********* ******. ***, it *** *** ********* for *****, *** ** you **** ***** * person's ***** ** *******, then ***** **'* *** such * ***** *****. I **** **** **** a *** ** ******* plate ****** ******* **********, such ** ** * parking ********* ******** ** exit ***. *** * think ***'** ***** ** see * *** ** properties **** **** ***** parking *** ***** ***** using ***** ***** ********** as ****. **’** ****** toward ********** ** ****** scans ** ******* ***** to ***** *** ****** to **** ****** ** they **** ** *** the ****** ********* **** know *** *** ****** is **** **** **** in. **’* *** ******** intelligence ** ***** *** casino ** ***** ** the ********’* *****. ** to ** ***** ** it ** * ******** walks ** *** *** drunk *** *** **** a *****.

****: '* ******'* ***** to *******': ** * casino ** ***** * right ** *******? ** in *** ****** *****, really. *****'* ** ********** expectation ** ******* ** there?

** ******, *****'* ** right ** ******* ** a ****** *****. **** I'm ******* ***** ** the ********* *** *****. As ****** *********** ******** becomes **** *************, ** will ** ******, ********** if ***'** ***** ********** intelligence **** ********** **** your *******' ****. **** data *** ** ** somewhere. *** ***** *** everyone ***** ** ***** that **** **** ** a ********** ****** *** times ** *** **** calendar ****. ***, **** places ***** **** **** kind ** *********** ************ but **** ***** ******* and **** ********? *** I'm ****** ** **** the ********* *** ***** is *****, *** ** has ** ** ******* against.

****: ** ****** *********** common? ******* *** ****** offenders ** *** ******** side? ** *** ******** who ***** * *** so *** *** ***** them ***** **** **** walk **? ** ******** who **** ** ****** one ********** ******** ******** or **** ******?

****** *********** ** *** that ****** ***, *** its ********** ** ***** recognized **** ***** **** by **********.

****: **** *** **** used ****** *********** ** a******.**** **** *** *** did ** ****?

* *****’* **** ** yet, *** ***** *** casinos **** ***. *** Winn, *'* ****** **** is ***** **. **** in *****—**’* * ** 4 ***** *** **** of *** *****—****’** ***** it *****, *’* *******. Facial *********** ** **** going ** **** **** and **** ** ********* speeds ********. ***’** ******** see ********** ***** ** share **** *********** ***** themselves ** **** ** a **** ********, ***. If * *** **** kicked *** ** *** casino *** **** ********, it’ll ** **** ******’* computer ******* *** **** and ******* *** ******* on ** ***** ***** casino.

Prevalent ****** ** *******

****: *** ***** ** you ******** *** ****** and ******** **********? *** is **** ******* ******* (i.e. *** ** *** evaluate/judge *** ******* *** software ****** *** ******** installed ******?)

********** ** *** ******** is ******** ** *******. I ***** *** ****** really ******** *** ** your ******** ** ** annual ***** **** *** submit **** ****** *** the **** ****** ****. However, *** ******* ** it ** **** ***'** not ***** ** ** able ** *** *** equipment ***** **** *** you ** ***** **** to **** **** *** things *** *** *****, to ** **** ** pitch **** ** ********* as * ********* ******** to * *******.

****: ** **** ********** what *** *** **** ubiquitous ****** ** ****** video ************?

****, ** ******, ***** is *** **** ******** when ** ***** ** PTZ ******* *** ****** (old ******) ****** ********. Their *********** ** *********** bulletproof. * ** *** as ********* **** ***** digital ********* *********, *******. I ** ******* ** a ****** ****** ************, out ** *** ******.

****: ************: **** ** it ***** ************ **** makes ** * ******** or *****?

*** **** ** *** for *** ********, *** the ******* ** *** administrator ** ********* ****** like ***** ********. ** is **** **** ** add ******, ****** *** react ** ***** ******. Every ****, *******, ***** are ****** *** ******* in *** ****. **** go ** *** ** Vegas *** *** **** usually **** ** ** more *** ************* **** coming **** *** ******. The *** ** ** find ** ********** ****'* experienced ****** ** ** it *** *** *** and ****** ******* *** with * ******* ******** with ***% ***********, *** a ******* ********** **** requires ** ** ****** to *** *** **********. The ******* **** **** digital ********* ******* ** that *** *** **** software **** *** *******, your *******, *** ************ from *******, *** ***'** storage ****** *** * network ****** **** *** another. *** ** **** have ******** ******* ********, everyone ****** ******* ** one ** *** ***** manufacturers.

Finding *** ***** ******** ********

****: *** ** *** select/evaluate ******** *********** ** work ****?

**** ***********, **** *** contractors, ******* ******* * reputation. * **** ** get ********* *** ***** to ******** *******, *** at ***** *** *** was ************, ** ********. It's **** **** ** interview ******* **** ***'** hiring ** ********. *** them ** **** *** about * *********** *********, or *** ***** **** flat-out ******. *** **** how **** ******** ** and *** **** *** assure *** **** ** will ***** ****** *****.

Integration: *** **** *****

****: **** **** * security/IT ******** **** ** consider ** ** ****** security ********?

******* *** *** ********* an ******** ****** ** starting **** ******* ** a ***** *** ********, the **** ** *** game ** ***********. *** Holy ***** ** ** be **** ** ********* everything ********, ********* ******** security, ****** *******, **** and ********** *** **** point ** **** ******* with **** ****** ********.

****: '*********** ** *** Holy *****': ** *** feel ** *** **** achieved? ** **, ***** and **** *** *** specifics?

**** ** *** ***** properties, **** ** *****, have ********** ***** ****** in ** **** ** they **** **. *** everyone ***** ** **** time *** **********, *** example, ********** **** * surveillance ******. *** *** every ******** ***** ** capture *** ******* ***** number ** ***** ****** vehicle **** ****** ** its ******** ** **** into * ******* ******. But **** ** *** big ******, **** **** the *********, *** ****** well **********.

Comments (40)

John Honovich

IPVM | 02/15/19 02:18pm

We'd like to do more interviews with operators going forward.

We don't generally do interviews, mainly because we want to avoid sales pitches (Axis salespeople urging not to buy cameras, Hikvision salespeople saying the Chinese government is no big deal, etc.)

However, operators / end users would be useful to give a better sense of what their needs and challenges are. If you are one and are interested or know of one, let us know, either in the comments or emailing us at dan@ipvm.com

Agree: 19
Disagree: 1
Informative: 5
Unhelpful: 1
Funny
Avatar

Ari Erenthal

Chesapeake & Midlantic
In reply to John Honovich | 02/15/19 03:57pm

I agree. I'd love to see more interviews with people who actually use our technology to do their jobs- burglary detectives, amusement park operators, and security managers of all types.

Agree: 16
Disagree: 1
Informative: 1
Unhelpful: 1
Funny

Undisclosed Integrator #1

02/15/19 02:40pm

Interesting interview. I would have liked to see a question regarding PTZ usage analog vs IP. Is he and most casinos still using analog PTZ's due to latency issues with IP PTZ's or has he found a solution for this.

 

Agree: 3
Disagree
Informative
Unhelpful
Funny
Avatar

Dan Gelinas

IPVM | In reply to Undisclosed Integrator #1 | 02/15/19 03:26pm

hello UD1. I'll pass those questions on to him and relate what I hear back.

Agree
Disagree: 1
Informative
Unhelpful: 1
Funny
Avatar

Ryan Karpilo

In reply to Undisclosed Integrator #1 | 02/15/19 03:54pm

"Is he and most casinos still using analog PTZ's due to latency issues with IP PTZ's or has he found a solution for this."

Great question!  Coming from a Casino Surveillance background, I see zero comparison between Old School Analog PTZ controls vs IP PTZ controls.  Going back to the old school days of Honeywell and Matrix, PTZ controls were always flawless and smooth.  From Pan and Tilt to Zooming, always perfect.

Now with USB joysticks, personally and professionally the experience is terrible.

Agree: 5
Disagree
Informative
Unhelpful
Funny
Avatar

Carl Lindgren

In reply to Ryan Karpilo | 02/15/19 05:03pm

Ryan, 

The property I used to work for recently completed a changeout of most of of their analog Pelco Spectra IV PTZs for IndigoVision IP PTZs (with a few Axis P5635's thrown in for good measure). The sole exception being the Parking Structure where the cabling cannot easily accommodate IP. They're still using Spectra IVs out there but before I left we did discuss IP replacement options.

Although there are a few "tricks" to getting smooth control of the IV IP PTZs, all of the users are very happy with the results.  One trick is to change settings to actually slow down input response from factory default, which tends to be too sensitive to joystick motions compared to the Pelcos. 

Of course, a key factor is latency. The PTZs and VMS must exhibit latency that is substantially under 200ms or control can be difficult. One surprising result of our testing of systems in 2012/2013 was that there were large variances in latency. The best systems had <150ms while the worst had >300ms. One actually had >500ms. The running gag was that with that combination, we would have trouble following someone using a walker.

So the conclusion I would take is that in applications where real time control of PTZs is a major factor, it pays to test that functionality before buying the system.

Agree: 2
Disagree
Informative: 4
Unhelpful
Funny
Avatar

Ryan Karpilo

In reply to Carl Lindgren | 02/15/19 07:22pm

Great Reply Carl,

That time frame of 2012/2013 was actually the last time I even looked at Latency of IP PTZ's.

The tricks you mentioned are spot on and still did not feel the same to me.  My expectations may be way off.

I'll have to do some current testing and see how they feel in 2019 and GO DEEP using Wireshark to watch the latency. 

Agree
Disagree
Informative
Unhelpful
Funny

Undisclosed End User #3

In reply to Carl Lindgren | 02/17/19 06:35pm

Have you all considered HD-Analog wherever you had/have Coax?

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Carl Lindgren

In reply to Undisclosed End User #3 | 02/18/19 04:52am

U#3,

I looked at the various flavors of HD, including HD-SDI, HD-TVI, HD-CVI and AHD over the years and the biggest problems I encountered were the lack of and/or price of suitable encoders. We tested one "encoder" in 2014 that was actually just a DVR with no HDD. It wasn't ONVIF-compliant and they had no compatible driver for our IndigoVision Camera Gateway so we talked to the mfg's support and they were unhelpful. 

I assume things are a bit better now but I saw no need to do any more tests because we found other ways to get around the distance limitations of Ethernet and our coax and analog UTP infrastructure. 

Although there are many other solutions, we liked the price vs. performance of Vigitron products for IP-over-coax and PoE high power insertion. We also used ComNet IP-over-single pair products when other signal transport methods were unavailable. 

 

Shortly before I left, we ran into another "problem" I'm sure everyone has encountered at one time or another: not having spare cable(s) to add a camera in one area or another. I did some research and found this product: CyberData 011187 3-port PoE-powered Switch. 

https://www.cyberdata.net/products/011187

We tested it with a number of camera combinations and, assuming the source PoE is sufficient to power the 3 devices (2 cameras + approximately 2W for the switch itself) it works flawlessly. The switch itself has a 28W input power limit so you wouldn't be able to connect two typical 20W PTZs but we have powered one indoor PTZ + one fixed camera from an 802.3at source (Dell 2000-series edge switch or Vigitron Vi2216 midspan).

Agree
Disagree
Informative: 3
Unhelpful
Funny

Undisclosed Integrator #5

In reply to Undisclosed End User #3 | 02/21/19 12:42am

No. This is almost never done in a Casino due to failover/redundancy along with price. If you drop an hdover coax box you're going to drop 8-16 cameras on the floor. AFAIK there are no hdovercoax boxes with redundant power supplies. For a few dollars more/parity on price you can put in an IP over Coax Converter and either DIN rail them in high density or use a converter in the rack with 16 slots with a failover power supply that were made specific to gaming. 

Agree
Disagree
Informative
Unhelpful
Funny

Shannon Davis

In reply to Ryan Karpilo | 02/18/19 01:39pm

I agree with you 100% on this. When I started switching out systems to digital that was one of the big hangups I had. Especially for the Pelco C control. Depending on the firmware version, distance and such these are still hard to control sometimes. Although when it came to joystick versus mouse most have preferred the mouse. The biggest part being they can drag the mouse over the object they want to see and the PTZ zooms right in.

In some cases if the camera had RS485 hooked up we would leave the joystick and other devices in place as long as we could get parts so the customer could still have that precise control over their cameras.

Agree
Disagree
Informative
Unhelpful
Funny

Undisclosed #2

IPVMU Certified | 02/15/19 03:40pm

Great interview, Dan.

A couple supplemental questions for you or James:

1) How many cameras are there in the larger systems you’ve been involved with?

2) Is virtually 100% of the gaming floor covered by at least one view?

3) Are panoramics used for an overview of the gaming floor?

4) How easy is it to track a suspected cheat as they move around the casino?

5) Are there any laws about minimum frame rate?

6) Is latency ever an issue with IP technology?

7) Are you banned for life from gambling in your casino because of your intimate knowledge of the system?

8) Do you know the iconic end-user, casino contractor, Pelco pundit and fellow IndigoVisionary Carl Lindgren?  You guys should have a beer or two at ISC, if you haven’t already :)

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Dan Gelinas

IPVM | In reply to Undisclosed #2 | 02/15/19 03:46pm

Thank you UD #2. Likewise, I'll pass your questions on.

Agree
Disagree: 1
Informative
Unhelpful: 1
Funny
Avatar

Carl Lindgren

In reply to Undisclosed #2 | 02/15/19 05:51pm

Undisclosed #2,

I can answer for my former place of employment. They are in the midst of a major expansion of the property. Much of the added floor space is back of house like restaurants, convention/meeting/live entertainment, a hotel and a spa, which typically require less camera coverage.

1) That said, the current system has approximately 1,000 cameras and approximately 800 will be added initially. I imagine that number will continue growing as they get a feel for what areas require additional coverage,

2) In that venue, we had >100% coverage since pretty much every square inch of the gaming floor had somewhat overlapping fixed camera coverage with strategically placed PTZs in addition. We used the theory that at the least, a fixed camera always covered every area and the operators could also, if needed, zoom in on activity anywhere. It wasn't cheap but over time, we rarely had an incident we had to deem "inconclusive".

3) We experimented with 180 and 360-degree cameras but found their coverage lacking. Having so few pixels viewing any given area made it difficult to identify activities very distant from the camera location. In addition, the way casino floors tend to be laid out (at least at that venue), camera views would be blocked by slot banks, slot "toppers" and signs for most camera locations so we used 3-9mm cameras with the zoom at its widest located where they could see the most unblocked area.

4) Tracking the movements of people is something any good operator learns early on. Maps can help that and IndigoVision's map function is a great help in that area. One caveat is that someone on the Surveillance Technician side must regularly update the maps since casinos constantly move things around.

5) There are and aren't laws about frame rates. The NIGC MICS (National Indian Gaming Commission Minimum Internal Control Standards) specifies 20fps for certain critical applications. That is a holdover from the days of "Real-time VCRs" that recorded single "fields" (only the odd or even lines) of video at 20fps for up to 24 hours on a t-160 videotape. Nevada rules just call for "real-time" recording of the same critical areas like count operations and table games. Many casinos in Nevada consider 15fps sufficient to meet that rule.

When we decided to go digital in 2003, many VMS manufacturers recommended 15fps or less but we operated under NIGC MICS so, with no digital option of 20fps, we chose 30fps. We also decided to run ALL cameras at that frame rate due to the ambiguity of the MICS when they used the term "Sufficient Clarity" in many sections after defining that term as at least 20fps in the first chapter.

6) As I stated above, latency can be an issue with PTZs and is something that should be evaluated prior to choosing a system. Unless a system has huge latency, it doesn't matter much with fixed cameras. One caveat is that a system should be able to switch from "Live" to "Playback" quickly and easily. At least one tested system was plagued by the spinning hourglass on initiating playback of many cameras. That type of latency was a deal breaker for us since we often have time constraints - especially when monitoring table game play.

7) No. Or at least I was not "banned for life" from my former employer's property. In fact, a couple of days after my retirement party, I joined their players' club and gambled there.  As a Gaming Commission employee, I was not allowed to gamble (or drink) anywhere on the property while employed there. The day I got my players' club card, I sat at the bar and played a bartop slot. That was the first time I gambled there in over 21 years (since the day of my job interview).

80 "Pelco pundit"? LOL

Agree: 1
Disagree
Informative: 4
Unhelpful
Funny

Undisclosed #2

IPVMU Certified | In reply to Carl Lindgren | 02/15/19 11:03pm

As I stated above, latency can be an issue with PTZs and is something that should be evaluated prior to choosing a system.

Yes, on the whole, I have found control of IP PTZ’s pales in comparison to analog PTZ’s, mainly due latency.

And with an IP only connection, there is latency possible from three sources,

1) encoding and network transmission of the joystick

2) decoding and translating the request by the SOC into analog servo motor control

3) encoding and network transmission of the image 

So when you say

The PTZs and VMS must exhibit latency that is substantially under 200ms

do you mean 200ms from the time you first move the joystick to the time you see the first movement on the screen?

If Indigo has 150ms total round-trip latency, that is quite impressive.

I would have been happy with 400ms.

For my setup, I ended up using the aux analog SD video feed with direct serial rs-485 control to a dedicated analog monitor w/ analog joystick, but the HD feed went to the VMS.

I was thinking maybe of trying an Analog HD (or higher res) PTZ to see how quickly they react.

 

Agree: 1
Disagree
Informative
Unhelpful
Funny
Avatar

Carl Lindgren

In reply to Undisclosed #2 | 02/16/19 02:59am

Undisclosed #2,

I developed a simple system to test camera-to-monitor latency, which is what I used. I could think of no real way to test joystick-to-camera latency accurately so I used camera control overrun to estimate if that path contributed much to overall latency. That test consisted of fast taps on the joystick and guesstimating if response time, both on initiation and on release, was an issue. 

My method was to aim a camera at a digital stop watch that displayed time to the hundredths of seconds held next to a monitor displaying the picture from that camera. I then aimed another camera in such a way as to capture both the stopwatch and the monitor displaying the stopwatch and recorded that camera at full frame 30fps.

By freezing a frame on playback of the second camera and comparing the time displayed on the stopwatch and monitor, the difference would be the latency. It took advancing a few frames to reveal usable images from both simultaneously but the results were repeatable and pretty accurate (probably within 1/2 frame or about 15ms).

Still, everyone agreed that the systems which exhibited the lowest camera-to-monitor latency were sufficiently responsive for our purposes. In fact, further testing with outdoor cameras following moving vehicles confirmed an operator could follow a vehicle travelling perpendicular to the camera at 25-30mph. That is good enough to follow people inside.

Agree
Disagree
Informative: 2
Unhelpful
Funny

Undisclosed #2

IPVMU Certified | In reply to Carl Lindgren | 02/16/19 04:50am

...probably within 1/2 frame or about 15ms).

Assuming you mean 150ms, that’s still some awesome speed.

Was the control to the camera via IP or serial?

btw and imho, variance is just as bad as sluggishness.  I’d rather have slightly slower ptz that responded at the same speed.  You start to anticipate the overrun after a bit.  But if it’s all over the place, it sucks.

So maybe “IP video killed the PTZ”?

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Carl Lindgren

In reply to Undisclosed #2 | 02/16/19 05:44am

No, 15ms, or rather 16.5ms if you want to be exact (since at 30fps a frame of video is 33ms). 

The point is that I often had to step through a few frames to be able to see both time displays simultaneously. I assume the stop watch also refreshed its screen at some unknown rate and that the cameras, monitor and display all had to be in sync to be able to see both times displayed.

We tested both analog PTZ cameras and IP PTZ cameras. The analog PTZs were Pelco Spectra IV's.  They were hooked up to encoders via coax for video and RS422 for control. If a manufacturer provided an encoder, we used that. If they didn't, we used Axis and/or Bosch encoders. IP cameras were all tested using network control from the manufacturer's demo system.

As of when I left, we had had no latency issues with our 200+ PTZs. In a casino environment, the Operators use them all of the time. They would be the first to complain if they were difficult to control, and they did if we forgot to change the control speed settings.

Agree
Disagree
Informative: 3
Unhelpful
Funny

Undisclosed #2

IPVMU Certified | In reply to Carl Lindgren | 02/16/19 06:44am

No, 15ms, or rather 16.5ms

Ok, now I’m confused, that’s about right for pure analog latency, but seems a bit fast for IP video:

From the seminal discussion from years ago:

The best systems yielded latencies between 140ms and 170ms while the worst were much higher. Pelco Endura encoders, for instance, yielded ~330ms while Avigilon encoders yielded >500ms. Dallmeier and IndigoVision were both better than 150ms.

What am I missing?

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Carl Lindgren

In reply to Undisclosed #2 | 02/16/19 07:03am

Oh, sorry. 15-16.5ms is what I think was the margin of error on my latency tests, not the latency itself. 

Agree
Disagree
Informative: 1
Unhelpful
Funny

Steve Keller

| 02/15/19 04:01pm

Very interesting and informative interview. Thank you. 

Agree: 1
Disagree
Informative
Unhelpful
Funny
Avatar

Ricardo Souza

IndigoVision Ltd
IPVMU Certified | 02/15/19 04:47pm

Really nice interview.

Would love to have more of those, especially for big systems like airports,metro, etc...

Agree
Disagree
Informative
Unhelpful
Funny: 1
Avatar

Clint Hays

Rasilient | 02/15/19 04:51pm

I started in gaming and it was one of the reasons why I stayed in the CCTV world. Each property had similiar issues but the level of concern and willingness to improve systems varied by a huge amount. 

 

I love that some properties are applying new technologies to security, but also to beyond that department. A casino is all about the experience and the public customer experience can be improved by using some of these topics. That in return has a positive affect ok revenue.

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Mark Palka

IPVMU Certified | 02/15/19 07:24pm

Great interview and insight 

Agree: 1
Disagree
Informative
Unhelpful
Funny
Avatar

Marty Major

FLIR Security
02/15/19 07:48pm

marty likey.

excellent piece Dan G.

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Dan Gelinas

IPVM | In reply to Marty Major | 02/15/19 08:04pm

Thank you, Marty!

Agree
Disagree: 1
Informative
Unhelpful: 1
Funny
Avatar

David Johnson

02/15/19 09:00pm

Enjoyable read!

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Dan Gelinas

IPVM | In reply to David Johnson | 02/15/19 09:04pm

Thank you, David!

Agree
Disagree: 1
Informative
Unhelpful: 1
Funny
Avatar

Sergey Bystrov

NetworkOptix
02/15/19 11:32pm

Informative interesting reading. Thank you!

Agree
Disagree
Informative
Unhelpful
Funny
Avatar

Dan Gelinas

IPVM | In reply to Sergey Bystrov | 02/18/19 01:48am

Thank you, Sergey!

Agree
Disagree: 1
Informative
Unhelpful: 1
Funny

Gary Girton

IPVMU Certified | 02/17/19 11:34pm

Cool interview! More Please!

Agree: 1
Disagree
Informative: 1
Unhelpful
Funny
Avatar

Dan Gelinas

IPVM | In reply to Gary Girton | 02/18/19 01:48am

Thank you, Gary. We're working on more.

Agree
Disagree: 1
Informative
Unhelpful: 1
Funny

Greg Nuckles

IPVMU Certified | 02/18/19 01:26am

Dan - great reporting.  Hope to see more of this..

Agree: 1
Disagree
Informative
Unhelpful
Funny
Avatar

Dan Gelinas

IPVM | In reply to Greg Nuckles | 02/18/19 01:49am

Thank you, Greg. We're looking at doing similar interviews soon.

Agree
Disagree: 1
Informative
Unhelpful: 1
Funny

Shannon Davis

02/18/19 01:41pm

Fantastic report Dan. It's always great when you can get such good information out of someone who tells it like it is about technology.

 

Agree: 1
Disagree
Informative
Unhelpful
Funny
Avatar

Joe Paletta

02/19/19 02:06pm

I've learned a lot with hands-on and reading interviews! This is a great addition and I'm looking forward to each new one! The only problem is the craving for more questions but I'm sure you guys will hit a thoughtful medium. Thanks!

Agree
Disagree
Informative: 1
Unhelpful
Funny

Lee Brown

02/20/19 02:21am

The closest that I can get to providing a near analog PTZ experience is to circumvent the VMS and have a network keyboard/joystick with built in display talk directly to the camera. This removes some of the intermediary connection traffic which is then exacerbated by the Onvif ContinuousMove operations. Recently we tested Dahua's NKB5000 network keyboard and found that it really delivers on the expectation of low latency operation. Unfortunately, there are a host of shortcomings preventing us from adopting it as our go to PTZ joystick controller.

Agree
Disagree
Informative
Unhelpful
Funny

Undisclosed Integrator #4

02/20/19 02:20pm

I worked at several Indian Casinos as a contractor 3-5 years ago.
They were not shy about spending money on video.
About 1500 cameras per site, all Axis, 75% being Axis PTZs. VMS was Genetec Security Center on Dell EMC Isilon Virtual Boxes with Isilon SAN Storage. About a Petabyte of storage per site. They also had 8 VMs spun up for fail over.

The access Control was Genetec also but the systems and departments were not connected with each other. Totally isolated with seperate employees and networks.

The work stations for reviewing video were pretty impressive too. Twin 4k Monitors with Titan Video cards. This was before 4k was common. The work stations were about 12k each from Dell.

 

 

 

Agree
Disagree
Informative: 2
Unhelpful
Funny
Avatar

Jordan Dobbs

02/26/19 12:35am

"The problem with many digital recording systems is that you buy your software from one company, your servers, and workstations from another, and you're storage arrays and a network switch from yet another. And if they have problems working together, everyone points fingers at one of the other manufacturers."

This... In so many ways is a fundamental challenge when you're piecing a system together. The 'total-package' offerings/manufacturers in my opinion have more pros than cons. 

Agree: 1
Disagree
Informative
Unhelpful
Funny

Undisclosed #6

In reply to Jordan Dobbs | 02/26/19 12:47am

Try using UCS & Catalyst for a VMS deployment. 95+ Mixed Servers, Blades, SAN and switches. 

 

(      )  <--toss wasted money into this hole.

 

The only sweet spot is Riverbed for WAN. 

Agree
Disagree
Informative
Unhelpful
Funny
Read this IPVM report for free.

This article is part of IPVM's 6,938 reports, 926 tests and is only available to members. To get a one-time preview of our work, enter your work email to access the full article.

Already a member? Login here | Join now
Loading Related Reports