'Best In Show' Fails

By John Honovich, Published Apr 19, 2018, 09:27am EDT

ISC West's "Best In Show" has failed.

For more than a decade, it has become increasingly irrelevant as the selections exhibit a cartoon level understanding of technology and the market. The 2018 winner is a case in point, as it is the same approach that has been available for years with minimal market impact.

In this note, we review the past decade's "Best In Show" winners, the 2018 winner's issues and the broader problems this presents.

2018 ****** - ***********

*********** *** '**** ** Show'*** ******* ***** *** inside ** ** **** camera.

VMS ****** ******* **** ******* / **** *******

******* ***** ****** *** VMSes ****** ** ***** cameras (unaware ** *** ****** this ****** **, *** running * *** ****** of * ****** *** been ********* *** **** years:

  • ******* *** **** **** for * ****** ** more (****** '*************')
  • ********* **** * *** bet * ***** *** on ***** ***** *** their *** ** ***** devices **** ** ******* (note: *** ******** *****, not *** **********)
  • **** ****** *** ***** own *** **** **** inside ** ***** *******
  • ***** *** *** ***** VMS ****** **** ******* for * ***** *** (see ******** **** ****) *** *** ** ACAP *********** **** **** IPConfigure

*** ******* **** *** 'VMS ****** ** * camera' ******** ** ****, for ***** *** **** low ****, ** *** is **** **** **********, and *** ***** *** want ******** ************* ** longer-term *******, ** ********** VMS ** **** **** attractive. **** ****** '*** inside ** ******' ** niche ************. *** ** particular ******* ** ******** of * '*** ****** of * ******' **** impact **** ********** *******.

No *********** **** ***

** ******* *** ** SIA, *** ********* *** the *****, ******* *** concerns *** ****** *** comment ** *********** ***** the *********. *** ************ the ******* *** *** not ******* *** ***********. Furthermore, ********* ******************** ** ***********.

10 **** *******

*** *** **** ***** is *** ** *******, reviewing *** **** ******'* 'Best ** ****' ****** shows ******* ********:

**** ** ***** ********* have **** ****** ***** industry ******. *** **** and **** ***** ******* have **** *******. *** 2013 ***** ******,*** ******* ** ******, *** **** *********. And *** ********* ****** have ****** ** ** bad *******, ****** ***** industry ****** **** **** even ** *** ****.

Good *** *** ******* *** ***********

*** ***** ******* ****, such ** ***********, ** is ********* **** *** them. ** ** *** the ******'* *****. **** present ***** ******** *** best **** *** *** if *** ****** ** not ********** ********** ** the ******, ** ** it.

** **********, *********** *** a **** ******* ** promotions, ************* * '******' *** their ***** ***** ***** acquired *********, **$**,*** **** *** ******** for *** *******, *********** ********** *** ******* for **** $**,***** *** ****** *** simply **** **** **********.

Little ******** ** *** ***** *******

** *** ****, ***** was ** ***** **** level ** ******** ********* in *** ***. **** has ******* **** ****. And **** ****, **** though *** ***** *** for * ***** ******* and * ******* ** have *******, ********* ** one ***** ** ***** it.

Good *** ***'* **********

*** ***** *** ****** program ******* *********** ******, it ** ****** **** money *** *** **** ever, **** ***+ ******** this ****, ****** ~$*,*** each, *** **** ~$***,*** from ****.

Bad *** *** ******** / *** *** **********

*** ******** ****** ** encouraging *** ********* **** innovation, ** ** ***** help ****** ** **** as ******** ********** ** spend **** ** *&*. And ***** *** **** is *** ******* ***** American **** *** *** most ********* *** ** the ****, **** ****** be **** ********* ** innovation **** *** ******* way ** **** ***** on *********.

Comments (24)

Related, periodically people ask about IPVM doing awards. On the one hand, we could obviously do much better identifying real innovations. On the other, as is our concern generally, we worry about the misuse of our analysis as promotion or marketing for companies. I am not sure what we would do. If anyone has thoughts on this, feel free to share.

Disclaimer: I am VP Engineering for IPConfigure

I think, respectfully, that John's post misunderstands the meaning -- or at least *a* meaning -- of "Best New Product". There is a semantic game at work here: is "Best New Product" the best product that was released in the past year, or the best product that's first to occupy a brand new market position? In my view and in the view of the SIA judges, this award addresses the former.

The best product in a class is rarely the first. Good work takes time, both in terms of product development and understanding customers' needs in a new or changing market space. IPConfigure Orchid Fusion VMS on ACAP for Axis cameras is a new product that offers a no-compromise embedded, web-based VMS with first-class user experiences and capabilities that are fundamentally identical to Orchid Fusion VMS running on traditional servers -- and in fact the two can integrate seamlessly in a way that's transparent to end-users and system administrators.

While intelligent and well-meaning folks can disagree on the degree to which this architecture is revolutionary, IPConfigure feels that our product represents the cleanest, most technologically sound, and highest value implementation of the concept and has likewise earned the judges' decision.

Cort, thanks for the feedback!

The best product in a class is rarely the first. Good work takes time

The 'class' is not the issue here but the entire industry.

Whether IPConfigure is the best product in the class of VMSes running in a camera is possible (it would best be determined by testing, which neither I nor the judges have done). Being the best in such a narrow and limited class is far different than 'best in show'

'Best in show' is not simply 'best in class' or best in a category. There are individual class awards for that (34 for this awards program). 'Best in show', if it means anything, has to mean it is better than any other product across all classes.

To be the best product of all products in an industry for a year means that the thing is 'revolutionary', to use your word. Your entry is not revolutionary (that's not a knock on your product, it may be a good product that sells well for you) nor does it solve or breakthrough fundamental barriers.

Moreover, my point is not simply about IPConfigure. Looking at the last decade, the show repeatedly picks products that have little market impact or even hope for that.

Most folks on the front-line of our industry are certainly interested in what all is going on and are most interested in what is new, hot and could have a good impact on  my business today. An honorable mention is always welcome for new things of all kinds like scratching on a rock a couple thousand years ago, (1) Man going to moon & (2) New substance discovered for killing devices. The second would most likely be the number one topic at the the monthly carnivore fest. Anyone spending a lot of time wanting to discuss going to the moon would probably raise a few eyebrows. 

The fact that 10 out of 10 previous year picks have faded into the woodwork would be more of a news story than anything else, which is exactly what you are pointing out. Another job well done, John.

Because users rarely run VMSes inside of their cameras (<0.1%), many are unaware of how common this option is, but running a VMS inside of a camera has been available for many years...

Agreed.  Isn’t every camera that has an SD card technically a VMS, even if only a rudimentary one?

Isn’t every camera that has an SD card technically a VMS, even if only a rudimentary one?

Yes, but there is a world of difference between an IP camera allowing navigating files of video clips vs a VMS.

Running a 'real' or 'full' VMS is materially different and if IPConfigure was the first to do that (say this was 2003, not 2018) you could make a real case that this could be revolutionary but not now, knowing what experts in the field should know.

Because users rarely run VMSes inside of their cameras (<0.1%), many are unaware of how common this option is, but running a VMS inside of a camera has been available for many years...

Here's the first IP camera I can remember with its own VMS, circa 2002.  Built-in hard drive and support for 4 cameras.  It did not win any best in, well... anything awards.


All the "awards" in this industry are a joke, mostly a marketing tool being paid for the manufacturers... 

It may be different stateside, but few people this side of the pond take a great deal of notice. There’s an award for everything and we have become accustomed to the money making industry behind it - the awards dinners, sponsorships of the wine etc......

I have never seen an RFP or spec call for "award winning products only", nor have I ever had a customer say "I called you because we saw your award for X".

To me, these awards are almost valueless. I say "almost" because they are not entirely valueless, but they have a very short half life.  The problem with the ISC West awards in particular is that there is too much going on at the show, and the awards are announced roughly halfway through the show, near what is basically the peak of show floor attendance.  This leaves winners practically zero chance to really promote and capitalize on their awards. By the week after ISC West, nobody cares who won what, everyone is back to grinding at sales, and "Wanna hear about the award we won" is a freshman's pickup line.

If SIA announced the award winners before the show, it would give manufacturers some opportunity to promote their wins, and possibly even adjust booth signage to say "Come see our award winning analytics-for-detecting-people-loitering-in-the-rain". This might help drive greater awareness of the awards in general, and actually allow winners to get some value from it.

Announcing awards before the show would also mean that you are not doing your product presentation to the committee the last day of pre-show setup, when (in my experience) you often need all hands on deck to finish things off in the booth. This is particularly true when you are showing a new product (which award entries are supposed to be), where you need to do final debugging and patching to make the damn thing run successfully :)

"Wanna hear about the award we won" is a freshman's pickup line.

Lol, that is funny. I did get 2 manufacturers citing their 'govies' awards in the past week. That was amusing.

If SIA announced the award winners before the show

I agree about that. Also, if they would promote the award nominees before the show that would help. We actually try to draw attention (see e.g., 102 New Product Showcase Entrants - ISC West 2018). It's 2018, they might as well have presentations as a webinar, which would make it easier for everyone.

Announcing awards before the show

Also, related, Savvy Marketers Announce Before Shows.

Take Axis, they had announced lots of new products but since they did most of them the first day of the show, it got lost in the commotion of that. Now, not even a week after, it is old news...

SIA used to be the good ole boys club and if your in you win. Don't know about the last ten years but seems to me the products that win are pretty weak. Or is it there isn't much inovation any longer ? 

Or is it there isn't much inovation any longer?

Come on, Larry! For example, Hikvision DarkfighterX was a submission this year and it got shut out. There were 36 total awards for 122 products and it got zero. Why? (that's my question to SIA, not you).

And you look over the years for big changes in video - Axis was first was super low light MP, first with smart codecs. And look at Avigilon, Genetec, Milestone, etc. The most used products in the industry never win best in show. Clearly, these companies have released innovative products over the years but SIA goes for these weird, almost destined to be irrelevant offerings.

So John you don't think politics has role in their choices? 

I’m intentionally the least political CEO in the industry. This was not politics for IPConfigure.  Our success won’t be attributed to politically savvy.  Anyone who knows me can attest to this.
IPConfigure CEO

Larry, the SIA / NPS judging process has minimal transparency so it is not feasible for me to determine the exact roles in their choices.

That said, when I look at the pattern over the last decade, the most common thread is companies whose ideas sound interesting to those with limited knowledge of the industry.

Look at the 2016 winner Lynx Tiger, I am not aware of them having any political connections. But if a person did not understand that various platforms have had that for years and that it remained a niche application (just like IPConfigure this year), they might conclude it's a big new thing.

I am a marketing guy. I am not a sales guy. I am not an engineer. Some of you can stop reading now.

Because John Honovich has such a significant following in the industry which my employer serves, I try to understand the underlying (or overarching) principles of his arguments. Why? By representing a manufacturer, I have a responsibility to try to understand what our potential customers think, or are being influenced to think.

As I understand it, John is not taking issue with running a VMS on the camera. John is taking an issue with the criteria used to choose winners of awards, specifically a "best of show" award at a security products convention.

To me, to understand this you must accept that John's purpose with IPVM is to educate and PROTECT integrators

When he states that the market for security cameras running the VMS on the camera is small, I think that he is telling us that he questions why such a niche product would be deemed significant. It could be the universally best VMS running on the universally best camera and he would still say that few people are going to need it or buy it. So, why should it win an award?

We sell a very, very good VMS and it is now available on very, very good cameras as "edge" offerings. That is why this issue is important to me.

John's argument acutely illustrates my challenge -- what is the use case for these cameras? Are there uses that integrators (and apparently, even John) are not recognizing, or is the application always going to be limited?

But to have a product named "best in show" where there is an old and ongoing argument about the usefulness of the product does not make much sense to me, either. But, congratulations. You won. If your win increases the market for the cameras running your VMS, it should influence our sales as well.

As several here have pointed out, it may just be that some of us disagree with the criteria for the award.

If the award was, as has been alluded to, "The Best New Product with the Most Potential to Make Integrators More Profitable," it might be easier to understand and to debate the winner.

As a marketing guy for a manufacturer, I never want to be accused of doing marketing for marketing's sake. Our engineers should not do engineering for engineering's sake. Innovation should not be innovative for innovation's sake.

Marketing and innovation for the customer's sake.

By choosing products for awards that solve niche issues, is the customer ultimately being served? Is the customer even considered in the criteria?

Mark, good feedback.

Related, we did ask SIA for an explanation of why IPConfigure won but they did not provide one nor is any explanation provided in the announcement.

This is typical for their awards. Because of that, it is impossible to know for sure what the judges were thinking. However, looking at the awards over the years leads me to the conclusion that the judges pick things that sound impressive to outsiders but are obviously insignificant or destined to be niche offerings to those in the field (look at the mobile streaming service or the Windows IP camera or the various complex and super expensive biometric offerings that have won over the years).

Mark - what a refreshing perspective - marketing that is focused on providing relevance on consumers (integrator).

I get that crass commercialization is the means-to-an-end for manufacturers - but sometimes, I'd just like an awesome product to be released with minimal fanfare that is widely adopted - because its a freaking awesome product.

Okay - let's throw that idealism out the window Ed and live in the real world. Marketing, despite what engineers think, is critical for placement / relevance of products and it helps to establish the target audience.

When I was at ISC West this year, I overheard a discussion at a booth. I was having one of the highly overpriced meals on the floor (standing at one of those lousy tables they put up at the Sands). While consuming my soggy, overpriced wrap (trying to lose weight) I heard a "customer" asking a camera vendor (who shall remain unnamed) ask if their "system" would protect perimeters by providing alerts. The customer described the need - 1.2 km of perimeter with the longest segment being over 550M. The vendor (still unnamed) indicated that their cameras would pick-up all movement on the perimeter, day or night. I chuckled, finished my wrap and then left.

That interaction, however, is primarily because the sales team is being told - move units (regardless if the solution is not needed or relevant to the customer).

I'm hopeful that the user had enough knowledge to call out that BS as what is was - but that's not a given.

This is where a subscription to IPVM would have helped the customer more than paying for a ticket to Vegas and the attendance fee.

So, is the customer really being served by overzealous sales people who are loathe to tell the "whole" truth. I prefer to quickly "fire" customers before them become customers IF my solution will not work for them. I'm happy to refer them to products that would meet their needs. I don't need to own the world, just a small chunk of it - unfortunately - so many vendors want it all. How much is enough? Just one more 1$ -or in this case, one more award leading to the false perception that because they received the award - it must be an excellent product - which is not always the case.


Wow - that was a ramble - sorry.

I'm hopeful that the user had enough knowledge to call out that BS as what is was - but that's not a given.

In a way, this is why I initially subscribed to IPVM.  There are many like-minded individuals on here who are anti-BS received from sales people.  I come from a family of sales people / chronic liars so I instantly distrust sales people until they prove otherwise.  I feel badly for people who are duped by price or persuasive sales pitches.  A good rule of thumb is to A) always test the product first and B) check IPVM for reviews or discussions.

Call me jaded, but I view the entire "association" (SIA, ASIS, PMI) as money machines and that "awards" and "pre-advertising" are just a means to an end - enriching the association.

I know that these shows combined with their membership pushes (and educational classes / materials) are their primary revenue sources and it's a necessary evil, but it does allow for agendas.

What criteria is used to establish the "best-in" label? Who's on the committee? Why did they get chosen and not others?

I think highlighting and spiking new technology - without consideration or remuneration - would give far more credibility to the SIA and selection committees. (Which is why I SO enjoy reading the reviews on IPVM - unbiased, relevant reviews of products - kudos).

I, as an end-consumer and reseller, would appreciate understanding "real" new technology in the market space. Additionally, I think that the SIA should take a focus annually as to where the market is trending - it certainly isn't with a VMS on the camera?

Looking forward to the recriminations of the industry groups to this post.



I’ll stand up for sales people here, since I am one.  

Depending on the size of the organization they can or do provide so much sales support that most of their sales team can only call someone to show you how to make it work. Is that their fault? Partially. 

Some provide no support beyond a bunch of glossy literature, power points created by product managers and mar-com who have never seen the product in the wild.  There you have good sales people passing bad info and not knowing.

Several provide the ability for a sales person to either sink or swim with so little support by team members or internal product resources. The swimmers usually know the product inside and out, but get trapped with after sale support calls.

On the sales person side, I’ve seen the people who won’t use external resources because they either lack the ability to manage them, schedule, communicate, or they just find doing themselves easier.

Then, there are the worst offenders.  They take bits and pieces of solutions as a total solution and just make it up on the go.  When caught, they move to another company claiming the last one didn’t really fit or their technology didn’t work as described.

In several instances I have been asked “BS Detection” questions geared to encourage me to make it up on the fly or just plain lie.  I’ve been told when others failed miserably after providing the correct answer, which can include “I have no idea”

Okay....done venting. 

Speaking of 'Best In Show' Fails, 2011 award winner FST has failed, ceasing operations.

In terms of the award, what was strange even at that time, given the complexity of technology involved and the price point they were trying to charge, this company had a high probability of failing and was virtually guaranteed to be, at best, a niche high-end play. 

And we called those concerns many years ago.

Read this IPVM report for free.

This article is part of IPVM's 6,736 reports, 909 tests and is only available to members. To get a one-time preview of our work, enter your work email to access the full article.

Already a member? Login here | Join now
Loading Related Reports