How to Evaluate New Technology

Author: John Honovich, Published on May 23, 2008

Most new technology fails but when it is successful, the business benefits can be enormous. The challenge then is how to efficiently determine what new technology is legit so that you simultaneously avoid disaster and reap the rewards of the rare gem.

You may have dozens of companies to review. Each new promising technology spurs the entrance of many companies hoping to enable the technology. So it's not just evaluating the technology, it's figuring out which companies, if any, has the winning solution.

You usually cannot make the evaluation based purely on your own knowledge. Most of the time when you are evaluating a new technology, you lack specific technical expertise in that area. As such, you need to figure out tactics and techniques to give yourself the best chance of projecting winners.

This article explores 5 key tips I have learned over the years working as an integrator and manufacturer. Here they are:

  1. Verify Marketing Materials Provide Technical Details
  2. Ask Specific Questions About Problems with the Product
  3. Verify that the Vendor is not a Pathological Liar
  4. Ask the Vendor how the product will work with all elements of your operations
  5. Test Under Stress

Does the Marketing Materials provide Technical Details?

The very first thing you should do is check how technical the marketing materials are. You do not need to know the technical jargon. At first, simply scan and notice how much of the marketing materials are prose (like an essay) versus how much are acronyms, numbers, diagrams, etc.

Few technical details are a strong indicator that the product is either conceptual or vaporware. Often, the lack of technical details arises because the company is promoting an idea but they are weak in engineering. Other times, their engineering is fine but the product is still so early that they have not gotten far enough to figure out a lot of the technical details.

I generally discard companies from further consideration that do not meet this criteria. On the other hand, just because a company does have technical details, does not mean it will definitely work. The company may be especially sophisticated in marketing or there may be more issues. As such, simply treat this as a first gate.

Are you asking Specific Questions about Problems?

Most people will not lie to you but are OK with not telling you the truth. Since people are generally uncomfortable lying, a common tactic is to ignore discussing damaging issues. If you ask a vendor "How many companies are using your product in production?", most vendors will tell something close to the truth. If you do not ask anything, almost no one will volunteer that the product has never been deployed or only deployed at 1 or 2 sites. Strictly speaking, they are not lying to you but the outcome is similar because it leads you to believe incorrectly about a key element in the decision making process. Unlike mature products where it is reasonable to take things for granted, this is a great risk with new technology products.

Get Video Surveillance News In Your Inbox
Get Video Surveillance News In Your Inbox

The challenge is new technology products always have problems. That does not mean you should not use them but you have to be aware of what those problems are. Be explicit and ask things like:

 

  • How many sites have the product deployed?
  • What was the cause of the last 3 failures of your product in the field?
  • What was the cause of the product failing in previous pilots? (all products fail in at least some pilots)
  • Can I have a reference? (Do not accept the excuse that they cannot tell you because of security issues. Any product with success has at least a few customers willing to talk, especially if you are a security manager.)

 

Just remember, do not takes things for granted, make sure to ask.

Is the Vendor a Pathological Liar?

Pathological liars are a very dangerous force in new technology products. Every once in a while, a vendor will consistently spin and deflect any problems or criticisms. They will be so good that you will relax your guard and in your enthusiasm for the benefits of the problem will overlook problems. This is doubly dangerous. First, this undermines your due diligence but, secondly, and much worse, pathological liars usually have worse products because they are too busy spinning rather than building.

I experienced this when I was an integrator. We would go into meetings and this guy would consistently spin our offerings, deflecting any legitimate issues and creating the perception of no risk and huge reward. One time, a customer asked a technical question like "Do you use Protocol X?" and this guy shot back "Of course." The customer, who was fairly technical, and myself were both taken aback. Unfortunately, what my colleague did not understand was that this was an outdated protocol that no one wanted to use anymore. When we left the meeting I asked him why he said that. His response was, "I was trying to tell them what they wanted to hear." Make sure vendors are not simply telling you what they want to hear.

The best tactic to handle this is to ask another person at the vendor (usually a technical person) questions away from the potential liar. Now most people know whether there colleagues are liars but they are going to be quite reluctant to say it directly. Talk to them about operational issues and ask this person direct questions. You will get a good sense of issues and discrepancies quite quickly this way.

How Does it affect the Elements of Your Operation?

New technology products usually fail because of unforeseen operational issues. Generally it is fairly easy to figure out if the technology solves a business problem. On the other hand, it is very hard to determine what the operational issues you might have deploying and using this technology.

This is the most important step in evaluating new technology products. Regardless of whatever has been said or promised, regardless of the potential, how the technology impacts your operations makes or breaks its viability. Very often, the technology results in hidden increases in cost or can simply not be made to work with your existing systems or procedures.

You must make sure you understand how new technology interacts with existing systems. You have existing systems and you want those systems to continue to work. You often find out that this technology does not work with a key component of your existing system. As an integrator, I once had a major problem designing a video analytic system because it did not integrate with the customer's existing matrix switch. A minor technical detail but it was a very serious operational issue. For all aspect of your system, go through them and make sure that there are no hidden operational incompatibilities.

Similarly, while it is easy to determine the direct cost of the new technology product, you must be careful about indirect costs this product might result in. Often new technologies will have requirements that cannot easily be met with your operations. This technology might require much greater amounts of bandwidth or client PCs that are much more powerful than your existing ones or significant amounts of training or maintenance. When you are estimating your costs, be sure to consider what the indirect costs can be - they often turn a promising project into an unrealistic one.

The technology may be good but not good enough for your business objectives. You have to be sure that it is truly good enough or you will cause a serious operational problem. Often, technology exists to automate existing processes managed by people. It is quite common that new technology can do a job 90% ot 95% as good as a person. However, in many situations, from an operational or customer support standpoint, sacrificing that 5% or 10% can be a significant business problem. If you use facial recognition to verify a person coming through a door (automating access control guard verification), if that facial recognition system makes a mistake only 5% of the time, that can be 5 to 20 people a day that are frustrated. This might be a very good system and strong technology but it may not be good enough to meet the other business objectives or your organization.

If you do a careful assesment of system interoperability, indirect costs and conformance with business objectives and it passes, you are very likely to have a winner.

How Does it work under Stress?

One key way to determine how the new technology product affects your operations is by doing a pilot. Pilots are common so I only have 2 pieces of advice here.

One, make sure your pilot places the system under the highest level of stress you expect the product to be used at in production. Often, the test is done in a lab or in your office. This is a very bad idea. Office or lab test hide issues and works to the advantage of unscrupulous vendors.

How capable a product is to handling extreme conditions and loads is a very common difference between new and mature products. It takes a lot of time and experience for a product to incorporate real world challenges and be optimized for performance in extreme conditions.

Placing the product in your toughest operational environment is the best way to show how ready the product is for production use. This way, any shortcomings are exposed quickly rather than months later after the project is well under way and it is very hard to adjust.

 

Using new technology products is the most powerful way to generate a business advantage. If you are a security manager, it can enable you to truly standout and advance in your career. If you are an integrator, it can drive incredible growth. I am a big proponent of using new technology products.

Making the right decisions about new technology products is critical. Consider using these steps and hopefully you will be able to make better decisions in less time.

Related Reports on Marketing

Beware The "Hit List" Ranking on Apr 21, 2017
The hit list. Kirschenbaum's recent newsletter complained about a 'hit list', bemoaning how a company took aim at ADT. Alas, that's the Google...
Failing at Marketing, "ALL HIKVISION PRODUCTS" On Sale on Apr 18, 2017
The ballerinas are out. The price cuts are back. Hikvision is struggling to build a premium brand (i.e., 'The Art of Video Surveillance') so...
Nvidia Deep Learning Developer Kit Examined on Apr 17, 2017
Can building a deep learning video analytics application be as easy as the typical VMS installation? Maybe not quite, but Nvidia is aiming to make...
A New Low For Knightscope - Just $3! on Apr 13, 2017
Take a look: Did you think, even for a fraction of a second, "$3 per share, could be a good deal"? If so, you should not invest in anything....
The Sleaziest Booth Of ISC West - Rapid Response on Apr 10, 2017
The use of 'booth babes' is way down overall, but one company, Rapid Response, continues to treat ISC West like it is an HBO brothel documentary...
Best and Worst - ISC West 2017 Show Report on Apr 10, 2017
IPVM went to Las Vegas, examining what vendors are showcasing and what is new. Attendance was up, according to the show, and was certainly well...
How HiSilicon Wasted $200,000 At ISC West on Apr 03, 2017
Huawei owned HiSilicon is a hidden key component of many of the cheapest cameras on the market. For the second year in a row, HiSilicon has wasted...
Hikvision Aims For #1 Brand In 2020 on Apr 03, 2017
Hikvision's Chairman and Communist Party Secretary told Xi Jingping his gift for the Communist Party's 100th anniversary in 2020 will be to make...
$10,000 Free VMS Licenses For New Dealers IPConfigure on Mar 29, 2017
Another gimmick or a real deal? The company that brought 'Unlimited' Enterprise VMS License for Just $32,000 and a cannon to ISC West are back with...
Hikvision Hires Crisis Communication Writer on Mar 21, 2017
Hikvision has hired a crisis communication writer as the company ramps up its efforts to deal with the 'crisis' it feels it is facing. 'Crisis...

Most Recent Industry Reports

Last Day - IP Networking Course May 2017 on Apr 27, 2017
Today is the last day to register for the May IP Networking Course. This is the only networking course designed specifically for video...
Hikvision EZVIZ Amazon Scam Revealed on Apr 26, 2017
Hikvision is violating US Federal Trade Commission guidelines and Amazon rules with a "Honest" Review Program scheme that provides gift cards to...
Anixter CEO Admits Price Deflation and Non-Exclusive Integrator Sales on Apr 26, 2017
Anixter's CEO has admitted to (1) price deflation impacting IP camera sales and (2) not always being 'exclusive' with security integrators. In...
Xandem Next Gen Intrusion Tested on Apr 26, 2017
Xandem's "full coverage motion tracking technology" is unlike any intrusion technology we have seen. We bought their new system and tested it...
Tri-Ed Favorability Results on Apr 25, 2017
Tri-Ed, owned by Anixter, far outranked Anixter, the lowest ranked company in our distributor favorability series. Still, Anixter's ownership did...
Eagle Eye Exec On Mountain Of Servers - VSaaS Growth Analysis on Apr 25, 2017
Eagle Eye VP of Operations, Hans Kahler, posted a picture of himself sitting on top of a shipment of new servers, as a testament to the companies...
Chinese 'Attacking Us From Every Direction', Says US FBI on Apr 25, 2017
"Chinese eating our lunch. Attacking us from every direction" said the US FBI's Deputy Director Andrew McCabe at the ASIS 2017 CSO Summit. .@FBI...
Axis Posts Strong Q1 2017 Financial Results on Apr 24, 2017
Axis posted strong numbers for Q1 2017, after having some challenges in 2016 (Q1 2016, Q3 2016). Inventory levels and overall spending show...
Axis Lowest Cost Outdoor IR Camera M2025-LE Tested on Apr 24, 2017
Axis has lagged offering low cost IR cameras while their Asian competitors have made IR standard even in their most entry level cameras. Recently,...
IPVM First Dean's List W2017 - Thomas Atkinson, Matt Hurly and Fredrik Lundqvist on Apr 24, 2017
IPVM is happy to congratulate and celebrate our first "Dean's List", the top students in our courses. For the Winter 2017 IP Networking course...

The world's leading video surveillance information source, IPVM provides the best reporting, testing and training for 10,000+ members globally. Dedicated to independent and objective information, we uniquely refuse any and all advertisements, sponsorship and consulting from manufacturers.

About | FAQ | Contact