Is IMS / IHS Research Reliable?

Well, what IMS is attempting to do is nearly impossible to do accurately, though certainly made worse that they typically hire people out of school with no industry experience nor contacts.

I have seen some of their numbers and talked to manufacturers, and we have wondered the same things - where did they get these numbers from? How could these numbers be correct given they obviously contradict X, Y and Z known through public filings, etc.

My understanding is that the bulk of the numbers are ascertained by contacting manufacturers and asking them how much is their revenue or how much is their revenue in encoders, DVRs, 5MP cameras, etc. Alternatively, they might say "We think you did X. How much do you think you did?" This is corroborated by quite a number of manufacturer's explanations to me.

Indeed, this makes sense. How can you possible know how many encoders (or SD cameras) ACTi or Axis or Vivotek or Verint sold last year without getting the numbers from them? Of course, then you face 2 risks: (1) the manufacturer overstates or (2) the manufacturer refuses and the firm is left to make up a number or leave them out.

The net/net is to take those numbers with a lot of skepticism.

In their defense, the other people that sort of try to do detailed market sizing in surveillance are far worse :)

I don't extravagant hope the data is acurate, but the maket trends should be correct at all catalogue level.

Here in brazil many companies bought it and the main association actually issue statements based on it... Even TV programs do their news reports based on what this association says. Its really getting out of hands lol!

Whenever possible I like to distill the market analysis, and trend projections, by triangulating across several "secondary research" market sources ( e.g., syndicated reports). This theoretically mitigates errors in any one, but of course it doesn't protect against them all being off. Then I temper the resulting analysis with primary research "proof points" which with any luck superimpose on the secondary research in a validating manner. It's usually difficult to look very far ahead or very far across products without applying some judgement and interpretation. There is art required with the science underscoring the importance of industry experience and insights.

We got involved in an IMS Report Study about a year and a half ago. The research they did and the questions they asked us were OK, but their vision and understanding of the market was, to be charitable, very limited. I completely agree with Skip's comments - the resulting report had to be analysed in the light of a lot of our own market intelligence and industry knowledge.

Our conclusions were that we didn't really gain much benefit from particpating, nor from the information contained in the resulting report, and that we can get almost as much information (if not more) from public sources, credit-rating agencies, financial reports and even, dare I say it . . . . IPVM.

The Frost & Sullivan Reports have long since become worthless, and their Innovation and Technology Awards are an industry joke. Without applying, we were informed that we had won an F&S Innovation award for one of our products (OK, it was a great product!). But the small print said that we weren't allowed to tell anyone nor use the F&S award logo unless we paid the $30k "Marketing Package" fee, which included all sorts of useless stuff, like being invited to a dinner to meet some boss of an F&S department. Laughable.

Apologies for posting this as "undisclosed", but the reasons are obvious.

John & all,

My experience as a buyer (in bulk, I might add - hence the undisclosed name) of global industry research is that several factors are hugely important.

One factor to understand is where the researcher "plants their flag" when attempting market sizing measured in currency (e.g. dollars, Euros, etc.). Are they measring from the ultimate end customer's wallet (including equipment at integrator cost plus installer labor)? Are they measuring at the distributor's shipping dock (equipment plus distributor markup, with no dealer involved?) Or, are they measuring from the manufacturer's shipping dock (no distributor markup, no integrator, no labor?)

Another factor is HOW the researchers did the research... I can do a lot of my own legwork using public data, so for me it's a "make vs. buy" decision. Reputable research outfits are happy to disclose the precise methods used; others view their method as a trade secret. Ask: and if the company won't share the method to your satisfaction, run away. (If the company has to give away fake awards to get noticed, run faster!)

Also: having some understanding of what researchers actually do will help you make these decisions. I break it all down into two groups: Qual and Quant.

"Qual" (qualitative) work is when the researcher performs interviews about a topic and summarizes the findings - an example would be to interview ten integrators and ask them what sorts of jobs they expect to be pursuing in the coming year. Qual research is important, but computationally "squishy"... that being said, it's helpful when you are tackling squishy issues such as consumer behaviors or preferences. Qual research is also helpful in informing "quant" (quantitative) research: Qual tells you where to look, Quant helps you see.

"Quant" (quantitative) research is when the researcher gathers data of a numeric sort: how many integrators pursued Canadian government contracts in 2012, how much work (in currency) was granted in 2012, etc. Quant data also comes from bulk surveys - one well-known consumer electronics research firm sends quarterly Web surveys to something north of 30,000 broadband equipped households... which may not tell the researchers what the customers are dreaming about, but can very specifically tell what they are buying. Compare the researcher's methods to your need for information and make sure you're speaking the same language.

Quant data on global video hardware, by the way, is the toughest data to come by in this space, because of the OEM issues. (The truth may be Out There, but none of us have it yet ;-)

Hope this helps, and apologies again for the undislosed name.