Is IMS / IHS Research Reliable?
Hi, I was looking at IMS market research studies yesterday and the numbers did not match not only my expectations but also the data that I have from the market. I would like to know how much can we rely on this research companies. Please share your thoughts and opinions.
Regards,
AL
Well, what IMS is attempting to do is nearly impossible to do accurately, though certainly made worse that they typically hire people out of school with no industry experience nor contacts.
I have seen some of their numbers and talked to manufacturers, and we have wondered the same things - where did they get these numbers from? How could these numbers be correct given they obviously contradict X, Y and Z known through public filings, etc.
My understanding is that the bulk of the numbers are ascertained by contacting manufacturers and asking them how much is their revenue or how much is their revenue in encoders, DVRs, 5MP cameras, etc. Alternatively, they might say "We think you did X. How much do you think you did?" This is corroborated by quite a number of manufacturer's explanations to me.
Indeed, this makes sense. How can you possible know how many encoders (or SD cameras) ACTi or Axis or Vivotek or Verint sold last year without getting the numbers from them? Of course, then you face 2 risks: (1) the manufacturer overstates or (2) the manufacturer refuses and the firm is left to make up a number or leave them out.
The net/net is to take those numbers with a lot of skepticism.
In their defense, the other people that sort of try to do detailed market sizing in surveillance are far worse :)
P.S. - What's even more amazing is the attempts to break up the numbers into regions - e.g., ranking NVR sales in America. This is even harder. Now you want a company like Bosch or Schneider to not only break out their surveillance number, but specifically their NVR sales (not DVR because that's another category, nor VMS software because that is yet another category) and then specifically by region - Americas, Europe, Asia.
It's impossibly hard. Indeed, I can't imagine why anyone would think such numbers could possibly be generally accurate. I am sure they get some right, (i.e., manufacturers who tell them the truth about every line item) but it's very hard to know how many and which ones are wrong. This, of course, is made much worse by the fragmentation in the surveillance market, so you need to get accurate numbers of hundreds of private companies or non public segments of companies.
Eduardo, IMS does have at least one person based in Asia and I am sure that helps somewhat but as you mention, with the hundreds of OEMs, tracking accurately is nearly impossible.
This raises one of the interesting nuances of IMS's tracking. Supposedly, it's branded sales only (i.e., when Milestone sells to OnSSI, the revenue from OnSSI should be backed out of Milestone's VMS total sales figure, or when Exacq sells to AD, etc. or when Dahua sells to its gazillion OEMs) but how do you figure out how much revenue should be adjusted?
That said, SIA and ASMAG, from what I have seen, are far less granular than the detailed numbers / segmentations and ranking that IMS offers. This is part of the attraction and problem of IMS. They try to measure things that are very nuanced and are therefore very easy to get wrong.
I don't extravagant hope the data is acurate, but the maket trends should be correct at all catalogue level.
Wang, that's actually an interesting point. There's two side to IMS. The 'where is the market now side' - how many encoders did Axis sell in Asia, how many cameras did CNB sell in Europe, etc. We've been discussing that.
The other side is 'where is the market going'. Here it's clear that IMS is just clueless. One example is their recent wireless projections, though we've examined some historical ones. The problem IMS has is no one on their team has any industry experience and they rotate mostly kids out of school in few year increments. When it comes to the future, IMS is doing little more than parroting manufacturer marketing claims as they have no experience to draw on of their own.
John,
Total agree with you. really an interesting topic
much much manufactures's marketing team buying this report but it's really useless, just use this data as a marketing tool to boast themself product, brand, business impact. this is not a good data for company business strategic direction.
That is a good point. Genetec didn't seem to like IMS reports but now that IMS ties them for #1 in VMS, they sing their praises :)
Earlier this month, the Verint marketing executive alluded to his IMS ranking as justification.
And it pays off, Milestone's clearly most powerful marketing claim for the last 5+ years had been their touting of the IMS ranking, which gets transformed from 'most revenue in a specific category' to world's best.
Lest I forget, Hikvision has done the same thing recently, though those numbers are meaningless, at best, to those looking to understand non China performance / adoption.
One clever thing that IMS has done in the last few years is expand the number of categories they rank manufacturers in - thus giving more manufacturers opportunities to say they are #1 or top 5 in a certain segment.
Btw, I am curious. Who's buying IMS reports? Are there any integrators buying them?
Here in brazil many companies bought it and the main association actually issue statements based on it... Even TV programs do their news reports based on what this association says. Its really getting out of hands lol!
TV programs! IMS does a good job of issuing press releases touting market growth for each new report. People like definitive statements, even if they are wrong.
The most absurd example comes from an IMS competitor, Frost & Sullivan, who got a ton of coverage for their insane PSIM $2.7 billion 'study'.
Whenever possible I like to distill the market analysis, and trend projections, by triangulating across several "secondary research" market sources ( e.g., syndicated reports). This theoretically mitigates errors in any one, but of course it doesn't protect against them all being off. Then I temper the resulting analysis with primary research "proof points" which with any luck superimpose on the secondary research in a validating manner. It's usually difficult to look very far ahead or very far across products without applying some judgement and interpretation. There is art required with the science underscoring the importance of industry experience and insights.
Skip, so you spend thousands (or tens of thousands) on multiple market reports and then you still need to analyze and compare against other sources / information?
We got involved in an IMS Report Study about a year and a half ago. The research they did and the questions they asked us were OK, but their vision and understanding of the market was, to be charitable, very limited. I completely agree with Skip's comments - the resulting report had to be analysed in the light of a lot of our own market intelligence and industry knowledge.
Our conclusions were that we didn't really gain much benefit from particpating, nor from the information contained in the resulting report, and that we can get almost as much information (if not more) from public sources, credit-rating agencies, financial reports and even, dare I say it . . . . IPVM.
The Frost & Sullivan Reports have long since become worthless, and their Innovation and Technology Awards are an industry joke. Without applying, we were informed that we had won an F&S Innovation award for one of our products (OK, it was a great product!). But the small print said that we weren't allowed to tell anyone nor use the F&S award logo unless we paid the $30k "Marketing Package" fee, which included all sorts of useless stuff, like being invited to a dinner to meet some boss of an F&S department. Laughable.
Apologies for posting this as "undisclosed", but the reasons are obvious.
$30k for a Frost award? Wow!
5 years ago, numerous surveillance manufacturers would 'win' Frost awards every year. Now, it's almost extinct. I think our report on Should You Trust Frost & Sullivan Awards played a role.
John & all,
My experience as a buyer (in bulk, I might add - hence the undisclosed name) of global industry research is that several factors are hugely important.
One factor to understand is where the researcher "plants their flag" when attempting market sizing measured in currency (e.g. dollars, Euros, etc.). Are they measring from the ultimate end customer's wallet (including equipment at integrator cost plus installer labor)? Are they measuring at the distributor's shipping dock (equipment plus distributor markup, with no dealer involved?) Or, are they measuring from the manufacturer's shipping dock (no distributor markup, no integrator, no labor?)
Another factor is HOW the researchers did the research... I can do a lot of my own legwork using public data, so for me it's a "make vs. buy" decision. Reputable research outfits are happy to disclose the precise methods used; others view their method as a trade secret. Ask: and if the company won't share the method to your satisfaction, run away. (If the company has to give away fake awards to get noticed, run faster!)
Also: having some understanding of what researchers actually do will help you make these decisions. I break it all down into two groups: Qual and Quant.
"Qual" (qualitative) work is when the researcher performs interviews about a topic and summarizes the findings - an example would be to interview ten integrators and ask them what sorts of jobs they expect to be pursuing in the coming year. Qual research is important, but computationally "squishy"... that being said, it's helpful when you are tackling squishy issues such as consumer behaviors or preferences. Qual research is also helpful in informing "quant" (quantitative) research: Qual tells you where to look, Quant helps you see.
"Quant" (quantitative) research is when the researcher gathers data of a numeric sort: how many integrators pursued Canadian government contracts in 2012, how much work (in currency) was granted in 2012, etc. Quant data also comes from bulk surveys - one well-known consumer electronics research firm sends quarterly Web surveys to something north of 30,000 broadband equipped households... which may not tell the researchers what the customers are dreaming about, but can very specifically tell what they are buying. Compare the researcher's methods to your need for information and make sure you're speaking the same language.
Quant data on global video hardware, by the way, is the toughest data to come by in this space, because of the OEM issues. (The truth may be Out There, but none of us have it yet ;-)
Hope this helps, and apologies again for the undislosed name.
Newest Discussions
| Discussion | Posts | Latest |
|---|---|---|
|
Started by
Jack Plunkett
|
4
|
less than a minute by Sean Patton |
|
Started by
John Honovich
|
7
|
about 1 hour by Duc Tran |
|
Started by
Carter Maslan
|
6
|
4 minutes by Undisclosed #1 |
|
Started by
Undisclosed Integrator #1
|
1
|
less than a minute by Undisclosed Integrator #1 |
|
Started by
Undisclosed Integrator #1
|
4
|
34 minutes by Undisclosed End User #2 |