Provide a list of part #'s, and list labor as "feet of wire pulled", "# of cameras hung", etc. Then have IPVM evaluate that list for completeness as part of the selection criteria (i.e. have they thought of everything). IPVM is usually able to estimate street price fairly well on their own.
There is going to be no fair way to evaluate labor prices. Is A low because he's a trunk slammer, or is B just a ripoff? It's hard to make that judgement call over the internet.
If an integrator argues for a more expensive device because it "saves on labor", then they should provide some math illustrating where the "break even" point is where it would just be cheaper to do it "the hard way".
Let's keep the focus on design, because that's the valuable part. Finding out mega integrator X gets way better prices than me doesn't help, nor does finding out that small guy Y is willing to work for free.
Perhaps they could submit pricing on a separate form that IPVM doesn't look at until they've declared a winner, and then do a follow up piece in light of pricing information (possibly declaring a different winner in the second round).
I also think it would be good to have them submit "good, better, best" proposals. What's ideal? What would you do to meet a restrictive budget?
I am not sure on compensation, but $200 seems low. I imagine this should take a lot of effort (at least as much as a standard bid, if not a lot more). Maybe if we are solving a really small problem, but small problems are not that interesting.
Also, will the evaluators know who they are evaluating? I think it would be interesting to evaluate blind, at least at first. We are all slaves to our own bias.
I really like the idea of solving real problems. It would be great if you could find real customers that could describe real projects that they don't yet know how to solve. I think you'd need to vet that situation to make sure participants weren't potential bidders.
In all, really interesting concept.