Voting - What Went Wrong
Voting on comments has been removed - at least for now.
Here are the 3 main issues:
- Comments that were witty / funny received the most votes. For example, Brian had a number of the 'top' voted comments including his 'Genetec is better than a mop' one and 'pimps and pricing'. By contrast, in depth technical or operational comments, by anyone, received very little votes. I think this is the opposite of what it should be.
- Not enough people were voting. While it became more diverse over time, even at the end, a dozen people probably counted for about half of the votes, making it not an accurate reflection of what the overall membership felt.
- A handful of people voted ideologically, i.e., if they supported a manufacturer, nearly all comments supporting their manufacturer got upvoted and any ones criticizing them got downvoted, regardless of the quality or depth of the comment.
What I want from voting is for votes to broadly reflect member's opinion the accuracy / value of the comment, not whether it is funny or it supports their business partner.
There are changes in how we present / list voting that could help. We also may need to do a little educational campaign with members to make it clearer the goal of voting.
Thoughts?
IPVMU Certified | 01/15/14 04:02pm
Probably just best to dispense with voting as it seems it will lead to more contentious issues and needless debates that will distract people from what should be the primary focus here: security related products and solutions and how they fit in our world and business.
By contrast, polls have worked well. We have done a lot of them that have been very informative - line item breakdowns for quotes, H.265 interest, 4K camera interest, Kansas gun law, etc.
There's precedent that member responses can be valuable. I do want to avoid the voting issues over the last month or so but I do see a possibility for better voting.
To that end, let's do a poll on it:
IPVMU Certified | | 01/15/14 04:38pm
I hate to sound pessimistic but I think your desire to have voting "reflect member's opinion [of] the accuracy / value of the comment" is a lost cause.
I think that, in general, internet users are too conditioned by Facebook "like" and voting in blogs to upvote comments that they find entertaining or that they agree with ideologically and downvote those that they disagree with ideologically (or posted by a commenter that they personally dislike), regardless of the accuracy/value of the comment.
Certainly the typical IPVM reader is a higher quality than most, but I doubt you will get many people to regularly upvote comments that they dislike or that go against their personal ideology just because the comment is technically accurate. It's even less likely that anyone would downvote a comment that they agree with because of inaccuracies.
True.
And yet we all develop over time a relative sense of the trustworthiness of a poster even if we disagree with them often. Marty for instance is unlikely to make a factual error even though his opinions may be quite extreme. Carlton is unlikely to spend much time on personal opinion and prefers to just tell it like it is without superfluous embellishment. Carl L. is likely to say anything at anytime and Luis C. just the opposite but still both get their points across and neither are you likely to impeach.
All these posters have well deserved trust that would be acknowledged by most everyone that has been around long enough.
I think the challenge is to capture that 'sense' so that others can rely on it and others be inspired to match it.
IPVMU Certified | | 01/15/14 05:29pm
"we all develop over time a relative sense of the trustworthiness of a poster even if we disagree with them often."
Very true. It occurs to me though that only applies to frequent readers reacting to the most frequent posters. Someone who is not a frequent reader does not develop that sense, except for the few posters that comment on just about every article/discussion. Likewise, even frequent readers can't develop a sense for posters that comment infrequently or anonymously. That makes it beneficial for only a small community of the most frequent readers/posters.
I completely agree with you Chris.
This is why I usually read all of the comments, since you never know where the next pearl of wisdom will come from. Regretfully, for every Marty, Carlton, Carl, or Luis comment, you probably have 20 other comments being upvoted for superfluous reasons.
I actually find the list over on the right of the page showing the top posters a better reference than any voting results. This gives me a quick reference on who are serious day-in-day-out contributors to the forum. Not sure if this would be a good idea for IPVM, but in another (non-surveillance) forum I belong to, they use a hiarchy of labels (i.e. senior member etc.) issued to members based on years as a member and amount of comments made. Being a new member myself, I am cetainly not saying new members have nothing good to say. On the contrary, having fresh blood in any organization adds to the vitality of the content, but I know that in the other forum having these labels right next to the name of the poster helps me further qualify the person making the comment.
"They use a hierarchy of labels (i.e. senior member etc.) issued to members based on years as a member and amount of comments made."
I think something like that is useful though sometimes the labels are unclear / weird - 'big kahuna', 'grandmaster', etc.
Maybe something like putting total comment count next or near to the commenter name would give a sense of how active they are?
The Big Kahuna....you're killing me. ROFL
Glad you haven't lost your sense of humor.
IPVMU Certified | | 01/15/14 06:34pm
No offense intended to the most frequent posters here but I would argue that the total number of posts has no direct correlation to the quality of those posts.
Would it be possible to include some kind of metric to rank the quality of a poster’s comments, kind of like a commenting batting average? If you go with the Helpful/Unhelpful type of presentation, maybe you could give a ranking based on the quality of posts vs. the total number of posts. Perhaps a formula like:
(# of “Helpful” votes - # of “Unhelpful” votes) / total # of comments
For example, a poster that has 100 posts but only 5 “Helpful” votes would have a ranking of 5/100=0.05. Conversely, someone with just 1 post but 5 “Helpful” votes would have a ranking of 5. The rogue manufacturer that pushes their own equipment without explaining how it solves the problem at hand might have more “Unhelpful” votes than “Helpful”, leading to a negative ranking.
On the plus side, something like this might cut down on the amount of superfluous comments that don’t have much technical value. On the negative side, it might also cut down on the comments that are interesting and/or entertaining but not necessarily “helpful”, lowering the entertainment value of the site.
+1
IPVMU Certified | | 01/15/14 05:46pm
Even though I'd be perfectly happy to see voting go away and stay away, I have been thinking about how it could be improved. My 2 ideas are:
1.) Present the voting as Agree/Disagree. Don't know how much this would help. Most of the same problems could still arise but maybe presenting it this way might keep those voting focused on the actual content of the comment, possibly getting rid of the "popularity contest" aspect.
2.) Present it like many online review site do it, i.e. "Was this comment helpful? Yes/No". Maybe this would lead to comments that actually provide useful information getting the most votes and those that confuse the issue getting downvoted. Humorous posts, while always appreciated, would probably not get many votes.
Richard, I like your idea. I do think making the wording / phrasing / question more clear and explicit would help reinforce what we are trying to do.
That and banning people who always vote ideologically :)
IPVMU Certified | | 01/15/14 05:58pm
John,
Are you saying that, were voting available, you would have given me an upvote. :-)
I do like Richard's idea. The "Was this comment helpful?" voting does seem to work well on other sites.
Reddit and Slashdot both have a mature form of comment voting / meta-moderation. While insightful comments do receive upvotes, your observations (especially first and third bullets) ultimately are what filter to the top. Trolls and humor.
Granted I am a relatively new member, but I did not even know we could vote on comments. Where were the voting buttons located?
Just curious, is the voting limited to the Pro-members?
My 2 cents on comment voting:
Conceptually I love the idea of comment voting, but in reality I usually find no worth in it. In other (non-surveillance) forums I belong to where you can vote on comments, my experience has been that the nature of the voting is exactly what you have found here at IPVM. It's either a popularity contest, an opportunity for revenge, or an open door for folks who just want to pump or dump a cause or product. Hence, I rarely even look at the voting results for comments.
In the end, if I like the subject of the article or discussion, then I will read all of the comments regardless if people found certain ones better or worst. Perhaps if the voting was an accurate representation of the actual worth of the comment content I would pay more attention to the ratings.
Not sure how to improve the voting system. You would think in a professional forum like this folks would apply more rigor to the voting process, but I think this is an indication of human nature in general, which is at times impossible to regulate or comprehend.
Jerome,
The voting buttons were to the immediate left of a person's name on a comment, like so:
Only members could vote.
I don't have any strong opinion about whether or not voting should be continued, but I have some thoughts on the matter.
It was clear that sometimes voting was a topic of conversation, which might have been either entertaining or an unnecessary distraction.
It wasn't clear that voting was necessarily negative, since IPVM appeared to function equally well both before and after voting.
Is it possible that you gained any interesting information at all from the votes, as for example undeclared affiliations? If the votes did not inform IPVM management of anything of value, then since anything generally costs SOME effort, they probably WERE an unnecessary distraction from the otherwise superb work you have been doing.
Just my 2 cents. PS keep up the great work!
I like Richard's idea, and perhaps that could be expanded on a bit... as I previously noted, Techdirt gives two voting options, "Insightful" and "Funny". Perhaps it would be good to have both, so the witty comments can still get an upvote without it skewing the results as to the post's actual usefulness.
(I couldn't get the screencap to pick it up, but when you hover over "insightful" the mouseover reads "rate this comment as insightful, informative, intelligent")
I like the voting, whatever form it takes. Many times I don't have anything to add other than "I agree" or "This information was useful". The votes gave me somewhat of a way to express this without making a post that really doesn't add value to the forum.
Now I wish I could go back and review my votes, but I do not see a summary of what I voted on. If I can recall, I generally upvoted those comments I found 1. Useful/ Insightful and 2. Amusing 3. Some Combination of the two. The review site www.yelp.com has system similar to the one outlined by Mat Ion. Perhaps a "mini poll" on each comment would prove beneficial. Each comment would have categories that could be assigned to it. The categories that I could see myself using include: Useful/Insightful, Funny, Trolling/Report, Need More Info/ Clarify. So then users who only want to see the comments ranked useful could filter out the comments with no "useful" votes. Of course this requires some administration, so there's that.
On ranking users- I think perhaps we are looking at similar rankings. So every "useful" vote gets you, the user a point towards a "piece of flare" or an icon after your name. If you get 200 useful votes you get a an icon which indicates you have 200 useful comments...etc.
Just a couple of thoughts. I would use the voting system, a categorized ranking system makes sense to me.
01/15/14 10:18pm
To echo what many others have said, the most useful voting system would allow users to designate replies as useful, insightful, funny, and so forth, similiar to Slashdot's system. Simple up and down voting will result in an echo chamber or popularity contest.
Brian is the most popular person on IPVM!
We'll take a look at having different categories. We'll need to reimplement the backend, but that shouldn't be a huge deal.
If it helps anyone, know that my wife typically downvotes my comments.
Any chance you will share who was the most voted person, not most voted for?
Back in the days when I had a pretty decent hand tally you were near the top vote getters, plus and minus combined, tho you were treading water at +3 or so...
01/15/14 11:57pm
I think I would downvote this decision. The voting shows what the population cares about, not what the administration thinks they should care about.
01/16/14 01:45am
+1 for Marty
My 2 cents for the voting feature....I would say it was simlar to "like"feature on Facebook. It was fun but pointless at the same time.
Newest Discussions
| Discussion | Posts | Latest |
|---|---|---|
|
Started by
Steve Stowe
|
12
|
less than a minute by Undisclosed Manufacturer #4 |
|
Started by
Undisclosed Integrator #1
|
6
|
less than a minute by Undisclosed End User #3 |
|
Started by
John Honovich
|
9
|
less than a minute by John Honovich |
|
Started by
Brian Rhodes
|
8
|
less than a minute by Undisclosed #6 |
|
Started by
John Honovich
|
1
|
13 minutes by John Honovich |
