How Do You Feel About Product Reviews Where the Reviewer is Paid Large Sums of Cash?

Checked out the reviews at the site. Well, technically, for example the one on GoToMyPC ( http://www.brighthub.com/Reviews/tabid/53/articleType/ArticleView/articleId/107/Default.aspx ) is in no way so deep and informative as my review available at http://www.smartphonemag.com/blogs/index.php?blog=3&p=1571&more=1&c=1&tb=1&pb=1 . For example, it has completely ignored comparing the products to the competing ones (not one software product exists in void), making some serious bandwidth usage benchmarks etc. Anyone can compare the amount of GoToMyPC-related information of my article to that of the article at brighthub - I don't think the info in my article were on the loser side :)

I might need to send them some of my reviews, just to show them how a really in-depth review should look like? ;)

As far as getting paid for a review is concerned, this is why I, mostly, don't do non-comparative reviews (I prefer full round-ups), particularly when I get the reviewed program for free.

It's very hard to openly speak about the really bad sides of an application / game if you know you'll bite the feeding hand. Even harder to completely "trash" it, as I often do in my comparative reviews / roundups. Of course, better reviewers do avoid writing a review than tried to make a product better than it really is. But, in cases, it's really hard to resist the temptation to write something that you know the developer will like and also know it's not really fair (because, for example, it doesn't compare the product to other, competing products, which is, again, essential in a real, non-advertisement review).

Wow, that's a lot of money for a review!

I think that if a developer is the one paying for a review than the review would be complete garbage. I wouldn't even read such a review.

However, if a publisher paid for a review and the developer was in no way connected with the publisher then there wouldn't be as much of a problem.

For example, if a magazine wanted a product reviewed and they offered to pay for a review, I don't think this would be too problematic so long as the magazine wasn't being paid by the developer for the review, etc.

Then again, any time a critique is paid for by anyone, the risk of bias is introduced.

Nice post!

Hal: have you ever considered adopting a similar policy? :)

(That was just a joke)

Reputable magazines have a clear "church and state" demarcation. That is editorial and advertising don't mix. I am quite proud of how we have stood our ground on this division.

The way we do it, is we contract advertising sales. That is, our editorial in the magazine is created / edited in Fairfield, IA by our company Thaddeus Computing. Thaddeus Computing contracts ad sales to Ascend Media in Overland Park, KS. I am the only person in the company that regularly communicates with both sides (hence my title "Exec Editor" and "Publisher".)

All of our independent, volunteer, bloggers also represent the magazine, and I have never told any of this team what they should or should not write. Our Best Software Awards are designed so that independent experts select winners - they are not influenced by who advertise.

We also publish a lengthy document about how content gets placed in our magazine: http://www.smartphonemag.com/publish

I am not saying we are perfect, and that there aren't gray areas. However, we really try hard to hold the line between editorial and advertising.

Having said that there are magazines, radio and TV shows where if you pay, you get your product covered. Actually, that is OK as long as it is clear that that is what is happening (like TV Infomercials). However, the problem in any media, including the Internet, is when that information is concealed, and the reader is not aware of influence.

The Internet is still the "Wild West", and editorial standards have not evolved yet.

"All of our independent, volunteer, bloggers also represent the magazine, and I have never told any of this team what they should or should not write. Our Best Software Awards are designed so that independent experts select winners - they are not influenced by who advertise."

Yeah, this is why I've come here to blog, on the first place :) When Hal asked me about my coming here, my first question was something like this: "would I be allowed to trash something I write about?" (I've been in the publishing / tech writing business for about 20 years and have met several editors that doesn't tolerate "negative" articles, so, I like clearing up this question before agreeing to writing.) When Hal answered I'm given completely freedom and I can trash anything, I agreed. This is why I'm really grateful for Hal (and yes, he's telling the truth - otherwise, articles like the one on IBE Backup couldn't have been publishd).

Werner, I really appreciate your comments.

Thanks for all the interesting comments. Specially Hal's comments makes good light on why this blog exists with the wide spectrum of views reflected here. I would not be here unless the open flow of opinions were reflected.

It is interesting how the second link I provided above is not directly accessible via the pages on the first link. Also the company states: "Get the Truth with Unbiased Reviews".

Seems to me that the large sums paid for negative reviews is a losing proposition for the reseller. Since the offer is only good until the end of October, Very few negative reviews would be part of the overall reviews posted though.

Seems like everyone that participates on the website earns points but no reference to what these points can be used for in the end. http://www.brighthub.com/tabid/121/default.aspx#2

I feel that the integrity of the reviews and the reseller gets lost in this form of marketing. In the end, I would browse over to the original software manufacturers website to place my online order for a given title in this case. :-)

"There is NO way to get a person to spend serious time on preparing a product reveiw without being sure there is a compensation mechanism in place to be sure they are properly investing their time."

I COMPLETELY disagree with this statement. I have been reviewing PDA software for almost 4 years now, and the ONLY compensation I receive is a free copy of the software I've reviewed. There are certainly times that I don't spend long with the software, but that's because there's either no reason for it (the game took 2 hours to complete, for instance), or the product is so bad that I don't feel like wasting my time. On the other hand, there are other products that I have probably spent way too much time with because I wanted to be sure that I gave it a fair shake before I wrote my review.

If someone is doing this strictly for the money, than I don't really care about their review, because they are simply going to say what it takes to get their next piece of software. I'm not interested in that kind of review. On the other hand, people who aren't getting paid for their reviews have nothing to lose, so they feel more open about writing what they really think about the software. Of course, I can't really prove that, but this is my feeling on the subject.

I will say that I am a bit baffled by reviewers who shy away from writing negative comments about software. I don't LIKE to write bad things about products, especially if it's a small operation that will probably make little to no money off of the product even without my review. However, as a reviewer I feel it's my obligation to make sure people are getting the best product for their money, even if it means NOT buying the game I'm reviewing. If I play a game that's particularly awful, I want people who actually have to spend their money on the product to know what they're getting into. In the end it's best for everyone except, unfortunately, the developer themselves.

I've had several people over the last few months tell me I should start charging for my reviews, but I don't see the point. If it ends up being a good review, I'll feel like they paid me to advertise, and if it's a bad review then they've solicited my prose to trash them. Either way I lose in the end. With no money exchanged, the only ill will I ever experience is when a developer can't come to grips with the fact that not everyone will like their product, and I can live with that.

Quoted from above.
"The serious consumer review magazines ALL compenstate their reviewers as a career path".

A respected magazine called "Consumers Report" has actual employed staff and the magazine purchases all products they review. This is a great unbiased opinion on the products "Consumer Report" reviews I feel.

I do not have a problem with magazines that clearly states that a reviewer is employed to do reviews and is paid as a job from the magazine and independent of the actual advertisers themselves.

I do have a problem when reviews are paid for and directly linked to the sales of the same products by the company commissioning the review.

I hate to say it but one major website that has advertising associated with the reviewed product directly linked is www.cnet.com.

There are others but this website seems to be in your face with advertising of the reviewed product plus sponsors directly relating to the reviewed product. It is very rare you will see a review on CNET that's under 6.0 out of 10 if at all for any product.

Not all products in any category can have a passing grade if the reviewers are truly involved in evaluating the products honestly from an experienced users point of view.

I have reviewed a good number of products over the years that have received 3 to 5 out of 10. It is important for the general consumer to be aware that some products are not fit for purchase. Otherwise the integrity of the whole reviewing process comes in to question. Regardless of what the product is, there will always be some lemons and we as reviewers need to honestly reflect this in our reviews and it becomes very difficult when compensation of $1000+ is received based on the reviewers resume and the reviews word count.

The old saying goes, "It's not what you say but how you say it". Some reviewers can write 2000 words and say little of any significance while others can get right to the meat of the matter in 200 words or less.

If reviews are paid for, I feel this should be indicated as part of each paid review to clearly separate the paid reviews from the unsolicited or unpaid reviews. This will allow the consumer to make a better decision in the end when considering any product for purchase based on any review.