When I asked our Arguably Redundant Senior Editor (ARSE) what I should write about this month, he immediately replied, "Are magazinr reviews believable?" If you think about it, there are a million different questions involved here, from the global 'are magazines believable?' all the way down to the specific 'are paper magazine kit reviews believable?'
Is somebody asking, "Why shouldn't they be?" at this point?
Let's restrict ourselves to reviews of kits, radios, etc. in the sort of magazine that you find on general sale in newsagents, hobby stores, etc. Right here the majority of you are probably thinking that it must be wonderful to be in the position of being offered model kits or radios to review. Well, the first one that you tackle can be like that. However, suppose that you are offered the latest helicopter kit to review. You now have the kit, but what about a motor, silencer, servos, gyro, battery pack, etc. Normally, you have to supply all of those yourself. Let's now suppose that you have done a few reviews before and really can't afford yet another set of all of the above. If you are the sort of person that doesn't get attached to things that you have made, you can remove the gear from another machine and dispose of it. Wait a moment now, did we say anything about it being yours to dispose of?
Let's go back to that first kit review. You are now obligated to produce a write-up for that magazine. Anything less than 5000 words or so is not doing it justice. If you have not done any writing before you will probably find that 500 seems like a hell of a lot. And don't forget those good quality photos of every phase of the assembly. 25 to 30 shots should do it. All you have to do now is go out and produce 15 to 20 flying shots and you are done. Make sure you don't ding it before these are taken.
By now you might just be getting the idea that the number of people who can produce a reasonable kit review are comparitively few. The number that magazine editors can trust to do a good job to a deadline are somewhat fewer. Halve that to give the number who are willing to tackle another review, halve that again for those that are willing to repeat the excercise more than once a year, and you are running out of reviewers.
Wait a minute now. That review kit was provided by someone, usually the manufacturer or importer. Shouldn't they be entitled to see a copy of the review before it is published and given the chance to comment? What if they don't like it? At this point, your editor, or more likely the magazines advertising department, may well decide that they cannot afford to upset a good advertiser (anyone who pays their bills is a good advertiser). Right here the editor may have to change things, whether he wants to or not.
If the review is a really bad one, the only possible action may be to simply not publish the review. This decision could be made by either the supplier or the editor. In either case, it certainly does not improve the relations between the two parties. Nor does it improve the reviewers chances of being asked to do another one.
Perhaps the magazine actually bought the review kit, probably at a good price. Contrary to what you may believe, this does not give them the go-ahead to say what they like. They are still dependant on advertisers for their existence. Back to our reviewer, he has a model that belongs to the magazine and it has all his equipment in it. He may have agreed with the editor that the kit is his payment for the review. If he needs the money and expects to get paid for the review then he better not be the type that gets attached to things that he has made.
Does it occur to you at this point that we need to halve our potential reviewers yet again if we want to use only those that are prepared to say what they really think?
Let's look at another scenario (sounds like a new Japanese helicopter). Suppose a regular writer/potential reviewer approaches the manufacturer/importer of a helicopter that he likes and offers to do a review of that machine for a magazine in exchange for a good 'deal'. If the machine turns out to be a real 'dog', can he afford to say so? Perhaps he did all of this without telling the magazines editor in advance (a real 'no-no' that one) and has now 'lumbered' the said editor with publishing a review which he knows is not very honest. Even worse, the editor may not realise the truth until after the review is published.
One possibility that we have not covered yet is that of a well-known reviewer giving a bad review to a new machine by a well-known manufacturer. Assume that the reviewer thought that the editor had sent the manufacturer a copy, while the editor thought the reviewer had done so. Perhaps the editor decided that he wanted to publish a 'hard' review for a change. The amount of criticism received by the poor old reviewer virtually ensures that he will learn the lesson and choose his words carefully next time. The fact that he may eventually turn out to have been right is largely irrelevant.
The answer to our original question is, "Of course they are". But, if you have read this far, you should have begun to realise why kit reviews tend to be done by the same old handful of writers over and over again and why magazine editors tend to change at regular intervals.