Limitations of Motorcycle Magazine Road Tests

When I decide to buy a bike, I like to know what the magazine testers have to say about it. In fact I like reading motorcycle magazines even when I am not about to buy a bike. But it is important to put motorcycle magazine road tests and opinions in perspective.

The good thing about magazine testers is that they are usually objective. Their objectivity, together with their unique opportunity to ride a wide variety of motorcycles, does make their opinions valuable to any prospective customer. They are also usually pretty good riders, and can better test the limits of braking, cornering, and acceleration than most amateurs. They also have a lot of knowledge about motorcycles and good observational skills.

But I don't completely trust magazine road tests for my decisions, and I would like to just give a few reasons why. Most obviously, magazines are subsidized by advertising, and much of the advertising is provided by the motorcycle manufacturers. And that's not all. The manufacturers also provide free test bikes, and trips to press launches. So the magazine is going to be very careful about writing anything blatantly negative. Sometimes you have to read between the lines to find the hidden caveats about a certain bike.

In a road test from the August '75 issue of Cycle, called "Eight for the Open Road", there is some Harley bashing going on the likes of which we have not seen in the motorcycle press for decades. Quote: "It could also be argued that the FLH [Harley-Davidson] on the basis of function is less than a motorcycle. We so argue." And they accordingly ranked it last in the test. You will not find that kind of black and white reporting today in any magazine still carrying advertising.

[Just by way of explanation, Harley Davidsons were not good bikes in 1975, and Harley Davidson got things turned around only when they copied (or invented for themselves) modern motorcycle manufacturing techniques that the Japanese makers were already using at that time]

Here is a comment from Cycle Canada, August 1999. This is about as strong a negative statement you are going to find, and I think Cycle Canada is one of the more hard line motorcycle publications. It refers to the Kawasaki Drifter 1500, a skirt-fendered Indian motorcycle lookalike.

"the quality of detailing and finish lags well behind the more conventionally appealing Yamaha Road Star....that makes for devastating competition for the drifter." and "...for the dedicated skirt chaser... the stiffest competition for the 1500 Drifter is the 800 Drifter, which we would prefer regardless of price."

Those are some harsh words, still not as harsh as the Cycle testers back in 1975. And yes, the Drifter 1500 has a full page ad in the same magazine that I got the quotes from.

My priorities are different from magazine testers. It is different when you own a bike, compared to test riding a bike. Magazine testers often complain about unusual controls, like the old BMW turnsignal buttons. An owner will get used to them quickly, because they usually ride only one, or at least a limited number, of bikes. An owner will be able to customize the motorcycle to their own comfort much more than a short term magazine tester. Even when the magazine has a bike for a year-long test, no one gets to modify the bike to their liking because it must change hands often. Magazines test new bikes, and rarely worry about or write about about the maintenance procedures. Sometimes the regular maintenance of a certain bike can be needlessly difficult compared to other bikes. Maintenance procedures usually remain a mystery until the bike has already been purchased for some time.

Finally, no magazine can fairly or accurately predict the reliability of a given bike. Sure they can tell you if their test bike blew up on the 401, but that rarely happens. Yet reliability is one of the qualities nearest to my heart. It is also the one you can predict the least.

Picture: A magazine from July 1970, the cover says "The time will come when only the medium weight bike makes sense. Enough, after all, is enough." Though the words are still true today, it is now apparent that motorcyclists are not all about making sense.