From time to time, gaming companies offer us products to review here at Campaign Mastery. Past such reviews include Tome Of Monsters from 4 Winds Fantasy Gaming in ‘Perfect Skin: Some Musing On The Design Of Monsters’, Players Option: Flaws in ‘On The Nature Of Flaws’ (also from 4 Winds), Nobis: The City-States from Pantheon Press in ‘Nobis: Going Renaissance and Loving It’, and Eureka! from Engine Publishing in ‘Eureka! Some Inspiring Notions’. We have also occasionally been invited to review websites and online utilities; one of our most popular articles is just such a review, ‘Building The Perfect Beast: A D&D 3.5 Online Monster Generator’.
I always get an ego-boost from such offers, but there have been a few such offers I wasn’t able to accept, and thinking about why was the initial inspiration behind this article.
We pride ourselves at Campaign Mastery on offering value for money – and, since time is money, this means that we want every article we post to be an honest attempt at rewarding anyone who takes the time to read it by offering a substantial contribution to their games.
That means that reviews take as much time to write, if not more, than a normal article – if they are to be done right. The time to write the article is going to much the same, and on top of that is the time to read, assess, and appraise the product being reviewed.
To achieve the value-for-reader’s-time targets that we have imposed on ourselves, we have established an unwritten set of standards – at least, unwritten until now, because the bulk of this article is to be a discussion of these standards:
- Value For Money,
- and Scope.
Why write this article?
I have two motives in writing this article – firstly, educating our readers on the principles that shape what we offer for them to read. Second, offering the writers of other sites some food for thought; while there are some excellent and ethical reviewers out there, there have also been all too many reviews that clearly violate these standards. The results of such violations are always a review that misleads the reader to some extent, eroding the credibility of the publishing site. There may be other motives as well – such as needing something to post this week – but those are secondary concerns.
Whatever opinions we present as part of a review will be our honest impressions of the product. If there are things we don’t like, or have trouble with, it’s our duty as reviewers to make those clear in the review, because it’s a reasonable expectation that some of our readers would also encounter those same issues. Equally, if there are things we like, you need to be told about those as well.
Sometimes, these opinions are misinformed – we may have missed a key paragraph of text, or a key menu option. Part of the honesty standard is to offer Mea Culpas when this occurs, and another part is to welcome replies to our reviews from the publisher of the product.
We’re also mindful that ANY product takes a long time to develop; people have invested long hours in its development and must be assumed to have done so to the best of their ability. You wouldn’t be happy if someone trash-talked something you had spent a lot of effort creating, so negative elements in a review are always delivered with an air of regret on our part. There’s nothing we like better than being able to laud a product without reservation.
Through twitter, I have friendly relations with a number of game product publishers, such as Robert Thompson from 4 winds and Jonathon from Nevermet Press. Johnn has been active in game product development as a professional writer for even longer, and has even more contacts within the gaming community.
No matter how much we may like a publisher personally, we try to leave that on the shelf when reviewing a product from by that publisher. The same is true of other sources of bias, such as how much we like past products from that publisher – we regard those as setting a standard against which the new offering can be measured, nothing more.
Those are both general sources of bias. Sometimes there is a potential bias relating specifically to the product being reviewed, such as being offered a free copy or reciprocal endorsements. While we do our best to maintain impartiality in such circumstances, we will always describe any such biases openly and prominently so that the reader can take them into account when assessing our review.
This criterion relates to our expectations of a product. If it relates to encounter generation, we don’t expect it to make omelets as well – so much the better if it does, that’s an unexpected bonus (and worthy of note). We don’t expect a product to do anything more than the publisher promises – on the cover, on their website, etc. Sometimes, that leads to pleasant surprises. There have been game products that I would not recommend for their primary purpose – but which are absolutely fantastic and worth purchasing for some extra that has been offered.
A key question that we always have to answer in a review is whether or not we would use the product in one of our own campaigns, and if so, in what capacity – and, if not, why not. The goal is always to present enough information that each reader can decide for themselves whether or not a product is something they want to consider buying.
A related question is whether or not we like a product enough to spend our own money on it. We have been quite positive about Holly Lisle’s ‘Create A Character Clinic’, reviewed in ‘Creating Alien Characters: Expanding the Create A Character Clinic’. This product’s advertising on Holly’s website was enough to persuade me to buy it – but I got a set from Johnn for my Birthday before I could do so.
This is, essentially, putting your money where your mouth is. It also plays into a related criterion: Value for Money.
Value For Money
There have been any number of products, gaming and otherwise, that I have come across over the years that I would happily have recommended – at half the price.
Now, setting the price of a product is a black art that no-one has ever fully mastered, because it hooks into all sorts of other criteria, especially in terms of expectations. Something not only has to BE value for money, for the price to be justified, it has to appear to be value for money before the buyer cracks open the covers. If it doesn’t do that, the price is so high that it will negatively impact sales.
The biggest service that reviewers can offer is to bring a product to an audience’s attention that is value for money even if it didn’t appear to be so from the product advertising, about which most modern audiences tend to be cynical – or warn them about products that sound great but that fall short of achieving what they promise by such an extent that they aren’t worth the asking price.
In my bio page here at Campaign Mastery, I state that I have never read a game supplement without thinking that I could have done at least some part of it better than it has been done (and I don’t except products like Assassin’s Amulet that I have co-authored).
Sometimes, that statement relates to some content that hasn’t been executed as well as the rest of the product, but at least as often, it refers to something that could have been included but has been left out – usually, I think, because the authors didn’t think of it.
I always like to add “something extra” to a review – part of the “value for money” ethos of Campaign Mastery – that carries the article beyond simply being “I liked this” and “I didn’t like that”. In other words, I like the article to have substance beyond simply being a review. That happens naturally, because when I read something, I’m always asking myself “How Can I Use This?”, and sometimes the answers are something original.
It’s a huge ego-boost when the publisher replies with a “great idea” comment about such ideas. It happened most notably with the Eureka! review.
I like to get my hands on the mechanics of anything I review. Why does it do things a certain way? What Else can be done that way? Is there anything I can learn about creating effective game mechanics from it?
Reverse-engineering products in this way adds depth and insight to a review, so it is worth doing for its own sake. I started doing it for the reasons nominated in the preceding paragraph, but these days I consider that a side-benefit.
The final criterion is to look at the whole, not just part of a product. This comes back to giving enough information that a reader can decide whether or not to buy the product based solely on the review, and on deciding whether or not a product is value for money.
It all takes time
None of these are achieved quickly. Some criteria are, nevertheless, relatively painless; others take time and effort and some deep thought on top.
It follows that if time is short, there are two options: publishing a substandard review (according to our own arbitrary standards, admittedly) or not publishing a review at all. Given the commitment to quality that we strive for at Campaign Mastery, that’s no choice at all – better no review than a substandard one. (As a side-note: we apply the same standard to weeding out spam comments – even marginal ones tend to go into the trash. Have some legitimate comments been thrown out with this bilgewater? Almost certainly. But that’s better option than polluting the contributions of our readers. Better no comments on a post than spam – no matter how flattering a superficial piece of spam might be, once language and grammar are cleaned up. Have we missed the occasional piece of spam? Again, almost certainly – but those occasions would be few and far between.)
But here, once again, my personal and professional ethics manifest themselves. I try never to do half a job. I take pride in what I write and the contributions that I’m able to make. It is my most sincere hope every week that someone’s campaign, somewhere, is improved by what I write.
If I accept an offer of a free product, I feel compelled to thank the person or company extending the offer by publishing a review of that product. I won’t accept an offer unless I have the time to provide recompense with a review that’s up to scratch. Usually, there is no insistence on such reciprocation, not even a request for it – the offers are made with no strings attached. My pride and ethics attaches the strings.
The ethics of paid articles
We frequently get offers of payment for publishing articles with links to gambling sites, and almost as frequently, we turn these down. If reviews have to meet our own ethical standards, non-review articles have to do so even more stringently. I couldn’t and wouldn’t write an article for no other reason than as an advertisement for a site offering online bingo or internet poker or whatever without being sure that the article had some intrinsic value to our readers.
Anything less than this approach is taking advantage of our readers in an unconscionable manner. Our readers come to Campaign Mastery to read something that will enlighten, empower, or improve their games, or at least make an honest attempt at doing so. No matter what other interests an article might have, it has got meet that primary objective.
Our articles must have inherent validity and value or we damage our most valuable asset – our credibility.
Nevertheless, there have been a few occasions when we have been able to naturally fold a paid link into an article that has genuine merit in its own right. There have even been one or two occasions where the site that we were being paid to promote has enhanced an article – for example, my article ‘A Different Perspective: changing the dynamic with a different metaphor’ which considers the benefits of using mechanisms other than die rolling to simulate complex situations, a principle that I put to good effect in a subsequent Ask-The-GMs about the best way to simulate a fishing tournament without reducing the game to a procession of die rolls (‘Ask-The-GMs: How to set up a fun fishing mini-game’).
Wikipedia defines Ethics as a branch of philosophy that involves systematizing, defending, and recommending concepts of right and wrong behavior.
In this article, I’ve tried to set out a summary of what we here at Campaign Mastery consider ethical behavior when it comes to reviews – and to explain to various publishers why I haven’t accepted their offers of free merchandise.
Bottom line: we want you to be able to trust what you read on this website. We don’t want you to have to rely on our legally-obligated disclaimer (‘Material Connections’).
Some other time, I’ll go into the actual process that I use to write a review. Just don’t get me started on the ethics of RPGs…