I think it is fair to say that we WAY underestimated the speed at which people would end up writing reviews and tips. It’s also fair to say that we way underestimated people’s response to this site as a whole. We originally thought that moving up the ranks would at least take a gamer a few months, and a lot of the rewards and achievements were based on that growth rate.
Incentives are a very tricky beasts to control. We’re not psychologists or economists, but perhaps we need some on the team. We’re not going to get rid of incentives, because that’s what we think makes this site fun, but we do need to tweak them, and that’s why we’re in beta.
It should be pretty obvious that we’d like to encourage high quality reviews rather than quantity. In theory, if we end up with tens of thousands of users, we could end up having tens of thousands of reviews for each game (1 per person).
Tens of thousands of reviews for a single game is not better than say 50 really good ones. In fact, it could be annoying to have that many reviews. But, hundreds of great reviews for a single game might be useful, if we can end up subdividing those reviews based on gamer types and other factors that people find useful.
Oh, and before everyone launches into a dicussion about wanting reviews to be sorted by rating … I’m working on that … so let’s not talk about that here.
Instead, what are some ways to use quests and incentives to:
- encourage quality reviews
- help weed out the less helpful reviews
- ensure that high quality critics get the rewards they deserve
Regarding point #2 … we want to be very careful that we do that in a way that is actually positive rather than offensive or embarrassing or too judgmental.
One thought we had was that when you are logged in and looking at your own profile, perhaps we could highlight the really good reviews and the least successful reviews and give the option of either deleting or editing the reviews. This might be based on the % of “Yes” grades after some minimum total votes. We’d need to implement a system where the ratings were erased if you deleted or edited a review, because past ratings would no longer be accurate if you edited a review.
I suspect that automatically deleting reviews that were poorly rated could make people mad, so that might not be a good option. Maybe they could be hidden or just not listed anywhere but in a person’t profile?
Another thing we could do is create a minimum length for a review, but maybe we should wait and analyze the statistics before implementing that type of restriction. Who knows, maybe there are some extremely good tips and reviews that are just single sentences.
Anyway … that should be enough to start the discussion.
Comments (39)