Post Rating and Member Reputation: An Explanation

Topic Tags:

HTML, for websites and MySpace:

BBcode, for forum posts and signatures:

If you have something you want to present to the entire community, post it here.
  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...

Post Rating and Member Reputation: An Explanation ( )

Postby Remæus on Thu Sep 11, 2008 10:30 am

Many of you have seen the re-introduction of our Post Rating System, as well as the matching Member Reputation that have replaced the user rank system. We've received hundreds of questions and comments about the system (and we thank you for all the feedback!), so we thought we'd take this opportunity to explain what everything means, why we've added it, and what you should be doing with it.

The first element you'll see is on the top right-hand side of all forum posts. There is a series of five stars that allow you to rate a post based on what you think of its contents, relevance to the topic, and insight.

A few key points about rating posts:
- When a post is made, voting will only be allowed for a period of time (between 7 and 14 days).
- When the voting period ends, we average out all the votes for that post to give it a final rating.
- If you do not rate a post, we assume that it was not a remarkable post and we give it three stars.

Our goal is to let users voice their opinion about posts around the site so in the future we can quickly identify posts that are very informative or entertaining, and feature that content around other areas of the site (which means, if you write posts that people like, we can make you famous!) . It is not our intent to belittle or judge anyone, so we have based this system entirely upon the opinions of our community members.

The next feature you'll see has replaced our user ranks (which were based on post count). We wanted to find a better way to measure user contribution to the community than post count, so we decided to add a Member Reputation metric in its place. These reputation scores are also displayed in the form of a five-star rating, and indicate the result of a very complex algorithm which calculates each member's reputation.

The algorithm that calculates the reputation scores take a large number of variables into consideration; average post rating, who rated the posts, average number of replies to threads they've created, what threads they've participated in, and even who they've rated (and how) - among many, many other things.

This algorithm is dynamic, and will automatically adjust itself based on what you've rated other people: the algorithm is intelligent enough to see that you're rating all your friends highly, but rating other posts and topics from other people lower. It will automatically adjust itself to rate users accurately, removing bias from social circles and (gasp) cheating schemes to prevent manipulation of the system.
User avatar
Remæus
Creator and Owner

A word from our sponsors...

Want to sponsor RolePlayGateway?


  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
thank you so much, I was entirely lost before reading this ^^
xenchantedx
FIVE coolio chickitas Lari Becky Lauren Illy and Jimmeh!!!^^ thats right fish Lady! Take that! lol
hehe beat THAT Lari ^^ mines huge and pink and evil, just like an overgrown bimbo! *pluuf* awww we got our own little fivesome going on ^^ wait...NOT LIKE THAT NUUU!!!!!!!!


SEXUAL HARRASMENT PANDA!!!!!!!!!!!!!!
"Shoot for the moon, even if you miss you'll be among the stars." - My Physics teacher
lol I am so happy ^^
User avatar
xenchantedx

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
I see that you took my advice, and probably others too, but anyways, thanks for such a detailed explanation of the whole thing.

Thanks,
Phouse
Image

(19:04:57) Surreal: I'm not STOOOOPIT!
(19:05:03) Surreal: >>
(19:05:09) Surreal: I'm just not very smart

Image <-Click it
User avatar
phouse512

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
Well, I guess then that after that... well integrated definition, it is nice to see that a rather larger than average amounts of my posts are highly regarded. This is a good selfconfidence boost, and I think this is a terrific system that you have inducted. May it drive fellow memebers to be more thurough with their posts, and help creat an even better Rp web site.
Nothing can be obtained in this world without some degree of Image
User avatar
Ketsueki Tenshu

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
Well, some people abuse the system and put good really good roleplayers down the ranking system and it dissapoints me sometimes. That system developed is it made to react to things like 1 person going on and rating everything 1 for the fun of it? :(
Darkness is only the absence of Light.


TWILIGHT
User avatar
Zero-Genesis

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
Auto-ranking a post at 3 stars just because it hasn't been ranked by a user seems rather unfair. If someone keeps mostly to themselves and a few small RPs and therefore doesn't get a lot of attention (or votes), it doesn't mean their posts are mediocre.

So far I am seeing a lot of talented RPers with low/mediocre ranks (possibly because people aren't ranking them), and a lot of very poor RPers with high ranks because they have a lot of buddies who rank them well. And God forbid you say something people disagree with in the discussion area and a flood of people rank your posts very low. This seems extremely flawed and prone to lots of abuse. The system isn't psychic. Regardless of how it considers individual ranks by individual voters, it won't be able to catch/discard every undue ranking. In the end, it seems like it had great potential to be very belittling to members.

If I enjoy RPing somewhere but every time I sign on I am met with a low/mediocre "member reputation," I'm not going to feel all too eager to keep trying.

But, thanks for posting a full explanation.
I am available as an RP mentor should you seek to improve.
I draw things that you may like.
I like Ronald Jenkees and you might, too.

I require a certain level of writing skill in my stories. Don't be insulted, I just prefer to RP novel-level material. (I, I, I, me, me, me. I sound like such a jerk.)
User avatar
ThatsNotPoetry
Global Moderator

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
I hate this system. I'm permanently stuck at 1.9, even if I give a five-star post. This system sucks and should be canned. The whole rating system should be canned because it's no indicator of how good you are. It's all opinion, which isn't good. And Member Reputation is just wrong. I have a 1.9 reputation now. While Remaeus and other high-ranking people have 4 or 5 star reputations. It isn't right, and needs to be canned. FOREVER.
"]Image
User avatar
True Grave

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
Overall, I think this is potentially an excellent member rating system.

At Surreal*Twilight (the role playing message board that I moderate), there was a very big issue with multiple people spamming useless nonsense in order to up their post count. It is still an ongoing problem, as it would seem spammers will never disappear as long as they have the rating incentive that is based on your number of posts made. The only thing we are able to do, without completely getting rid of post counts, is to enforce our no spam rule and to warn them (or ban them if they do not cease after being warned a certain amount of times).

There is a flaw in the post count ratings - even for those who are non-spammers and post meaningful replies. Your rank is entirely dependent on simply you talking. There are quite a lot of people who prefer to simply just role play and not participate in much of any OOC content (for whatever reason that may be). So, as a result, their rank is significantly lower because their post count is lower. Does that make them less of a member? Of course not.

However, with this rating system you're now enforcing here, it completely eliminates that issue. A person's rank now has actual significance. Other members around the site are able to participate and add their own input into the posts made by their fellow members, which allows them the ability to, in essence, voice their opinion. It makes a lot more sense when you're dealing with ranks to have something more in depth as this than simply a post count.

So, all-in-all, I really like this reputation system.
Image
DeviantArt | Facebook | EthereaLights | Surreal*Twilight

Stars fall like dust
Our lips will touch
We speak too much

-Breathe
Angels and Airwaves
User avatar
Syren Song

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Don't forget to rate this post! Comment on this post...
ThatsNotPoetry wrote:Auto-ranking a post at 3 stars just because it hasn't been ranked by a user seems rather unfair. If someone keeps mostly to themselves and a few small RPs and therefore doesn't get a lot of attention (or votes), it doesn't mean their posts are mediocre.


If a post's voting period ends without any votes, it doesn't get calculated into the reputation score.
User avatar
Remæus
Creator and Owner

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Don't forget to rate this post! Comment on this post...
Remæus wrote:
ThatsNotPoetry wrote:Auto-ranking a post at 3 stars just because it hasn't been ranked by a user seems rather unfair. If someone keeps mostly to themselves and a few small RPs and therefore doesn't get a lot of attention (or votes), it doesn't mean their posts are mediocre.


If a post's voting period ends without any votes, it doesn't get calculated into the reputation score.

That was my question. Thanks for clearing that up Remæus.
ImageImageImageImageImage
Image
Image
Now, maybe my signature shall clear up all this Forky being male business... Haha Surreal.

(13:18:17) Surreal: Chat is so addictive. Makes you high on fail
A Yugioh Roleplay~Click Here!
User avatar
TheAlmightyForkNinja

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Don't forget to rate this post! Comment on this post...
I have a question, and I am not sure whether it has been addressed or not, so please forgive me if I am repeating.

On my screen there are two stars on one line and three on the next down it looks like this:
* *
* * *

So, my question is this: On my screen would this be the five:
* X
* * *

Or would it be this:
* *
* * X

Basically, I'm not sure if it's two on the first line and then the next three stars on the next line, or if a glitch puts the last two stars higher than the others on my screen.

...sorry if this is a dumb question!
Well I’m not there all the time, you know : Some people, some people, some people,
call it insane, yeah they call it insane. I play Russian roulette everyday, a man’s sport,
with a bullet called life, yeah called life!:
:You know that every time I try to go : where I really want to be,
It’s already where I am, : cause I’m already there!

<KyojoKen>: What does riding a bicycle have to do with making love on a couch?
<Vlad>: NOBODY

Image
User avatar
The Rogue Doll

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Don't forget to rate this post! Comment on this post...
Remæus wrote:
ThatsNotPoetry wrote:Auto-ranking a post at 3 stars just because it hasn't been ranked by a user seems rather unfair. If someone keeps mostly to themselves and a few small RPs and therefore doesn't get a lot of attention (or votes), it doesn't mean their posts are mediocre.


If a post's voting period ends without any votes, it doesn't get calculated into the reputation score.


I'm talking about what you said here:
Remæus wrote:If you do not rate a post, we assume that it was not a remarkable post and we give it three stars.


Average votes, be it for reputation or for post quality, sounds awful, because both reflect the RPer. The person.

Of course, that was not the only issue I had with it.
User avatar
ThatsNotPoetry
Global Moderator

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Don't forget to rate this post! Comment on this post...
The Rogue Doll -- If you hover over the stars when wanting to rate a post, it should provide words that describe the star's rank. From worst to best:
"Meh."
"Okay."
"Average."
"Good."
"Epic."
User avatar
Syren Song

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
I don't like this system. It makes me feel like an idiot because I'm stuck in the 1. something..and then I see all these new comers who only have one post and they're 3 stars and it makes me mad..so yeah I agree with True Grave that this system isn't a very good idea because it makes people think they're not that good.
BTF Fanatic Girls: Angel(Craig)~Kirai(Mike)~Puppy(Matt) <333
Image
Craig:"MYSPACE IS UNAVALIABLE!" Mike:"You guys..." Craig:"WE'RE ALL GOING TO DIE!" Eric:"MYSPACE WORKS!! WOO!!" XD BTF rules.
User avatar
Angel Of Darkness

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
Angel Of Darkness wrote:I don't like this system. It makes me feel like an idiot because I'm stuck in the 1. something..and then I see all these new comers who only have one post and they're 3 stars and it makes me mad..so yeah I agree with True Grave that this system isn't a very good idea because it makes people think they're not that good.

Either the newcomers you are referring to simply did not get rated, or they had something worthwhile to say. Either way, just because they are new to RPG doesn't make them any less of a person and mean they aren't beneficial to the site.

The only issue I have with this is honesty. There are some people on here who would probably prefer to dish out unjustified low ratings to others in order to make themselves feel better or to bring down those who have high ratings out of jealousy. It is those types of people that are the only real threat to an otherwise decent reputation system.
User avatar
Syren Song

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
Angel Of Darkness wrote:I don't like this system. It makes me feel like an idiot because I'm stuck in the 1. something..and then I see all these new comers who only have one post and they're 3 stars and it makes me mad..so yeah I agree with True Grave that this system isn't a very good idea because it makes people think they're not that good.


Well, let's look at your posting record: you registered on July 14th, 2008 - and you've already got over 4,000 posts. That's 70 posts per day! If you're posting that much, you probably aren't spending very much time on each post.

Look at some of the posts you've made, like in this thread, for example. I can see why people are rating you lower - you're using the forums like they were a chat, and making super-short posts.

Syren Song wrote:The only issue I have with this is honesty. There are some people on here who would probably prefer to dish out unjustified low ratings to others in order to make themselves feel better or to bring down those who have high ratings out of jealousy. It is those types of people that are the only real threat to an otherwise decent reputation system.


As the algorithm learns and adapts, these dishonest votes will have less effect on the system.
User avatar
Remæus
Creator and Owner

  • 1 star
  • 2 stars
  • 3 stars
  • 4 stars
  • 5 stars
Comment on this post...
I've seen shorter posts then that. And it's called WRITERS BLOCK. Not everyone can sit in front of the computer and be able to write the first thing that comes to mind. And if someone can then they're damn good. All I said was I didn't like the system because it's stupid. You don't have to be so mean about it dude.
User avatar
Angel Of Darkness

  • 1 star
  • 2 stars