Open Discussion Thread of New Rating System

[sirhatter]

This is meant to provoke dialogue more than anything.

I am deeply indebted to TheOnlySaneOne, for reasons that will be clear soon enough. This idea had been rolling around in my head, and i started on a 15000 word article over christmas (its actually finished, but is horribly disjointed and needs revision).

As many of you know, I am of the stance that every battler should try to maximize their own pet and their potential. This means spending their neopoints as best as they can between training and weapons, so they can beat the widest range of opponents possible.

Too many battlers are almost forced to choose exclusively between 1 player and 2 player fights. The reason for this is the league system implemented 7 years ago when people did not know as much about the battledome. This has resulted in some people basically runining their pets from achieving their potential (training 100 strength, 100 hitppoints and 500 defense for example). This has also stopped other less crazy people from really training their pet properly, for fear of getting squished in 2 player fights. Even on battledome chats you see people looking for opponents with similar stats. When it gets really crazy people even put obscure weapons restrictions on: no freezing, no multi heal, etc.

People then maximize their pet for the restrictions, but if a war comes around or plot they end up losing to the hard core 1 player grinders, who wont enter 2 player tournaments because their pet is too weak weapon wise.

It ends up being a rather screwy system. The worst aspects of this are in the n00binator movment. I think I explained my personal (and the unofficial but almost official IDB) position on why n00binators are not something we are promoting here: http://forums.finalhit.org/thread-21027-page-2.html


In any event, if we could some how boil down a battlers ability, comprising of stats, weapons, and tactical ability into a single number, it might promote fairer matchups. There are several advantages to this system.

1) it gives people something to strive for. If they are unhappy with their 1200 rating, they can train their pet, upgrade their weapon set, or get smarter about tactics. In other words, by doing something, and then proving it on the battlefield, they can move their rating up over time.

2) It avoids people sandbagging. People get aggravated when they feel they are in a mismatch, whether its the league system that have created these tank behemoths, or they feel that something shady happened with weapons being lent or whatnot.. With this system, if you are a 1350 level pet, and trouncing other pets in the 1300-1400 range, within 10 or so fights, you will be rated 1500 or thereabouts, and will be fighting much tougher opponents. The idea is to have a truly fair fight between opponents if their ratings are the same.

Now how do we determine the rating?

The initial seeding is dramatically troublesome. If you join yahoo or microsoft games (say spades or scrabble or really almost any game), you start with a default rating of 1200 or 1600 or whatnot.

The advantage these game servers have is they have tens of thousnads of users in some cases, all anonymous, and while they can quickly build a large database of matches to make sure the match results work out a players true rating over time, there is no way to seed people initially.

We have a disadvantage and an advantage. Our disadvantage is that our community is small, and not growing at any rapid rate. Viacom in general has put next to nothing into the battledome, and hasnt even really fixed the glitches that plague the system from time to time. We are a self-built community, and a small one at that.

Our advantage is we have a relative idea before people set on teh field what their rating should be based off of a number of things. Controversially we initially seeded rankings based off of 1 plaeyr wins and not initial statistics. 1 player wins, i hoped reflected weapon strength and tactical ability and not just raw force. In retrospect though it also reflects laziness. Most peopel didnt max out so some seedings were screwy (sorry!)

The actual formula for each match, well this is a timeless formula and works really well from game to game

Its as follows

If you win:

New rating = old rating + (400 + your opponents rating - your old rating)
-------------------------------
25


Not sure if that will translate quite right or not, but the idea is if you are rated evenly with your opponent going into a match, and you win, you will gain 16 points . If you are 100 points weaker (an underdog), you will gain 20 points. If you are say 300 points stronger (a big favorite), you will only gain 4 points.

For extreme values in rating differences > 400 the gain or loss is 0 or 1 point.

Anyway the formula is tried and true. However many points you gain from a win your opponent will lose from a loss. The opposite of course holds true. If you are a big favorite and lose, you might drop 28 points, where that win would only gain you 4 points. The idea being that a 300 point difference in ratings means the stronger pet should win about 7 out of 8 times. (maybe the time they lose they blunder and get their best weapon stolen, or get double-froze, or some other unlucky or foolish scenario happens)

A draw counts as half a win and half a loss for ratings calculation purposes, so the lower rated pet will get a slight increase. In the end rating points cannot be destroyed, nor created, except when new battlers enter the fray.



The problem is we dont know exactly where everyone should be rated to begin with, and as a result we ran this tournament, to try to test waht a system might look like. Whats doubly challenging is that I dont think our ratings are truly reflective of the ability of the pets in the first place, at least not now. After 10 or 20 fights, sure. After 0 points when we took a guess off of flimsy 1 player scores? Not sure.


Either way, if your rating and your opponents rating were equal, we are expecting an equal fight.

What are the advantages of this? Over time, instead of people advertising league 7 fights, and secretly having enough weapons to blow up the old Soviet Union, they will advertise a range of ratings from say, 1100-1200, their own pet being 1140. They will hopefully get a truly even fight, and someone fighting them can have some confidence that they are in a fair fight. Tournaments can be held sorted by rating groups. The beauty of it is the sandbagger that maybe gets lucky once and wins a tournament in the 1500-1800 division might gain enough points, that their new rating is 1850. Next tournament they enter they are now in a higher division.

There are some flaws. For one, if the system does go live, I am worried about throwing matches. Especially if it means they throw matches in meaningless situations so they can get into a lower tournament bracket for future matches. This is a bothersome problem for me. The online gaming communitys have the opposite problem. I used to play othello at a reasonably high level (went to the US Championship once and played a few other live board tournaments occasionally travelling a few hundred miles. I was not world class by any means but in spite of my limited abilities I was good enough to make very strong opponents sweat out victories).

Online in othello, you would nto only have the cheaters (not sure what the equivalent of that is in neopets, but maybe double burrowers or weapons glitchers or even just autobuyers to fund huge pets), but also sometimes youd get a friend to join, throw matches against them to knock your rating down, and then play your vaunted enemy, and if they lost to you they might lose 30 points off their rating instead of 15. If they went "on tilt" (crazy) and played you 4-5 times and lost all 5 games they might take a real beating to their rating that might take a month to recover from. Of course over time they would recover.

You would alos see peopel create phantom accounts to feed wins to make sure that a rating stayed above a certain minimum.

I dont know how these issues translate to us, but we have the advantage as a self-policed community that we are small enough to catch shady behavior.

I also hope that we have the right mix of people ethically as to not cheat.

Anyway, the tournament so far has been a mixed success. I know the brackets have seemed helter skelter (we had no idea how many people were entering), and some matchups have been absurdly unfair (sorry).

Over time though, we are hoping that we can launch an online database that will store ratings, and make the idb THE destination for 2 player battling, and encourage people to really maximize their pet from a holistic approach.

Hopefully that makes sense.

If you have any suggestions as to how to improve things, I am all ears.

TOSO has kept this tournament from being a failure. He originally suggested the power pool idea to avoid total slaughters of pets like Kemito fighting the weakest pets. He also has kept me on track from implementing crazy things. He has been a good leader, and I can tell you most of the times hes suggested something I have come to his way of thinking on it. Alot of the chagnes were minor and logistical, but they made sense.

This is his baby as much as (if not mroe than) mine as he is our new tournament director, so feedback for tournametns in general would be appreciated. Going forward we are hoping to avoid the antiquated league system and get some good matchups. I have already seen some pets that would be considered 2 or 3 leagues lower in the old system win matches, so i know things are possible.

Thank you in advance for reading this, and I will read every comment, and hopefully respond to it too, either publicly or at least via idb pm.

Cheers,

[chicken]

It'd be awesome if it works out, and would be nice for people like me who have devoted themselves to 1p and completely abandoned 2p, but it seems awfully complicated... Also, the BD does seem to be dying a bit so it may be too late in time for this to work. I wonder as well if IDB will be able to really run this, since there aren't really very many active mods here, and even fewer people who have much experience running BD tournaments and organizing things. IDB hasn't done much tournament wise in years. Apparently only sirhatter and TOSO are running this one, and I think more leaders and organizers are going to be needed than that.
But yeah, if it works... 'Twould be pretty cool.

[Shattered Rift]

(04-15-2009 09:09 PM)chicken Wrote:  [T]here aren't really very many active mods here... Apparently only sirhatter and TOSO are running this one

It's division of labor. Some of us are busy with outside things (such as myself), and others have been working on other projects. Though the rest of your post is true: it's been a good while since we've hosted a Two Player Tournament.

I wasn't able to follow this (and I haven't touched the Two Player sphere in years), but what was the exact range that the seeding allowed? It seems like the answer would be to either increase the range or hope that it leads to a natural desire to max out before entering this scene. Given the newness of this system, it's no surprise that people didn't care enough to fully max. Also, you could work backwards from the ratings where various Pets landed after this tournament and figure out how those pets would have performed in One Player (at their limits). Though I realize that one is skewered by the initial seeding not being the flat 1200(?).

[theonlysaneone]

(04-15-2009 09:09 PM)chicken Wrote:  It'd be awesome if it works out, and would be nice for people like me who have devoted themselves to 1p and completely abandoned 2p, but it seems awfully complicated... Also, the BD does seem to be dying a bit so it may be too late in time for this to work. I wonder as well if IDB will be able to really run this, since there aren't really very many active mods here, and even fewer people who have much experience running BD tournaments and organizing things. IDB hasn't done much tournament wise in years. Apparently only sirhatter and TOSO are running this one, and I think more leaders and organizers are going to be needed than that.
But yeah, if it works... 'Twould be pretty cool.

We're working on a very automated system for this. One would be able to report a battle, the other person would confirm it, and the rating would be automatically updated afterward.

[chicken]

If it's automated then it would seem to have a great chance... Of course it will still come down to whether or not people buy into it and adopt it over the old league system. The league system has the great advantage of being entrenched already, having been around for so many years. I don't have any idea what will happen but if some major guilds adopt it then it'll probably be a success.
OTOH... so many people have already based their training of their pet and their wep selection because of the league system. It's going to be hard to get them to like this method. Although, probably the ideal would be having some tournaments using the rating sytem and others leagues.

[final_phyre]

People fought against the league system like mad when we switched over to it from using boosts, but leagues still eventually took over. I say that if a few good tournaments can be run on this system, then you'll get the higher levels of battlers to switch over. If you can get them to make the change, then the rest of the community will follow eventually.

[Lounge Pants]

This raiting system seems really intesresting to me, it seems like a much better idea then the league system. I was wondering if there was significant movement on this? But i have just a small suggestion in terms of a raiting system, although your formula provides a change. I think that a formula involving an expernetial drop off or power law would be more appropriate then your linear one since you'll have to have an odd cut off and the potential for raiting fixing, as people with over a 400 raiting difference have nothing to loose so once people are past a certain rank they would gain nothing from contiued playing.

[e27182]

The system used here is mostly arithmetic. A geometric system is more mathematically sound (of course, over time, it will all average out). I like this system much better than the old league system, but maybe a similar rating type system involves a more geometric system, which allows for more variation, so that, say, a 100 point rating difference will have a theoretical odds of the higher seeded person scoring, maybe 3.5 points out of 5 or so (with wins being 1 point, draws being 0.5 points, and losses 0).