Generated Aug-07 17:41:26 GMT

You are not logged in.

Name: Password:

Or: Recover Account (Forgotten Password) / Register new user / Re-send validation email / Main Page

Maintenance is currently disabled.

ADVERTISEMENT

Website Discussion, Bug Reports and Abuse Reports

Cutoff point for player ratings

This thread is open.

Posted by Hammerite This user is an administrator. 2009-05-27 11:51:57 GMT


At present, player ratings (top 30 may be viewed on the statistics page; your own rating may be viewed on your user details page) are calculated based on "competitive" games that finished in the last 6 months. Is this too long a period for it to consider? Should it be something more like 4 months? 5 months? Or do you think it is too short a time? I am interested to know people's views.

Note that at present, the "rating" is only based on how many games you won, and it doesn't take into consideration at all what the number of players was in your games, nor where you placed if you didn't win, nor how your score compared to the winner's, nor what the calibre of your opponents was. These too are valid criticisms of the current system but are less trivial to address! You can post about these shortcomings in this thread but the only thing I am planning to address just now is what I talked about in the first paragraph.

Posted by Tub This user is a translator. 2009-05-27 13:17:00 GMT


More than a period of time, I think a number of game played would be more interesting. For instance, the rating only takes into account your last 30 "competitive" games played.
The reasons are :
- people plays at very different paces. Whatever period you will choose, it will always be too short for some and too long for others.
- number of game played is directly related to the experience of the player. After 50 games played, you should be better than after 10. You could have played one game per week or one per hour, it wouldn't change that.
The last x games played will reflect your true value much better than all games played in the last x months.

So my proposal is : keep the last x "competitive" games played in the last x months to compute the rating. Then the period of time would only prevent someone who haven't played for ages to keep his/her rating until the end of time, and wouldn't be a factor in the player's progression anymore.

As I said, it seems impossible to me to give you a period that would be "good" or "better" for everyone. The only thing I can tell you is that it is way too long for me. 2 months would do fine...

Posted by greg909 2009-05-27 15:05:42 GMT


Would it be possible for you to implement some kind of ELO rating instead?

No matter the period or the number of games played, just based on your opponents value.

Posted by Tub This user is a translator. 2009-05-28 08:48:30 GMT


I don't know much about the ELO system.
The problem I have with systems based only on opponents value is that it does not apply very well to 3+ players situations.

In 2 player games, if your only opponent makes a mistake you will be the one to benefit from it. In 3+ player games, it can not only benefit another player but it can also be very detrimental to yourself, even more than for the one who did the mistake.
Generaly speaking, being beaten by a player with a weaker rating in a 3+ player game doesn't necessarily mean that you played a weaker game than him, whereas it is necessarily the case in a 2 player game. And I'm not even talking about kingmaking problems...

I'm not sure that a system based only on opponents value is the best one in our case. Final position and final score seem as important to me.

Posted by renegade 2009-05-28 19:54:17 GMT


I wonder how many players are even aware of the statistics page. I was playing for over a month before I was told about my ranking. I had to ask him how to access the statistics page since it does require a bit of work to get there.

I would like to see a more prominent link to the statistics page off of the main page. I think you would see the number of "competitive" games increase as more people become aware that a ladder ranking exists for competitive games only.

Also, it would be nice to expand the rankings to include all players, not just the top 30.

Using a model as proposed by Tub above which takes into consideration recent games (max number and max time frame) seems reasonable.

Posted by Tub This user is a translator. 2009-06-19 10:20:57 GMT


Up !

Hammerite, have you made any change on the period for player ratings ? Are you planning to ?

Also, I have a proposal for the rating system.
Currently, based on the FAQ, the formula is " 10000 * y/x * (1-1/x)? " where x is the number of competitive game played and y is the number of victory in those games. My proposal is to modify how the "y" is computed.

If we generalize this "y" term, we can see it as the sum, for all the competitive games that are taken into consideration for your rating, of a "i*j%/k" ratio :
- "i" is your final score
- "j%" is the weight applied to your score depending on your final position
- "k" is the final score of the winning player
If you apply the following values for "j%" :
- 1st place : 100%
- 2nd, 3rd and 4th places : 0%
you have exactly what your current system does (per game : 1 point for the winning players, 0 for the others).

By modifying the values of "j%", I think we can have a nice rating system taking into account position and final score. 3 proposals for "j%" :
- 1st 100% / 2nd 90% / 3rd 75% / 4th 50%
- 1st 100% / 2nd 85% / 3rd 60% / 4th 25%
- 1st 100% / 2nd 80% / 3rd 50% / 4th 10%

I don't know if it is easy to implement. Even if it is not perfect (it still doesn't take into account the opponents' value), I really like the idea. And so I shared :)

Posted by Tub This user is a translator. 2009-06-19 11:11:10 GMT


I would like to add 2 things.

The way I described the "j%" paramater is 4er-oriented. The generalized description :
- 1st 100% / 2nd (4er games only) 90% / penultimate 75% / last 50%
- 1st 100% / 2nd (4er games only) 85% / penultimate 60% / last 25%
- 1st 100% / 2nd (4er games only) 80% / penultimate 50% / last 10%
For instance, in a 2-players game :
- 1st 100% / 2nd 50%
- 1st 100% / 2nd 25%
- 1st 100% / 2nd 10%
This implies that the rating is dependant on the number of player in the game, which seems totally relevent to me.

Also, I think my favorite is the second one (1st 100% / 2nd (4er games only) 85% / penultimate 60% / last 25%).
The first one seems too kind with a bad player : even if you only score half the points of the winner and end up being 4th, which is really a very bad score, you will still earn 25% (half the score weighted half) of the number of points the winner earned.
The third one seems too harsh with a good player : even if you scored as much as the winner but end up being 4th, which is really a great score, you will only earn 10% (same score weighted 10%) of the number of points the winner earned.
The second one seems to be a good balance between score and position.

Posted by Hammerite This user is an administrator. 2009-07-03 00:52:15 GMT


I have made an attempt at a stopgap improvement to the rating calculation along the lines of Tub's suggestions. I have specified here how it is calculated. (You may need to refresh the page or clear your cache to see the new formula image.)

In the long term I would like to implement an Elo or similar system, but this makes it slightly fairer in the short-to-medium term. You no longer get no credit at all for coming 2nd in a 3er, or 3rd in a 4er. I have reduced the weight given to 2-player games in a way I think is fair.

Since the new calculation gives more points, the mean rating has risen, and the top ratings have inflated slightly from what they were. I've increased the top "minimum rating to join" you can choose for a game from 5000 to 6000.

Posted by renegade 2009-07-17 18:05:19 GMT


Just for fun (and because I had too much free time) I tried to use an Excel spreadsheet to calculate my player rating using the formula you provided. Unfortunately I could never get my Excel calculated player rating to match my player rating calculated here. There was about a 20-40 point difference.

My guess is that the difference has to do with how "5 months ago" is determined and/or whether "the game I was automatically kicked out of" counts. Philip, can you clarify exactly how those events are factored into the equation?

Thanks.

Posted by Hammerite This user is an administrator. 2009-07-17 18:27:04 GMT


Yes, games you quit or get kicked out of count. Games that get aborted don't. Have you tried clicking the "Recalculate my rating" link? If games grow too old to be counted, this isn't reflected in your rating until either



SELECT CASE WHEN NumPlays IS NULL THEN 0 
WHEN NumPlays = 0 THEN 0
ELSE ROUND(10000*POW((1-1/NumPlays), 2)*NumWins/NumPlays)
END AS RankingScore
FROM (
SELECT SUM(CASE Game.MaximumPlayers WHEN 2 THEN 0.8 ELSE 1 END) AS NumPlays,
SUM(CASE WHEN PlayerGameRcd.GameResult = \'Finished 1st\' AND
Game.MaximumPlayers = 2 THEN 0.8
WHEN PlayerGameRcd.GameResult = \'Finished 1st\' AND
Game.MaximumPlayers > 2 THEN 1
WHEN PlayerGameRcd.GameResult = \'Finished 2nd\' AND
Game.MaximumPlayers = 3 THEN 0.44
WHEN PlayerGameRcd.GameResult = \'Finished 2nd\' AND
Game.MaximumPlayers = 4 THEN 0.63
WHEN PlayerGameRcd.GameResult = \'Finished 3rd\' AND
Game.MaximumPlayers = 4 THEN 0.26
ELSE 0
END) AS NumWins
FROM PlayerGameRcd
LEFT JOIN Game
ON PlayerGameRcd.Game = Game.GameID
WHERE PlayerGameRcd.User = [your user ID] AND
PlayerGameRcd.GameResult <> \'Game Aborted\' AND
PlayerGameRcd.GameCounts = 1 AND
Game.Friendly = 0 AND
TIMESTAMPADD(MONTH, 5, Game.LastMove) > UTC_TIMESTAMP()
) AS QRYA

Posted by Hammerite This user is an administrator. 2009-07-21 15:09:01 GMT


Maybe I should explain that in the above,

MaximumPlayers
doesn't store the "maximum number of players" setting once the game begins. It stores a number that is usually the number of players in the game, but in the case of a game that has been downsized, may vary from it.

Posted by renegade 2009-07-28 00:32:57 GMT


Hey, Hammerite, I finally got my spreadsheet calculations to match yours. The discrepancy seemed to be those downsized games. In one case ( http://wargamessoc.union.shef.ac.uk/brass/board.php?GameID=1102 ) the MaximumNumber of players is reduced from 4 to 3. But in another case ( http://wargamessoc.union.shef.ac.uk/brass/board.php?GameID=1592 ) the MaximumNumber of players remained at 3 (or 4) even though the game finished with only 2 players.

Is there a specific algorithm you use to decide what the final MaximumPlayers will be? Does the time a player drops out of a game matter?

Posted by Hammerite This user is an administrator. 2009-07-28 01:29:29 GMT


Oh yes, sorry. I should have been more explicit about that. Yes, it matters when the downsizing happens. If a game starts with a certain number of players and finishes with fewer, a decision has to be made as to what number of players it "counts as" having had. It is arbitrary, but the number of players it "counts as" having is the number of players it had at the end of Round 3 of the Rail Phase.

Posted by renegade 2009-08-18 21:21:35 GMT


The help for ratings says, "At the current time, a player's rating is recalculated only when he finishes playing in a game (whether competitive or not); this means that a player's rating might, if recalculated immediately, change owing to some games having passed the five-month threshold since the last time it was calculated. The Administrator can cause all users' ratings to be recalculated on demand."

Looks like there are some players in the top 30 who haven't played a game recently so their rating is not accurate. If the recalculation operation isn't too time consuming, couldn't you schedule this task to run automatically once a day at 3 a.m. or something like that?

Posted by renegade 2009-08-26 17:40:48 GMT


It has been a week with no response so I'll ask this again. Can the recalculation occur automatically on a daily basis?

Posted by Hammerite This user is an administrator. 2009-08-26 22:31:27 GMT


Oh, sorry. Yes, that is an idea. Consider it on a "to-do list"

Posted by Hammerite This user is an administrator. 2009-09-18 16:03:54 GMT


Players' ratings are now recalculated every day at 7:05am GMT.

Posted by renegade 2009-09-25 21:03:24 GMT


Thanks!

You must log on in order to post messages.

Click here to return to the Board Page, or here to return to the Main Page.