Since we released The Justice Review a few months ago, weve been monitoring The Tribunal and would like to share a few changes weve made that are subtle, but critical. These changes are aimed at improving accuracy, providing voters more relevant information, and helping to identify the most toxic players even faster. I want to quickly highlight these changes and explain the logic behind them:
- Previously, our Player Support team reviewed every Tribunal case before approving punishments and pardons. After reviewing the past 12 months of data, weve seen that cases that are unanimously pardoned or punished tend to be extremely accuratePlayer Support rarely overturns these decisions. In light of this data, were now going to focus our attention on the cases that are more ambiguousthe slight majorities and split decisions. This should improve The Tribunals accuracy by focusing on the cases that need the most attention.
- Weve tweaked how The Tribunal builds cases: Instead of 100% random sampling, The Tribunal will attempt to find the games that ultimately sent each player to The Tribunal. By selecting recent games that had a big impact in sending the player to The Tribunal, you should see the average number of reports against a player increase per game.
- The Tribunal system is a constant balance between speed and accuracy. In the past, we were concerned that forcing The Tribunal to find more games per case meant that toxic players were left unchecked for longer than necessary. However, we realize that more information for our judges means more accurate verdicts, and recent optimizations have allowed us to change The Tribunal so that every case will have at least two games to review instead of one.
- Players have often said they forget how many cases theyve done on a given dayweve added a progress bar that will outline how close you are to maxing out your daily case limit.
These changes are already live, so go check them out and let us know what you think! Were confident that these changes will improve the accuracy of The Tribunal system without compromising on speed.
Visit the Tribunal today and help us make a difference in the community!
Quote:So Player Support was still checking cases overall? Or just the Permabans? I think that support should be checking every case before marking someone as a criminal. I hope we'll get a feature against Punish spammers soon.
As a second data point, depending on the month, punish rates in the Tribunal range from 60-70%. We have to remember that the punish rate actually doesn't matter that mean--what matters is whether those players deserve the bans.
When we measure the success or accuracy of the Tribunal, we actually look at statistics like: "What is the false positive rate?" In other words, how many players are being punished when it is undeserved? We try to ensure that the false positive rate is low as possible, which is why the Tribunal is actually designed to be quite conservative.
Quote:Dr. Lyte, I was doing tribunal when a thought occurred to me: what is your stance on behavior progression in relationship to tribunal cases? Let me elaborate: say a player has a track record of raging in game and get reported consistently enough to get into tribunal. Now consider that this player has actually shown behavioral progression in the positive- that is this player is more considerate of how he is behaving and how it effects other players' playing experiences. They get reported in some games still, but overall shows slight progression. I'm wondering if Tribunal already takes this into consideration, because now I have the nagging though in the back of my head that I may, in fact, be helping to punish a player that is actually showing progression without having been punished first. Although on the other hand, one could say that the punishment is still deserved for past grievances.
What is your stance, or rather the stance of the Tribunal, on this perspective?
Quote:1. Occasionally the consequences of spam punish was overlooked. We thought that was awesome and have found ways to make it happen more.
2. Wow, and premade teams can abuse the tribunal system on their poor team even more. Did you even think on this one more than a minute?
3. So it's more likely now that no matter how nice the player the odds are greater the one game they did anything wrong will make it onto the report. Instantly justifying a ban for all the nonsense reports that came with it. All hail spam punish.
4. Sitting around waiting to spam punish is hard without opening other windows, and we understand you lose track of what you're doing by not paying attention. Here is a counter.
Quote:Italics are my added emphasis.
First off I'd like to say I love the Tribunal system and the idea of having the community involvement.
As far as the changes go I could not be happier for having games that will more likely show the reason for the player being reported. That will make our job so very much easier. I think one additional change could help with accuracy even more, and perhaps weed out false reports, and that is to either A. Not include reports where the player hasn't typed in the reason for the report, or B. Force players to type at least a certain number of characters if they are going to report. I feel as a player, that if someone is toxic enough for me to be reporting them that I should take the time to give the reason why I am doing so which should also make it easier for Tribunal judges to determine the accuracy of my report and the toxicity of the player.
1) If we make the limitation "any number of characters," then many players will simply scribble random nonsense just to file a report.
2) If we make the limitation too long, then many players might not report players they should because they don't really want to deal with writing a comment.
It's a tough balance between #1 and #2, but it's something that's not entirely off the table.
Secondly, as has been said by many a red post, the people who end up in the Tribunal are reported and reported often. So, I have a very hard time understanding why there would be cases with only two games. I've seen it myself where each game has just one report(with almost always no additional information), and yet this player is supposedly so toxic that they need to be punished. Now as you've said that the Tribunal will be looking for the most recent and egregious offenses I can possibly see a case with just two games where there are maybe 3+ reporters per game, but I still feel as if this person is in the Tribunal there will (or at least in my mind should be) be AT LEAST 5 games in which the player has been reported so that we can always have the most information when making our decisions.
We are going to experiment longer term with 2, and monitor the data and go from there.
I've actually had another idea which I'm sure many other players have had as well, and that is based on the fact that it has been said that people who falsely report have their reports disregarded by the system. Now that is great, but what if you were to, besides each report put some sort of rating system, or perhaps that reporters previous accuracy in finding toxic players. That way when I look at a game with one report and the reporter's accuracy is rather low, I can go in being skeptical of the person actually being guilty.
Quote:"Previously, our Player Support team reviewed every Tribunal case before approving punishments and pardons."
So you guys were checking each and every tribunal case before approving punishes and pardons? Am I understanding this right? I was under the impression that it was only permaban cases that were reviewed by player support.
Quote:If your team checks each tribunal case, why do we even have the tribunal?
Our philosophy on the player behavior team is to develop systems and tools to empower our players to directly shape the community and to add more accountability to online gaming. Could we simply remove the Tribunal and just keep reviewing every case? Of course. But, if our goal is to change the culture of online games... we need systems like the Tribunal and Honor, and we need players who want to make a difference. This is why we always say that fixing player behavior is a collaboration between Riot Games and you--the players.
If I go for 1 year without being banned will I still be permed?
Quote:So Lyte, as a avid Tribunal Ban Inquiries goer as you know,
What is the update with adding Pre/Post game chat into Tribunal logs? Is that still on the list of priorities or is PB&J team and you putting it on hold while you put in changes that are easier to implement?
I believe in what we are working on, but it's not related to pre- and post-game chat logs which is why I haven't had much to say about the recent threads around this issue.
Quote:I have another question.
Is the hall of justice still planed?
Does the rating about tribunal worries you, how people in generall will react to this or don't you have enough contributer to release it?
Do alot of things speak against the hall of justice, that you haven't considered before?
Do you need more time or is it planned to release the hall of justice with possible rewards and you want to avoid missing some people who deserve it?
I just wonder why, since i think no word was fallen about it and it was announced together with the overworked tribunal.
Halls of Justice might re-appear in a different form, or we might come up with something else--we'll have to see.