How someone could be permabanned - or punished in any way - for literally showing no semblance of any toxic behavior is astonishing. I get that there are supposed to be Riot employees doing checks on these cases before they are decided for a permaban, but if that is the case they either have an extremely lazy employee on their hands, or do not actually review cases as thoroughly as we have been lead to believe.
Every player that has seen this thread and has a moderate level of intelligence has the same fear in the back of their head as I do: the possibility of receiving tribunal punishment for no reason at all. Riot, I am sure that you can see why this is a problem for both the player base and your income from this game. If word spread around to more members of the community than just those who frequent the forums, many would share this same fear. I doubt that many players would feel safe pouring money into a game that there is an apparent chance of being permanently banned from for following the summoner's code. The Tribunal - while very clever and fair in concept - is far more toxic to the community than the players that it removes. A system which allows players to abuse the report system, then allows those same players to hold an account's future in their hands cannot exist in such a large player base.
On top of the abuse of the system from players (which I'm sure that you are already working on something to help end), the responses from your end in regards to problems of this nature have been negligible at best. A quick review of the first few pages of the forums shows that not a single post about the tribunal unjustly punishing someone has had a red post. Not a single one. This is just not acceptable. The message I am getting from this is that The Tribunal was just a clever way for you to avoid hiring a large customer support team to fairly deal with problems of this nature. It also gives me the feeling that the lingering possibility of an undeserved permaban for any player doesn't bother you in the slightest.
Here is what I would like to know:
1. What (if any) changes are planned for The Tribunal and report systems to avoid abuse of them?
2. What safety measures are currently in place to prevent players from being unjustly banned? (Specifics on this matter would be great)
3. When a player (such as the aforementioned one) is unjustly banned, what flaws have caused a slip-up of this magnitude?
4. How can these mistakes be prevented in the future?
5. Are there any actions being taken against players who, for example, have punished 100% of their cases in the tribunal?
6. What assurance can you offer (other than the public knowledge of how a case ends up in The Tribunal) that a player will not be randomly banned while following the summoner's code?
While I am here, I would also like to offer a suggestion or of my own to help remove the toxic aspect of this device.
The justice system was a great addition to The Tribunal. It eliminated IP farmers who would simply punish every case. It also created people who would punish-spam just so they can brag about their high justice rating. I believe that this system can be used as a strong base to help fix the unwarranted bans. Maybe a player's justice rating should be more than just a number. If a certain rating or ranking were required to even review a case which would possibly result in a permaban, I'm sure that you would instantly see many undeserving bans disappear. If a case is less serious, allow lower ranked tribunal users to review it, allowing them to build up over time to the bigger cases.
I am really hoping that you will come through with changing this in some way, Riot, because The Tribunal is heading in a terrible direction. If left on this track, this system could eventually reach a point where it has become too toxic and must be eliminated completely. Please work with the community to help fix this tool.
I'm sure other people have concerns with The Tribunal that I haven't mentioned so please comment so Riot can see them as well.
The forums tend to paint an inaccurate picture. When a player is justly banned and learns from it, he rarely posts on the forums to complain about it--these are the vast majority of Tribunal cases.
Secondly, when a player has never been punished by Tribunal... he rarely posts on the forums a thread like "Hey, never been banned by Tribunal, great system!" This is actually over 98% of the population on the NA server right now.
However, toxic players that are banned by the Tribunal have a reasonably high chance of going to the forums and complaining about it.
Quote:
How someone could be permabanned - or punished in any way - for literally showing no semblance of any toxic behavior is astonishing. I get that there are supposed to be Riot employees doing checks on these cases before they are decided for a permaban, but if that is the case they either have an extremely lazy employee on their hands, or do not actually review cases as thoroughly as we have been lead to believe.
We have a lot more information than Tribunal reviewers do. This includes pre- and post-game chat logs, report histories, ban histories and in-game chat logs for every game. We want to get more of this information to the reviewers but this takes development time. We also have to consider the possible dangers and side effects of releasing any extra details and how it interacts with privacy of our players.
The question is, if the Tribunal verdict is "Pardon" and a Riot team reviews the case, what do we do if we feel the player deserves a punishment for information not in the Reform Card? Overturn the case and punish the player even if the Reform Card appears neutral, or keep the pardon? We're always improving on policies like this but right now we've decided to keep the punishments in if we feel the player deserves it.
Quote:
Every player that has seen this thread and has a moderate level of intelligence has the same fear in the back of their head as I do: the possibility of receiving tribunal punishment for no reason at all.
It is nearly impossible for a positive player to be punished by the Tribunal. There's so many checks in place all along the way that the 'miracle' chance of a positive player getting a punishment is so trivial that we accept that cost. If it happens, we'll correct it by hand and personally apologize to the affected players.
Quote:
Here is what I would like to know:
1. What (if any) changes are planned for The Tribunal and report systems to avoid abuse of them?
There's a lot in place actually.
1) Players who false report a lot lose 'power' in their reports
2) We've removed IP from the Tribunal, which stopped a lot of spam punishers from visiting
3) We've added Justice Reviews that encourages discussion in the community
4) Players that have been extremely inaccurate in the Tribunal can be banned from Tribunal use
Quote:
2. What safety measures are currently in place to prevent players from being unjustly banned? (Specifics on this matter would be great)
1) For a player to get into the Tribunal, they have to be reported by dozens (sometimes hundreds) of players and be in the bottom 2% of the server in terms of toxic behavior.
2) The player also has to receive 'valid' reports, assessed by our systems.
3) The player then has to be punished by a supermajority of the playerbase.
4) The player then might be randomly reviewed by Riot staff in Player Support or Player Behavior.
Those aren't even all the barriers, it's just the ones we've talked about before. For a player to legitimately be positive and slip through all of the checks... it's such a low rate that we're willing to accept it as a 'known' cost.
I honestly do not think you guys are strict enough in checking on the tribunal voters themselves.
You say people can be banned from the tribunal, but if they spam punish they're almost guaranteed a 75% accuracy rating which only encourages more punish spamming.
Do you think it would be possible to put blatantly non-punishable cases on the tribunal and insta-revoke tribunal privileges from those who vote punish on it? Just something to make sure players are at least trying to be accurate and there is a fear of acting like a jerk in game and in the tribunal
It's extremely easy to detect spam punishers and revoke their Tribunal privileges. The question we should be asking is, if 75% of cases are punished and 100% of those cases deserve it, is there a problem?
We definitely know the concept of 'catch' cases; these are obvious cases that should be pardoned to randomly 'test' our reviewers.
if you keep getting rid of the 2% most toxic people doesn't a new 2% just keep taking their place?
when tribunal was first implemented every single case that was a punish was painfully obvious: someone saying im gonna feed and go 0-23-0, blatant racism, sexism, homophobia, all caps raging, etc. Now, I see someone get banned for calling someone a noob or saying stop feeding. Those aren't constructive things to say but in the heat of playing a game people can say things that they regret down the road.
This is a great question, but it doesn't because the system is not designed just to scrape the bottom 2%.
In fact, when the player behavior team first started, it was scraping a much higher %, but now it's been decreasing to the bottom 2%. For this month, it's only been scraping the bottom 0.7%.
On other servers, the numbers vary a lot.
Quote:
Additionally:
What if their pardon/punish metrics are different from the rest of the community? What if they feel that "noob" "stop feeding" "you're bad" don't constitute punishable offenses unless used every other word? If anything that seems like it forces a mob mentality.
It's a bit more complicated than that. Let's say a Tribunal case has 98% Punish votes. In this case, the players who voted "Pardon" are probably wildly off or trying to break the system. If a Tribunal case has 55% punish votes, then the case is probably much more subtle and we are less worried about who voted "Pardon" as an abuse case.
how about riot BITES THE BULLET and pays people to do this? it would cost ****ing nothing. it would be so much better because.. you know.. those people would actually know the code.
This is a common misconception that players have. We still have a large Player Support team. Rioters still review Tribunal cases (not all of them, but the severe bans). We actually incur high costs because we also have a development team that is solely working on the Tribunal on maintenance and development of new features.
By the way, if we only had Riot staff review every case... we'd be far, far more strict than the current reviewers.
Lyte, from the countless posts in the forum explaining things about the Tribunal, and the FAQ, I have come to the conclusion that there is one major fault in the way the system works.
It punishes people who play more. It is always mentioned how people need to get reported a lot of times, but naturally, people who play 12 hours a day will get reported way more than people who play 1 game a day. The system doesn't treat the total amount of time spent on the game as equal. Let's say I get reported for mild flaming 1 out of 10 games by playing playing 1 game a day for 10 days. I am sure I won't even get into the Tribunal because of this. However, if I play 10 games a day, for 10 days, and have mild flaming in 1 game each day, the Tribunal will see me as a toxic player and punish me, even though the % of games I slip with the Summoner's Code is the same.
Not at all, the Tribunal system automatically normalizes reports against the number of games you play. Players who play 2000 games have just as equal chance to enter Tribunal as players who play 200 games.
Quote:
Another problem is the following.
I'm scared of playing a lot of games now because I know I have my days where I am moody, and frankly, I'm entitled to having my own opinions and emotions, and expressing them freely, as long as I'm not heavily insulting anyone. The report system is flawed because of the fact that you can be reported for anything, be it as little as gg noobs or even less than that. I personally don't use the chat at all when I play this game because I don't want to risk saying something "bad". People abuse the report system and will use ANYTHING they can possibly have against you to report you, especially if they are bitter because they lost the game.
The amount of reports a player can give out per day NEED to be limited, so people don't spam reports on ridiculous stuff and have to think about what to report. Think about it, would you run to the police and press charges against someone who called you a meanie, a loser, a noobie? Or would you go to the police because of someone who threatened your life? The problem is that the police obviously doesn't have the time and resources to deal with minor issues like that, but the Tribunal ENCOURAGES to abuse the system to report for ANYTHING, because no resources are needed. The resource is the community, which is a contradiction in itself, because everyone is reporting anybody for anything. It's like trying to turn the community and Leage of Legends into some Fairytale Land where everybody loves everyone, but in my humble opinion, people have their damn right to be harsh as long as it's not real, true harrassment. The true harrassment lies in the fact that people will ACT all nice and pretend to be innocent while actually sitting behind their monitor trying their hardest to find ANYTHING to get you reported and banned, thereby abusing you, and taking away your freedom of speech and your permission to express having a personality other than being a hippie who loves everybody and everything in this world. I don't want to be friends with everybody who plays this game.
98% of players have never even seen the Tribunal. If you are neutral or neutral positive with a few outbursts here and there, you will probably never see the Tribunal. No one is there because of 1 bad day, or a few bad days after a few months.
To all the people using legal analogies, keep in mind, each and every person who had been deprived of their "property" had clicked a legal agreement to not only follow the rules of the frame, but also agreeing to riot's ability to take away this property as they see for. Riot does not need to adhere to an stipulations of due process.
@Lyte
Do you see check the toxicity levels of the people complaining about the tribunal? I'm especially interested in the toxicity levels of their who attempt to justify losing you cool and taking it out on others by rationalizing it, claiming that everyone does it. Just to provide rebuttal to the seeming consensus, I get mad and lose my cool often, but I never per myself take it out on others by raging.
Also, if I managed to grab you attention, Lyte, I'm actually wondering if you could tell me how my report weight is at the moment. I try to only report when I'm fairly positive there will be a punish, but I worry that my use of"report unskilled player" will result in a decrease in the weight of my reports.
We do actually regularly assess the toxicity levels of players who complain about the Tribunal--the results pretty predictable
I can't give out report weights for the typical positive or neutral player. The last time I did Player Support was swamped with requests and it was my fault
Do you review every case of the player that goes to tribunal after they are flagged for being banned?
Only the more severe punishments are reviewed.
Quote:
Do you check to see if its becoming a statistical trend or just spiking?
To even end up in the Tribunal, the behavior has to be a statistical trend.
Quote:
At how many games of bad behavior compared to games of good behavior do you draw the line at?
Don't know the answer to this one off the top of my head. The large majority of your games should be at least neutral.
Quote:
Do you think a low game count would increase your chances of being banned?
Nope, it's an equal chance for players who play a high number of games or a low number of games.
Quote:
If a person trolls another into retaliation do you mark that guy as a possible subject to look into?
If a toxic player initiates and a player retaliates in a toxic manner, both are reviewed in our systems. Retaliation is not OK! It actually makes it a toxic experience for the rest of the players in the game.
It's alarming how much secondary education you guys seem to end up doing about your player behavior systems over and over again. Every time you or another PB&J representative ends up in a thread about it, you end up going over the same basic points (Tribunal is fairly lenient, Tribunal isn't systemically biased, Tribunal has defense mechanisms against trolling voters, etc.)
As ridiculous as it sounds, do you think a project to raise awareness about the workings of the current player behavior systems would help? Maybe tieing information of the tribunal and honor systems more directly into the game's tutorials or new player help tooltips?
It just seems awfully redundant that we have to keep going over this again and again, and the more seriously those initiatives are taken by the playerbase, the more effective (presumably) they might become.
I agree with you and we're brainstorming on some projects. Nothing we can discuss here yet, but players will be the first to know.
This is my problem, you say that you have to be in the bottom 2% to be sent to tribunal, but many people are making it there with one bad game and tons of false reports. Could you clarify this?
A player has to receive dozens of reports over dozens of games to even have a chance to enter the Tribunal. Currently, your report metrics have to be in the bottom 0.7% to 2.0% to even have a chance to enter the Tribunal.
The only reason players see a smaller number of games in a Tribunal Case is because the Tribunal randomly samples 1-5 recent games from the player's match history.
@Riot; help us understand The Tribunal
http://na.leagueoflegends.com/board/....php?t=2865799
How someone could be permabanned - or punished in any way - for literally showing no semblance of any toxic behavior is astonishing. I get that there are supposed to be Riot employees doing checks on these cases before they are decided for a permaban, but if that is the case they either have an extremely lazy employee on their hands, or do not actually review cases as thoroughly as we have been lead to believe.
Every player that has seen this thread and has a moderate level of intelligence has the same fear in the back of their head as I do: the possibility of receiving tribunal punishment for no reason at all. Riot, I am sure that you can see why this is a problem for both the player base and your income from this game. If word spread around to more members of the community than just those who frequent the forums, many would share this same fear. I doubt that many players would feel safe pouring money into a game that there is an apparent chance of being permanently banned from for following the summoner's code. The Tribunal - while very clever and fair in concept - is far more toxic to the community than the players that it removes. A system which allows players to abuse the report system, then allows those same players to hold an account's future in their hands cannot exist in such a large player base.
On top of the abuse of the system from players (which I'm sure that you are already working on something to help end), the responses from your end in regards to problems of this nature have been negligible at best. A quick review of the first few pages of the forums shows that not a single post about the tribunal unjustly punishing someone has had a red post. Not a single one. This is just not acceptable. The message I am getting from this is that The Tribunal was just a clever way for you to avoid hiring a large customer support team to fairly deal with problems of this nature. It also gives me the feeling that the lingering possibility of an undeserved permaban for any player doesn't bother you in the slightest.
Here is what I would like to know:
1. What (if any) changes are planned for The Tribunal and report systems to avoid abuse of them?
2. What safety measures are currently in place to prevent players from being unjustly banned? (Specifics on this matter would be great)
3. When a player (such as the aforementioned one) is unjustly banned, what flaws have caused a slip-up of this magnitude?
4. How can these mistakes be prevented in the future?
5. Are there any actions being taken against players who, for example, have punished 100% of their cases in the tribunal?
6. What assurance can you offer (other than the public knowledge of how a case ends up in The Tribunal) that a player will not be randomly banned while following the summoner's code?
While I am here, I would also like to offer a suggestion or of my own to help remove the toxic aspect of this device.
The justice system was a great addition to The Tribunal. It eliminated IP farmers who would simply punish every case. It also created people who would punish-spam just so they can brag about their high justice rating. I believe that this system can be used as a strong base to help fix the unwarranted bans. Maybe a player's justice rating should be more than just a number. If a certain rating or ranking were required to even review a case which would possibly result in a permaban, I'm sure that you would instantly see many undeserving bans disappear. If a case is less serious, allow lower ranked tribunal users to review it, allowing them to build up over time to the bigger cases.
I am really hoping that you will come through with changing this in some way, Riot, because The Tribunal is heading in a terrible direction. If left on this track, this system could eventually reach a point where it has become too toxic and must be eliminated completely. Please work with the community to help fix this tool.
I'm sure other people have concerns with The Tribunal that I haven't mentioned so please comment so Riot can see them as well.
http://na.leagueoflegends.com/board/....php?t=2865799
Secondly, when a player has never been punished by Tribunal... he rarely posts on the forums a thread like "Hey, never been banned by Tribunal, great system!" This is actually over 98% of the population on the NA server right now.
However, toxic players that are banned by the Tribunal have a reasonably high chance of going to the forums and complaining about it.
The question is, if the Tribunal verdict is "Pardon" and a Riot team reviews the case, what do we do if we feel the player deserves a punishment for information not in the Reform Card? Overturn the case and punish the player even if the Reform Card appears neutral, or keep the pardon? We're always improving on policies like this but right now we've decided to keep the punishments in if we feel the player deserves it.
1. What (if any) changes are planned for The Tribunal and report systems to avoid abuse of them?
1) Players who false report a lot lose 'power' in their reports
2) We've removed IP from the Tribunal, which stopped a lot of spam punishers from visiting
3) We've added Justice Reviews that encourages discussion in the community
4) Players that have been extremely inaccurate in the Tribunal can be banned from Tribunal use
2) The player also has to receive 'valid' reports, assessed by our systems.
3) The player then has to be punished by a supermajority of the playerbase.
4) The player then might be randomly reviewed by Riot staff in Player Support or Player Behavior.
Those aren't even all the barriers, it's just the ones we've talked about before. For a player to legitimately be positive and slip through all of the checks... it's such a low rate that we're willing to accept it as a 'known' cost.
You say people can be banned from the tribunal, but if they spam punish they're almost guaranteed a 75% accuracy rating which only encourages more punish spamming.
Do you think it would be possible to put blatantly non-punishable cases on the tribunal and insta-revoke tribunal privileges from those who vote punish on it? Just something to make sure players are at least trying to be accurate and there is a fear of acting like a jerk in game and in the tribunal
We definitely know the concept of 'catch' cases; these are obvious cases that should be pardoned to randomly 'test' our reviewers.
when tribunal was first implemented every single case that was a punish was painfully obvious: someone saying im gonna feed and go 0-23-0, blatant racism, sexism, homophobia, all caps raging, etc. Now, I see someone get banned for calling someone a noob or saying stop feeding. Those aren't constructive things to say but in the heat of playing a game people can say things that they regret down the road.
In fact, when the player behavior team first started, it was scraping a much higher %, but now it's been decreasing to the bottom 2%. For this month, it's only been scraping the bottom 0.7%.
On other servers, the numbers vary a lot.
What if their pardon/punish metrics are different from the rest of the community? What if they feel that "noob" "stop feeding" "you're bad" don't constitute punishable offenses unless used every other word? If anything that seems like it forces a mob mentality.
By the way, if we only had Riot staff review every case... we'd be far, far more strict than the current reviewers.
It punishes people who play more. It is always mentioned how people need to get reported a lot of times, but naturally, people who play 12 hours a day will get reported way more than people who play 1 game a day. The system doesn't treat the total amount of time spent on the game as equal. Let's say I get reported for mild flaming 1 out of 10 games by playing playing 1 game a day for 10 days. I am sure I won't even get into the Tribunal because of this. However, if I play 10 games a day, for 10 days, and have mild flaming in 1 game each day, the Tribunal will see me as a toxic player and punish me, even though the % of games I slip with the Summoner's Code is the same.
I'm scared of playing a lot of games now because I know I have my days where I am moody, and frankly, I'm entitled to having my own opinions and emotions, and expressing them freely, as long as I'm not heavily insulting anyone. The report system is flawed because of the fact that you can be reported for anything, be it as little as gg noobs or even less than that. I personally don't use the chat at all when I play this game because I don't want to risk saying something "bad". People abuse the report system and will use ANYTHING they can possibly have against you to report you, especially if they are bitter because they lost the game.
The amount of reports a player can give out per day NEED to be limited, so people don't spam reports on ridiculous stuff and have to think about what to report. Think about it, would you run to the police and press charges against someone who called you a meanie, a loser, a noobie? Or would you go to the police because of someone who threatened your life? The problem is that the police obviously doesn't have the time and resources to deal with minor issues like that, but the Tribunal ENCOURAGES to abuse the system to report for ANYTHING, because no resources are needed. The resource is the community, which is a contradiction in itself, because everyone is reporting anybody for anything. It's like trying to turn the community and Leage of Legends into some Fairytale Land where everybody loves everyone, but in my humble opinion, people have their damn right to be harsh as long as it's not real, true harrassment. The true harrassment lies in the fact that people will ACT all nice and pretend to be innocent while actually sitting behind their monitor trying their hardest to find ANYTHING to get you reported and banned, thereby abusing you, and taking away your freedom of speech and your permission to express having a personality other than being a hippie who loves everybody and everything in this world. I don't want to be friends with everybody who plays this game.
@Lyte
Do you see check the toxicity levels of the people complaining about the tribunal? I'm especially interested in the toxicity levels of their who attempt to justify losing you cool and taking it out on others by rationalizing it, claiming that everyone does it. Just to provide rebuttal to the seeming consensus, I get mad and lose my cool often, but I never per myself take it out on others by raging.
Also, if I managed to grab you attention, Lyte, I'm actually wondering if you could tell me how my report weight is at the moment. I try to only report when I'm fairly positive there will be a punish, but I worry that my use of"report unskilled player" will result in a decrease in the weight of my reports.
I can't give out report weights for the typical positive or neutral player. The last time I did Player Support was swamped with requests and it was my fault
As ridiculous as it sounds, do you think a project to raise awareness about the workings of the current player behavior systems would help? Maybe tieing information of the tribunal and honor systems more directly into the game's tutorials or new player help tooltips?
It just seems awfully redundant that we have to keep going over this again and again, and the more seriously those initiatives are taken by the playerbase, the more effective (presumably) they might become.
The only reason players see a smaller number of games in a Tribunal Case is because the Tribunal randomly samples 1-5 recent games from the player's match history.