TBM FAQ

General Questions

What is TheBetterMeta?

The Better Meta (TBM) is a website and a Discord bot dedicated to analyzing stats related to Hi-Rez’s free-to-play, team-based, first-person shooter Paladins. The website focuses on aggregate stats (stats about groups of people), while the discord bot lets you look at stats about individual players and matches.

What game modes does TBM support?

At the moment, TBM only looks at Casual and Ranked Siege.

What platforms does TBM support?

TBM collects data about all platforms. Graphs will say specifically what platform/input device they reflect; any graph that doesn’t mention platform shows Keyboard/Mouse Ranked data.

How do I help support TBM?

TBM is entirely user-funded; there are no ads or paywalls. But, TBM takes an enormous amount of time to create and maintain. You can become a supporter of TBM on our Patreon page, which will enable us to keep putting out interesting data.

What if I have questions not answered in this FAQ?

You can pretty much always find me (@pseucrose) on the TBM Discord, where I’m more than happy to talk to anyone about anything stats related. You can also tweet at me, or email me at pseucrose@thebettermeta.com.

How TBM Works

Where does the data come from?

All the raw data comes from the publicly available Hi-Rez API. Hi-Rez provides a list of all non-custom matches that are played in-game, who was in them, what champions they used, who won, etc. Anyone can create an API account and get the same raw data. TBM then applies statistical analysis and machine learning techniques on top of the raw data to create its analyses.

How often do things get updated?

The website updates each morning around 6 AM Eastern time. In terms of the Discord bot, matches get pulled roughly an hour after they take place. It crunches new ratings every thirty minutes. The leaderboard gets updated every four hours. All of these updates can be out of sync from one another.

What programming language is TBM written in?

All the code that archives matches and calculates ratings, as well as the bot itself, is written in C#. The neural network used by the Chance-To-Win calculation (see this question) is written in Python, using Keras. The data is stored in a PostgreSQL database.

Why does TBM say champion X is good/bad, when everyone actually knows X is bad/good?

The actual effectiveness of different champions or talents is not necessarily reflected in how they feel. Some things champions do are much harder to notice (e.g., helping your team make space) than other things (raw damage or CC).

Assuming you know the answer before you look at the data is not a very good way of figuring out what’s true.

Why should I believe anything TBM says?

TBM has an accurate picture of Paladins siege, because we look at every (siege) match that has taken place since early 2017. As of July 2019, that’s over 100 million matches. Any particular graph may not answer your particular question, but in terms of what the graphs do say, they are accurate.

Ratings

I don't care about averages; I want to know what works at my skill level. How do you deal with skill?

TBM estimates skill by calculating ratings. All new graphs now come with dropdowns where you can select ratings. Some older graphs include ratings, or skill, as one of the axes.

What is a rating?

A rating is a measure of how good someone is at winning. TBM calculates separate ratings for every combination of player, champion, and division (Crossplatform Casual, PS4 Casual, Controller Ranked, PS4 Ranked, and Keyboard/Mouse Ranked). TBM often distinguishes between a “player rating” (the skill a player has overall, across all the champions they choose to play) and a “champion rating” (the skill a player has with a particular champion).

Ratings are sometimes called MMR, Elo, or SR by different games or players. TBM generally just calls them “ratings”.

TBM ratings are calculated by us; they are not the same as Hi-Rez’s internal MMR or TP, and they are not the same as those on any other Paladins website. TBM uses the Glicko algorithm, adapted for teams, to calculate its ratings.

What exactly do ratings represent?

TBM ratings are just numbers, ranging from 800 to 4800, that reflect how good someone is at winning. An average player will have a rating near 2000. Each additional 100 points of rating roughly corresponds to a 1.5% advantage (so a 2100 player will beat a 2000 player [playing in an otherwise balanced match] roughly 51.5% of the time).

The fundamental idea behind a rating is that if you were to play with and against people who have the same rating as you, then which team wins should be the same chance as a coin-flip, 50/50.

How do ratings get made?

Every time you play a match, TBM makes an adjustment to your rating based on whether you won or lost. Every time you win a match, your rating goes up, and every time you lose a match, your rating goes down; only winning or losing matters (see this question). The amount you gain or lose depends on how much better or worse your team is than the enemy: beating a team that has higher ratings than your team will cause your rating to rise much more than beating a worse team. Similarly, losing against a team with higher ratings causes your rating to drop only a small amount, while losing against a worse team causes your rating to drop more sharply.

After enough matches, these adjustments cause similarly skilled players to have similar ratings numbers.

Why don't ratings take into account performance?

Ultimately, a good player or strategy is one that wins. Maybe that means doing a lot of damage, or maybe that means doing low damage but applying it to the right people. Or maybe that means sacrificing damage for healing, or healing for damage, or knocking enemies off their horses in exchange for your life. Or, maybe instead of zoning, you should have died on point to extend overtime. Or, maybe you’re good at calling targets for your teammates, but your aim isn’t great. Etc., ad nauseam. There’s no way to list all the ways you might be contributing to the outcome of the match, but by looking at whether you tend to win or lose more compared to someone else in similar situations, we can figure out how good you are.

Since there’s 10 people in every match, it takes a lot of matches to tease out how important your contributions are. But, in the end, your rating will reflect those contributions, even if we’re not really sure why what you’re doing works. A ratings algorithm that looked at things like damage, healing, kills, etc., would necessarily miss or downplay any other kinds of actions.

How accurate are ratings overall?

Overall, we know that ratings are decently accurate because they can be used to predict which of two teams will win a match, before the match is even played (see this question).

How accurate are individual ratings? Or, What is RD?

Individual ratings really come in two parts, like this: 1500±100 (read 1500 plus-or-minus 100). The first part is the rating proper, and the second part is called the 2⋅RD. The 2⋅RD measures how uncertain the rating is. The more matches you play, the less uncertain your rating becomes. It starts at 400 and decreases with each match (down to a minimum of 100). Each patch, everyone’s 2⋅RD rises partially back up towards 400. Your rating plus-or-minus 2⋅RD gives an idea of a range of skill levels a person probably falls into (technically the Rating±2⋅RD carves out a 95% confidence interval). So for 1500±100, this range would be from 1400 to 1600.

For the mathematically inclined, the RD is the same thing as a standard deviation. We use 2⋅RD to get a 95% CI, but we could use a 1⋅RD for a 68% CI, or 3⋅RD to get a 99.7% CI.

Why bother with ratings when we have Winrates?

Straight winrates aren’t a very good indication of how strong a player or a champion is, because Paladin’s matchmaker means that not everyone is playing the same kinds of matches. Some people are playing against easy opponents, while others are playing against the best players in the world. Having a 60% winrate against Bronze-level opponents is “worth less” than having a 60% winrate against Diamond-level opponents.

Why bother with ratings when we have Tiers (Diamond, Platinum, etc.)?

Tiers are only for ranked, so they wouldn’t help when talking about skill in casual matches. They also get reset very frequently. Finally, the primary role of Tiers is to define a competitive ladder (giving people something to play for), not to denote skill.

The Discord Bot

What do I need to know about the bot in order to use it?

Start by asking it for !help. Try out the different commands and see what they do. You can get specific help about the different commands by asking for !help stats or !help champ.

If you want to look up someone with a space in their name, you need to surround their name with quotes, like !stats "cool guy".

To look up people on Xbox, prefix the command with an x, like x!stats playername (or similarly p for PS4 and s for Switch).

How do I look up data for custom matches?

You can’t; Hi-Rez doesn’t provide any match data for custom matches.

How do the leaderboards work?

The TBM leaderboards simply take all the people who’ve played (a champion, or a division) in the last 30 days, and order them by Rating - 2⋅RD (see this question). If we instead ordered the leaderboard by just the Rating, then we couldn’t distinguish someone who had a 2000 rating after 20 matches from someone with a 2000 rating after 200 matches. We have good reason to think someone is at least as good as their Rating - 2⋅RD, so it acts as a conservative estimate of skill.

Which matters more, my rating or my rank on the leaderboard?

The rating is always our best guess at how good you are. Your position on the leaderboard reflects the recent popularity of a champion (it’s easier to get to the top of a less popular champion’s leaderboard), so by itself it’s not a great measure of which champion you do the best with.

Imagine you play two champions, X and Y. You’re the #1 X in the world with a rating of 2500, and you’re the #1000 Y in the world with a rating of 3000. You are probably more effective with Y than with X.

My position on the leaderboard just moved in a direction I didn't expect. What happened?

There’s basically three ways this can happen.

The first way is, other people are also playing matches while you are, and they are also moving around you.

The second way is, maybe the leaderboard updated after you lost a match, but before your three-win streak immediately afterwards. Give it a few hours and your most recent matches will be included.

The third way is, you can climb the leaderboard even after losing a match. This happens because the leaderboard is sorted by Rating - 2⋅RD (see this question). Imagine you lost a match and went from 1500±200 to 1490±180 (2⋅RD always decreases after each match, because uncertainty always decreases). In terms of the leaderboard you went from 1500-200 (or 1300) to 1490-180 (or 1310), which is actually an increase.

How is "Chance to win" calculated?

The commands !current and !match commands both predict the winners of a match. This prediction is done by a neural network trained on the last few million matches. It knows game mode, input devices, champion, and skill data (both player ratings and champion ratings), and looks for patterns in previous matches to guess what will happen in new matches.

The bot successfully predicts the winners of 2/3 of botless matches. This has been tested both in terms of the queries people run on Discord, as well as the overall population. Furthermore, when the bot says a match has an X% chance to win, it really works out that X% of similar matches have won historically, and will win in the future. That is to say, if you look at all the times the bot said “70% chance to win”, it was right 70% of the time.

How do I invite the TBM bot to another server?

At this moment, you can’t. Currently, the TBM bot is a public resource, free for anyone to use as much as they’d like, but only available on the TBM Discord. However, getting the bot running on other servers is a goal we’re working towards on Patreon. At a certain level of funding, we can get the bot a proper host (which is quite expensive because of the enormous volume of data). At that point we’ll do the work it takes to let people run the bot on their Discord servers.