2021 Regionals By The Numbers
Wed Jan 27, 2021 12:05 am
Regional assignments are out at long last and that means it’s time for some analysis! We will have our week one analysis out as soon as possible, but in the meantime, we have crunched some numbers to bring you our initial thoughts on the difficulty of each regional.
Different strength breakdowns:
As usual, we can break this down in a lot of different ways:
First, we can use our rank metrics to get a sense of the overall strength of each region. Ranks are the simplest metric for measuring the strength of teams, a lower average rank indicates better teams on the whole. The first column on our chart represents the average rank of the teams at each regional.
It’s also important to look at how strong the top teams at a tournament are because these are the teams you have to displace if you want to get a bid. Thus, the second column looks at the average rank of the top n teams at each regional where “n” is the number of bids allocated to that regional.
TPR points look at the raw data that AMTA uses to assign those ranks (sometimes there are large jumps in rank so this can be a little more accurate). For TPR we can look at the sum of the TPR for all of the teams there which gives us a sense of the total power at the tournament. Our third column looks at the total number of TPR points present in a region.
We can also look at the average number of TPR points for a team in a region to take into account the fact that different tournaments have different numbers of teams. This looks at how strong the average team at the tournament is. Our fourth column looks at the average TPR points of a team in the region.
Once again, we can use TPR to look at just the average of the top teams. This tells us how much power you have to displace to get a bid. The fifth column looks at the average number of TPR points held by the top n teams in a region where “n” is the number of bids allocated to that regional.
And finally, we can look at the number of teams from various difficulty levels at each tournament. Our chart shows the number of Top 200 teams, the number of Top 100 teams and the number of Top 50 teams in each region.
A note on C+ teams: since C+ teams can’t get a TPR, but some of them are quite good, we assigned C teams ½ the TPR points held by the corresponding B teams. D teams got ½ the C team TPR and so on. We then assigned ranks based on where those point values would place the teams.
We have taken the liberty of color coding our chart so that it’s easy to see where each region stands relative to the rest of the field in each category. Red regions are harder than others in that category. Green regions are easier than others.
The MAIMD Official Ranking:
And now, averaging the results from our different ranking methods, we have the official Mock Analysis regional rankings from most to least difficult:
1. 2D
2. 3F
3. 1F
4. 4B
5. 3C
6. 1D
7. 2A
8. 2F
9. 3H
10. 1G
11. 1C
12. 1E
13. 3D
14. 4E
15. 3B
16. 3G
17. 4D
18. 2C
19. 4A
20. 4C
21. 3E
22. 4H
23. 2H
24. 4G
25. 3A
26. 2E
27. 2B
28. 4F
29. 1H
30. 1A
31. 2G
32. 1B
The New Geographically Balanced Regionals:
The next question at the very top of our mind is how this year’s regionals compare to last year’s regionals in terms of numerical fairness. Over the last few years there has been a general view that any disparity in terms of difficulty between the regions is a result of geographic limitations on what AMTA can do. So we wanted to see if things got better when geography was no longer a concern. In order to do that we looked at the disparity between the hardest regional and the easiest regional and compared AMTA’s work from this year to last year in each metric.
At first glance this looks like by getting rid of geographic concerns AMTA managed to eliminate between ¼ and ½ of the strength disparity in every category this year.
However there was one really major issue that was affecting last year’s regionals and it was both Cornell teams dropping out of Buffalo very late making Buffalo the numerically weakest regional we have seen in our time doing this by a long shot. So in order to make sure this wasn’t just the Buffalo effect we eliminated last year’s Buffalo regionals from our computations and did them again. This time, they don’t look so rosy:
Interestingly it looks like the regional disparity this year is pretty much the same as it was last year. On some metrics this year is better, and on some metrics this year is worse, The changes were all pretty small. In other words, the balance of regionals didn’t get any better when AMTA could balance across the whole country and didn’t have to worry about drive times, regionals are just as imalanced as they ever were.
This is particularly striking because they didn’t have to be imbalanced numerically. Suppose for example AMTA has selected regionals by the simple method of going down the ranks list, puting one team in each regional until each had one and then doubling back and putting one team in each regional sequentially in the opposite direction (that is if Rank 1 went in 1A, Rank 2 went in 1B, and so on until Rank 32 when in 4H, and then Rank 33 when in 4H, Rank 34 went in 4G, and so on until Rank 64 went in 1A). There would still be some slight variation in the regionals due to the fact that there are only 293 ranked teams which don't fit evenly into 32 regionals and currently 655 teams which also don’t fit evenly into 32 regionals. But the disparity in average rank between the hardest and easiest region would be between 226.70 and 229.86 for a difference of 3.16. This would have eliminated 93.17% of the disparity from 2020 (using the nonBuffalo numbers). Meanwhile the disparity in Average Rank of Top teams would be 0, eliminating 100% of the disparity. The TPR based metrics would have slightly wider gaps using that metric. The TPR Sum disparity would be 147.08124.08=23.00 eliminating 74.23% of the 2020 disparity. The TPR Average disparity would be 7.356.43=1.00 eliminating 78.20% of the 2020 disparity. The TPR Average for Top Teams disparity would be 22.6819.10=3.58 eliminating 58.52% of the 2020 disparity. Notably this isn’t even the most even way of splitting up the teams numerically speaking, it's just the easiest way to split them up and make them pretty even. Obviously AMTA has more considerations than just numerical balance (some teams can’t compete on certain weekends, etc). But it is notable how little has changed this year given how much could have.
Different strength breakdowns:
As usual, we can break this down in a lot of different ways:
First, we can use our rank metrics to get a sense of the overall strength of each region. Ranks are the simplest metric for measuring the strength of teams, a lower average rank indicates better teams on the whole. The first column on our chart represents the average rank of the teams at each regional.
It’s also important to look at how strong the top teams at a tournament are because these are the teams you have to displace if you want to get a bid. Thus, the second column looks at the average rank of the top n teams at each regional where “n” is the number of bids allocated to that regional.
TPR points look at the raw data that AMTA uses to assign those ranks (sometimes there are large jumps in rank so this can be a little more accurate). For TPR we can look at the sum of the TPR for all of the teams there which gives us a sense of the total power at the tournament. Our third column looks at the total number of TPR points present in a region.
We can also look at the average number of TPR points for a team in a region to take into account the fact that different tournaments have different numbers of teams. This looks at how strong the average team at the tournament is. Our fourth column looks at the average TPR points of a team in the region.
Once again, we can use TPR to look at just the average of the top teams. This tells us how much power you have to displace to get a bid. The fifth column looks at the average number of TPR points held by the top n teams in a region where “n” is the number of bids allocated to that regional.
And finally, we can look at the number of teams from various difficulty levels at each tournament. Our chart shows the number of Top 200 teams, the number of Top 100 teams and the number of Top 50 teams in each region.
A note on C+ teams: since C+ teams can’t get a TPR, but some of them are quite good, we assigned C teams ½ the TPR points held by the corresponding B teams. D teams got ½ the C team TPR and so on. We then assigned ranks based on where those point values would place the teams.
We have taken the liberty of color coding our chart so that it’s easy to see where each region stands relative to the rest of the field in each category. Red regions are harder than others in that category. Green regions are easier than others.
The MAIMD Official Ranking:
And now, averaging the results from our different ranking methods, we have the official Mock Analysis regional rankings from most to least difficult:
1. 2D
2. 3F
3. 1F
4. 4B
5. 3C
6. 1D
7. 2A
8. 2F
9. 3H
10. 1G
11. 1C
12. 1E
13. 3D
14. 4E
15. 3B
16. 3G
17. 4D
18. 2C
19. 4A
20. 4C
21. 3E
22. 4H
23. 2H
24. 4G
25. 3A
26. 2E
27. 2B
28. 4F
29. 1H
30. 1A
31. 2G
32. 1B
The New Geographically Balanced Regionals:
The next question at the very top of our mind is how this year’s regionals compare to last year’s regionals in terms of numerical fairness. Over the last few years there has been a general view that any disparity in terms of difficulty between the regions is a result of geographic limitations on what AMTA can do. So we wanted to see if things got better when geography was no longer a concern. In order to do that we looked at the disparity between the hardest regional and the easiest regional and compared AMTA’s work from this year to last year in each metric.
At first glance this looks like by getting rid of geographic concerns AMTA managed to eliminate between ¼ and ½ of the strength disparity in every category this year.
However there was one really major issue that was affecting last year’s regionals and it was both Cornell teams dropping out of Buffalo very late making Buffalo the numerically weakest regional we have seen in our time doing this by a long shot. So in order to make sure this wasn’t just the Buffalo effect we eliminated last year’s Buffalo regionals from our computations and did them again. This time, they don’t look so rosy:
Interestingly it looks like the regional disparity this year is pretty much the same as it was last year. On some metrics this year is better, and on some metrics this year is worse, The changes were all pretty small. In other words, the balance of regionals didn’t get any better when AMTA could balance across the whole country and didn’t have to worry about drive times, regionals are just as imalanced as they ever were.
This is particularly striking because they didn’t have to be imbalanced numerically. Suppose for example AMTA has selected regionals by the simple method of going down the ranks list, puting one team in each regional until each had one and then doubling back and putting one team in each regional sequentially in the opposite direction (that is if Rank 1 went in 1A, Rank 2 went in 1B, and so on until Rank 32 when in 4H, and then Rank 33 when in 4H, Rank 34 went in 4G, and so on until Rank 64 went in 1A). There would still be some slight variation in the regionals due to the fact that there are only 293 ranked teams which don't fit evenly into 32 regionals and currently 655 teams which also don’t fit evenly into 32 regionals. But the disparity in average rank between the hardest and easiest region would be between 226.70 and 229.86 for a difference of 3.16. This would have eliminated 93.17% of the disparity from 2020 (using the nonBuffalo numbers). Meanwhile the disparity in Average Rank of Top teams would be 0, eliminating 100% of the disparity. The TPR based metrics would have slightly wider gaps using that metric. The TPR Sum disparity would be 147.08124.08=23.00 eliminating 74.23% of the 2020 disparity. The TPR Average disparity would be 7.356.43=1.00 eliminating 78.20% of the 2020 disparity. The TPR Average for Top Teams disparity would be 22.6819.10=3.58 eliminating 58.52% of the 2020 disparity. Notably this isn’t even the most even way of splitting up the teams numerically speaking, it's just the easiest way to split them up and make them pretty even. Obviously AMTA has more considerations than just numerical balance (some teams can’t compete on certain weekends, etc). But it is notable how little has changed this year given how much could have.
MockTrialWritersGuild likes this post
Re: 2021 Regionals By The Numbers
Wed Jan 27, 2021 7:31 am
TPR is inherently geographically biased. They have a dependence on locationallybased tournament performance (regionals > ORCS). Yes, ORCS have always had wider, geographically broader pools, but still a strong bias.
I understand that most recent year TPR is probably best estimate of current strength that you have available. But in order to best estimate your "disparity" eliminated, you should do some "geographic normalization" based on past X national championships. For example, for the last five NCT, if I split the pool up to four broad groups by geography, their relative performance compared to expected baseline performance if NCT were perfectly balanced could be one way to normalize. You can look at specific matchups, overall record, etc and arrive at a way to quantify geographical bias thus disparity that way and adjust for that in your calculation.
I understand that most recent year TPR is probably best estimate of current strength that you have available. But in order to best estimate your "disparity" eliminated, you should do some "geographic normalization" based on past X national championships. For example, for the last five NCT, if I split the pool up to four broad groups by geography, their relative performance compared to expected baseline performance if NCT were perfectly balanced could be one way to normalize. You can look at specific matchups, overall record, etc and arrive at a way to quantify geographical bias thus disparity that way and adjust for that in your calculation.
TheRealMockProdigy likes this post
Re: 2021 Regionals By The Numbers
Wed Jan 27, 2021 9:20 am
I also think AMTA made these regionals using data other than TPR from 2019. My understanding is that they calculated TPR for teams that they were able to for last year, and are factoring that into the regionals assignments. Having more accurate data might be helpful here, although since AMTA hasn't shared that with us there isn't much we can do, short of calculating it ourselves.
briefly_your_honor likes this post
Re: 2021 Regionals By The Numbers
Wed Jan 27, 2021 10:32 am
Given what you have to work with this is super impressive. An interesting experiment this year, if nothing else.
Re: 2021 Regionals By The Numbers
Wed Jan 27, 2021 10:45 am
TheRealMockProdigy wrote:I also think AMTA made these regionals using data other than TPR from 2019. My understanding is that they calculated TPR for teams that they were able to for last year, and are factoring that into the regionals assignments. Having more accurate data might be helpful here, although since AMTA hasn't shared that with us there isn't much we can do, short of calculating it ourselves.
This is really interesting if its true. Do you know if AMTA plans to take the same approach for the ORCS tiers? Seems very strange to sort 2021 teams based on 2019 TPR at ORCS.
Re: 2021 Regionals By The Numbers
Wed Jan 27, 2021 10:52 am
I don't see why they wouldn't. This is certainly a situation where some transparency from AMTA would be really beneficial.
Permissions in this forum:
You cannot reply to topics in this forum

