Posted by Dr Bracket on March 27, 2025 at 11:31:37:
So CIF released a bracket analysis, but their stats are highly unsophisticated. So being a Statistician by trade I did a deep dive, comparing 23-24 and 24-25 playoff results. CIF talked a lot about point differential in terms of averages, but most people should understand that is not the best measure, especially when you see large disparity in data set like this (product of talent disparities in the sport in general).
So we ran the following statistical analysis of the brackets (first round only). A two sample T test, an effect size calculation to quantify the magnitude of any difference and finally a chi-square test to compare proportions of competitive vs. non-competitive games. (We did this for the first round only)
in the T test we saw
The critical t-value:
For a two-tailed test with α = 0.05 and df = 261, the critical t-value is approximately ±1.97
Calculate the p-value:
The p-value for t = 1.55 with 261 degrees of freedom is approximately 0.122.
The t-statistic (1.55) is less than the critical t-value (1.97), and the p-value (0.122) is greater than the significance level (0.05). Therefore, we fail to reject the null hypothesis.
This means that while there is a difference in the mean point differentials between 2024 (30.48 points) and 2025 (27.04 points), this difference is not statistically significant at the 0.05 level.
In other words, although we observed a decrease in the average point differential from 2024 to 2025, suggesting potentially more competitive games in 2025, this difference is not large enough to conclude that it represents a genuine change in competitiveness rather than random variation.
Next we ran a Cohen D effect size calculation:
Step 1: Calculate the pooled standard deviation (sp)
We already calculated this in the t-test:
sp = 17.85
Step 2: Calculate Cohen's d
d = (mean1 - mean2) / sp
d = (30.48 - 27.04) / 17.85
d = 0.19
Interpretation:
Cohen suggested the following guidelines for interpreting effect sizes:
Small effect: d = 0.2
Medium effect: d = 0.5
Large effect: d = 0.8
Our calculated effect size is d = 0.19, which is just below the threshold for a small effect.
The difference in competitiveness between 2024 and 2025 represents a small effect. While there is a noticeable difference, it's not substantial.
The average point differential in 2024 was about 0.19 standard deviations higher than in 2025. This indicates a slight increase in competitiveness in 2025 compared to 2024.
While the t-test didn't show statistical significance, the effect size calculation suggests there is a real, albeit small, difference in competitiveness between the two years.
Lastly the Chi square test:
Find the critical value
For α = 0.05 and df = 1, the critical value is 3.84
Calculate p-value
The p-value for χ² = 1.26 with df = 1 is approximately 0.262
Interpretation:
The calculated Chi-square statistic (1.26) is less than the critical value (3.84).
The p-value (0.262) is greater than the significance level (0.05).
Therefore, we fail to reject the null hypothesis. This means that there is no statistically significant association between the year and the proportion of competitive vs. non-competitive games.
In other words, while we observed a slight increase in the number of competitive games from 2024 (34 out of 130) to 2025 (43 out of 133), this difference is not statistically significant. The change in the proportion of competitive games could be due to random chance rather than a real shift in competitiveness between the two years.
However, it's worth noting that:
The proportion of competitive games did increase from 26.15% in 2024 to 32.33% in 2025.
This increase aligns with our previous observations of a trend towards more competitive games in 2025, even though it's not statistically significant.
The lack of statistical significance doesn't mean there's no real difference; it just means we can't be confident that the observed difference isn't due to chance.
To summarize
Considering all three tests (two-sample t-test, effect size calculation, Chi-square test), we can summarize the first round competitiveness between 2024 and 2025 as follows:
Trend towards increased competitiveness:
All tests consistently indicate a trend towards more competitive games in 2025 compared to 2024. The average point differential decreased from 30.48 in 2024 to 27.04 in 2025.
Small but noticeable effect:
The effect size calculation (Cohen's d = 0.19) suggests that the increase in competitiveness is real but small. This is just below the threshold for what's typically considered a "small" effect.
Lack of statistical significance:
Both the t-test (p = 0.122) and Chi-square test (p = 0.262) failed to show statistical significance at the conventional 0.05 level. This means we can't rule out that the observed differences could be due to random chance.
Increased proportion of competitive games:
The Chi-square test revealed an increase in the proportion of "competitive" games (point differential ≤15) from 26.15% in 2024 to 32.33% in 2025, though this increase wasn't statistically significant.
The first round of the 2025 tournament showed a consistent trend towards increased competitiveness compared to 2024. While this increase was not large enough to be statistically significant in traditional hypothesis tests, the effect size calculation suggest a real, albeit small, improvement in competitiveness. This change was most noticeable in games with moderate point differentials and resulted in a higher proportion of close games (point differential ≤15) in 2025.
These findings suggest that any changes made to the tournament structure or seeding process between 2024 and 2025 may have had a positive, though modest, impact on first-round competitiveness. Tournament organizers should view this as a step in the right direction, while also recognizing that there's still room for further improvements if increased competitiveness is a goal.
All that being said we did look at Division placement, ranking, and finish and found that the ranking system was inconsistent at placing lower divisions teams appropriately.
Gabrielino (Division 5A)
Ranking: 443
Record: 13-9-0
Performance: Won the championship, with wins by margins of 31, 22, 25, and 15 points
Analysis: One of the lowest-ranked teams in 5A, yet they dominated the division, indicating a severe underranking.
Rosemead (Division 5A)
Ranking: 388
Record: 21-6-0
Performance: Reached the final, winning games by margins of 42, 24, and 22 points
Analysis: Despite an excellent record, they were ranked very low. Their playoff performance confirms they were significantly underranked.
Hillcrest (Division 5AA)
Ranking: 311
Record: 17-6-0
Performance: Won the championship, with wins by margins of 21, 23, 4, and 10 points
Analysis: Their performance suggests they should have been ranked significantly higher than 311th.
This suggests that the ranking algorithm may be undervaluing factors such as:
Late-season improvements
Performance in key games
Strength of non-league schedule
Historical program strength
Conversely, it may be overemphasizing:
League difficulty
Early-season results
Margin of victory in weaker leagues
These discrepancies indicate that while the algorithmic approach provides a baseline, there might be a need for some level of human oversight or additional factors to be considered in the ranking and division placement process, especially for teams in lower divisions or with records that don't align with their ranking.
| Name: | |
| E-Mail: | |
| Subject: | |
| Comments: | |
| Optional Link URL: | |
| Link Title: | |
| Optional Image URL: | |