2024 Grand Nationals Recap: Evaluating how the b(AI)ndScores Model performed

Initial Evaluation

Perhaps the first thing you noticed is that the model picked the correct champion! The 2024 season ended with Avon HS being crowned Grand National Champions yet again, as they became the first repeat winners in six years. Their winning score of 97.3 is only the ninth highest in a Grand Nationals finals round since 2012, however them and Flower Mound HS earned the title of being the only two bands to break 97 this season.

The model also correctly picked 11 out of 12 finalists including Cedar Park HS who appeared at Grand Nationals for the first time since 2016, American Fork HS who made finals for the first time since 1995, and Bridgeland HS who made their first ever appearance at Grand Nationals. It also picked the correct placements for Carmel HS and Dobyns-Bennett HS, and came within one spot of correct placements for Broken Arrow HS, Tarpon Springs HS, and William Mason HS. Overall, 9 finalists finished within +/- 2 spots of their predicted placement and the 12 finalists finished on average within +/- 1.6 spots of their predicted placement. When evaluating scores, 8 bands finished with scores within one point of their predicted score and the 12 finalists finished on average +/- 1.3 points away from their predicted score.

The model was only marginally successful when evaluating the 13 semifinalists that finished in the top 25. Of these bands, 8 were correctly predicted as top 25 semifinalists, while 4 finished in the top 25 after the model predicted them to finish lower, and 1 finished as a semifinalist after being predicted to finish as a finalist. The variance in their scores and placements were much higher than the finalists bands as well. Only four bands finished within two spots of their predicted placement and three bands within a point of their predicted score.

Where the Model was Right

As mentioned earlier, the model was right in picking Avon HS as champion and also Carmel HS as the runner up. This is largely unsurprising - the two Indiana bands have placed in the top 2 at the Indianapolis Super Regional every year since 2009 and have both placed in the top 3 at 10 of the last 13 Grand Nationals - trends that continued in 2024. Additionally the two bands have won a combined nine championships since 2008, while the winner of the Indianapolis Super Regional has gone on to also win Grand Nationals five times. Simply put, the winner of the Indianapolis Super Regional was always going to be the model’s favorite going into Grand Nationals with the runner up favorited for second place.

We always anticipated that the grouping of Broken Arrow HS and Tarpon Springs HS was going to be tricky to predict and that was confirmed by the bands’ final placements. In their last head-to-head competition at the 2023 Orlando Regional, Broken Arrow HS finished second and this year, the Oklahoma band didn’t win the St. Louis Super Regional for the first time since 2007. Meanwhile, Tarpon Springs HS repeated as champions at this year’s Orlando Regional but didn’t face any significant competition. The model slightly favorited Tarpon Springs HS by two tenths due to these trends, though the results ultimately swung in Broken Arrow HS favor by 0.35. In fact, Broken Arrow HS even placed above Carmel HS by a slim margin in semifinals, and tied them by score in finals, although second place was awarded to Carmel HS due to the tiebreaker in the General Effect caption.

Coming into 2024, William Mason HS had placed in the top four at the past three Indianapolis Super Regionals and in the top six at the past two Grand Nationals. As this trend also continued this season, the model predicted another top six finish for them, though they finished below Cedar Park HS and one place lower than their predicted spot. Cedar Park HS was picked in seventh largely due to their strong, top ten finish at the San Antonio Super Regional and the model favoriting them as the best Texas band attending Nationals; they ultimately exceeded expectations and matched their fifth place finish at their last appearance in 2016.

As alluded to earlier, some of the scores that are hardest to predict are for bands that only attend one regional earlier in the season and/or bands who don’t compete against recent Grand National finalists. Three such examples this year were Bridgeland HS, Prosper HS, and American Fork HS. Bridgeland HS and Prosper HS were similar in that both placed in the top four at regionals in Texas in early October however neither attended the San Antonio Super Regional. Making predictions off of early season scores gets tricky as some bands may not perform their full show and have not yet reached their full potential at that point in the season. That said, both Bridgeland HS and Prosper HS both scored in the mid 80s and within a few spots of 2023 Grand Nationals finalists at their respective regionals, so the model was comfortable and correct in picking the pair to make finals.

American Fork HS was a slightly different story. In 2024, the band once again won the Utah Regional for the 11th time in the past 12 seasons and set a new record score for the show with a 92.45, breaking their own record set last year. Despite this accomplishment, it was still a question of whether or not the band could finally make the leap into finals after top 20 finishes at their five previous Grand National appearances dating back to 2008. And for good reason - the Utah Regional rarely features other bands that are prominent Grand Nationals or Super Regionals finalists, so it is often difficult to gauge where American Fork HS ranks nationally. That said, their performances at Grand Nationals backed up their scores and the model’s predictions as the band found themselves in finals and with an eighth place finish.

The remaining two correctly-picked finalists - Castle HS and Dobyns-Bennett HS - both finished their seasons as expected. Although Dobyns-Bennett HS finished lower than second at a regional for the first time since 2016, they won the Chattanooga Regional. Based on this accomplishment and their previous, recent successes, the model correctly picked them to finish in 10th. Meanwhile, Castle was predicted to finish 12th after a respectable seventh place finish at the Indianapolis Super Regional, and ultimately exceed expectations by three places to finish ninth.

Where the Model was Wrong

We’ll iterate again - the model wasn’t perfect and had a few misses. Perhaps the most obvious was that it picked Brownsburg HS to make finals and left out Leander HS. And for good reason - in the case of Brownsburg HS, the band made finals in the previous two seasons and finished sixth at this year’s Indianapolis Super Regional. While it was not a lock, it seemed likely that the band was destined for another appearance in finals, however they ultimately fell and finished in 17th.

Leander HS was a band that was very much on the bubble coming into Grand Nationals. A seventh place finish and score in the mid 70s at the Austin Regional earlier in the season put their status in question, and again a week before Nationals when they finished 16th and outside of finals at the San Antonio Super Regional. Previous results at the San Antonio Super Regional shows high correlation with results at Grand Nationals - from 2012 to 2023, of the 41 Texas bands that made Grand Nationals finals, 31 also attended and made finals at San Antonio, while only three bands made Grand Nationals finals while missing finals at San Antonio. The model favorited these odds and picked Leander HS to finish qs a semifinalist, though the band ultimately beat this prediction to finish 12th.

Remember when we discussed earlier that it’s really difficult to predict how bands are going to place when they perform at less competitive shows? That fact reared its ugly head when two bands - Stoneman Douglas HS and Westlake HS - missed semifinals after the model predicted them to finish in the top 25. In both bands’ cases, each had strong showings at earlier shows and finished two places behind a pair of bands that were ultimately named finalists - Tarpon Springs HS and Stoneman Douglas HS. In these cases, the model saw that the point gaps between the champions and these bands were close enough that they stood a fair chance of breaking into the top 25, though this prediction ultimately fell short.

Conversely, four bands - Jenks HS, Argyle HS, Union HS, and Rockford HS - exceeded expectations and finished in the top 25 after the model predicted that they wouldn’t. Union HS placed four spots higher than expected though the other three saw impressive finishes of 18 or more places higher than expected. In the case of Jenks HS and Argyle HS, both were predicted to finish lower after each missed finals at their only regional competitions of the year. While Rockford HS did make finals and finished third at the Toledo Regional in the first week of the season, their lower score at this show indicated a possibility that they wouldn’t make semifinals.

Final Thoughts

As you can tell, the model had an equal amount of right and wrong predictions and if we had to give it a letter grade, we’d give it a solid B. It did a great job predicting the correct champion, correctly choosing 11 out of the 12 finalists, and predicting rankings within two spots of the actual placement for 75% of the finalists. When it came to semifinalists, it correctly picked a number of bands that ultimately finished within the top 25, however it missed on a few. That said, the reasoning for some of the incorrect predictions were well supported by results seen earlier in the season. As we’ve said all along, it’s incredibly difficult to make these predictions solely based on data when there are a number of intangible circumstances that factor into a band’s performance, score, and final placement. However, we’re pleased with the final results of the 2024 b(AI)ndScores model and hope you enjoyed following along this season!

If you’ve been following our content on our blog and Instagram throughout the 2024 Bands of America season, you’re likely familiar with our b(AI)ndScores rankings that we’ve been sharing during the fall. Earlier in the season, we also posted a piece describing some of the data and methodology that powered the model and explained how it made its predictions.

Using the model’s output, we also posted our predictions for the 2024 Grand Nationals before the event started. Essentially, we took the model’s predicted score for every band attending Grand Nationals and ranked the top 25 bands using those scores. We only posted rounded scores and placements for the top 12 bands while naming the other 13 as semifinalists.

Now that Grand Nationals and the season is over, we thought it’d be fun to look back at our predictions, evaluate how they compared to the actual results, and share our thoughts on where and why the model went right and wrong. Below is a table listing the final top 25 bands from Grand Nationals. The left two columns are their actual final scores and placements, the middle two columns were their predicted final scores and placements, and the right two columns show the difference between the actual and predicted results. Click on the image for a full screen view.