Reintroducing the 2024 BOA b(AI)ndScores Rankings

With the completion of this past weekend’s regionals, we are half way through the 2024 season - five weekends down, five to go. Through the first half of the season 353 different bands have competed in 18 regionals combining for a total of 583 scores in all rounds and events.

The back half of the season is sure to draw even more intrigue. Included in that slate are the three super regionals in Indianapolis, St Louis, and San Antonio with a combined 229 bands competing at those events alone. Of course all eyes will be on the final weekend of the season at Grand National championships, the biggest ever in BOA history.

Although a number of bands have yet to make their 2024 BOA debuts, the competitive landscape is beginning to take shape. Given this and where we are in the season schedule, we are excited to announce the return of our b(AI)ndScores rankings - our predictive rankings powered by an artificial intelligence model using historic BOA score data and score data from the 2024 season.

b(AI)ndScores made its debut last year for the 2023 season during which we published weekly rankings as well as our predictions for the top 25 placing bands at Grand Nationals. The model performed quite well and by the end of the season was operating at close to 90% accuracy with historic data. In context of the 2023 Grand Nationals, it correctly predicted 10 of the 12 finalists and came within a point of predicting the correct final score for a handful of finalist bands.

Although it’s been a year since the initial launch, the model is nowhere close to being in a "permanent" state. We’re constantly tweaking it, introducing it to more historic data, and calculating and including additional metrics all with the goal of making improvements and increasing accuracy. Even so, the model and its predictions will never be perfect - after all, competitive marching arts is very difficult to predict when it’s entirely dependent on subjective opinions from a panel of judges viewing a single performance of every band. That said, you can perhaps take our rankings with a grain of salt knowing that bands in the same general level of ability and execution will differ in score and placement at any given competition.

This year, our model predictions will be expanded to ranking the top 100 bands and will be updated weekly on our blog for the remaining five weeks of the season. Additionally, as part of this blog post, we’d like to share more insight and transparency into our model, how it comes up with its rankings, and even some steps we take to further adjust and improve it. If you love following scores, rankings, and band data in general (which hopefully you do if you’re here in the first place), we’re sure you’ll find the rest of this post fascinating. Let’s dive in.

The Data behind the Model

Artificial intelligence and machine learning have exploded in popularity in recent years, especially after OpenAI made AI tools such as Chat GPT and DALL-E publicly available in 2022. More simple AI and machine learning models are used in many areas of the world as people are building these tools to help generate predictions and make sense of the world.

Behind every AI model is one common thing - they all use varying amounts of real data from events generated in the past to make predictions about future events. The b(AI)ndScores model is no different - it uses years of historical BOA scores and data to predict (and then rank) how bands will finish in the current season. More specifically, we use the same data set that powers our BOA Historic Scores page with over 10,000 scores dating back to the 2012 season.

Using the historic data, we filter for regional scores by bands that later attended Grand Nationals in the same season - 1348 total scores to be exact. The band’s final score at Grand Nationals becomes the target variable - we tell the model to calculate how scores from regionals influenced what each band ultimately received at Grand Nationals. It then use those calculations from past seasons, applies them to scores from the 2024 season, and predicts what each band may receive as a final score at 2024 Grand Nationals, even if the band is not scheduled to attend. Where the band ranks within all of these predicted final scores is ultimately the rankings that we share (Well….sort of. More on that in a bit).

The Important Stuff

It goes without saying the obvious, but a band’s score at a regional is the most important data point that can predict a band’s score at Grand Nationals. If you’ve followed the activity for a while, you know that bands that do well at regionals - especially regionals that are bigger and with a number of notable bands - tend to do well at Grand Nationals. While some bands are able to make significant improvements over the course of a season, the bands that generally do well earlier in the season tend to place highly at the end of the season.

Along those lines, caption scores hold strong predictive value and are included in the model. Depending on how you consider BOA's score breakdown, General Effect is worth 60% of a band’s score (three GE judges whose scores are worth 20 points each) while Music is also worth 60% (two music GE judges whose scores are worth 40 points total plus two music performance judges whose combined scores are worth an additional 20 points). That said, GE and music scores tend to hold more weight and value in the model, though visual scores still hold some value too.

Where a band’s score falls in the season schedule is another important factor. A band that competes later in the season is much more likely to have a cleaner and more complete show on the field versus a band that competes in the first few weeks of the season. Scores from earlier in the season do hold predictive value, but the model tends to favor later season scores and is more likely to make a more accurate prediction with that data. By that logic, the number of weeks a score occurs before Grand Nationals is included as a variable. Additionally, our rankings use the predicted score that is generated from each band’s most recent score.

Historic Success is a Factor Too

We also decided for the model to consider a band’s successes in recent seasons. While scores for many bands fluctuate between seasons depending on a variety of factors (student experience and ability, staff and director changes, and show design to name a few), in general, bands do not jump or drop a significant number of placements between seasons. It is unlikely that a band that places below 50th at Grand Nationals to make finals the following season, or vice versa, for a band that wins Grand Nationals to fall out of finals at their next appearance. Case in point, certain bands that attend Grand Nationals every year or every other year such as Carmel HS, Avon HS, Broken Arrow HS, Tarpon Springs HS, and The Woodlands HS have become notable programs that have consistently placed highly over the past decade and a half. Using placements from previous seasons can help the model identify bands that are more likely to continue to stay in the top echelon of the activity.

Additionally, using prior placements at bigger shows such as Super Regionals can help accurately rank bands that attend Grand Nationals on a less regular basis. Bands from Texas are a perfect example of this - despite the state being known as one of the most competitive in the marching band world, a number of programs are only able to attend Grand Nationals every few years. For these bands that consistently place highly at the San Antonio Super Regional, using recent placements from that show can help the model more fairly rank those programs more accurately even though they compete on the national level less frequently.

Not All Scores are Created Equally

Perhaps the most difficult part of building the BOA b(AI)ndScores model is accounting for the fact that it is incredibly difficult to compare scores between different regionals. While BOA has criteria that helps judges determine their scores, the final scores are always in the context of the competition itself and shouldn’t necessarily be compared between shows. For bands that compete at multiple regionals a year, there are often cases of scores increasing or even decreasing by significant amounts between shows. Even at Grand Nationals, there are cases of bands finishing with a score that is a few points lower than where they scored at a regional only a few weeks earlier.

To account for these factors, the model considers some additional metrics about the regionals themselves such as the total number of bands that competed, the average point spread between each band, and the score of the regional champion. We also attempt to determine a numeric “show quality” index of a regional by considering the number of Grand National and Super Regional finalists and their placements from the prior season at a given show, and using this weighted value in the model.

No one is Perfect

As you can tell, there’s a number of data points that the model considers, and yet, it is far from 100% perfect. So why does the model rank certain bands ahead of others even though a head to head match up in real life tells a different story? For many of the reasons described above - perhaps the model believes a band will be scoring higher based on a better GE or music score, a score from a more recent regional, or better successes in recent season. Remember, these rankings are based on where bands are predicted to score at the end of the season, not necessarily where they currently rank during the season.

We will also add three final points of context:

1. The final rankings are based on a weighted average that considers 75% of the predicted score from the model and 25% of a band’s adjusted score. The adjusted scores are available on our 2024 BOA Scores page and takes a band’s score at their most recent regional and adds points equal to the number the weeks the score occurs before Grand Nationals. Using this weighted average helps balance out some of the model’s rankings, for example, in cases where it ranks a band that is scoring well but is perhaps is newer to BOA and is being punished by lack of historical success.

2. We may continue to adjust and improve the model in a variety of capacities as the season progresses. As such, band’s rankings may fluctuate week to week, even if they do not compete in a more recent BOA event.

3. Always consider that the model only ranks bands that have competed in at least one event during the 2024 season. As we update the rankings, we will include a list of bands that were highly ranked in the 2023 season but have not yet competed.

At the end of the day, these rankings are merely for fun and to provide some insight as where bands might finish the 2024 season. View them with the understanding that scoring is always subjective and that anything can happen.