The VFX Predictinator, Part 1

Avid readers of this site may have noticed a curious omission during last year's awards season. I didn't run the numbers to try and predict which film would win the Academy Award for Best Visual Effects.

In early 2008, I took an exhaustive (and exhausting) look at the previous 23 years of visual effects nominees and tried to determine what was a better predictor of the film that wins the Oscar: critical acclaim or box office popularity. (If you're interested in all of the explanations and data, be sure to read the full articles, "Predicting the VFX Oscar Part 1," which has links to Parts 2 through 5.)

Here is a brief summary for those who need to catch up. I've always wondered if there exists a statistical relationship between the winner of the visual effects Academy Award and some sort of quantifiable criteria which informs the Academy's voting. I wondered if one could accurately predict which nominee wins the award. And, most importantly, which is the more accurate predictor of the visual effects Oscar - critical acclaim or box office success?

And please keep in mind that this is a game of prediction; this process does not take into account the actual quality or achievement of each years' nominees. We are looking at Academy voting trends, and what criteria influences Academy voters' choices.

Analysis of 23 years of Academy Award nominees for Best Visual Effects brought me to this conclusion: critical acclaim is generally a better predictor of the winner of the Academy Award than box office popularity. And, to state the obvious, this theory surmises that critical acclaim (which initially drives the wave of publicity, such as "For Your Consideration" advertisements, Oscar "buzz," and the self-fulfilling prophecy of "If people are saying it's Oscar-worthy, then it must be Oscar-worthy") ultimately informs Academy voters and influences their vote.

For the 80th Academy Awards, I ran the numbers (in Part 4), and the data overwhelmingly indicated that "Transformers" would bring home the gold. The silly giant robot film had the greatest critical acclaim (a modest 57% on the Tomatometer, but still higher than its competitors) and earned the most money at the box office (barely edging out "Pirates 3" but destroying "The Golden Compass"). In addition, in eleven years of the 23 years charted the film that earned the most acclaim and box office won the Oscar ten times. Armed with this ammunition, I felt pretty confident predicting "Transformers" would bring home the gold. Of course, I-- and the rest of the moviegoing world-- was a bit stunned with "The Golden Compass" win.

Huh?

Not only did my prediction fall flat on its face, but my heart was broken (not because I'm particularly fond of shiny robots, but because I was a sequence supervisor at ILM for the film). The "Transformers" loss left my theory dead.

As the 81st Academy Awards nominations were announced, I had a sinking feeling that the theory would fall apart yet again. The three nominees for the Oscar were "The Dark Knight," "Iron Man" and "Benjamin Button." Looking at the numbers, all three films earned some fairly solid acclaim ("Dark Knight" earned 94% on the Tomatometer, "Iron Man" earned 93% and "Button" garnered 72%), and although "Dark Knight" destroyed at the box office, they each made a respectable amount of money ($513, $318 and $125 million, respectively). According to the numbers, with its domination of the box office and the highest amount of critical acclaim (by a hair), "The Dark Knight" should have wrapped up the statuette.

As we drew closer to the awards, I realized it was not meant to be. The stinging, theory-destroying win of "The Golden Compass" would probably be repeated, with "Button," the least acclaimed nominee and the least earning film winning the Oscar. (On a personal note, I actually felt that "Button" deserved the Oscar by a hair, just slightly more than "Dark Knight." But to quote Clint Eastwood in "Unforgiven," 'deserve's got nothin' to do with it.')

So I decided not to run the numbers and not write up a blog post. Ultimately, I feel somewhat vindicated since my fears came true; "Button" won the award, once again foiling my theory that the film with the most amount of critical acclaim has the best chance of winning the award.

Just before the Oscars, I heard through the grapevine that my humble "Predict The Oscar" methodology was mentioned on The VFX Show podcast, show #68, from February 13, 2009. As an aside - thanks, guys, for the mention! When my wife heard about FXRant's mention on an actual, respected podcast, her immediate response was "Wow, somebody actually reads your blog? And then talked about it?!" Later, after thinking about it for a few moments, she said, "You know, you're doing this all wrong."

"Doing what all wrong?" I asked.

She said that I was looking for correlation between statistics and the ultimate winner of the Academy Award-- and getting upset when the correlation breaks between the data and reality. A far more interesting exercise would be to devise a formula to actually predict the winner, based on quantifiable, known criteria. We could look at several years' worth of data, craft a series of calculations based on certain values, and weight those values accordingly to skew the formula in the right direction.

She, like always, was correct.

After analyzing the data and looking for the trends in the winning films, we added several new criteria and weighted the criteria until we hit paydirt. Using nine carefully weighted scores, we devised a formula which was able to predict the visual effects Academy Award winner with a strong degree of accuracy.

How strong?

In 20 years of Academy Awards data, the formula is 100% accurate. It correctly predicted the winner of the visual effects Academy Award every single year. We're calling the formula The VFX Predictinator.

Stay tuned for Part 2 coming soon! It will be worthy of these exclamation points! Trust me!

And now... here's Part 2!