Wednesday, April 9, 2008

Peformance vs Expectation

What a roller coaster ride! There were highs and lows, "The Slump" followed by a strong resurgence, roaring comebacks against LSU, Syracuse (second game) and Clemson (NCAA round 1) and the referee robberies (NC State and the first Georgetown game). Just about everything this fan could handle. And about on par for a freshmen-oriented team? I decided to take a leaf from The Hoya Prospectus Blog and plot the scoring differential against Pomeroy's projected scoring differential to see how the team actually performed. I looked at the last 24 games of the season -- the most difficult portion of the schedule which included the entire Big East regular season, the last Big 5 game (versus St. Joseph's), the Big East Tournament and the 3 rounds of the Big East Tournament. After a 10-1 start (and about 9 weeks ranked in the Top 25), Villanova went 12-12 over those last 24 games. The scatter diagram below plots the performance versus projected outcome difference chronologically, from the first game in the series (the away game at DePaul which opened Villanova's Big East regular season) to the last game in the series (the Sweet Sixteen game versus Kansas in Detroit Michigan). Wins and losses, as shown in the legend are noted by black rectangles and red circles respectively. The five game losing streak ("The Slump") is highlighted in the red shaded rectangle. Games which fall on (or near) the X-Axis are those games which Pomeroy correctly predicted the scoring difference. The Seton Hall game for example, in which Pomeroy predicted a 2 point Villanova victory lies squarely on the X-Axis because the scoring difference was indeed 2 points (Villanova won 72-70 on a last minute Dwayne Anderson field goal). Proximity to the X-Axis indicates the closeness of the projection to the actual score. Note that I have (space limitations & an attempt limit the clutter within the plot area) only selectively labeled the games. The complete list of games plotted is given (in chronological order) below.

Most of the scatter diagram agrees with my (and I am sure many Nova fans) impressions of the season. The team was wildly inconsistent in the games before "The Slump" -- note how the scoring difference swings through the X-Axis on nearly every one of the 5 preceding games, first below and then above on the very next game -- DePaul, Pittsburgh, Cincinnati, then DePaul (second game). Syracuse (on the road no less...) was probably their best game to that point in the season. Followed immediately by what was clearly their worst game of the season (and arguably Villanova's worst game in 4 seasons) -- the road game at Rutgers. Ironically a similar swing occurred after the Wildcats eliminated the Orange from the Big East Tournament. The very next game was a significant downward swing (against the Hoyas). According to the scatter chart the four games that most exceeded expectations were the two Syracuse games, the road game versus St. John's and the home game, against the Mountaineers, which followed immediately on the St. John's game. While the Nova Nation may have been disappointed in the outcome of the first Georgetown game, the scatter chart suggests the team played a super game. The four "Most Underachieving" games included (chronologically) the first DePaul game, Rutgers (on the road), St. Joseph's (on the road) and the second (BET) Georgetown game. The data:

at DePaul-18
vs Pittsburgh7
at Cincinnati-8
vs DePaul (2)2
at Syracuse19
at Rutgers-22
vs Notre Dame-1
at Pittsburgh (2)-1
vs Syracuse (2)-9
at St. Joseph's (PA)-18
vs Seton Hall0
at Georgetown15
at St. John's19
vs West Virginia28
vs Connecticut5
vs Marquette-5
at Louisville-1
vs South Florida5
at Providence11
vs Syracuse (BET)17
vs Georgetown (BET)-11
vs Clemson (NCAA)13
vs Siena (NCAA)5
vs Kansas (NCAA)2

A few caveat emptors, and other thoughts...
1. Measuring performance against the expectation, as projected by the Pomeroy system, may be a bit misleading. The expectation that Pomeroy's system can predict the winner is a central assumption. But it may not, even accounting for the "usual" caveats (player injuries/suspensions, fatigue from travel, playing in unfamiliar venues, etc.), the system is still not 100% accurate at predicting winners/margins of victories, etc. I have not seen data lately on the accuracy of the possession-based ratings that Pomeroy implemented 2 seasons ago. His system bases his projections on a comparison of those ratings (adjusted for pace and has calculated the degree of confidence on Bill Jame's Pythagorean Winning Percentage formula).
2. The trendline, by the way, had a positive slope, suggesting the team improved, relative to expectations, as they played through this part of the season. The (linear) trendline starts at -3 (approximately) on the Y-Axis for the first DePaul game, and finishes at +8 (approximately) on the Y-Axis for the Kansas game, suggesting a significant improvement through the period.
3. The staff moved Dwayne Anderson into the starting lineup in the Seton Hall game, the only game in this series in which the team "played to expectations" (as a win, it is symbolized with a square that sits directly upon the x-axis). The team went 9-5 from that point to the end of the season, but more interestingly, went 10-3-1 with respect to performance versus expectations from that same point.
4. Anderson's numbers before he started versus as a starter are quite interesting. I have reproduced those which contrast greatly below. Note that as his time doubled, his points scored nearly tripled. While Dwayne's PPWS rose very slightly, this nevertheless came with a much higher volume of shooting. His eFG% actually declined from an excellent .590 down to a very good .571, the decline in his 2 point shooting offset by greater accuracy in his 3 point shooting and free throw shooting. While Average Rebounds is an admittedly crude metric here, the assist-to-turnover ratio really caught my eye. He was functioning within the flow of the offense, and distributing the ball very well for a frontcourt player.

Stony Brook - St. Joseph's (Before)
Seton Hall - Kansas (During)

5. Corey Stokes also had a run at the end of the year similar to Anderson's. He seemed to struggle through much of the season looking for his place on the team (and in the offense), but appears to have found it with a vengence as the season moved past the St. John's game. His stats, offered below, show that as the Wildcats approached that point in the season when freshmen are supposed to suffer the effects of the longer season, Stokes was getting started.

Stony Brook - Georgetown (Before)
St. John's - Kansas (During)24.312.11.2055.12.80.43:1