[We'd like to thank Paul Jakus for this analysis of recent Phish.net ratings. Coincidentally, we've been analyzing ratings with him for a future blog series digging deeper into how Phish fans rate shows. Stay tuned for more on ratings soon! —Ed.]

At 3:42 p.m. on the afternoon of January 3, 2024 the ratings function of Phish.Net was disabled due to unusual patterns in ratings behavior. Here we’ll explain those patterns, but first let’s establish what a “normal” Holiday Run ratings pattern looks like.

For comparison, let’s look at ratings submitted between 1:00 a.m. January 1, 2023 through 3:42 p.m. January 3, 2023 (a time period that will match that of the 2023/24 NYE Run.) Some 1,004 ratings were submitted over nearly 63 hours, for 116 different shows. Of these ratings, 838 (84%) were for the four holiday shows, leaving 166 ratings to be spread across the remaining 112 non-holiday shows. The most new ratings any non-holiday show received was six.

So, what happened after the 2023/24 Run? Read on for more.

The first rating for 12/31/23 was at 1:12 a.m. on January 1, 2024. A total of 3,779 ratings were submitted for 442 different shows until ratings were suspended at 3:42 p.m. on January 3. Only 2,103 of these ratings (56%) were for the 2023/2024 NYE Run, leaving 1,676 ratings spread over the remaining 438 shows.

There is no doubt that something odd happened.

Now let’s take a closer look at the ratings for the Gamehendge show (Figure 1). Nearly 1,800 ratings were posted, with a very high proportion of ‘5’ ratings (87%), neither of which would be unexpected for an instantly classic performance. Almost 7% of ratings were a ‘1’, though, which was a bit higher than the norm for all modern era shows (5.8%). Peculiar, yes, but nothing immediately identified as highly unusual.

But if nothing was obviously wrong with ratings for the Gamehendge show, why suspend the ratings function?

Recall the same time period after the 2022/23 run: we saw lots of ratings for the Holiday shows and relatively little activity on non-holiday shows. Things were different this year.

In 2024, newly submitted ratings of notably historic shows were pervasive, with more than 30 older shows receiving at least 10 new ratings in the first three days of the New Year. For example, here are all the older shows with 30 or more new ratings (and the ratings distribution) during the January 1-3, 2024 period.

Table 1: Older Shows and Newly Submitted Ratings

Before January 1, 2024*

After January 1, 2024*

Date

Location

# of Ratings

Average

# of New Ratings

Average of New Ratings

12/31/1999

Big Cypress, FL

1,325

4.767

259

4.232

12/30/1997

MSG

770

4.700

69

3.783

11/22/1997

Hampton, VA

854

4.678

55

3.873

4/3/1998

Nassau, NY

832

4.669

49

3.932

8/2/2003

Limestone IT

490

4.680

39

3.769

12/31/1995

MSG

942

4.634

37

4.297

8/17/1997

Limestone Went

650

4.652

33

4.030

8/22/2015

Watkins Glen Magnaball

1,648

4.649

30

3.233

*Before/After 1:12 a.m. EST, through 3:42 p.m. January 3

It’s clear that Big Cypress was the primary historic show affected, but it was definitely not the only one. Every show in the table above was well-known, highly rated, and listed in The Phish Companion as a “Top 100” performance.

Did all of these performances really get perceived as “worse” starting on January 1?

Let’s take a closer look at ratings for Big Cypress, 12/31/1999. The 259 post-January 1, 2024 ratings happened within about 51 hours. For comparison, the previous 259 ratings for this show were submitted over 1,336 days. The number of new ratings for this show was highly unusual.

Table 2 depicts the number and percentage of each rating for NYE Big Cypress, before and after January 1.

Table 2: Big Cypress (12/31/1999), Before and After January 1, 2024

Rating

Before January 1, 2024*

After January 1, 2024*

1

51 (3.85%)

42 (16.22%)

2

7 (0.53%)

3 (1.16%)

3

14 (1.06%)

4 (1.54%)

4

56 (4.23%)

14 (5.41%)

5

1,197 (90.34%)

196 (75.34%)

*Before/After 1:12 a.m. EST

First, the 12/31/99 ratings distribution looks remarkably similar to that of the 2023 NYE Gamehendge show, except for fewer “1s’. Second, the ratings distribution for the post-January 1 period is markedly different. Is it reasonable to expect 259 ratings to pour in for a show that’s 24 years old, in a matter of hours, and with such different ratings?

A nice feature of the Phish.Net ratings function is that it generates debate among fans, but its primary purpose is to help guide fans, both new and old, through a large body of recorded music. Given this primary goal, and given what appears to be manipulation of numerous highly-regarded shows, it was decided to suspend the ratings function.

The ratings function will return, but Phish.Net administrators are currently mulling possible changes in how ratings are calculated and presented in the future. To begin with, though, it’s necessary to answer the thousand dollar question: whether the ratings submitted this week reflect an honest re-appraisal of historic shows, or if they were submissions designed to affect show rankings.

If you liked this blog post, one way you could "like" it is to make a donation to The Mockingbird Foundation, the sponsor of Phish.net. Support music education for children, and you just might change the world.