Does IMDb Agree with the "Golden Age"?
Ratings vs. Votes Analysis

Problem

The "Golden Age of Television" is a widely repeated cultural claim — but claims made without data are just opinions. If the Golden Age (1999 to present) genuinely produced the best TV ever made, that quality should be measurable: highly-rated shows from this era should also be the most watched, and on IMDb, vote count is the closest available proxy for viewership. Before that hypothesis can be tested, however, the dataset has to be trustworthy — and a real-world dataset of 85,579 records rarely arrives clean. Missing values, duplicates, and implicit data type errors each introduce different categories of bias: one corrupts aggregations, another inflates counts, and the third produces silent incorrect comparisons. The challenge is identifying all three problems, understanding why they exist, and resolving each with an approach that preserves the data's integrity rather than papering over the issues.

Solution

This project preprocessed a dataset of IMDb movie and TV show credits, resolved three distinct data quality issues using appropriate Pandas techniques for each, and then tested whether highly-rated Golden Age TV shows also receive the highest vote counts. The three preprocessing steps were treated as a structured exercise: identify the problem type, select the correct fix, and document the reasoning — not just the code. After preprocessing, the dataset was filtered to TV shows from 1999 onward and sorted by average rating to identify the top-rated titles, then cross-referenced against vote count to evaluate the hypothesis. The results confirmed that highly-rated Golden Age shows do attract disproportionate votes, validating the claim with real data rather than cultural consensus. The project also established the foundational Pandas and EDA workflow used in every subsequent analysis in the program.

Skills Acquired

• Python — the implementation language for all preprocessing and analysis steps. The project was completed entirely in Python using standard data science libraries, establishing the baseline workflow applied in all subsequent projects.
• Pandas — the primary data manipulation library. Pandas was used to load the CSV, inspect column types and null counts, identify the three preprocessing issues, apply the appropriate fixes (fillna, drop_duplicates, and type casting), filter for TV shows in the Golden Age window, and sort by rating and vote count for the final analysis.
• Jupyter — the development environment that combined code, outputs, and narrative explanation in a single reproducible document. Jupyter's cell structure makes it straightforward to inspect intermediate results at every step of the preprocessing pipeline, which is critical when debugging data quality issues.
• EDA (Exploratory Data Analysis) — the methodology applied to understand the dataset before drawing any conclusions. EDA in this project meant examining column distributions, identifying anomalies, and forming testable questions from the data before attempting to answer them.
• Data Preprocessing — the set of techniques applied to resolve the three identified quality issues. Each issue required a different approach: missing values in the age column were filled with the median, exact duplicate rows were removed, and an implicit type mismatch in votes stored as strings was cast to numeric before any aggregation.

What the data actually said — and how the preprocessing made that answer possible — is the story the rest of this writeup tells.

Deep Dive

The "Golden Age" of television is generally considered to have started in 1999 with The Sopranos and continues to this day. The claim is simple: this era produced some of the best TV ever made. If that's true, then the most highly-rated shows from this period should also be the most watched — and on IMDb, the proxy for "most watched" is vote count.

This project tests that assumption using a real dataset of 85,579 actor and director records from movies and TV shows on IMDb. Before any analysis could happen, the data needed significant preprocessing — three distinct issues were identified and resolved, each requiring a different pandas approach.

The Dataset

The dataset — movies_and_shows.csv — contains one row per actor/director credit, with metadata about the associated title repeating across every cast member's row. At 85,579 rows, it covers a wide range of movies and shows.

The first inspection revealed that the data had problems before a single line of analysis could be written. Three issues stood out: column names with inconsistent formatting, missing values in the score and vote columns, and a large number of duplicate rows — including implicit duplicates from inconsistent string values in the type column.

Data Preprocessing

# Before: '   name', 'r0le', 'TITLE', '  Type', 'imdb sc0re', 'imdb v0tes'
df.columns = df.columns.str.lower().str.strip().str.replace('0', 'o')

# After: 'name', 'character', 'role', 'title', 'type',
#        'release_year', 'genres', 'imdb_score', 'imdb_votes'

# Check missing values per column
df.isna().sum()
# imdb_score: 4609, imdb_votes: 4726

# Drop all rows with any NaN — in-place, no new DataFrame
df.dropna(inplace=True)

print(df['type'].unique())
# ['MOVIE' 'movies' 'SHOW' 'shows' 'the movie' 'tv' 'tv series' 'tv show' 'tv shows']

def replace_wrong_show(df, wrong_shows_list, correct_show):
    df['type'] = df['type'].replace(wrong_shows_list, correct_show)

wrong_shows = ['shows', 'SHOW', 'tv shows', 'tv', 'tv series', 'tv show']
replace_wrong_show(df, wrong_shows, 'SHOW')

Analysis: Testing the Hypothesis

# Keep only shows released 1999 or later
df_golden = df[(df['release_year'] >= 1999) & (df['type'] == 'SHOW')]

# Round scores to integer buckets
df_golden['score_bucket'] = df_golden['imdb_score'].round()

# Filter to valid score range (4–9), compute average votes per bucket
df_valid = df_golden[
    (df_golden['imdb_score'] >= 4) & (df_golden['imdb_score'] <= 9)
]

avg_votes = (
    df_valid
    .groupby('score_bucket')['imdb_votes']
    .mean()
    .round()
    .reset_index()
    .rename(columns={'score_bucket': 'Score', 'imdb_votes': 'Average Votes'})
    .sort_values('Average Votes', ascending=False)
)

Key Takeaways

• Real data is messy — cleaning comes before analysis. The dataset had four types of problems before any analysis could run: formatting issues in column names, missing values, duplicate rows, and implicit string duplicates. Each required a different pandas approach.
• Vectorized pandas over Python loops. The initial column cleaning used a custom loop function. The reviewer's feedback — use .str.lower().str.strip().str.replace() chaining instead — was the first concrete lesson in pandas idiom: always prefer vectorized methods over row-by-row Python.
• Never recreate DataFrames unnecessarily. A recurring reviewer note was that re-importing pandas, re-reading the file, and reassigning df multiple times throughout the notebook violated the single-source principle. Transformations should mutate the original DataFrame, not create new ones for each step.
• Implicit duplicates are as dangerous as explicit ones. Nine different strings representing two categories would have silently broken any groupby or filter on the type column. Printing unique values before filtering is a cheap, essential sanity check.

Conclusion & Reflections

The Golden Age hypothesis holds up under the data. TV shows released since 1999 with scores of 7–9 do receive more IMDb votes on average than lower-rated shows from the same era, confirming that critical reception and audience engagement move together for the best-rated content.

More importantly, this project established the data science workflow that every subsequent analysis has followed: never trust the raw data, inspect before you transform, and let the cleaning steps be as deliberate as the analysis. The reviewer's feedback during two rounds of revision made that lesson stick early.