Data Analysis Tutorial: U.S. Minimum Wage by State (1968-2017)

Introduction

• Focus on data analysis and data science using Python and pandas.
• Dataset: U.S. Minimum Wage by State (1968-2017).
• Key concepts: Minimum wage values, CPI (cost of living adjustments), data cleaning, and correlation analysis.

Dataset Overview

• Contains high and low minimum wage values by state.
• Values adjusted to 2018 terms using CPI.
• Objective: Work with the lowest minimum wage data to analyze trends and correlations.

Initial Steps

1. Import Libraries
   • Import pandas as pd.
2. Read Data
   • Load dataset using pd.read_csv().
   • Dataset path: datasets/minimum_wage_data.csv.
   • Encountered a Unicode decode error (likely encoding issue).
3. Fix Encoding
   • Try reading data with latin1 encoding if utf-8 fails.
   • Save corrected data to avoid future issues: df.to_csv('dataset_min_wage.csv', encoding='utf-8')

Data Grouping and Analysis

Grouping by State

• Use groupby to analyze data by state: gb = df.groupby('state')
• Retrieve specific state's data (e.g., Alabama).

Iterating Over Groups

• Can iterate over groups for concise data handling: for name, group in df.groupby('state'):
• Set year as index for better organization.
• Rename columns for clarity and organization.

Descriptive Statistics

• Use df.describe() to overview dataset statistics:
  • Count, mean, standard deviation, minimum, percentiles, and maximum values.

Correlation Analysis

• Use df.corr() for correlation matrix:
  • Identify relationships between state minimum wages.

Handling Missing Data

• Debug missing data issues (NaN values) in the dataset:
  • Check which states have missing values and consider their relevance.
• Replace zeros with NaN for states with no data. df.replace(0, np.nan, inplace=True)
• Drop columns with NaN values: df.dropna(axis=1, inplace=True)

Conclusion and Next Steps

• Resolve data issues to prepare for correlation visualization in future tutorials.
• Encourage engagement from the audience for further questions and comments.

Ending Notes

• Thank the audience for their support.
• Anticipate the next tutorial focusing on visualization of correlation data.