before we start this portion of the lesson:

check if you have pip installed since we are going to be installing some libraries today!!!!!! if you arnt sure if you have pip, check it by running this command:

pip

if your terminal says "command not found" or something else on linux, run this:

python3 -m ensurepip --default-pip

Overview:

Pandas is a powerful tool in Python that is used for data analysis and manipulation. In this lesson, we will explore how to use Pandas to work with datasets, analyze them, and visualize the results.

Learning Objectives:

By the end of this lesson, students should be able to:

  • Understand what Pandas is and why it is useful for data analysis
  • Load data into Pandas and create tables to store it
  • Use different functions in Pandas to manipulate data, such as filtering, sorting, and grouping
  • Visualize data using graphs and charts

Question

Who here has used numpy????

(should be all odf you because all of you have used it in this class before. )

what is pandas?

image

no not this

this:

  • Pandas is a Python library used for data analysis and manipulation.
  • it can handle different types of data, including CSV files and databases.
  • it also allows you to create tables to store and work with your data.
  • it has functions for filtering, sorting, and grouping data to make it easier to work with.
  • it also has tools for visualizing data with graphs and charts.
  • it is widely used in the industry for data analysis and is a valuable skill to learn.
  • companies that use Pandas include JPMorgan Chase, Google, NASA, the New York Times, and many others.

Question #2 & 3:

  • which companies use pandas?
  • what is pandas?

but why is pandas useful?

  • it can provides tools for handling and manipulating tabular data, which is a common format for storing and analyzing data.
  • it can handle different types of data, including CSV files and databases.
  • it allows you to perform tasks such as filtering, sorting, and grouping data, making it easier to analyze and work with.
  • it has functions for handling missing data and can fill in or remove missing values, which is important for accurate data analysis.
  • it also has tools for creating visualizations such as graphs and charts, making it easier to communicate insights from the data.
  • it is fast and efficient, even for large datasets, which is important for time-critical data analysis.
  • it is widely used in the industry and has a large community of users and developers, making it easy to find support and resources.

Question #4:

  • why is pandas useful?

how do i flipping use it? its so hard, my puny brain cant understand it

it is actually really simple

here is numpy doing simple math:

import pandas as pd

df = pd.read_csv('yourcsvfileidcjustpickoneidiot.csv')

print(df.head())

print("Average age:", df['Age'].mean())

females = df[df['Gender'] == 'Female']
print(females)

sorted_data = df.sort_values(by='Salary', ascending=False)
print(sorted_data)

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[2], line 1
----> 1 import pandas as pd
      3 # Read the CSV file into a Pandas DataFrame
      4 df = pd.read_csv('example.csv')

ModuleNotFoundError: No module named 'pandas'

uh oh!!! no pandas 😢

if see this error, enter these into your terminal:

pip install wheel
pip install pandas

on stack overflow, it said pandas is disturbed through pip as a wheel. so you need that too.

link to full forum if curious: https://stackoverflow.com/questions/33481974/importerror-no-module-named-pandas

ps: do this for this to work on ur laptop:

wget https://raw.githubusercontent.com/KKcbal/amongus/master/_notebooks/files/example.csv

example code on how to load a csv into a chart 

import pandas as pd

# read the CSV file
df = pd.read_csv('example.csv')

# print the first five rows
print(df.head())

# define a function to assign each age to an age group
def assign_age_group(age):
    if age < 30:
        return '<30'
    elif age < 40:
        return '30-40'
    elif age < 50:
        return '40-50'
    else:
        return '>50'

# apply the function to the Age column to create a new column with age groups
df['Age Group'] = df['Age'].apply(assign_age_group)

# group by age group and count the number of people in each group
age_counts = df.groupby('Age Group')['Name'].count()

# print the age group counts
print(age_counts)

           Name  Age  Gender Occupation
0      John Doe   32    Male   Engineer
1    Jane Smith   27  Female    Teacher
2  Mike Johnson   45    Male    Manager
3      Sara Lee   38  Female     Doctor
4     David Kim   23    Male    Student
Age Group
30-40    7
40-50    4
<30      7
Name: Name, dtype: int64

how to manipulate the data in pandas. 

import pandas as pd

# load the csv file
df = pd.read_csv('example.csv')

# print the first five rows
print(df.head())

# filter the data to include only people aged 30 or older
df_filtered = df[df['Age'] >= 30]

# sort the data by age in descending order
df_sorted = df.sort_values('Age', ascending=False)

# group the data by gender and calculate the mean age for each group
age_by_gender = df.groupby('Gender')['Age'].mean()

# print the filtered data
print(df_filtered)

# print the sorted data
print(df_sorted)

# print the mean age by gender
print(age_by_gender)

           Name  Age  Gender Occupation
0      John Doe   32    Male   Engineer
1    Jane Smith   27  Female    Teacher
2  Mike Johnson   45    Male    Manager
3      Sara Lee   38  Female     Doctor
4     David Kim   23    Male    Student
                Name  Age  Gender               Occupation
0           John Doe   32    Male                 Engineer
2       Mike Johnson   45    Male                  Manager
3           Sara Lee   38  Female                   Doctor
6       Robert Green   41    Male                Architect
7        Emily Davis   35  Female        Marketing Manager
8   Carlos Hernandez   47    Male             Entrepreneur
10         Kevin Lee   31    Male               Accountant
12     Jacob Johnson   34    Male                   Lawyer
13   Maria Rodriguez   39  Female               Consultant
15    Victoria Brown   42  Female  Human Resources Manager
17        Sophie Lee   30  Female          Project Manager
                Name  Age  Gender               Occupation
8   Carlos Hernandez   47    Male             Entrepreneur
2       Mike Johnson   45    Male                  Manager
15    Victoria Brown   42  Female  Human Resources Manager
6       Robert Green   41    Male                Architect
13   Maria Rodriguez   39  Female               Consultant
3           Sara Lee   38  Female                   Doctor
7        Emily Davis   35  Female        Marketing Manager
12     Jacob Johnson   34    Male                   Lawyer
0           John Doe   32    Male                 Engineer
10         Kevin Lee   31    Male               Accountant
17        Sophie Lee   30  Female          Project Manager
5        Anna Garcia   29  Female       Software Developer
14       Mark Taylor   28    Male             Web Designer
1         Jane Smith   27  Female                  Teacher
11      Rachel Baker   26  Female               Journalist
9     Melissa Nguyen   25  Female         Graphic Designer
16        Ethan Chen   24    Male       Research Assistant
4          David Kim   23    Male                  Student
Gender
Female    32.333333
Male      33.888889
Name: Age, dtype: float64

how do i put it into a chart 😩

here is how:

import pandas as pd
import matplotlib.pyplot as plt

# read the CSV file
df = pd.read_csv('example.csv')

# create a bar chart of the number of people in each age group
age_groups = ['<30', '30-40', '40-50', '>50']
age_counts = pd.cut(df['Age'], bins=[0, 30, 40, 50, df['Age'].max()], labels=age_groups, include_lowest=True).value_counts()
plt.bar(age_counts.index, age_counts.values)
plt.title('Number of people in each age group')
plt.xlabel('Age group')
plt.ylabel('Number of people')
plt.show()

# create a pie chart of the gender distribution
gender_counts = df['Gender'].value_counts()
plt.pie(gender_counts.values, labels=gender_counts.index, autopct='%1.1f%%')
plt.title('Gender distribution')
plt.show()

# create a scatter plot of age vs. income
plt.scatter(df['Age'], df['Income'])
plt.title('Age vs. Income')
plt.xlabel('Age')
plt.ylabel('Income')
plt.show()

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[55], line 2
      1 import pandas as pd
----> 2 import matplotlib.pyplot as plt
      4 # read the CSV file
      5 df = pd.read_csv('example.csv')

ModuleNotFoundError: No module named 'matplotlib'

uh oh!!!! another error!??!!??!?! install this library:

pip install matplotlib

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np

# read the CSV file
df = pd.read_csv('example.csv')

# define age groups
age_groups = ['<30', '30-40', '40-50', '>50']

# create a new column with the age group for each person
df['Age Group'] = pd.cut(df['Age'], bins=[0, 30, 40, 50, np.inf], labels=age_groups, include_lowest=True)

# group by age group and count the number of people in each group
age_counts = df.groupby('Age Group')['Name'].count()

# create a bar chart of the age counts
age_counts.plot(kind='bar')

# set the title and axis labels
plt.title('Number of People in Each Age Group')
plt.xlabel('Age Group')
plt.ylabel('Number of People')

# show the chart
plt.show()

magic!!!!!!

Hacks

  1. make your own data using your brian, google or chatgpt, should look different than mine.
  2. modify my code or write your own
  3. output your data other than a bar graph.
  4. answer the questions below, the more explained the better.

Questions

  1. What are the two primary data structures in pandas and how do they differ?
  2. How do you read a CSV file into a pandas DataFrame?
  3. How do you select a single column from a pandas DataFrame?
  4. How do you filter rows in a pandas DataFrame based on a condition?
  5. How do you group rows in a pandas DataFrame by a particular column?
  6. How do you aggregate data in a pandas DataFrame using functions like sum and mean?
  7. How do you handle missing values in a pandas DataFrame?
  8. How do you merge two pandas DataFrames together?
  9. How do you export a pandas DataFrame to a CSV file?
  10. What is the difference between a Series and a DataFrame in Pandas?

note

all hacks due monday 3:45, the more earlier you get them in the higher score you will get. if you miss the due date, you will get a 0. there will be no tolerance.

wdfasdf

2.9 if you finish pandas hacks by tuesday