A few more notes on lists¶
copying¶
- if L is a list, then setting S=L creates a new NAME (S) for L, but not a new object
- If you apply an operation to L, then you get a new element.
In [52]:
## methods vs functions
- methods act on a list in place
- append ```L.append(x)```
- del ```del L[3]```
- sort ```L.sort()```
- functions create a new list, leaving the original one alone
- S = L + [x]
- Y = sorted(L)
In [ ]:
In [32]:
primes = [2,3,5,7,11,13,17,19]
new = primes.append(23)
print('returned value = ',new,'original = ',primes)
In [31]:
primes = [2,3,5,7,11,13,17,19]
new = primes + [23]
print('returned value = ',new,'original = ',primes)
In [4]:
del primes[3]
In [5]:
primes
Out[5]:
In [41]:
primes = [2,3,5,7,11,13,17,19]
new = primes.sort(reverse=True)
print('returned value = ',new,'original = ',primes)
In [38]:
primes = [2,3,5,7,11,13,17,19]
new= sorted(primes,reverse=True)
print('returned value = ',new,'original = ',primes)
copying¶
- if L is a list, then setting S=L creates a new NAME (S) for L, but not a new object
- If you apply an operation to L, then you get a new element.
In [53]:
primes = [2,3,5,7,11,13,17,19]
S = primes
primes.append(23)
print(S)
taking a slice is one way to force a copy
In [55]:
primes = [2,3,5,7,11,13,17,19]
S = primes[:]
primes.append(23)
print(S)
so is using the .copy() method
In [59]:
primes = [2,3,5,7,11,13,17,19]
S = primes.copy()
primes.append(23)
print(S,primes)
lists and strings¶
- list
- join and split
In [11]:
','.join(['a','b','c'])
Out[11]:
In [12]:
list('ABCDEF')
Out[12]:
In [15]:
commas = ','.join(list('ABCDEF'))
print(commas)
In [16]:
commas.split(',')
Out[16]:
In [19]:
data = ".5,.2,1.8,17,21"
In [23]:
data.split(',')
Out[23]:
In [25]:
print('\t'.join(data.split(',')))
Iteration¶
- iteration means doing an operation repeatedly with varying parameters
- the
forkeyword is the fundamental iterator in python - INDENTATION MATTERS IN PYTHON SYNTAX
- you can indent using tabs or spaces, but blocks must line up
- notice the colon at the end of the first line
In [62]:
for x in [2,3,5,7,11,13,17,19]:
print(x,'is a prime')
print(x**2,'is its square')
In [64]:
for x in [2,3,5,7,11,13,17,19]:
print(x,'is a prime')
print(x**2,'is its square')
See Tabs vs Spaces
iterating over a range, or over a list¶
- range(100)
- range(10,20)
- range(10,20,30)
- np.arange(.1,1.1,.05)
In [65]:
for x in range(10):
print(x)
In [66]:
L=[]
for x in range(10):
L.append(x)
print(L)
In [68]:
for x in range(10,0,-1):
print(x)
In [69]:
import numpy as np
In [70]:
for x in np.arange(1,5,.1):
print(x,np.sin(x))
accumulating¶
In [74]:
sum = 0
for x in range(20):
sum = sum + x
print(sum)
In [79]:
sentence = 'Now is the time for all good me to come to the aid of their party'
words = sentence.split()
sum = 0
for x in words:
print(x,len(x))
sum = sum + len(x)
print('Total length is',sum)
In [86]:
# iterating over files
import pandas as pd
for file in ['../data/gapminder_gdp_africa.csv','../data/gapminder_gdp_asia.csv']:
data = pd.read_csv(file,index_col='country')
print(file, data['gdpPercap_1957'].idxmin())
reading a directory¶
glob gives access to UNIX wild cards
In [87]:
from glob import glob
In [105]:
for file in glob('../data/*gdp*.csv'):
data = pd.read_csv(file,index_col='country')
print(file, data['gdpPercap_1957'].idxmin())
In [106]:
for file in glob('../data/*gdp*.csv'):
data = pd.read_csv(file,index_col='country')
continent = file.split('_')[2].split('.')[0].upper()
print(continent, '\nPoorest in 1957:',data['gdpPercap_1957'].idxmin(),'\nRichest in 1957',data['gdpPercap_1957'].idxmax(),'\n\n')
nested loops¶
In [107]:
for fruit in ['apple','pear','grape']:
for color in ['green','orange','yellow']:
print('I wish I had a ',color,fruit)
putting some stuff together:¶
- string operations to extract the continent from the file name
- making a column nameIn [104]:
for file in glob('../data/*gdp*.csv'):
for year in ['1957','2002']:
data = pd.read_csv(file,index_col='country')
continent = file.split('_')[2].split('.')[0].upper()
key = 'gdpPercap_'+year
print(continent, '\nPoorest in '+year+':',data[key].idxmin(),'\nRichest in '+year+':',data[key].idxmax(),'\n\n')
In [111]:
import glob
import pandas as pd
import matplotlib.pyplot as plt
fig, ax = plt.subplots(1,1)
for filename in glob.glob('../data/gapminder_gdp*.csv'):
dataframe = pd.read_csv(filename)
# extract <region> from the filename, expected to be in the format 'data/gapminder_gdp_<region>.csv'.
# we will split the string using the split method and `_` as our separator,
# retrieve the last string in the list that split returns (`<region>.csv`),
# and then remove the `.csv` extension from that string.
region = filename.split('_')[-1][:-4]
dataframe.mean().plot(ax=ax, label=region)
plt.legend()
plt.show()
Functions¶
We have seen many examples of built-in functions (print, math functions, read_csv, etc are all functions). You can make your own function: takes certain inputs (arguments) and returns a value.
In [113]:
def print_date(year, month, day):
joined = str(year)+'/'+str(month)+'/'+str(day)
print(joined)
In [114]:
print_date(2007,11,31)
You can return the value for use in other computations¶
In [116]:
def format_date(year,month,day):
joined = str(year)+'/'+str(month)+'/'+str(day)
return joined
In [117]:
format_date(2007,12,10)
Out[117]:
In [118]:
dates = []
for year in range(2007,2010):
for month in range(5, 8):
for day in range(12,20):
dates.append(format_date(year,month,day))
In [119]:
dates
Out[119]:
functions and data analysis¶
Recall that we worked with the pandas dataframe gapminder_data.csv.
In [130]:
import matplotlib.pyplot as plt
plt.style.use('ggplot')
In [131]:
data = pd.read_csv('../data/gapminder_data.csv',index_col='country')
In [132]:
percapita = pd.pivot_table(data,index='country',columns='year',values='gdpPercap')
In [134]:
percapita.loc['Afghanistan'].plot(title='Afghanistan GDP Per capita over Time')
Out[134]:
In [151]:
def country_gdp(country):
title = country + ' GDP Per Capita over Time'
percapita.loc[country].plot(figsize=(8,8),legend=True,title='GDP Per Capita')
In [152]:
country_gdp("United States")
country_gdp("Germany")
country_gdp("Switzerland")
In [153]:
for country in ['United States','China','Germany','Australia','Brazil']:
country_gdp(country)
A random walk¶
In [154]:
import numpy.random as rnd
In [155]:
x = rnd.choice([-1,1])
In [156]:
def random_walk(N):
spot = 0
L = []
for i in range(N):
spot = spot + rnd.choice([-1,1])
L.append(spot)
return L
In [162]:
end_spots = []
for i in range(200):
walk = random_walk(100)
plt.plot(range(100),walk)
end_spots.append(walk[-1])
In [166]:
s=plt.hist(end_spots,bins=10)
Variable scope¶
variables inside functions are "local" to the function and changes to them don't last after the function ends
- the scope of a variable is the region where it is defined -- basically, in a function block and anything inside that.
In [174]:
def lister(x):
L = [x]*10
v = 47
print(L)
In [173]:
print(L)
lister(4)
print(L)
There are subtleties to variable scope. If a variable is NOT NAMED as an argument, then it is assumed to come from "outside" the function.
In [186]:
L = ['a','b','c']
def j(x):
L.append(x)
In [187]:
j('u')
In [188]:
L
Out[188]:
And structures (like lists, and arrays) CAN be modified inside a function EVEN IF they are given as an argument.
In [193]:
L = ['a','b','c']
def j(x,L):
L.append(x)
j('x',L)
print(L)
The full set of rules is a bit complicated and we won't go into all the details.
Conditionals¶
In [194]:
def my_abs(x):
if x<0:
return -x
else:
return x
In [195]:
my_abs(-1)
Out[195]:
This function returns a "tuple" which is a pair of lists. You can get the two lists with subscripts or with the A, B = construction
In [196]:
def split_threshold(threshold,L):
Low = []
High = []
for item in L:
if item<threshold:
Low.append(item)
else:
High.append(item)
return Low, High
In [203]:
T = split_threshold(0,[-1,-5,2,-13,11,100])
print(T)
In [204]:
print(T[0])
In [205]:
print(T[1])
This is the syntax for unpacking a tuple
In [198]:
Low, High = split_threshold(0,[-1,-5,2,-13,11,100])
In [206]:
Low
Out[206]:
In [207]:
High
Out[207]:
Python admits and and or operators
In [209]:
data['pop'].head()
Out[209]:
In [217]:
for x in 'Jeremy Teitelbaum':
if (x>='r' and x<='u'):
print(x)
applying a function in pandas¶
In [223]:
population_size(300000000)
Out[223]:
In [227]:
data['pop_class']=data['pop'].apply(population_size)
In [242]:
data.head()
Out[242]:
In [241]:
data[(data['year']==2002)].groupby('pop_class')['lifeExp'].mean().plot(kind='bar')
Out[241]:
In [243]:
def life_exp_by_class_and_year(year):
data[(data['year']==2002)].groupby('pop_class')['lifeExp'].mean().plot(kind='bar')
In [245]:
life_exp_by_class_and_year(1957)
In [246]:
data[(data['year']==2002) & (data['pop_class'] == 'large')]
Out[246]:
docstrings¶
In [254]:
def threshold(x,L):
"""Returns (Low, High) where Low is the list of elements in L less than x,
and High is a list of those greater than or equal to x"""
Low, High = [], []
for item in L:
if item<x:
Low.append(item)
else:
High.append(item)
return Low, High
In [255]:
?threshold
default arguments¶
In [256]:
def threshold(L,x=0):
"""Returns (Low, High) where Low is the list of elements in L less than x,
and High is a list of those greater than or equal to x. x defaults to zero."""
Low, High = [], []
for item in L:
if item<x:
Low.append(item)
else:
High.append(item)
return Low, High
In [257]:
threshold([1,-3,2,5])
Out[257]:
In [258]:
?threshold
In [259]:
?print
In [ ]: