Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
13a2f5e
project proposal added
segerphilip Feb 24, 2016
9cfe54c
working on making a data frame
frackleton Feb 25, 2016
c8ae31f
attempt to load the dat file, needs pruning
segerphilip Feb 26, 2016
ec3d13e
ignoring checkins and dat file
segerphilip Feb 26, 2016
bcf4ffb
data added
segerphilip Mar 2, 2016
4834401
new csv files, able to load and clean data
segerphilip Mar 2, 2016
646b7e8
working on graphs
segerphilip Mar 3, 2016
7086da0
tracked life expectancy in Africa
frackleton Mar 3, 2016
de5647c
working on making a meaningful plot
segerphilip Mar 3, 2016
c6bdd41
more refinement, graph looks bad
segerphilip Mar 3, 2016
67e9610
added a meaningful graph
segerphilip Mar 3, 2016
510d154
mid project graphics
frackleton Mar 4, 2016
bc9943f
added comments to notebook
segerphilip Mar 4, 2016
d2efea2
mid project checkin file
frackleton Mar 4, 2016
f21e46b
added images and mid project checkin
segerphilip Mar 4, 2016
df7971f
Merge pull request #1 from segerphilip/pip
segerphilip Mar 4, 2016
f9dc90a
Merge pull request #2 from segerphilip/mack
segerphilip Mar 4, 2016
b99197a
fixed paragraphs
segerphilip Mar 4, 2016
e06a725
new graph that shows some trend
segerphilip Mar 4, 2016
e036203
plots for each individual region
segerphilip Mar 5, 2016
731aaf3
Merge pull request #3 from segerphilip/regions
segerphilip Mar 5, 2016
c18b0b3
looked into 1st iteration of trend tracking
frackleton Mar 7, 2016
3deb624
finished first visualizations of categories
frackleton Mar 8, 2016
4e04017
cleaned up graphs
frackleton Mar 8, 2016
3259e7e
cleaning data and saving dataframes
segerphilip Mar 8, 2016
77d9d6f
working on final cumulative notebook
frackleton Mar 9, 2016
0c3ff78
cleaned up dieAlive graphs and make functions of repetitive code
frackleton Mar 10, 2016
e1a876e
finished mack's contribution to the final notebook
frackleton Mar 10, 2016
8efc9db
attempting mapping, not currently working
segerphilip Mar 11, 2016
8bc019c
updated where I found mapping information
segerphilip Mar 11, 2016
2b8011b
Merge pull request #4 from segerphilip/maps
segerphilip Mar 11, 2016
a38ea00
Merge pull request #5 from segerphilip/mackTrends
segerphilip Mar 11, 2016
320e19e
organizational changes and rerun all notebooks
segerphilip Mar 11, 2016
5efc141
final reflection added
segerphilip Mar 11, 2016
ea772e4
updated README with links
segerphilip Mar 11, 2016
548f9dc
slight tweaks to final reflection
segerphilip Mar 11, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
nispuf14.dat
.ipynb_checkpoints/
*.DS_Store
468 changes: 468 additions & 0 deletions Exploring Trends for plotting.ipynb

Large diffs are not rendered by default.

705 changes: 705 additions & 0 deletions Final Overview of CTW Project.ipynb

Large diffs are not rendered by default.

17 changes: 17 additions & 0 deletions FinalReflection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# CTW Project Final Reflection


## Backstory
We as a team had many lofty ideas when we initially started this project, however things definitely did not work out as well as expected. A large pain point that we discovered about halfway through this project was that the initial dataset we were working on was difficult to properly clean and setup, as it required a SAS (proprietary data-science software for businesses) component that we were unable to properly utilize. As such, right before the mid-project checkin, we pivoted. We switched to a new dataset (the WHO data on mortality rates and such) and attempted to pickup right where we left off. As such, we probably did not accomplish all of the initial goals we set out for, but we felt that we were able to accomplish enough to educate others and explain a fairly large dataset to any individual.

## Assessment Evidence and Interpretation
Our main deliverable can be viewed in our repo (Jupyter notebook called Final Overview of CTW Project). This document contains the cleaning functions that we created, along with mapping functions and other graphing tools. We have 3 distinct types of graphs: bar graphs showing the number of people alive in an age group vs. the number of people dying in that same age group, life expectancies across time by age for individual regions, and the probability of dying specific to certain age brackets. We also attempted to develop a world map showing different statistics in relation to their country, however this functionality had many hiccups (the shapefile we originally needed for world data was unavailable, along with the new shapefile being corrupt after a certain number of entries). In the end, we were unable to complete a proper map, however our other graphs were fully functional and available in the notebook. These graphs show primarily age and region, in relation to the numeric values for living/dying/expectancies. This notebook provides the most full-picture idea of what we worked on, however a fourth and final graph type we used was a scatter plot consisting of life expectancies across time by age, in a .gif format for a different way to visualize aspects of the data. This dynamic visualization has more impact in grabbing an audience’s attention, along with helping fulfill our goal of different types of visualization and exploration into that space.

## Changing the World
In total, we feel our project gets partway to changing the world. If we were to continue, we would need to break the statistics down into countries or some breakdown more specific than “regions.” We realize this may mean we would need a more complete, or broader, dataset. A lot of the computing has already been done for the dataset we used, which means much of our work was done with post-processing data. We would need a broken down dataset because the Americas contain 1st and 3rd world countries that reasonably shouldn’t be grouped together, among other similar issues. If the data is better separated, we could see making similar visuals that show how much longer Europeans, North Americans, and Australians can expect to live compared to someone in Africa or SouthEast Asia. Changing the world is about finding the actual shocking differences (for example, how many years lost in life expectancy) in hopes of educating and motivating individuals to join initiatives.

## Learning Goals
### Mack:
I did learn things I wanted to know in this project. I gained comfort with matplotlib as a visualization tool and feel comfortable creating different kinds of visualizations, 3D and 2D. I think I was successful because I spent a majority of my time creating failed graphs that taught me how these tools were supposed to be used, how I was using them incorrectly, and what tools were correct for representing data the way I wanted. Unfortunately I deleted most of these early representations before I realized they were a sign of progress. At the end of this project, I was able to confidently clean up our code to be replicable, readable, and functional.
### Philip:
I feel that I did learn quite a bit, but it was less in the strain of visualization and more in the experiment/fail/experiment again space. Similar to Mack, many of the things I worked on did not fully pan out (looking specifically at the mapping functions, along with the foray into SAS and that failure), however I felt that any step, even in the wrong direction, was teaching me a bit more about the intricacies of matplotlib or python in the data science context. Along with that, I really enjoyed learning a lot about playing with a fairly unformatted dataset. The WHO had their own dataset standards, and when we got to making it readable for our implementation, we had to play around quite a bit to make everything work. Cleaning and formatting data was a new challenge, but it was up to us what we wanted to focus on, which made it more interesting to me in the end.
Loading