CO2 vs. Temperature Exercise (Databricks)

This repository contains exercises in Databricks that ingests Global Temperature and Global Temperature By Country data from Kaggle and CO2 Emissions data from OWID and transforms it. The goal of this exercise is to teach some basics about data wrangling and Spark with respect to real world questions.

• Which countries are worse-hit (higher temperature anomalies)?
• Which countries are the biggest emitters?
• What are some attempts of ranking “biggest polluters” in a sensible way?

Data Sources

In order to answer some of the questions of the exercise, we picked open-source data from Open World in Data (OWID) and Kaggle.

The specific datasets:

• CO2 Emissions (2020).csv (OWID)
• GlobalLandTemperaturesByCountry (Kaggle)
• GlobalTemperatures.csv (Kaggle)

Data Sources (Modified!)

Since the point of this exercise is to learn how to work with data and the datasets from OWID and Kaggle are both too clean and curated, a set of dirtied data is provided.

They can be found at:

• EmissionsByCountry.csv
• GlobalTemperatures.csv
• TemperaturesByCountry.csv

Prerequisites

• Basic knowledge of Python
• Basic knowledge of Spark
• Databricks Community Edition (free) account

Create a Databricks Community Account

1. Navigate the Databricks Community Login Page
2. Click Signup

3. Fill in your details, click "Get Started For Free"

   

4. SCROLL TO THE BOTTOM to create a Community Account

   

Data Ingestion

1. Clone this repo if you haven't already done so
2. Open Data Ingestion CO2 vs Temperature.py in Databricks Community Edition
3. Follow instructions, move on to following exercises once all tests pass.
4. Solutions can be found here.

Data Transformation

1. Clone this repo if you haven't already done so
2. Open Data Transformation CO2 vs Temperature.py in Databricks Community Edition
3. Follow instructions, move on to following exercises once all tests pass.
4. Solutions can be found here.