Big Data mini-projects using Apache Spark
- Extracted top10 relatively popular tokens from 100 GB twitter raw JSON data within 80s using Spark RDD
- Applied PCA to analyze 38,436 pieces 1-year-length weather station records and accomplish reconstruction
- Implemented Linear Regression, K-means++ to predict weather data coefficients using the top eigenvectors
- Built Gradient Boosted Trees and Random Forest model on RDD to train a Tree Cover Type classifier