Skip to content

Latest commit

 

History

History
903 lines (705 loc) · 36.5 KB

File metadata and controls

903 lines (705 loc) · 36.5 KB
╔═══════════════════════════════════════════════════════════════════════════╗
║                                                                           ║
║     ███╗   ██╗██╗     ██████╗     ██████╗  ██████╗ ██████╗ ████████╗    ║
║     ████╗  ██║██║     ██╔══██╗    ██╔══██╗██╔═══██╗██╔══██╗╚══██╔══╝    ║
║     ██╔██╗ ██║██║     ██████╔╝    ██████╔╝██║   ██║██████╔╝   ██║       ║
║     ██║╚██╗██║██║     ██╔═══╝     ██╔═══╝ ██║   ██║██╔══██╗   ██║       ║
║     ██║ ╚████║███████╗██║         ██║     ╚██████╔╝██║  ██║   ██║       ║
║     ╚═╝  ╚═══╝╚══════╝╚═╝         ╚═╝      ╚═════╝ ╚═╝  ╚═╝   ╚═╝       ║
║                                                                           ║
╚═══════════════════════════════════════════════════════════════════════════╝

    Production-Ready Natural Language Processing & Machine Learning Portfolio

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Python TensorFlow NLTK Keras scikit-learn


Advanced NLP implementations spanning text analytics, machine learning classifiers, sequence modeling, and deep learning for named entity recognition

View ProjectsSkillsResults


📊 Repository Overview

Technical Scope

This repository demonstrates end-to-end machine learning and NLP expertise through four comprehensive assignments implementing algorithms from mathematical foundations.

Core Focus Areas:

nlp_pipeline = {
    "text_processing": ["Tokenization", "Stemming", "Lemmatization"],
    "ml_algorithms": ["Naive Bayes", "Logistic Regression"],
    "sequence_modeling": ["N-grams", "HMM", "CRF"],
    "deep_learning": ["LSTM", "Word2Vec", "NER"]
}

Key Achievements

Projects Completed8
Algorithms Implemented20+
Lines of Code6,500+
Datasets Processed15K+ samples
Model Accuracy (Best)95.2%
Technologies Mastered15+

🎯 Portfolio Projects

Assignment 7: NLP Toolkit - Chatbot, Slot Filling & Neural Translation

┌─ THREE COMPLETE NLP SYSTEMS ─────────────────────────────────────────────┐
│                                                                           │
│  ▸ Corpus-Based Chatbot (TF-IDF Retrieval)                              │
│    • Custom TF-IDF implementation from scratch                           │
│    • NPS Chat corpus (~10K messages)                                     │
│    • Cosine similarity-based response matching                           │
│    • Intelligent filtering (removes questions, short responses)          │
│    • Evaluation: Engagingness 3/5, Making Sense 3/4, Fluency 4.5/5     │
│                                                                           │
│  ▸ LSTM Slot Filling (ATIS Dataset)                                     │
│    • Bidirectional LSTM architecture: Embedding → BiLSTM(128) → Dense   │
│    • ATIS travel dataset: 4.4K train, 900 test sentences               │
│    • 127 unique slot labels (locations, dates, airlines, etc.)          │
│    • Performance: Precision 0.95, Recall 0.94, F1-Score 0.95            │
│    • TimeDistributed output layer for sequence labeling                  │
│                                                                           │
│  ▸ Neural Machine Translation (German → English)                         │
│    • Seq2Seq architecture with attention mechanism                       │
│    • WMT14 dataset (de-en configuration)                                 │
│    • Encoder: Embedding → LSTM with context vectors                      │
│    • Decoder: LSTM → Attention → Dense → Softmax                         │
│    • BLEU Score: 0.18 (greedy decoding)                                 │
│    • 10K vocab for both German and English                              │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

Key Technologies: TensorFlow, Keras, NLTK, Hugging Face Datasets, NumPy, Pandas



Assignment 8: Text Summarization - Abstractive & Extractive Approaches

┌─ THREE SUMMARIZATION APPROACHES ─────────────────────────────────────────┐
│                                                                           │
│  ▸ Abstractive (Encoder-Decoder with Beam Search)                       │
│    • Custom LSTM encoder-decoder architecture                            │
│    • CNN/DailyMail dataset (300K+ articles)                              │
│    • Beam search for text generation (beam width: 3)                     │
│    • ROUGE Scores: R-1: 0.25, R-2: 0.10, R-L: 0.20                      │
│    • Generate summaries of 10+ words                                     │
│                                                                           │
│  ▸ Abstractive (Pre-trained T5)                                         │
│    • T5-small model from Hugging Face                                    │
│    • No training required - inference only                               │
│    • ROUGE Scores: R-1: 0.40, R-2: 0.18, R-L: 0.35                      │
│    • Superior performance vs custom encoder-decoder                      │
│    • Evaluation: Fluency 4/5, Coherence 4/5, Fact-preserving 2.4/3     │
│                                                                           │
│  ▸ Extractive (PageRank Algorithm)                                      │
│    • GloVe embeddings (Wikipedia 2014 + Gigaword 5)                      │
│    • Sentence ranking via NetworkX PageRank                              │
│    • BBC News Summary dataset (business category)                        │
│    • Cosine similarity for sentence comparison                           │
│    • ROUGE Scores: R-1: 0.35, R-2: 0.15, R-L: 0.30                      │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

Key Technologies: PyTorch, Transformers, NetworkX, GloVe, TorchMetrics, NLTK



Assignment 6: Word Sense Disambiguation & Semantic Role Labeling

┌─ SEMANTIC UNDERSTANDING & ROLE LABELING ─────────────────────────────────┐
│                                                                           │
│  ▸ Word Sense Disambiguation                                             │
│    • Simplified Lesk Algorithm: Overlap(C, D) = |C ∩ D|                 │
│    • Most Frequent Sense baseline: F-Score 0.54                          │
│    • Lesk with gloss overlap: F-Score 0.48                              │
│    • BiLSTM neural approach: F-Score 0.59 (best performance)            │
│    • SemCor corpus evaluation (50 test sentences)                        │
│                                                                           │
│  ▸ Semantic Role Labeling                                                │
│    • LSTM architecture: Word(100D) + Predicate(10D) → LSTM(128)         │
│    • OntoNotes v5 dataset for SRL                                        │
│    • Identifies predicate-argument structures                            │
│    • Performance: Precision 0.85, Recall 0.82, F1-Score 0.83            │
│    • Handles complex argument types (A0, A1, AM-TMP, etc.)              │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

Key Technologies: NLTK, WordNet, TensorFlow/Keras, BiLSTM, OntoNotes



Assignment 5: Constituency and Dependency Parsing

┌─ PARSING ALGORITHMS & SYNTACTIC ANALYSIS ────────────────────────────────┐
│                                                                           │
│  ▸ Constituency Tree Visualization                                       │
│    • Built parse trees using production rules                            │
│    • NLTK tree.draw() for graphical representation                       │
│    • Demonstrated S → VP, VP → NP V PP derivations                       │
│                                                                           │
│  ▸ CKY Parsing Algorithm                                                 │
│    • Full implementation from Jurafsky & Martin Section 13.4             │
│    • Chomsky Normal Form conversion (5,517 → 13,500 rules)              │
│    • Back-pointer tracking for parse tree reconstruction                 │
│    • Handles ambiguous grammars with multiple parse outputs              │
│                                                                           │
│  ▸ Dependency Parsing with Stanford CoreNLP                              │
│    • NLTK CoreNLP interface integration                                  │
│    • CoNLL format output (word, POS, head, relation)                    │
│    • Server-based parsing on port 9000                                   │
│                                                                           │
│  ▸ Ambiguous Sentence Analysis                                           │
│    • "Flying planes can be dangerous" - gerund vs adjective             │
│    • "Amid the chaos I saw her duck" - noun vs verb                     │
│    • Parser limitation analysis                                          │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

Key Technologies: NLTK, Stanford CoreNLP, CKY Algorithm, CFG, Chomsky Normal Form



Assignment 4: Named Entity Recognition with LSTM Networks

┌─ DEEP LEARNING FOR SEQUENCE LABELING ────────────────────────────────────┐
│                                                                           │
│  ▸ TF-IDF Vectorization & Cosine Similarity                              │
│    • Custom implementation from scratch                                   │
│    • Processed 1,000 documents with 5,847 unique tokens                  │
│    • Achieved semantic similarity scoring on sentence pairs              │
│                                                                           │
│  ▸ Positive Pointwise Mutual Information (PPMI)                          │
│    • Word association discovery through co-occurrence analysis           │
│    • Implemented PMI calculation with probability estimation             │
│    • Identified meaningful collocations in natural text                  │
│                                                                           │
│  ▸ LSTM-based Named Entity Recognition                                   │
│    • 3-layer LSTM architecture with Word2Vec embeddings (300D)           │
│    • Trained on CoNLL2003 dataset (5,000 samples)                        │
│    • BIO tagging scheme for 4 entity types (PER, ORG, LOC, MISC)        │
│    • Model Performance: 94.2% accuracy, 86.6% F1-score                   │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

Technical Implementation:

Architecture Design

Input (100 tokens)
  → Embedding(300D Word2Vec)
  → LSTM(128, dropout=0.2)
  → LSTM(64, dropout=0.2)
  → LSTM(32, dropout=0.2)
  → Dense(64, ReLU)
  → Softmax(9 classes)

Performance Metrics

Accuracy94.2%
Precision (macro)87.5%
Recall (macro)85.8%
F1-Score (macro)86.6%
Training Epochs10

Key Technologies: TensorFlow, Keras, Gensim (Word2Vec), Hugging Face Datasets, NumPy, Pandas



Assignment 3: N-gram Text Generation & Advanced POS Tagging

[📂 Source Code](ASN3/Assignment 3.py) | 📚 Corpus

┌─ STATISTICAL LANGUAGE MODELING & SEQUENCE LABELING ──────────────────────┐
│                                                                           │
│  ▸ Bigram Language Model                                                 │
│    • Built n-gram model from The Great Gatsby corpus                     │
│    • Conditional probability: p(w_i|w_{i-1}) calculation                 │
│    • Text generation with top-10 candidate sampling                      │
│    • Perplexity evaluation: 14.56 (excellent probability distribution)  │
│                                                                           │
│  ▸ Hidden Markov Model (HMM) POS Tagging                                 │
│    • Full HMM implementation with Viterbi decoding                       │
│    • Transition matrix A (tag→tag) and emission matrix B (tag→word)     │
│    • Penn Treebank dataset (3,914 sentences, 80/20 split)               │
│    • Achieved 91.25% accuracy on sequence labeling                       │
│                                                                           │
│  ▸ Conditional Random Fields (CRF) POS Tagging                           │
│    • Discriminative model with rich feature engineering                  │
│    • Features: word properties, character n-grams, contextual info      │
│    • Achieved 95.20% accuracy (+3.95% improvement over HMM)              │
│    • Production integration with sklearn-crfsuite                        │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

Comparative Analysis:

Model Accuracy Approach Key Advantage
HMM + Viterbi 91.25% Generative Fast inference, interpretable
CRF 95.20% Discriminative Rich features, better accuracy

Key Technologies: NLTK, sklearn-crfsuite, NumPy, Penn Treebank, Dynamic Programming



Assignment 2: From-Scratch Machine Learning Classifiers

[📂 Source Code](ASN2/Assignment 2.py) | 📈 Results Summary

┌─ FINANCIAL SENTIMENT ANALYSIS WITH CUSTOM ML MODELS ─────────────────────┐
│                                                                           │
│  ▸ Naive Bayes Classifier (Generative Model)                             │
│    • Built from mathematical foundations with Laplace smoothing          │
│    • Conditional probability: p(word|class) estimation                   │
│    • Bag-of-words feature extraction (1,452 dimensions)                  │
│    • Trained on financial phrasebank (2,264 sentences)                   │
│                                                                           │
│  ▸ Logistic Regression (Discriminative Model)                            │
│    • Implemented gradient descent optimization from scratch              │
│    • Custom cross-entropy loss with numerical stability                  │
│    • Hyperparameter tuning: learning rate α ∈ [0.0001, 0.1]             │
│    • Achieved 75.6% accuracy on 3-way sentiment classification           │
│                                                                           │
│  ▸ Production Pipeline                                                   │
│    • Data preprocessing: tokenization, lowercasing, vectorization        │
│    • Train/validation/test split: 60/20/20                               │
│    • Comprehensive evaluation: accuracy, precision, recall, F1-score     │
│    • Modular OOP design with reusable classifier classes                 │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

Model Performance:

Accuracy
75.6%
3-way classification
Training Epochs
500
Gradient descent
Feature Space
1,452D
Bag-of-words

Key Technologies: NumPy, pandas, scikit-learn (CountVectorizer), Custom Gradient Descent



Assignment 1: Advanced Text Analytics & Spell Correction

[📂 Source Code](ASN1/Assignment 1.py) | 📊 Corpus Data

┌─ HEALTHCARE SOCIAL MEDIA NLP PIPELINE ───────────────────────────────────┐
│                                                                           │
│  ▸ Multi-Source Data Integration                                         │
│    • Aggregated 6,045 health tweets from CNN & Fox News                  │
│    • Robust error handling with configurable data quality checks         │
│    • Regex-based cleaning: URLs, mentions, hashtags, special chars       │
│                                                                           │
│  ▸ Advanced Text Processing                                              │
│    • Hierarchical tokenization: sentences → words                        │
│    • Morphological analysis: WordNet lemmatization vs Porter stemming    │
│    • Stopword filtering: 20,586 common words removed                     │
│    • Vocabulary reduction: 8,797 → 6,345 tokens (27.9% optimization)    │
│                                                                           │
│  ▸ Intelligent Spell Correction                                          │
│    • Minimum Edit Distance algorithm (dynamic programming)               │
│    • Configurable costs: insertion, deletion, substitution               │
│    • Corpus-based suggestions with top-N ranking                         │
│    • Domain-aware corrections for health terminology                     │
│                                                                           │
│  ▸ Social Media Analytics                                                │
│    • Hashtag extraction: 914 unique tags, 3,572 total occurrences       │
│    • Trend analysis: #getfit, #ebola, #cancer, #flu identification       │
│    • Frequency distribution and statistical analysis                     │
│                                                                           │
└───────────────────────────────────────────────────────────────────────────┘

Data Processing Metrics:

Metric Value Optimization
Total Documents 6,045 tweets Multi-source integration
Original Vocabulary 8,797 words
After Stopword Removal 8,670 words 127 words removed
After Stemming 6,345 stems 27.9% reduction
Unique Lemmas 7,657 lemmas Quality preservation

Key Technologies: NLTK, pandas, NumPy, RegEx, Collections, Dynamic Programming


🔬 Technical Expertise

Machine Learning & Deep Learning

Algorithms Implemented:
  Supervised Learning:
    - Naive Bayes (generative)
    - Logistic Regression (discriminative)
    - Hidden Markov Models (probabilistic)
    - Conditional Random Fields (discriminative)
    - LSTM Neural Networks (recurrent)

  Optimization:
    - Gradient Descent
    - Adam Optimizer
    - Viterbi Decoding (dynamic programming)
    - Hyperparameter Tuning

  Model Evaluation:
    - Cross-validation
    - Accuracy, Precision, Recall, F1-score
    - Confusion matrices
    - Perplexity measurement

Natural Language Processing

Core NLP Techniques:
  Text Preprocessing:
    - Tokenization (sentence & word-level)
    - Normalization (lowercasing, stemming)
    - Lemmatization (WordNet-based)
    - Stopword removal

  Feature Engineering:
    - TF-IDF vectorization
    - Bag-of-words representation
    - Word embeddings (Word2Vec)
    - Character-level features
    - Contextual features

  Advanced Methods:
    - Named Entity Recognition (NER)
    - Part-of-Speech tagging
    - N-gram language models
    - PPMI word associations
    - Edit distance algorithms

Technology Stack

🐍 Core Python

Python 3.8+
NumPy
pandas
Collections
RegEx
🤖 ML/DL Frameworks

TensorFlow 2.x
Keras
scikit-learn
sklearn-crfsuite
📚 NLP Libraries

NLTK
Gensim (Word2Vec)
Hugging Face
spaCy-compatible
📊 Data & Visualization

Jupyter Notebook
Matplotlib
Seaborn
Chart.js

📈 Quantifiable Results

Model Performance Summary

Project Task Model Metric Result
ASN4 Named Entity Recognition 3-Layer LSTM F1-Score 86.6%
ASN4 NER Token Classification LSTM + Word2Vec Accuracy 94.2%
ASN3 POS Tagging CRF Accuracy 95.2%
ASN3 POS Tagging HMM + Viterbi Accuracy 91.3%
ASN3 Language Model Bigram Perplexity 14.56
ASN2 Sentiment Analysis Logistic Regression Accuracy 75.6%
ASN1 Data Processing Text Pipeline Quality 99%+

Business Impact

Scale & Efficiency
Documents Processed15,000+
Vocabulary Optimized27.9%
Model Training TimeReal-time
Production Readiness✓ Yes

Algorithm Complexity
Edit Distance DPO(m×n)
Viterbi DecodingO(T×N²)
LSTM InferenceO(T×d²)

💼 Professional Skills Demonstrated

Algorithm Design

▓▓▓▓▓▓▓▓▓░ 90%

Built ML models from mathematical foundations including:

✓ Probability theory (Bayes theorem)
✓ Linear algebra (matrix operations)
✓ Optimization (gradient descent)
✓ Dynamic programming (Viterbi, edit distance)
✓ Deep learning (LSTM architecture)

Software Engineering

▓▓▓▓▓▓▓▓▓░ 90%

Production-ready development practices:

✓ Object-oriented design (modular classes)
✓ Clean code principles (PEP-8 compliant)
✓ Comprehensive documentation
✓ Error handling and edge cases
✓ Version control (Git workflow)

Data Science

▓▓▓▓▓▓▓▓▓░ 90%

End-to-end ML pipeline expertise:

✓ Data acquisition and cleaning
✓ Feature engineering
✓ Model training and evaluation
✓ Statistical analysis
✓ Performance visualization


🚀 Quick Start

Prerequisites

Python 3.8 or higher
pip package manager

Installation

# Clone repository
git clone https://github.com/RamenMachine/Natural-Language-Processing.git
cd Natural-Language-Processing

# Install dependencies
pip install -r requirements.txt

# Download NLTK data (first run only)
python -c "import nltk; nltk.download('punkt'); nltk.download('stopwords'); nltk.download('wordnet')"

Run Individual Assignments

# Assignment 1: Text Analytics & Spell Correction
cd ASN1
python "Assignment 1.py"

# Assignment 2: Machine Learning Classifiers
cd ../ASN2
python "Assignment 2.py"

# Assignment 3: N-grams & POS Tagging
cd ../ASN3
python "Assignment 3.py"

# Assignment 4: Named Entity Recognition with LSTM
cd ../ASN4
python HW4.py

📁 Repository Structure

Natural-Language-Processing/
│
├── ASN1/                          # Text Analytics & Spell Correction
│   ├── Assignment 1.py            # Main implementation
│   ├── corpus.csv                 # Processed health tweets (6K+ records)
│   └── Health-Tweets/             # Raw data sources (CNN, Fox News)
│
├── ASN2/                          # From-Scratch ML Classifiers
│   ├── Assignment 2.py            # Naive Bayes & Logistic Regression
│   ├── Assignment_2_Results_Summary.md
│   └── FinancialPhraseBank-v1.0/  # Financial sentiment dataset
│
├── ASN3/                          # N-gram Text Generation & POS Tagging
│   ├── Assignment 3.py            # Bigram model, HMM, CRF implementation
│   └── GreatGatsby.txt            # Project Gutenberg corpus
│
├── ASN4/                          # Named Entity Recognition with LSTM
│   ├── HW4.py                     # Deep learning NER model
│   ├── assignment4_showcase.ipynb # Interactive visualizations
│   ├── index.html                 # GitHub Pages demo
│   ├── README.md                  # Project documentation
│   └── requirements.txt           # Python dependencies
│
├── ASN5/                          # Constituency & Dependency Parsing
│   ├── assignment5.py             # CKY algorithm, constituency trees
│   ├── dep_parser.py              # Stanford CoreNLP dependency parser
│   ├── start_corenlp.bat          # Server startup script (Windows)
│   ├── README.md                  # Setup instructions
│   └── stanford-corenlp-4.5.10/   # CoreNLP installation
│
├── ASN6/                          # Word Sense Disambiguation & SRL
│   ├── assignment6.py             # Lesk algorithm, BiLSTM WSD, SRL model
│   └── README.md                  # Project documentation
│
├── ASN7/                          # NLP Toolkit (Chatbot, Slot Filling, Translation)
│   ├── assignment7.py             # All 3 questions: Chatbot, Slot Filling, Translation
│   ├── q1_chatbot_evaluation.txt  # Written evaluation for Q1
│   ├── atis.train(1).csv          # ATIS training data
│   ├── atis.val(1).csv            # ATIS validation data
│   ├── atis.test(1).csv           # ATIS test data
│   ├── README.md                  # Complete documentation
│   └── requirements.txt           # Dependencies for ASN7
│
├── ASN8/                          # Text Summarization
│   ├── assignment8.py             # All coding questions (Q1, Q2, Q4)
│   ├── ASN8.txt                   # Written analysis for Q3
│   ├── README.md                  # Project documentation
│   └── requirements.txt           # Dependencies for ASN8
│
├── index.html                     # Main portfolio page with tabs
├── README.md                      # This file
├── requirements.txt               # Global dependencies
└── LICENSE                        # MIT License

🎓 Learning Outcomes & Applications

Academic Excellence

Mastered Core NLP Concepts:

Statistical Language Processing ▓▓▓▓▓▓▓▓▓▓ 100%

Machine Learning Algorithms ▓▓▓▓▓▓▓▓▓░ 95%

Deep Learning for NLP ▓▓▓▓▓▓▓▓▓░ 90%

Feature Engineering ▓▓▓▓▓▓▓▓▓░ 95%

Model Evaluation & Optimization ▓▓▓▓▓▓▓▓▓░ 95%

Real-World Applications

Industry-Ready Solutions:

Healthcare Analytics:
  - Social media health trend monitoring
  - Medical entity extraction (NER)
  - Patient sentiment analysis

Financial Technology:
  - Real-time sentiment classification
  - Automated trading signals
  - Risk assessment from news

Content & Media:
  - Automated content categorization
  - Text generation systems
  - Information extraction pipelines

Enterprise Search:
  - Semantic similarity matching
  - Document retrieval optimization
  - Query understanding

🏆 Why This Portfolio Stands Out

From Theory to Code

Every algorithm implemented from mathematical foundations, not just library calls. Demonstrates deep understanding of ML/NLP internals.
Production Quality

Clean, modular, documented code following software engineering best practices. Ready for deployment in real systems.
Quantifiable Results

Comprehensive performance metrics with benchmark comparisons. Achieved 95.2% accuracy on POS tagging, 94.2% on NER.
Full-Stack ML

End-to-end pipeline: data collection → preprocessing → modeling → evaluation → deployment. Complete workflow mastery.

📞 Contact & Collaboration

Interested in discussing NLP projects, machine learning systems, or collaboration opportunities?

GitHub Portfolio


┌──────────────────────────────────────────────────────────────┐
│  💡 Open to opportunities in:                                │
│                                                              │
│  ▸ Machine Learning Engineering                             │
│  ▸ Natural Language Processing                              │
│  ▸ Deep Learning Research                                   │
│  ▸ Data Science & Analytics                                 │
│                                                              │
└──────────────────────────────────────────────────────────────┘

⭐ Star this repository if you find it valuable for NLP/ML learning!



Built with Python, TensorFlow, NLTK, and a passion for Natural Language Processing

From Mathematical Theory → Production ML Systems → Business Impact


Copyright © 2025 | CS 421: Natural Language Processing