╔═══════════════════════════════════════════════════════════════════════════╗
║ ║
║ ███╗ ██╗██╗ ██████╗ ██████╗ ██████╗ ██████╗ ████████╗ ║
║ ████╗ ██║██║ ██╔══██╗ ██╔══██╗██╔═══██╗██╔══██╗╚══██╔══╝ ║
║ ██╔██╗ ██║██║ ██████╔╝ ██████╔╝██║ ██║██████╔╝ ██║ ║
║ ██║╚██╗██║██║ ██╔═══╝ ██╔═══╝ ██║ ██║██╔══██╗ ██║ ║
║ ██║ ╚████║███████╗██║ ██║ ╚██████╔╝██║ ██║ ██║ ║
║ ╚═╝ ╚═══╝╚══════╝╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝ ╚═╝ ║
║ ║
╚═══════════════════════════════════════════════════════════════════════════╝
Production-Ready Natural Language Processing & Machine Learning Portfolio
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Advanced NLP implementations spanning text analytics, machine learning classifiers, sequence modeling, and deep learning for named entity recognition
|
This repository demonstrates end-to-end machine learning and NLP expertise through four comprehensive assignments implementing algorithms from mathematical foundations. Core Focus Areas: nlp_pipeline = {
"text_processing": ["Tokenization", "Stemming", "Lemmatization"],
"ml_algorithms": ["Naive Bayes", "Logistic Regression"],
"sequence_modeling": ["N-grams", "HMM", "CRF"],
"deep_learning": ["LSTM", "Word2Vec", "NER"]
} |
|
┌─ THREE COMPLETE NLP SYSTEMS ─────────────────────────────────────────────┐
│ │
│ ▸ Corpus-Based Chatbot (TF-IDF Retrieval) │
│ • Custom TF-IDF implementation from scratch │
│ • NPS Chat corpus (~10K messages) │
│ • Cosine similarity-based response matching │
│ • Intelligent filtering (removes questions, short responses) │
│ • Evaluation: Engagingness 3/5, Making Sense 3/4, Fluency 4.5/5 │
│ │
│ ▸ LSTM Slot Filling (ATIS Dataset) │
│ • Bidirectional LSTM architecture: Embedding → BiLSTM(128) → Dense │
│ • ATIS travel dataset: 4.4K train, 900 test sentences │
│ • 127 unique slot labels (locations, dates, airlines, etc.) │
│ • Performance: Precision 0.95, Recall 0.94, F1-Score 0.95 │
│ • TimeDistributed output layer for sequence labeling │
│ │
│ ▸ Neural Machine Translation (German → English) │
│ • Seq2Seq architecture with attention mechanism │
│ • WMT14 dataset (de-en configuration) │
│ • Encoder: Embedding → LSTM with context vectors │
│ • Decoder: LSTM → Attention → Dense → Softmax │
│ • BLEU Score: 0.18 (greedy decoding) │
│ • 10K vocab for both German and English │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Key Technologies: TensorFlow, Keras, NLTK, Hugging Face Datasets, NumPy, Pandas
┌─ THREE SUMMARIZATION APPROACHES ─────────────────────────────────────────┐
│ │
│ ▸ Abstractive (Encoder-Decoder with Beam Search) │
│ • Custom LSTM encoder-decoder architecture │
│ • CNN/DailyMail dataset (300K+ articles) │
│ • Beam search for text generation (beam width: 3) │
│ • ROUGE Scores: R-1: 0.25, R-2: 0.10, R-L: 0.20 │
│ • Generate summaries of 10+ words │
│ │
│ ▸ Abstractive (Pre-trained T5) │
│ • T5-small model from Hugging Face │
│ • No training required - inference only │
│ • ROUGE Scores: R-1: 0.40, R-2: 0.18, R-L: 0.35 │
│ • Superior performance vs custom encoder-decoder │
│ • Evaluation: Fluency 4/5, Coherence 4/5, Fact-preserving 2.4/3 │
│ │
│ ▸ Extractive (PageRank Algorithm) │
│ • GloVe embeddings (Wikipedia 2014 + Gigaword 5) │
│ • Sentence ranking via NetworkX PageRank │
│ • BBC News Summary dataset (business category) │
│ • Cosine similarity for sentence comparison │
│ • ROUGE Scores: R-1: 0.35, R-2: 0.15, R-L: 0.30 │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Key Technologies: PyTorch, Transformers, NetworkX, GloVe, TorchMetrics, NLTK
┌─ SEMANTIC UNDERSTANDING & ROLE LABELING ─────────────────────────────────┐
│ │
│ ▸ Word Sense Disambiguation │
│ • Simplified Lesk Algorithm: Overlap(C, D) = |C ∩ D| │
│ • Most Frequent Sense baseline: F-Score 0.54 │
│ • Lesk with gloss overlap: F-Score 0.48 │
│ • BiLSTM neural approach: F-Score 0.59 (best performance) │
│ • SemCor corpus evaluation (50 test sentences) │
│ │
│ ▸ Semantic Role Labeling │
│ • LSTM architecture: Word(100D) + Predicate(10D) → LSTM(128) │
│ • OntoNotes v5 dataset for SRL │
│ • Identifies predicate-argument structures │
│ • Performance: Precision 0.85, Recall 0.82, F1-Score 0.83 │
│ • Handles complex argument types (A0, A1, AM-TMP, etc.) │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Key Technologies: NLTK, WordNet, TensorFlow/Keras, BiLSTM, OntoNotes
┌─ PARSING ALGORITHMS & SYNTACTIC ANALYSIS ────────────────────────────────┐
│ │
│ ▸ Constituency Tree Visualization │
│ • Built parse trees using production rules │
│ • NLTK tree.draw() for graphical representation │
│ • Demonstrated S → VP, VP → NP V PP derivations │
│ │
│ ▸ CKY Parsing Algorithm │
│ • Full implementation from Jurafsky & Martin Section 13.4 │
│ • Chomsky Normal Form conversion (5,517 → 13,500 rules) │
│ • Back-pointer tracking for parse tree reconstruction │
│ • Handles ambiguous grammars with multiple parse outputs │
│ │
│ ▸ Dependency Parsing with Stanford CoreNLP │
│ • NLTK CoreNLP interface integration │
│ • CoNLL format output (word, POS, head, relation) │
│ • Server-based parsing on port 9000 │
│ │
│ ▸ Ambiguous Sentence Analysis │
│ • "Flying planes can be dangerous" - gerund vs adjective │
│ • "Amid the chaos I saw her duck" - noun vs verb │
│ • Parser limitation analysis │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Key Technologies: NLTK, Stanford CoreNLP, CKY Algorithm, CFG, Chomsky Normal Form
┌─ DEEP LEARNING FOR SEQUENCE LABELING ────────────────────────────────────┐
│ │
│ ▸ TF-IDF Vectorization & Cosine Similarity │
│ • Custom implementation from scratch │
│ • Processed 1,000 documents with 5,847 unique tokens │
│ • Achieved semantic similarity scoring on sentence pairs │
│ │
│ ▸ Positive Pointwise Mutual Information (PPMI) │
│ • Word association discovery through co-occurrence analysis │
│ • Implemented PMI calculation with probability estimation │
│ • Identified meaningful collocations in natural text │
│ │
│ ▸ LSTM-based Named Entity Recognition │
│ • 3-layer LSTM architecture with Word2Vec embeddings (300D) │
│ • Trained on CoNLL2003 dataset (5,000 samples) │
│ • BIO tagging scheme for 4 entity types (PER, ORG, LOC, MISC) │
│ • Model Performance: 94.2% accuracy, 86.6% F1-score │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Technical Implementation:
|
Architecture Design Input (100 tokens)
→ Embedding(300D Word2Vec)
→ LSTM(128, dropout=0.2)
→ LSTM(64, dropout=0.2)
→ LSTM(32, dropout=0.2)
→ Dense(64, ReLU)
→ Softmax(9 classes) |
Performance Metrics
|
Key Technologies: TensorFlow, Keras, Gensim (Word2Vec), Hugging Face Datasets, NumPy, Pandas
[📂 Source Code](ASN3/Assignment 3.py) | 📚 Corpus
┌─ STATISTICAL LANGUAGE MODELING & SEQUENCE LABELING ──────────────────────┐
│ │
│ ▸ Bigram Language Model │
│ • Built n-gram model from The Great Gatsby corpus │
│ • Conditional probability: p(w_i|w_{i-1}) calculation │
│ • Text generation with top-10 candidate sampling │
│ • Perplexity evaluation: 14.56 (excellent probability distribution) │
│ │
│ ▸ Hidden Markov Model (HMM) POS Tagging │
│ • Full HMM implementation with Viterbi decoding │
│ • Transition matrix A (tag→tag) and emission matrix B (tag→word) │
│ • Penn Treebank dataset (3,914 sentences, 80/20 split) │
│ • Achieved 91.25% accuracy on sequence labeling │
│ │
│ ▸ Conditional Random Fields (CRF) POS Tagging │
│ • Discriminative model with rich feature engineering │
│ • Features: word properties, character n-grams, contextual info │
│ • Achieved 95.20% accuracy (+3.95% improvement over HMM) │
│ • Production integration with sklearn-crfsuite │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Comparative Analysis:
| Model | Accuracy | Approach | Key Advantage |
|---|---|---|---|
| HMM + Viterbi | 91.25% | Generative | Fast inference, interpretable |
| CRF | 95.20% | Discriminative | Rich features, better accuracy |
Key Technologies: NLTK, sklearn-crfsuite, NumPy, Penn Treebank, Dynamic Programming
[📂 Source Code](ASN2/Assignment 2.py) | 📈 Results Summary
┌─ FINANCIAL SENTIMENT ANALYSIS WITH CUSTOM ML MODELS ─────────────────────┐
│ │
│ ▸ Naive Bayes Classifier (Generative Model) │
│ • Built from mathematical foundations with Laplace smoothing │
│ • Conditional probability: p(word|class) estimation │
│ • Bag-of-words feature extraction (1,452 dimensions) │
│ • Trained on financial phrasebank (2,264 sentences) │
│ │
│ ▸ Logistic Regression (Discriminative Model) │
│ • Implemented gradient descent optimization from scratch │
│ • Custom cross-entropy loss with numerical stability │
│ • Hyperparameter tuning: learning rate α ∈ [0.0001, 0.1] │
│ • Achieved 75.6% accuracy on 3-way sentiment classification │
│ │
│ ▸ Production Pipeline │
│ • Data preprocessing: tokenization, lowercasing, vectorization │
│ • Train/validation/test split: 60/20/20 │
│ • Comprehensive evaluation: accuracy, precision, recall, F1-score │
│ • Modular OOP design with reusable classifier classes │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Model Performance:
Accuracy75.6%3-way classification |
Training Epochs500Gradient descent |
Feature Space1,452DBag-of-words |
Key Technologies: NumPy, pandas, scikit-learn (CountVectorizer), Custom Gradient Descent
[📂 Source Code](ASN1/Assignment 1.py) | 📊 Corpus Data
┌─ HEALTHCARE SOCIAL MEDIA NLP PIPELINE ───────────────────────────────────┐
│ │
│ ▸ Multi-Source Data Integration │
│ • Aggregated 6,045 health tweets from CNN & Fox News │
│ • Robust error handling with configurable data quality checks │
│ • Regex-based cleaning: URLs, mentions, hashtags, special chars │
│ │
│ ▸ Advanced Text Processing │
│ • Hierarchical tokenization: sentences → words │
│ • Morphological analysis: WordNet lemmatization vs Porter stemming │
│ • Stopword filtering: 20,586 common words removed │
│ • Vocabulary reduction: 8,797 → 6,345 tokens (27.9% optimization) │
│ │
│ ▸ Intelligent Spell Correction │
│ • Minimum Edit Distance algorithm (dynamic programming) │
│ • Configurable costs: insertion, deletion, substitution │
│ • Corpus-based suggestions with top-N ranking │
│ • Domain-aware corrections for health terminology │
│ │
│ ▸ Social Media Analytics │
│ • Hashtag extraction: 914 unique tags, 3,572 total occurrences │
│ • Trend analysis: #getfit, #ebola, #cancer, #flu identification │
│ • Frequency distribution and statistical analysis │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Data Processing Metrics:
| Metric | Value | Optimization |
|---|---|---|
| Total Documents | 6,045 tweets | Multi-source integration |
| Original Vocabulary | 8,797 words | — |
| After Stopword Removal | 8,670 words | 127 words removed |
| After Stemming | 6,345 stems | 27.9% reduction |
| Unique Lemmas | 7,657 lemmas | Quality preservation |
Key Technologies: NLTK, pandas, NumPy, RegEx, Collections, Dynamic Programming
Algorithms Implemented:
Supervised Learning:
- Naive Bayes (generative)
- Logistic Regression (discriminative)
- Hidden Markov Models (probabilistic)
- Conditional Random Fields (discriminative)
- LSTM Neural Networks (recurrent)
Optimization:
- Gradient Descent
- Adam Optimizer
- Viterbi Decoding (dynamic programming)
- Hyperparameter Tuning
Model Evaluation:
- Cross-validation
- Accuracy, Precision, Recall, F1-score
- Confusion matrices
- Perplexity measurement |
Core NLP Techniques:
Text Preprocessing:
- Tokenization (sentence & word-level)
- Normalization (lowercasing, stemming)
- Lemmatization (WordNet-based)
- Stopword removal
Feature Engineering:
- TF-IDF vectorization
- Bag-of-words representation
- Word embeddings (Word2Vec)
- Character-level features
- Contextual features
Advanced Methods:
- Named Entity Recognition (NER)
- Part-of-Speech tagging
- N-gram language models
- PPMI word associations
- Edit distance algorithms |
🐍 Core PythonPython 3.8+NumPypandasCollectionsRegEx
|
🤖 ML/DL FrameworksTensorFlow 2.xKerasscikit-learnsklearn-crfsuite
|
📚 NLP LibrariesNLTKGensim (Word2Vec)Hugging FacespaCy-compatible
|
📊 Data & VisualizationJupyter NotebookMatplotlibSeabornChart.js
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Built ML models from mathematical foundations including: ✓ Probability theory (Bayes theorem) |
Production-ready development practices: ✓ Object-oriented design (modular classes) |
End-to-end ML pipeline expertise: ✓ Data acquisition and cleaning |
Python 3.8 or higher
pip package manager# Clone repository
git clone https://github.com/RamenMachine/Natural-Language-Processing.git
cd Natural-Language-Processing
# Install dependencies
pip install -r requirements.txt
# Download NLTK data (first run only)
python -c "import nltk; nltk.download('punkt'); nltk.download('stopwords'); nltk.download('wordnet')"# Assignment 1: Text Analytics & Spell Correction
cd ASN1
python "Assignment 1.py"
# Assignment 2: Machine Learning Classifiers
cd ../ASN2
python "Assignment 2.py"
# Assignment 3: N-grams & POS Tagging
cd ../ASN3
python "Assignment 3.py"
# Assignment 4: Named Entity Recognition with LSTM
cd ../ASN4
python HW4.pyNatural-Language-Processing/
│
├── ASN1/ # Text Analytics & Spell Correction
│ ├── Assignment 1.py # Main implementation
│ ├── corpus.csv # Processed health tweets (6K+ records)
│ └── Health-Tweets/ # Raw data sources (CNN, Fox News)
│
├── ASN2/ # From-Scratch ML Classifiers
│ ├── Assignment 2.py # Naive Bayes & Logistic Regression
│ ├── Assignment_2_Results_Summary.md
│ └── FinancialPhraseBank-v1.0/ # Financial sentiment dataset
│
├── ASN3/ # N-gram Text Generation & POS Tagging
│ ├── Assignment 3.py # Bigram model, HMM, CRF implementation
│ └── GreatGatsby.txt # Project Gutenberg corpus
│
├── ASN4/ # Named Entity Recognition with LSTM
│ ├── HW4.py # Deep learning NER model
│ ├── assignment4_showcase.ipynb # Interactive visualizations
│ ├── index.html # GitHub Pages demo
│ ├── README.md # Project documentation
│ └── requirements.txt # Python dependencies
│
├── ASN5/ # Constituency & Dependency Parsing
│ ├── assignment5.py # CKY algorithm, constituency trees
│ ├── dep_parser.py # Stanford CoreNLP dependency parser
│ ├── start_corenlp.bat # Server startup script (Windows)
│ ├── README.md # Setup instructions
│ └── stanford-corenlp-4.5.10/ # CoreNLP installation
│
├── ASN6/ # Word Sense Disambiguation & SRL
│ ├── assignment6.py # Lesk algorithm, BiLSTM WSD, SRL model
│ └── README.md # Project documentation
│
├── ASN7/ # NLP Toolkit (Chatbot, Slot Filling, Translation)
│ ├── assignment7.py # All 3 questions: Chatbot, Slot Filling, Translation
│ ├── q1_chatbot_evaluation.txt # Written evaluation for Q1
│ ├── atis.train(1).csv # ATIS training data
│ ├── atis.val(1).csv # ATIS validation data
│ ├── atis.test(1).csv # ATIS test data
│ ├── README.md # Complete documentation
│ └── requirements.txt # Dependencies for ASN7
│
├── ASN8/ # Text Summarization
│ ├── assignment8.py # All coding questions (Q1, Q2, Q4)
│ ├── ASN8.txt # Written analysis for Q3
│ ├── README.md # Project documentation
│ └── requirements.txt # Dependencies for ASN8
│
├── index.html # Main portfolio page with tabs
├── README.md # This file
├── requirements.txt # Global dependencies
└── LICENSE # MIT License
|
Mastered Core NLP Concepts: Statistical Language Processing
Machine Learning Algorithms
Deep Learning for NLP
Feature Engineering
Model Evaluation & Optimization
|
Industry-Ready Solutions: Healthcare Analytics:
- Social media health trend monitoring
- Medical entity extraction (NER)
- Patient sentiment analysis
Financial Technology:
- Real-time sentiment classification
- Automated trading signals
- Risk assessment from news
Content & Media:
- Automated content categorization
- Text generation systems
- Information extraction pipelines
Enterprise Search:
- Semantic similarity matching
- Document retrieval optimization
- Query understanding |
|
From Theory to Code Every algorithm implemented from mathematical foundations, not just library calls. Demonstrates deep understanding of ML/NLP internals. |
Production Quality Clean, modular, documented code following software engineering best practices. Ready for deployment in real systems. |
Quantifiable Results Comprehensive performance metrics with benchmark comparisons. Achieved 95.2% accuracy on POS tagging, 94.2% on NER. |
Full-Stack ML End-to-end pipeline: data collection → preprocessing → modeling → evaluation → deployment. Complete workflow mastery. |
Interested in discussing NLP projects, machine learning systems, or collaboration opportunities?
┌──────────────────────────────────────────────────────────────┐
│ 💡 Open to opportunities in: │
│ │
│ ▸ Machine Learning Engineering │
│ ▸ Natural Language Processing │
│ ▸ Deep Learning Research │
│ ▸ Data Science & Analytics │
│ │
└──────────────────────────────────────────────────────────────┘
⭐ Star this repository if you find it valuable for NLP/ML learning!
Built with Python, TensorFlow, NLTK, and a passion for Natural Language Processing
From Mathematical Theory → Production ML Systems → Business Impact
Copyright © 2025 | CS 421: Natural Language Processing