Skip to content

Commit 3c365ab

Browse files
cameronsjoclaude
andcommitted
feat: add 24 new reference pages via parallel agents
Machine Learning: - Vector Databases (Pinecone, Weaviate, Qdrant, pgvector, Chroma) - Embeddings (OpenAI, Cohere, Voyage, BGE models) - Model Serving (vLLM, TGI, Triton, Ollama) - Fine-tuning (LoRA, QLoRA, PEFT) - LLM Evaluation (RAGAS, benchmarks, evals) Security: - AI Security (OWASP LLM Top 10, prompt injection, guardrails) - Cryptography (symmetric, asymmetric, hashing, TLS) - Secrets Management (Vault, AWS/Azure/GCP, Doppler) - Supply Chain Security (SBOM, Sigstore, SLSA) Infrastructure/Tools: - CI-CD Pipelines (GitHub Actions, GitLab CI, ArgoCD) - Infrastructure as Code (Terraform, Pulumi, CDK) - Message Queues (Kafka, RabbitMQ, NATS) - Search Engines (Elasticsearch, Meilisearch, Typesense) - Caching Strategies (Redis, Memcached, patterns) - Service Mesh (Istio, Linkerd, Cilium) - CDN (Cloudflare, Fastly, CloudFront) - Feature Flags (LaunchDarkly, Unleash) - CLI Frameworks (Click, Typer, clap, cobra) Computer Science: - Database Internals (B-trees, WAL, MVCC, query planning) - Dynamic Programming (patterns, classic problems) - Backtracking (N-Queens, Sudoku, permutations) - DNS (records, DNSSEC, DoH/DoT) Frameworks: - Unreal Engine (UE5, Blueprints, Nanite, Lumen) - HTMX (hypermedia, progressive enhancement) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent dbefb96 commit 3c365ab

24 files changed

Lines changed: 9233 additions & 0 deletions

Computer Science/Backtracking.md

Lines changed: 428 additions & 0 deletions
Large diffs are not rendered by default.

Computer Science/DNS.md

Lines changed: 404 additions & 0 deletions
Large diffs are not rendered by default.

Computer Science/Database Internals.md

Lines changed: 641 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 376 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,376 @@
1+
---
2+
title: Dynamic Programming
3+
aliases:
4+
- DP
5+
- Dynamic Programming
6+
tags:
7+
- cs
8+
- algorithms
9+
- fundamentals
10+
type: reference
11+
status: complete
12+
created: 2025-11-30
13+
---
14+
15+
# Dynamic Programming
16+
17+
An algorithmic optimization technique that solves complex problems by breaking them down into simpler subproblems, storing solutions to avoid redundant computation.
18+
19+
## Overview
20+
21+
| Aspect | Details |
22+
|--------|---------|
23+
| **Core Principle** | Store solutions to subproblems and reuse them |
24+
| **When to Use** | Problems with optimal substructure and overlapping subproblems |
25+
| **Time Complexity** | Typically O(n) to O(n³) depending on problem dimensions |
26+
| **Space Complexity** | O(n) to O(n²), often reducible with optimization |
27+
| **Key Benefit** | Reduces exponential time to polynomial time |
28+
29+
## Core Concepts
30+
31+
### Optimal Substructure
32+
33+
A problem has optimal substructure if its optimal solution can be constructed from optimal solutions of its subproblems.
34+
35+
**Example:** Shortest path from A to C through B = shortest path from A to B + shortest path from B to C.
36+
37+
### Overlapping Subproblems
38+
39+
The problem can be broken down into subproblems that are reused multiple times. Without memoization, these would be recalculated repeatedly.
40+
41+
**Example:** Computing Fibonacci(5) requires Fibonacci(3) twice when calculated recursively.
42+
43+
## Approaches
44+
45+
| Approach | Description | When to Use | Space | Complexity |
46+
|----------|-------------|-------------|-------|------------|
47+
| **Memoization** | Top-down recursive with caching | Intuitive, doesn't compute all states | ✅ Can skip unneeded states | O(n) to O(n²) |
48+
| **Tabulation** | Bottom-up iterative | All states needed, iterative preference | ❌ Computes all states | O(n) to O(n²) |
49+
50+
### Memoization (Top-Down)
51+
52+
Recursive approach with caching of computed results.
53+
54+
```python
55+
def fib_memo(n: int, memo: dict[int, int] = None) -> int:
56+
if memo is None:
57+
memo = {}
58+
if n in memo:
59+
return memo[n]
60+
if n <= 1:
61+
return n
62+
memo[n] = fib_memo(n - 1, memo) + fib_memo(n - 2, memo)
63+
return memo[n]
64+
65+
# Time: O(n), Space: O(n)
66+
```
67+
68+
### Tabulation (Bottom-Up)
69+
70+
Iterative approach building from base cases.
71+
72+
```python
73+
def fib_tab(n: int) -> int:
74+
if n <= 1:
75+
return n
76+
dp = [0] * (n + 1)
77+
dp[1] = 1
78+
for i in range(2, n + 1):
79+
dp[i] = dp[i - 1] + dp[i - 2]
80+
return dp[n]
81+
82+
# Time: O(n), Space: O(n)
83+
```
84+
85+
### Space Optimization
86+
87+
Many DP problems can reduce space by keeping only needed previous states.
88+
89+
```python
90+
def fib_optimized(n: int) -> int:
91+
if n <= 1:
92+
return n
93+
prev2, prev1 = 0, 1
94+
for _ in range(2, n + 1):
95+
current = prev1 + prev2
96+
prev2, prev1 = prev1, current
97+
return prev1
98+
99+
# Time: O(n), Space: O(1)
100+
```
101+
102+
## Classic Problems
103+
104+
### 1. Fibonacci Sequence
105+
106+
**State:** `dp[i]` = ith Fibonacci number
107+
108+
**Recurrence:** `dp[i] = dp[i-1] + dp[i-2]`
109+
110+
**Base Cases:** `dp[0] = 0`, `dp[1] = 1`
111+
112+
**Complexity:** O(n) time, O(1) space optimized
113+
114+
### 2. Coin Change
115+
116+
**Problem:** Minimum coins to make amount using given denominations.
117+
118+
**State:** `dp[amount]` = minimum coins to make amount
119+
120+
**Recurrence:** `dp[i] = min(dp[i - coin] + 1)` for each coin
121+
122+
**Base Case:** `dp[0] = 0`
123+
124+
```python
125+
def coin_change(coins: list[int], amount: int) -> int:
126+
dp = [float('inf')] * (amount + 1)
127+
dp[0] = 0
128+
129+
for i in range(1, amount + 1):
130+
for coin in coins:
131+
if i >= coin:
132+
dp[i] = min(dp[i], dp[i - coin] + 1)
133+
134+
return dp[amount] if dp[amount] != float('inf') else -1
135+
136+
# Time: O(amount * len(coins)), Space: O(amount)
137+
```
138+
139+
### 3. 0/1 Knapsack
140+
141+
**Problem:** Maximize value with weight constraint, items used once.
142+
143+
**State:** `dp[i][w]` = max value using first i items with weight limit w
144+
145+
**Recurrence:**
146+
```
147+
dp[i][w] = max(
148+
dp[i-1][w], # don't take item i
149+
dp[i-1][w-weight[i]] + value[i] # take item i
150+
)
151+
```
152+
153+
**Complexity:** O(n × capacity) time, O(capacity) space optimized
154+
155+
```python
156+
def knapsack(weights: list[int], values: list[int], capacity: int) -> int:
157+
n = len(weights)
158+
dp = [[0] * (capacity + 1) for _ in range(n + 1)]
159+
160+
for i in range(1, n + 1):
161+
for w in range(capacity + 1):
162+
if weights[i-1] <= w:
163+
dp[i][w] = max(
164+
dp[i-1][w],
165+
dp[i-1][w - weights[i-1]] + values[i-1]
166+
)
167+
else:
168+
dp[i][w] = dp[i-1][w]
169+
170+
return dp[n][capacity]
171+
172+
# Time: O(n * capacity), Space: O(n * capacity)
173+
# Space optimizable to O(capacity) with 1D array
174+
```
175+
176+
### 4. Longest Common Subsequence (LCS)
177+
178+
**Problem:** Find length of longest subsequence common to two strings.
179+
180+
**State:** `dp[i][j]` = LCS length of text1[0:i] and text2[0:j]
181+
182+
**Recurrence:**
183+
```
184+
if text1[i-1] == text2[j-1]:
185+
dp[i][j] = dp[i-1][j-1] + 1
186+
else:
187+
dp[i][j] = max(dp[i-1][j], dp[i][j-1])
188+
```
189+
190+
**Complexity:** O(m × n) time, O(min(m, n)) space optimized
191+
192+
### 5. Longest Increasing Subsequence (LIS)
193+
194+
**Problem:** Find length of longest strictly increasing subsequence.
195+
196+
**State:** `dp[i]` = length of LIS ending at index i
197+
198+
**Recurrence:** `dp[i] = max(dp[j] + 1)` for all j < i where nums[j] < nums[i]
199+
200+
**Complexity:** O(n²) time with DP, O(n log n) with binary search optimization
201+
202+
```python
203+
def longest_increasing_subsequence(nums: list[int]) -> int:
204+
if not nums:
205+
return 0
206+
207+
dp = [1] * len(nums)
208+
209+
for i in range(1, len(nums)):
210+
for j in range(i):
211+
if nums[j] < nums[i]:
212+
dp[i] = max(dp[i], dp[j] + 1)
213+
214+
return max(dp)
215+
216+
# Time: O(n²), Space: O(n)
217+
```
218+
219+
### 6. Edit Distance (Levenshtein Distance)
220+
221+
**Problem:** Minimum operations (insert, delete, replace) to convert word1 to word2.
222+
223+
**State:** `dp[i][j]` = edit distance for word1[0:i] to word2[0:j]
224+
225+
**Recurrence:**
226+
```
227+
if word1[i-1] == word2[j-1]:
228+
dp[i][j] = dp[i-1][j-1]
229+
else:
230+
dp[i][j] = 1 + min(
231+
dp[i-1][j], # delete
232+
dp[i][j-1], # insert
233+
dp[i-1][j-1] # replace
234+
)
235+
```
236+
237+
**Complexity:** O(m × n) time, O(min(m, n)) space optimized
238+
239+
### 7. Matrix Chain Multiplication
240+
241+
**Problem:** Find optimal order to multiply sequence of matrices.
242+
243+
**State:** `dp[i][j]` = minimum operations to multiply matrices from i to j
244+
245+
**Recurrence:**
246+
```
247+
dp[i][j] = min(
248+
dp[i][k] + dp[k+1][j] + dimensions[i-1] * dimensions[k] * dimensions[j]
249+
)
250+
for all k in range(i, j)
251+
```
252+
253+
**Complexity:** O(n³) time, O(n²) space
254+
255+
## DP Patterns
256+
257+
### Linear DP (1D)
258+
259+
Problems where state depends on previous elements in a sequence.
260+
261+
**Examples:** Fibonacci, House Robber, Climbing Stairs, Decode Ways
262+
263+
**Pattern:**
264+
```python
265+
dp = [base_case] * n
266+
for i in range(start, n):
267+
dp[i] = function(dp[i-1], dp[i-2], ...)
268+
```
269+
270+
### Grid DP (2D)
271+
272+
Problems involving paths, grids, or two sequences.
273+
274+
**Examples:** Unique Paths, Minimum Path Sum, LCS, Edit Distance
275+
276+
**Pattern:**
277+
```python
278+
dp = [[0] * cols for _ in range(rows)]
279+
for i in range(rows):
280+
for j in range(cols):
281+
dp[i][j] = function(dp[i-1][j], dp[i][j-1], ...)
282+
```
283+
284+
### Interval DP
285+
286+
Problems involving ranges or intervals.
287+
288+
**Examples:** Matrix Chain Multiplication, Palindrome Partitioning, Burst Balloons
289+
290+
**Pattern:**
291+
```python
292+
# Process by increasing interval length
293+
for length in range(2, n + 1):
294+
for i in range(n - length + 1):
295+
j = i + length - 1
296+
for k in range(i, j):
297+
dp[i][j] = optimize(dp[i][k], dp[k+1][j])
298+
```
299+
300+
### State Machine DP
301+
302+
Problems with discrete states and transitions.
303+
304+
**Examples:** Best Time to Buy/Sell Stock (with cooldown/fees), Paint House
305+
306+
**Pattern:**
307+
```python
308+
# Multiple states per position
309+
state1, state2 = initial_values
310+
for element in sequence:
311+
new_state1 = function(state1, state2, element)
312+
new_state2 = function(state1, state2, element)
313+
state1, state2 = new_state1, new_state2
314+
```
315+
316+
## Complexity Analysis by Pattern
317+
318+
| Pattern | Typical Time | Typical Space | Space Optimizable |
319+
|---------|--------------|---------------|-------------------|
320+
| Linear 1D | O(n) | O(n) | ✅ Often to O(1) |
321+
| Grid 2D | O(m × n) | O(m × n) | ✅ To O(min(m,n)) |
322+
| Interval DP | O(n³) | O(n²) | ❌ Usually not |
323+
| Knapsack | O(n × W) | O(n × W) | ✅ To O(W) |
324+
| LIS (DP) | O(n²) | O(n) | ❌ Already minimal |
325+
| String DP | O(m × n) | O(m × n) | ✅ To O(min(m,n)) |
326+
327+
## Problem-Solving Framework
328+
329+
1. **Identify DP applicability:**
330+
- Does it have optimal substructure?
331+
- Are there overlapping subproblems?
332+
333+
2. **Define the state:**
334+
- What information is needed to represent a subproblem?
335+
- How many dimensions are required?
336+
337+
3. **Find the recurrence relation:**
338+
- How does the current state relate to previous states?
339+
- What are the transition options?
340+
341+
4. **Determine base cases:**
342+
- What are the simplest subproblems with known answers?
343+
344+
5. **Choose approach:**
345+
- Memoization: intuitive, handles sparse states
346+
- Tabulation: iterative, potentially more efficient
347+
348+
6. **Optimize space:**
349+
- Can you reduce dimensions?
350+
- Do you only need the last k states?
351+
352+
## When to Use Dynamic Programming
353+
354+
| Use When | Avoid When |
355+
|----------|------------|
356+
| ✅ Optimization problems (min/max) | ❌ Need to find all solutions (use backtracking) |
357+
| ✅ Counting problems (number of ways) | ❌ Problem requires actual path/solution construction only |
358+
| ✅ Overlapping subproblems exist | ❌ Subproblems are independent (use divide & conquer) |
359+
| ✅ Optimal substructure present | ❌ Greedy approach suffices |
360+
| ✅ Decision problems (yes/no) | ❌ Real-time constraints (DP has setup cost) |
361+
362+
## Common Mistakes
363+
364+
1. **Off-by-one errors:** Carefully handle array indices and loop bounds
365+
2. **Incorrect base cases:** Ensure base cases cover all edge scenarios
366+
3. **Wrong state definition:** State must capture all information needed for transitions
367+
4. **Space optimization errors:** Ensure you don't overwrite needed values
368+
5. **Integer overflow:** Use appropriate data types for large values
369+
370+
## Related
371+
372+
- [[Big O Notation]] - Understanding time and space complexity
373+
- [[Sorting Algorithms]] - Foundational algorithmic techniques
374+
- [[Graph Algorithms]] - Advanced algorithmic problem solving
375+
- [[Recursion]] - Foundation for memoization approach
376+
- [[Greedy Algorithms]] - Alternative optimization technique

0 commit comments

Comments
 (0)