Notes 16-26 for 334

Author: localhost433

notes/courses/CSCI-UA-310/R-1.md

@@ -0,0 +1,105 @@
+---
+title: Recitation 1 - Mathematical Background
+date: 2026-01-23
+---
+
+## Roadmap
+
+This recitation serves as a **mathematical toolbox** review. We cover the essential discrete mathematics concepts required for analyzing algorithm correctness and runtime.
+
+1. **Induction**: The standard template for proofs.
+2. **Summations**: Arithmetic and Geometric progressions.
+3. **Logarithms**: Identities and approximations.
+4. **Asymptotic Bounds**: Proving limits.
+
+---
+
+## 1. Mathematical Induction
+
+Induction is the primary tool for proving the correctness of loop invariants and properties of recursive algorithms.
+
+### 1.1 The Template
+
+To prove a statement $P(n)$ is true for all integers $n \ge k$:
+
+1. **Base Case**: Prove that $P(k)$ is true.
+2. **Inductive Hypothesis (IH)**: Assume $P(n)$ is true for some arbitrary $n \ge k$.
+3. **Inductive Step**: Show that $P(n) \implies P(n+1)$.
+4. **Conclusion**: By the principle of mathematical induction, $P(n)$ is true for all $n \ge k$.
+
+### 1.2 Example: Sum of Cubes
+
+**Claim**: $\sum_{i=1}^n i^3 = \left( \frac{n(n+1)}{2} \right)^2$.
+
+**Proof**:
+
+* **Base Case ($n=1$)**:
+    LHS: $1^3 = 1$.
+    RHS: $\left( \frac{1(2)}{2} \right)^2 = 1^2 = 1$.
+    LHS = RHS.
+* **Inductive Hypothesis**: Assume $\sum_{i=1}^k i^3 = \left( \frac{k(k+1)}{2} \right)^2$.
+* **Inductive Step**: Consider $n = k+1$.
+    $$\sum_{i=1}^{k+1} i^3 = \left( \sum_{i=1}^k i^3 \right) + (k+1)^3$$
+    Substitute IH:
+    $$= \left( \frac{k(k+1)}{2} \right)^2 + (k+1)^3$$
+    $$= \frac{k^2(k+1)^2}{4} + \frac{4(k+1)^3}{4}$$
+    $$= \frac{(k+1)^2}{4} \left( k^2 + 4(k+1) \right)$$
+    $$= \frac{(k+1)^2}{4} (k^2 + 4k + 4)$$
+    $$= \frac{(k+1)^2 (k+2)^2}{4} = \left( \frac{(k+1)(k+2)}{2} \right)^2$$
+    This matches the formula for $n=k+1$.
+* **Conclusion**: The claim holds for all $n \ge 1$.
+
+---
+
+## 2. Summations
+
+You will frequently encounter these series in runtime analysis.
+
+### 2.1 Arithmetic Progression
+
+An arithmetic series increases by a constant amount $d$.
+$$\sum_{i=0}^{n-1} (a + id) = \frac{n}{2}(a + a_{last})$$
+**Standard Case**: Sum of integers $1$ to $n$:
+$$\sum_{i=1}^n i = \frac{n(n+1)}{2} = \Theta(n^2)$$
+where $a_{last} = a + (n-1)d$.
+*Usage*: Appears in Insertion Sort worst-case analysis (nested loops).
+
+### 2.2 Geometric Progression
+
+A geometric series changes by a constant ratio $r$.
+$$\sum_{i=0}^n r^i = \frac{r^{n+1} - 1}{r - 1}$$
+
+* If $r < 1$ (decreasing series): The sum converges to $\frac{1}{1-r}$, which is $\Theta(1)$.
+* If $r > 1$ (increasing series): The sum is dominated by the last term, $\Theta(r^n)$.
+
+*Usage*: Appears in recursion trees where the work per level grows or shrinks geometrically.
+
+### 2.3 Harmonic Series
+
+$$H_n = \sum_{i=1}^n \frac{1}{i} = \ln n + \gamma \approx \ln n$$
+where $\gamma$ is the Euler-Mascheroni constant.
+$$H_n = \Theta(\log n)$$
+*Usage*: Appears in Randomized QuickSort analysis.
+
+---
+
+## 3. Logarithms
+
+In computer science, $\log n$ usually denotes $\log_2 n$ (binary logarithm).
+
+### Key Identities
+
+1. $a = b^{\log_b a}$
+2. $\log_c(ab) = \log_c a + \log_c b$
+3. $\log_b a = \frac{\log_c a}{\log_c b}$ (Change of base)
+4. $a^{\log_b n} = n^{\log_b a}$ (Important for Master Theorem)
+
+**Stirling's Approximation**:
+$$n! \approx \sqrt{2\pi n} \left(\frac{n}{e}\right)^n$$
+$$\log(n!) = \Theta(n \log n)$$
+
+---
+
+## References
+
+* **CLRS**: Appendix A (Summations), Appendix B (Sets, Etc.), Chapter 3 (Growth of Functions).

notes/courses/CSCI-UA-310/R-2.md

@@ -0,0 +1,99 @@
+---
+title: Recitation 2 - Exact Analysis & Recurrences
+date: 2026-01-30
+---
+
+## Roadmap
+
+In this recitation, we perform a precise runtime analysis of a simple algorithm (`FindMax`) to understand the difference between exact instruction counting and asymptotic analysis. We also solve a common recurrence found in Binary Search.
+
+1. **Exact Analysis**: `FindMax` algorithm.
+2. **Comparison**: $\log n$ vs $n$.
+3. **Recurrence**: $T(n) = T(n/2) + c$.
+
+---
+
+## 1. Runtime of FindMax
+
+### 1.1 The Algorithm
+
+```text
+FindMax(A)
+1. max = A[1]
+2. for i = 2 to A.length
+3.     if A[i] > max
+4.         max = A[i]
+5. return max
+
+```
+
+### 1.2 Exact Cost Analysis
+
+We assign a cost $c_k$ to line $k$.
+
+* Line 1: Executed 1 time. Cost $c_1$.
+* Line 2: The loop header tests $i$ from $2$ to $n$. Executed $n$ times. Cost $c_2 n$.
+* Line 3: The body runs $n-1$ times. Cost $c_3(n-1)$.
+* Line 4: Assignment runs $n-1$ times (depends on data). Worst case: $n-1$ times (strictly increasing array). Best case: 0 times (strictly decreasing). Cost $c_4(n-1)$.
+* Line 5: Executed 1 time. Cost $c_5$.
+
+**Total Worst-Case Time**:
+$$ T(n) = c_1 + c_2 n + c_3(n-1) + c_4(n-1) + c_5 $$
+$$ T(n) = (c_2 + c_3 + c_4)n + (c_1 - c_3 - c_4 + c_5) $$
+This is of the form $an + b$, which is linear, $\Theta(n)$.
+Asymptotic analysis allows us to skip the detailed $c_k$ accounting and jump straight to the linear structure.
+
+---
+
+## 2. Comparing Growth: $\log n$ vs $n$
+
+**Claim**: $\log n < n$ for all $n \ge 1$.
+
+**Proof (Induction)**:
+
+* **Base Case**: $n=1$. $\log 1 = 0 < 1$. True.
+* **Hypothesis**: Assume $\log k < k$.
+* **Step**: Prove for $n = k+1$.
+We know $\log(k+1) < \log k + 1$.
+By IH, $\log k < k$.
+Therefore, $\log(k+1) < k + 1$.
+
+---
+
+## 3. The Binary Search Recurrence
+
+Consider the recurrence:
+$$ T(n) = T(n/2) + c $$
+This arises in algorithms like **Binary Search**, where we do constant work to discard half the input.
+
+### 3.1 Recursion Tree
+
+* Level 0: Cost $c$. Problem size $n$.
+* Level 1: Cost $c$. Problem size $n/2$.
+* Level 2: Cost $c$. Problem size $n/4$.
+* ...
+* Level $h$: Cost $c$. Problem size $1$.
+
+Height $h = \log_2 n$.
+Total Cost = (Number of levels) $\times$ (cost per level)
+$$ T(n) \approx c \log_2 n = \Theta(\log n) $$
+
+### 3.2 Substitution Method
+
+Guess $T(n) \le d \log n$.
+$$
+\begin{align*}
+T(n) &= T(n/2) + c \\\\
+&\le d \log(n/2) + c \\\\
+&= d(\log n - 1) + c \\\\
+&= d \log n - d + c
+\end{align*}
+$$
+We need $-d + c \le 0$, so $d \ge c$.
+The guess holds.
+
+---
+
+## References
+
+* **CLRS**: Chapter 2 (Analysis of Algorithms), Chapter 4 (Recurrences).

notes/courses/CSCI-UA-310/R-3.md

@@ -0,0 +1,77 @@
+---
+title: Recitation 3 - Merge Sort Variants & QuickSort
+date: 2026-02-06
+---
+
+## Roadmap
+
+We explore variations of standard algorithms to deepen our understanding of recurrences and worst-case scenarios.
+
+1. **Unequal Split Merge Sort**: Analyzing a 1/3-2/3 split.
+2. **QuickSort Worst Case**: Constructing an input that triggers $O(n^2)$ behavior.
+
+---
+
+## 1. Merge Sort: Unequal Split
+
+Standard Merge Sort splits $n$ into $n/2$ and $n/2$. What if we split into $n/3$ and $2n/3$?
+
+### 1.1 The Recurrence
+
+The algorithm still sorts two subarrays and merges them in linear time.
+$$T(n) = T(n/3) + T(2n/3) + cn$$
+
+### 1.2 Recursion Tree Analysis
+
+* **Work per level**:
+    Level 0: $cn$.
+    Level 1: $c(n/3) + c(2n/3) = cn$.
+    Level $k$: Sum of costs is $cn$.
+* **Depth**:
+    The tree is unbalanced.
+  * **Minimum Depth**: Along the $n/3$ branch. $n/3^h = 1 \implies h = \log_3 n$.
+  * **Maximum Depth**: Along the $2n/3$ branch. $n(2/3)^h = 1 \implies h = \log_{3/2} n$.
+
+Since the work per level is $cn$ and the max depth is logarithmic ($\log_{3/2} n \approx 1.7 \log_2 n$), the total time is still:
+$$T(n) = \Theta(n \log n)$$
+The base of the log affects the constant factor, but not the asymptotic class.
+
+---
+
+## 2. QuickSort Worst-Case Construction
+
+We want to construct an input of size $n$ that causes a deterministic QuickSort (pivot = last element) to run in $\Omega(n^2)$.
+
+### 2.1 The Goal
+
+We need the pivot to always be the maximum (or minimum) of the current subarray. This produces partitions of size $k-1$ and $0$.
+
+### 2.2 Construction Strategy
+
+Let's work backwards from the desired execution trace.
+
+* **Step 1**: We want the pivot (last element) to be the largest, say $n$.
+    Array: $[ \dots, n ]$. Partition leaves $[ \dots ]$ and empty.
+* **Step 2**: In the remaining array of size $n-1$, we want the last element to be the largest remaining, $n-1$.
+    Array: $[ \dots, n-1, n ]$.
+
+**Resulting Input**: Sorted array $[1, 2, 3, \dots, n]$.
+Trace:
+
+1. Pivot $n$. Partition: $[1, \dots, n-1]$ vs [].
+2. Pivot $n-1$. Partition: $[1, \dots, n-2]$ vs [].
+3. ...
+
+**Reverse Sorted Input**: $[n, n-1, \dots, 1]$ with pivot = last ($1$).
+
+1. Pivot $1$. Partition: [] vs $[n, \dots, 2]$.
+2. Pivot $2$. Partition: [] vs $[n, \dots, 3]$.
+This also yields $O(n^2)$.
+
+To protect against this, we use **Randomized QuickSort**, which makes it impossible to design a single "killer input".
+
+---
+
+## References
+
+* **CLRS**: Chapter 4 (Recurrences), Chapter 7 (Quicksort).

notes/courses/MATH-UA-334/13-hypothesis-testing.md

@@ -59,13 +59,10 @@ Suppose we have a sample $X\_1, \dots, X\_n \sim \mathcal{N}(\mu, \sigma^2)$ wit
 
 Whenever we make a decision using a statistical test, we risk making one of two distinct types of errors:
 
-1. **$\alpha$ - Type I Error (False Positive):** We incorrectly reject the null hypothesis $H\_0$ when it is actually true.
-    - The probability of committing a Type I Error is called the **Significance Level**, denoted by $\alpha$.
-    - $\alpha = \prob(\text{Output } H\_1 \mid H\_0 \text{ is true})$.
-2. **$\beta$ - Type II Error (False Negative):** We incorrectly accept the null hypothesis $H\_0$ when the alternative $H\_1$ is actually true.
-    - The probability of a Type II error is denoted by $\beta$.
-    - The **Power** of the test is defined as $1 - \beta$, which is the probability of correctly rejecting $H\_0$ when $H\_1$ is true.
-    - $1 - \beta = \prob(\text{Output } H\_1 \mid H\_1 \text{ is true})$.
+| Truth \ Output | $H\_0$ | $H\_1$ |
+| :--- | :--- | :--- |
+| **$H\_0$ is true** | Correct decision; $\prob(\text{Output } H\_0 \mid H\_0 \text{ is true}) = 1 - \alpha$ | **Type I Error** (False Positive); $\alpha = \prob(\text{Output } H\_1 \mid H\_0 \text{ is true})$ |
+| **$H\_1$ is true** | **Type II Error** (False Negative); $\beta = \prob(\text{Output } H\_0 \mid H\_1 \text{ is true})$ | Correct decision (**Power**); $1 - \beta = \prob(\text{Output } H\_1 \mid H\_1 \text{ is true})$ |
 
 In rigorous statistical practice, it is mathematically impossible to simultaneously minimize both $\alpha$ and $\beta$ for a fixed sample size $n$. The standard frequentist paradigm dictates that we fix the significance level $\alpha$ at a pre-determined, strictly controlled threshold (such as $0.05$ or $0.01$) and then actively seek the specific test that maximizes the statistical power $1 - \beta$.
 

…rses/MATH-UA-334/14-p-value-conf-sets.md …TH-UA-334/14-p-values-confidence-sets.mdnotes/courses/MATH-UA-334/14-p-value-conf-sets.md renamed to notes/courses/MATH-UA-334/14-p-values-confidence-sets.md notes/courses/MATH-UA-334/14-p-value-conf-sets.md renamed to notes/courses/MATH-UA-334/14-p-values-confidence-sets.md

@@ -1,5 +1,5 @@
 ---
-title: Generalized LR Tests
+title: Generalized LR Test
 date: 2026-03-23
 ---
 
@@ -221,4 +221,4 @@ Here, the full space has $n$ parameters and the null space has $1$ parameter. By
 ## References
 
 1. Rice, J. A. (2007). *Mathematical Statistics and Data Analysis* (3rd ed.). Thomson Brooks/Cole.
-2. Han, Y. (2026). Lecture 15: Generalized LR Test.
+2. Han, Y. (2026). Lecture 15: Generalized LR Test.