initial preparation for decision trees

Author: mibur1

book/2_models/7_SVM.md

@@ -135,9 +135,13 @@ clf = SVC(kernel='linear')
 clf.fit(X, y);
 ```
 
-We can then write a little helper function to visualize the decision function and supports:
+With a little helper function we can visualize the decision function and supports:
 
 ```{code-cell} ipython3
+---
+tags:
+  - hide-input
+---
 def plot_svc_decision_function(model, ax=None):
     """Plot the decision boundary and margins for a trained 2D SVC model."""
     # Set up grid
@@ -181,7 +185,7 @@ fig, ax = plt.subplots()
 sns.scatterplot(x=X[:, 0], y=X[:, 1], hue=y, s=60, ax=ax)
 ax.set(xlabel="Feature 1", ylabel="Feature 2")
 
-# Custom egend
+# Custom legend
 legend_elements = [
     Line2D([0], [0], marker='o', linestyle='None', markersize=8, label='Class 0', markerfacecolor="#0173B2", markeredgecolor='None'),
     Line2D([0], [0], marker='o', linestyle='None', markersize=8, label='Class 1', markerfacecolor="#DE8F05", markeredgecolor='None')]
@@ -317,6 +321,10 @@ As always, hyperparameters should be tuned using [cross-validation](book/1_basic
 
 
 ```{code-cell} ipython3
+---
+tags:
+  - hide-input
+---
 import pandas as pd
 from sklearn.model_selection import GridSearchCV
 

book/2_models/8_Trees.md

@@ -21,4 +21,36 @@ https://mlu-explain.github.io/decision-tree/
 
 https://animlbook.com/classification/trees/index.html
 
-https://mlu-explain.github.io/random-forest/
+https://mlu-explain.github.io/random-forest/
+
+
+
+```{code-cell}ipython3
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+import seaborn as sns
+from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor, plot_tree
+from sklearn.model_selection import (train_test_split, cross_val_score, GridSearchCV, RandomizedSearchCV, cross_validate, KFold)
+from sklearn.metrics import accuracy_score, mean_squared_error
+from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
+from xgboost import XGBClassifier
+```
+
+We will work with two datasets:
+
+- Breast Cancer (binary classification)
+  - Features: 30 real-valued measurements of tumors
+  - Target: malignant (1) vs. benign (0)
+- Diabetes (continuous regression)
+  - Features: 10 baseline variables (age, sex, BMI, …)
+  - Target: a quantitative measure of disease progression
+
+
+```{code-cell}ipython3
+X = [[0, 0], [1, 1]]
+Y = [0, 1]
+
+clf = DecisionTreeClassifier()
+clf = clf.fit(X, Y)
+```

requirements.txt

@@ -1,6 +1,7 @@
 jupyter-book
 numpy
 pandas
+scipy
 matplotlib
 seaborn
 plotly
@@ -12,4 +13,5 @@ statsmodels
 mlxtend
 patsy
 ipywidgets
-ISLP
+ISLP
+xgboost