Updating workshop to include all the dependencies we will need and to ensure it is able to build without error. Pushing changes now, but will need to continue updating content.

Author: salernos

DESCRIPTION

@@ -19,12 +19,12 @@ Authors@R:
 Description: |
     A workshop demonstrating how to perform downstream statistical inference 
     when the outcome has been partially imputed via a machine learning model.  
-    The central R/Bioconductor package is ipd (Inference with Predicted Data).
+    The central R package is the ipd package (Inference with Predicted Data).
 License: MIT + file LICENSE
 Encoding: UTF-8
 LazyData: true
 Roxygen: list(markdown = TRUE)
-RoxygenNote: 7.1.0
+RoxygenNote: 7.3.2
 Depends:
     R (>= 4.0),
     ipd,
@@ -40,7 +40,22 @@ Imports:
     mgcv,
     pROC,
     ALL,
-    BiocGenerics
+    golubEsets,
+    AnnotationDbi,
+    hgu95av2.db,
+    hu6800.db,
+    BiocGenerics,
+    DALEX,
+    broom,
+    GGally,
+    MLInterfaces,
+    here,
+    janitor,
+    keggorthology,
+    neuralnet,
+    partykit,
+    randomForest,
+    scales
 Suggests:
     DT
 URL: https://salernos.github.io/ipdworkshop/

README.md

@@ -1,52 +1,118 @@
-# BuildABiocWorkshop
+# Introduction and Motivation
 
-This package is a template for building a Bioconductor workshop. The package
-includes Github actions to:
+Artificial intelligence and machine learning (AI/ML) have become essential tools in biomedical research, enabling large-scale analyses across diverse domains such as genomics, structural biology, and electronic health records-based research. Increasingly, researchers rely on model-generated predictions, rather than directly measured variables, as inputs for downstream statistical analyses. For example, predicted gene expression values or polygenic risk scores are often used in place of experimental assays, allowing researchers to expand cohort sizes and explore hypotheses when traditional data collection is infeasible, costly, or time-consuming.
 
-1. Set up bioconductor/bioconductor_docker:devel on Github resources
-2. Install package dependencies for your package (based on the `DESCRIPTION` file)
-3. Run `rcmdcheck::rcmdcheck`
-4. Build a pkgdown website and push it to github pages
-5. Build a docker image with the installed package and dependencies and deploy to [the Github Container Repository](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#pulling-container-images) at the name `ghcr.io/gihub_user/repo_name`, all lowercase. 
+While this practice of "using predictions as data" holds promise for accelerating scientific discovery, it presents significant challenges for statistical inference. When predicted values are used in place of true variables, the resulting estimates of association can be biased and misleading if uncertainty in the prediction step is not properly accounted for. 
 
-## Responsibilities
+In this workshop, we explore the consequences of inference on predicted data across several biomedical applications.  Drawing from classical approaches to measurement error and recent developments in bias correction, we will present a suite of prediction-based inference methods that adjust for prediction-related uncertainty and improve inference validity and efficiency. We will also introduce {ipd}, a user-friendly Bioconductor R package that implements several of these correction methods through a unified interface. The package supports modular integration into existing workflows and includes tidy methods for model inspection and diagnostics.
 
-Package authors are primarily responsible for:
 
-1. Creating a landing site of their choosing for their workshops (a website). This website should be listed in the `DESCRIPTION` file as the `URL`.
-2. Creating a docker image that will contain workshop materials and the installed packages necessary to run those materials. The name of the resulting docker image, including "tag" if desired, should be listed in a non-standard tag, `DockerImage:` in the `DESCRIPTION` file. 
 
-Both of those tasks can be accomplished using the Github actions included in this template package. The vignette accompanying this package describes how to accomplish both of these tasks.
 
-## Details
 
-For detailed instructions, see the `How to build a workshop` article/vignette.
+################################################################################
 
-## Results of successful deployment
 
-- A working docker image that contains the installed package and dependencies.
-- An up-to-date `pkgdown` website at https://YOURUSERNAME.github.io/YOURREPOSITORYNAME/
-- Docker image will be tagged with `latest`, `sha-XXXXXX` where `XXXXXX` is the hash of the current `master` commit, and `master`. 
 
-## To use the resulting image:
 
-```sh
-docker run -e PASSWORD=<choose_a_password_for_rstudio> -p 8787:8787 YOURDOCKERIMAGENAME
-```
-Once running, navigate to http://localhost:8787/ and then login with `rstudio`:`yourchosenpassword`. 
+In many modern data science applications, it is common to encounter settings 
+where measuring a particular outcome, $Y$, is expensive or time-consuming, 
+whereas predictions, $\hat{Y} = f(X)$, from a machine learning model are 
+readily available on largedatasets. The `ipd` package provides a suite of 
+methods to perform valid  statistical inference on when some outcomes are 
+observed (labeled) and others are only predicted.
+
+In this workshop, you will learn:
+
+* The theoretical foundation behind prediction-powered inference (PPI) and its extensions.
+* How to use **ipd** functions to simulate data, fit models, and extract inference results.
+* Practical exercises to compare naive estimators with IPD methods.
+
+By the end, you should be able to design analyses that leverage large unlabeled datasets while maintaining correct uncertainty quantification.
+
+## The Augmented Data Scheme
+
+Consider three sets of observations:
+
+* **Training set**: ${(X_i, Y_i)}*{i=1}^{n*\text{train}}$, used to fit a predictive model $f(\cdot)$.
+* **Labeled set**: ${(X_i, Y_i)}*{i=1}^{n*\ell}$, smaller sample with true outcomes.
+* **Unlabeled set**: ${X_i}*{i=n*\text{train}+n_\ell+1}^{n_\text{train}+n_\ell+n_u}$, only features available.
 
-To try with **this** repository docker image:
+After fitting $f$ on the training set, we apply it to the labeled and unlabeled sets to obtain predictions $f_i = f(X_i)$. We then construct an **augmented dataset**:
 
-```sh
-docker run -e PASSWORD=abc -p 8787:8787 ghcr.io/bioconductor/buildabiocworkshop
+```plaintext
++---------------+      +---------------+      +----------------+
+| Training (T)  | ---> | Labeled (L)   | ---> | Unlabeled (U)  |
+| (X, Y)        |      | (X, Y, f)     |      | (X, f)         |
++---------------+      +---------------+      +----------------+
 ```
 
-*NOTE*: Running docker that uses the password in plain text like above exposes the password to others 
-in a multi-user system (like a shared workstation or compute node). In practice, consider using an environment 
-variable instead of plain text to pass along passwords and other secrets in docker command lines. 
+We treat $f_i$ in the unlabeled set as surrogate outcomes and combine them with observed $Y_i$ in the labeled set to estimate regression parameters $\beta$.
+
+## Key Formulas
+
+### Naive Estimator
+
+Using only the unlabeled predictions, the naive OLS estimator solves
+
+$$
+\hat\beta_{\text{naive}} = \arg\min_\beta \sum_{i\in U} \bigl(f_i - X_i^T\beta\bigr)^2.
+$$
+
+
 
+# This Workshop
 
-## Whatcha get
+Welcome! This workshop provides a brief introduction to performing valid statistical inference when your outcome has been partially imputed by a machine learning model. The central package is [**ipd**](https://bioconductor.org/packages/ipd/), which implements several recent methods for conducting inference with predicted data (IPD).
+
+> **Prerequisites**  
+> 1. R (≥ 4.1) and Bioconductor installed  
+> 2. The `ipd` package:  
+>    ```r
+>    if (!requireNamespace("BiocManager", quietly = TRUE))
+>        install.packages("BiocManager")
+>    BiocManager::install("ipd")
+>    ```
+> 3. Supporting packages:
+>    ```r
+>    install.packages(c("tidyverse", "patchwork", "NHANES", "rashomonquartet", "ranger", "mgcv", "pROC", "ALL"))
+>    BiocManager::install(c("BiocStyle", "Biobase", "BiocGenerics"))
+>    ```
+
+---
+
+## Workshop Structure
+
+There are four hands‐on tutorials (R Markdown vignettes). Each vignette loads or simulates data, trains a prediction model, applies `ipd::ipd()`, and includes exercises:
+
+1. [Chapter 1: Simulated Data](vignettes/01-simulated-data.html)  
+   - Fully synthetic linear‐regression example  
+   - Compare naïve, classical, and PPI/PPI++/PostPI/PSPA methods  
+   - Residual diagnostics and bootstrap coverage
+
+2. [Chapter 2: Rashomon Quartet](vignettes/02-rashomon-quartet.html)  
+   - Illustrate how four datasets with identical summary statistics can behave very differently  
+   - Show why naïve regression on predictions can fail under nonlinearity or outliers  
+   - Compare IPD corrections (PPI, PPI++, PSPA) across R1–R4 scenarios
+
+3. [Chapter 3: NHANES Body Fat vs BMI](vignettes/03-nhanes-bodyfat.html)  
+   - Real‐world data from NHANES (DXA percent body fat vs BMI)  
+   - Fit a linear (or nonlinear) prediction model on labeled participants  
+   - Use IPD to estimate the effect of Age on true percent body fat (correcting for bias)
+
+4. [Chapter 4: Genetic Data (Bioconductor `ALL`)](vignettes/04-genetic-data.html)  
+   - A binary IPD example using leucemia microarray data (`ALL` package)  
+   - Fit a logistic model (CD19‐based) to predict BCR/ABL labels  
+   - Apply IPD to estimate the log‐odds effect of CD38 expression on true BCR/ABL status
+
+---
+
+## How to Build & View Locally
+
+1. Clone or download this repository:
+
+```bash
+git clone https://github.com/salernos/ipdworkshop.git
+cd ipdworkshop
+```
 
-- https://bioconductor.github.io/BuildABiocWorkshop
-- A Docker image that you can run locally, in the cloud, or (usually) even as a singularity container on HPC systems. 

_pkgdown.yml

@@ -1,18 +1,21 @@
 url: https://salernos.github.io/ipdworkshop
 
 template:
-  params:
+  bootstrap: 5
+  lightswitch: true
+  bslib:
     bootswatch: flatly
-    #ganalytics: UA-99999999-9
+    pkgdown-nav-height: 100px
+    primary: "#1B365D"
 
 home:
   title: "Inference with Predicted Data Using the ipd Package"
   type: inverse
 
-
 navbar:
   right:
    - icon: fa-github
      href: https://github.com/salernos/ipdworkshop
 
-
+project:
+  render: ['*.qmd']