log_rubric.json

{
   "1. Model Overview": 
            "4: Clear purpose, scope, and context defined with specific objectives.\n3 : Well‑defined but could be more specific.\n2 : Vague or incomplete definition of purpose and scope.\n1 : No clear definition of purpose or scope.",
   
   "2. Stakeholder Identification and Fairness Definition": 
            "4: Diverse stakeholders identified with specific fairness goals set.\n3 : Most stakeholders identified, goals mostly clear.\n2 : Limited stakeholder involvement and fairness goals.\n1 : No stakeholder involvement or unclear fairness goals.",
   
   "3. Data Collection and Processing": 
            "4: Implements appropriate resampling or augmentation strategies that fully corrects class imbalance in the scoped data.\n3 : Applies resampling or augmentation to address the major imbalances, but some subgroups remain underrepresented.\n2 : Identifies class imbalance but uses no or only simple mitigation techniques.\n1 : No recognition or handling of dataset imbalance.",
   
   "4. Bias Detection and Mitigation": 
            "4: Comprehensive bias mitigation strategies (preprocessing, post‑processing, fair representation learning) implemented.\n3 : Some bias mitigation strategies used with minor gaps.\n2 : Only basic bias mitigation approaches used.\n1 : No bias mitigation strategies used.",
   
   "5. Fairness Metric Selection" :  
            "4: Clear, context‑specific metrics chosen with transparent, actionable decisions.\n3 : Good selection of metrics with reasonable transparency.\n2 : Metrics chosen without full alignment with context.\n1 : No clear fairness metrics or criteria selection.",
     
   "6. Model Selection and Training" : 
            "4: Both (a) Fairness mitigation techniques (e.g., bias correction, constraints, adversarial debiasing) built into model selection/training **and** (b) full transparency and explainability through interpretability tools or clear documentation of decision logic(eg. comments).\n3 : (a) Some mitigation techniques applied during training **and** (b) basic transparency (e.g., feature importance reports or high‑level model descriptions), but not both comprehensively.\n2 : (a) Mitigation techniques added late **or** (b) minimal transparency/explainability (e.g., no interpretability tools, only brief comments).\n1 : Neither fairness mitigation techniques during training nor any transparency/explainability measures implemented.",
   
   "7. Evaluation and Testing" : 
            "4: Both (a) comprehensive disaggregated evaluation across all relevant demographic groups and (b) scheduled, regular bias audits with documented follow‑up actions.\n3 : Either (a) solid disaggregated evaluation with minor group gaps or (b) periodic bias audits, but not both fully; documentation is mostly complete.\n2 : Only one activity performed in a limited way (e.g., disaggregated evaluation for a few groups or irregular bias audits); little or no documentation of findings.\n1 : Neither disaggregated evaluation nor bias auditing conducted."
   
}