Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Contributing to AdLint

Thanks for helping improve AdLint. The project is local-first decision-support software for preflight ad, landing-page, brand-safety, privacy, and disclosure checks.
Thanks for helping improve AdLint. The project is local-first decision-support software for preflight ad, landing-page, brand-safety, privacy, and disclosure checks. The current OSS project goal is documented in `docs/open_source_goal.md`.

## Good first contributions

Expand Down
15 changes: 8 additions & 7 deletions PRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,8 +165,9 @@ Implemented platform modules:
risk.
- TikTok Ads: misleading content, weight-management claims, and disclosure risk.
- LinkedIn Ads: sensitive targeting, discrimination, and professional claims.

Meta remains a future platform module.
- Meta Ads: selected personal-attribute heuristics, health and appearance results, health/wellness
age-targeting review, financial-services authorization, Special Ad Category
review, private-information requests, and branded-content disclosure.

### 7.3 Health privacy risk

Expand Down Expand Up @@ -387,7 +388,7 @@ Representative output shape:
| FR-1 | Implemented | Users can submit ad copy with headline, body, and CTA through CLI, API, or the Python engine. |
| FR-2 | Implemented | Users can provide an optional `landing_page_url` or `landing_page_html`. |
| FR-3 | Partial | The system extracts title, headings, visible claims, forms, pricing text, disclaimers, and trackers from static HTML. JavaScript rendering and richer extraction are future work. |
| FR-4 | Implemented | Users can select `google`, `tiktok`, or `linkedin` policy behavior through platform metadata. Meta is future work. |
| FR-4 | Implemented | Users can select `google`, `tiktok`, `linkedin`, or `meta` policy behavior through platform metadata. |
| FR-5 | Implemented | Users can select industries such as `health`, `wellness`, `finance`, `saas`, `creator`, or `general`. |
| FR-6 | Implemented | Deterministic rule checks run first using policy signals, regexes, keyword patterns, and heuristics. |
| FR-7 | Partial | Ollama classification can run as a hybrid pass behind `model_enabled` or `--enable-model`; deterministic rules always run, and live model quality is not yet benchmarked. |
Expand Down Expand Up @@ -599,7 +600,7 @@ The current eval runner reports:

### 14.2 Future benchmark

Create 200-500 labeled examples over time across several axes.
Maintain and expand the 209-example labeled benchmark over time across several axes.

Decision labels:

Expand Down Expand Up @@ -643,7 +644,7 @@ Future benchmark reports should include:
- Risk scoring.
- JSON and Markdown reports.
- Deterministic safer rewrites.
- 50 curated seed eval examples.
- 58 curated seed eval examples.
- Documentation and legal boundary notes.
- Opt-in JSONL logging.

Expand Down Expand Up @@ -671,7 +672,7 @@ Future benchmark reports should include:

### Phase 4: Evals and benchmark

- 200-500 labeled examples.
- Maintain and expand the 209-example labeled benchmark.
- Confusion matrix.
- False positive and false negative review notes.
- Rule-only vs. model-only vs. hybrid comparison.
Expand Down Expand Up @@ -754,7 +755,7 @@ AdLint becomes successful beyond the MVP if:
- A user can analyze an ad and landing page in under 60 seconds on a local
Apple Silicon workstation with adequate memory.
- The Web UI makes the main review workflow accessible to non-engineers.
- A 200-500 example benchmark shows stable recall for high-severity health,
- A maintained 209+ example benchmark shows stable recall for high-severity health,
privacy, and safety categories.
- Teams can tune scoring with `scoring.yml` without editing code.
- Local model use is benchmarked and documented with clear limitations.
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ make dev # install and run the high-risk example, writing reports/
make scan # install and run the wellness example
make api # start uvicorn with adlint.api:app
make eval # run the seed evals and write evals/results/latest.json
make benchmark # run the 200-row synthetic policy regression benchmark
make benchmark # run the 209-row synthetic policy regression benchmark
make policy-coverage # refresh docs/policy_coverage_matrix.md
make policy-coverage-validate # check the committed coverage matrix
make rewrite-quality # run the deterministic rewrite-quality rubric eval
Expand Down Expand Up @@ -318,7 +318,7 @@ Run the seed evals:
make eval
```

The seed dataset has 54 examples across health, wellness, finance, SaaS,
The seed dataset has 58 examples across health, wellness, finance, SaaS,
creator disclosure, privacy, landing-page mismatch, brand-safety, and Meta
platform-policy cases. It is a development sanity check, not a production
benchmark.
Expand Down Expand Up @@ -464,15 +464,17 @@ platform-specific examples, documentation, and tests for edge cases. Start with

High-value contribution areas:

- Meta Ads policy coverage.
- Deeper Meta Ads parity, including additional restricted verticals and placement-specific cases.
- More public-source/paraphrased eval cases.
- Landing-page extraction improvements.
- Safer rewrite-quality evaluation.
- Docs, examples, screenshots, and launch polish.

## Related docs

- `docs/open_source_goal.md`
- `docs/policy_design.md`
- `docs/meta_ads_scope.md`
- `docs/legal_disclaimer.md`
- `docs/local_models.md`
- `docs/eval_report.md`
Expand Down
87 changes: 87 additions & 0 deletions adlint/policies/platform_meta_ads.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Initial Meta Ads heuristic coverage for local preflight review.
# Source notes: see docs/meta_ads_scope.md for official Meta references, reviewed scope,
# and explicit non-goals. These rules are decision support, not Meta approval guarantees.

policies:
- id: meta_personal_attributes_health
severity: high
Expand Down Expand Up @@ -47,9 +51,92 @@ policies:
- melts fat
- belly fat
- dramatic results
- body-shaming
- perfect body
- hate your body
recommended_action: Avoid transformation framing and use qualified wellness-support language.
rewrite_strategy: wellness_support

- id: meta_health_wellness_age_targeting_review
severity: medium
category: platform_policy
description: Meta health, weight-loss, cosmetic, sexual-health, and reproductive-health ads can require 18+ targeting review.
modules: [platform]
platforms: [meta]
industries: [health, wellness]
requires_review: true
signals:
- weight loss pills
- weight loss supplement
- cosmetic procedure
- botox
- reproductive health
- contraception
- family planning
- sexual health
recommended_action: Confirm age targeting and health/wellness policy eligibility before launch.
rewrite_strategy: add_review_reminder

- id: meta_financial_services_authorization_review
severity: medium
category: platform_policy
description: Meta financial product ads can require authorization, licensing, disclosures, and 18+ targeting review.
modules: [platform]
platforms: [meta]
industries: [finance]
requires_review: true
signals:
- credit card application
- apply for credit
- apply for a loan
- get a loan
- loan approval
- insurance quote
- investment opportunity
- mortgage application
- refinance today
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Widen refinance trigger for Meta finance authorization checks

This signal is matched as an exact phrase, and _signal_to_regex escapes literals before compiling the pattern, so only copy containing refinance today will trigger this rule. That creates false negatives for common refinance ads such as “refinance your mortgage” that should still be covered by meta_financial_services_authorization_review based on its description.

Useful? React with 👍 / 👎.

- cash advance
recommended_action: Confirm financial-services authorization, required disclosures, and 18+ targeting before launch.
rewrite_strategy: add_review_reminder

- id: meta_special_ad_category_review
severity: medium
category: platform_policy
description: Meta campaigns for housing, employment, or financial products/services may require Special Ad Category configuration and targeting limits.
modules: [platform]
platforms: [meta]
requires_review: true
signals:
- housing opportunity
- apartment rental
- mortgage application
- job opening
- hiring now
- apply for this role
- employment opportunity
- financial products and services
- credit offer
recommended_action: Confirm Special Ad Category selection, country settings, and audience restrictions before launch.
rewrite_strategy: add_review_reminder

- id: meta_private_information_request
severity: high
category: platform_policy
description: Meta ads should not request financial, health, or other private information without appropriate permission and review.
modules: [platform]
platforms: [meta]
requires_review: true
signals:
- enter your credit score
- share your bank account
- provide your income
- submit your medical history
- upload your diagnosis
- tell us your symptoms
- provide your health information
recommended_action: Remove private-information requests from ad creative and review the landing-page intake flow.
rewrite_strategy: add_review_reminder

- id: meta_branded_content_disclosure
severity: medium
category: platform_policy
Expand Down
2 changes: 1 addition & 1 deletion docs/adlint_hybrid_eval_paper.tex
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ \section{Evaluation Design}
\toprule
\textbf{Layer} & \textbf{Purpose} & \textbf{Gate type} \\
\midrule
\texttt{rule\_benchmark\_v1.jsonl} & Fast policy-engine regression coverage over 200 deterministic examples. & CI suitable \\
\texttt{rule\_benchmark\_v1.jsonl} & Fast policy-engine regression coverage over 209 deterministic examples. & CI suitable \\
\dataset{} & Public-source diagnostic set with exact 25/25/25 decision balance and strict provenance metadata. & CI schema and rule-only diagnostic \\
\texttt{real-cases-model-quality} & Full live \modelname{} all-mode comparison with model-only, rule-only, and hybrid metrics. & Manual or scheduled quality run \\
\bottomrule
Expand Down
2 changes: 1 addition & 1 deletion docs/adlint_paper_figure_proposals.tex
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ \section*{\sffamily AdLint Paper Figure Replacement Proposals}
};
\end{axis}
\end{tikzpicture}
\caption{Decision-label distribution in the 200-example rule benchmark.}
\caption{Decision-label distribution in the 209-example rule benchmark.}
\label{fig:decision-distribution}
\end{figure}

Expand Down
10 changes: 5 additions & 5 deletions docs/eval_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ Status: deterministic rule benchmark v1.

AdLint includes four labeled JSONL datasets:

- `evals/datasets/seed_ads.jsonl`: the 54-example smoke set.
- `evals/datasets/rule_benchmark_v1.jsonl`: a 200-example benchmark generated
- `evals/datasets/seed_ads.jsonl`: the 58-example smoke set.
- `evals/datasets/rule_benchmark_v1.jsonl`: a 209-example benchmark generated
from the seed set plus policy-author authored synthetic variants.
- `evals/datasets/real_cases_v1.jsonl`: a 75-example public-source diagnostic
set balanced across 25 approved, 25 needs-review, and 25 high-risk expected
Expand All @@ -29,7 +29,7 @@ Rebuild the committed benchmark dataset:
make benchmark-data
```

Run the 200-example benchmark and write JSON plus Markdown reports:
Run the 209-example benchmark and write JSON plus Markdown reports:

```bash
make benchmark
Expand Down Expand Up @@ -201,8 +201,8 @@ Current category-level precision and recall:

Interpretation: the 1.000 score is strong evidence that the deterministic
rules and current benchmark labels are internally consistent. It is not a
claim that future ads will pass review with 100% accuracy. If the 200 examples
were a representative random sample, 200/200 correct decisions would imply an
claim that future ads will pass review with 100% accuracy. If the 209 examples
were a representative random sample, 209/209 correct decisions would imply an
approximate 95% Wilson lower bound of 0.981, but this benchmark is authored
regression coverage rather than a random production sample.

Expand Down
70 changes: 70 additions & 0 deletions docs/meta_ads_scope.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Meta Ads policy scope

AdLint's Meta module is an initial, conservative preflight surface for campaign
review. It is not a complete implementation of Meta's Advertising Standards and
it does not guarantee Meta approval.

## What the current module covers

The bundled `platform_meta_ads.yml` rules focus on high-signal patterns that are
useful before a growth team sends creative to review:

- **Selected personal attributes**: health/body and vulnerable-finance wording that
can imply knowledge of the viewer's condition or status. The current module
does not yet cover every Meta personal-attribute class such as religion, race,
disability, gender identity, sexual orientation, or trade-union membership.
- **Health and appearance results**: transformation, weight-loss, and negative
self-perception framing.
- **Health/wellness age-targeting review**: weight-loss, cosmetic, sexual-health,
and reproductive-health terms that should trigger human review before launch.
- **Financial services authorization review**: credit, loan, insurance,
investment, refinance, and cash-advance offers that may require 18+ targeting,
disclosures, licensing, or authorization checks.
- **Special Ad Category review**: housing, employment, and financial-products
contexts that may require Meta campaign-level category settings and targeting
limits. This check is intentionally not gated by advertiser vertical because
healthcare, SaaS, education, or other advertisers can still run employment,
housing, or credit campaigns.
- **Private information requests**: ad copy asking for health, financial, or
similarly private information. This check is intentionally not gated by
advertiser vertical because sensitive-data requests can appear in otherwise
general campaigns.
- **Branded content disclosure**: sponsorship, affiliate, promo-code, and paid
partnership language.

## Source references

Use these official Meta references when extending the module:

- Meta Advertising Standards overview:
<https://transparency.meta.com/policies/ad-standards/>
- Financial and Insurance Products and Services:
<https://transparency.meta.com/policies/ad-standards/restricted-goods-services/financial-services/>
- Discriminatory Practices and Special Ad Category context:
<https://transparency.meta.com/policies/ad-standards/unacceptable-content/discriminatory-practices>
- Marketing API Special Ad Categories:
<https://developers.facebook.com/docs/marketing-api/audiences/special-ad-category/>
- Branded Content Policies:
<https://www.facebook.com/business/help/221149188908254>

## Deliberate limitations

- Rules are deterministic phrase and pattern checks, not a legal or platform
approval model.
- Synthetic benchmark rows are regression coverage, not production accuracy
claims.
- The module intentionally routes ambiguous regulated-category copy to
`needs_review` rather than trying to decide eligibility automatically.
- Campaign-level fields such as actual Meta objective, placement, age targeting,
country targeting, and `special_ad_categories` are not modeled yet.
- Landing-page mismatch is currently handled by AdLint's generic landing-page
module rather than a Meta-specific policy id.

## Good next contributions

- Add public-source, paraphrased Meta Ad Library and Meta policy examples.
- Add explicit campaign metadata fields for age range, country, and special ad
category selection, then make review rules conditional on those fields.
- Add Meta-specific landing-page and destination-quality policy ids beyond the generic landing-page mismatch rule.
- Split regulated finance review into offer, education, and brand-awareness
subcases with stronger false-positive fixtures.
58 changes: 58 additions & 0 deletions docs/open_source_goal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Open-source project goal

## Working prompt

Make AdLint a legitimately useful open-source, local-first preflight tool for
growth and marketing teams before they ship ads or landing pages.

Prioritize work that makes the project more trustworthy to a stranger landing on
GitHub:

1. **Useful out of the box**: clear install path, runnable examples, CLI/API/Web
UI workflows, and reports a marketer can understand.
2. **Evidence-backed policies**: platform, disclosure, privacy, health, finance,
and brand-safety rules with source notes, scoped claims, and conservative
review language.
3. **False-positive discipline**: every broad rule needs near-miss tests or eval
rows so benign education, tooling, or planning content is not overflagged.
4. **Transparent quality gates**: benchmark, seed eval, real-case eval, policy
coverage, and PR preflight should make regressions obvious.
5. **Local-first trust**: no default raw ad persistence, no secret data in tests,
no live ad-account mutations, and no legal/platform approval guarantees.

## Near-term OSS roadmap

### 1. Meta Ads credibility

- Keep the Meta module framed as initial heuristic coverage, not policy parity.
- Add source-linked docs and reviewed-date notes when new Meta rules land.
- Expand from synthetic triggers to paraphrased, public-source examples where
safe and legally usable.
- Split broad regulated-category checks into higher-precision subcases.

### 2. Contributor-friendly policy work

- Add one example, one positive eval, and one near-miss eval for each new policy.
- Prefer policy IDs that describe the review reason, not a vague platform bucket.
- Require recommended actions that tell a marketer what to change or verify.

### 3. Product relevance

- Improve first-run experience: demo configs, screenshots/GIFs, and concise
report examples.
- Make landing-page mismatch and disclosure checks easy to demo in the CLI and
Web UI.
- Keep README language practical: who uses this, what it catches, what it does
not promise.

### 4. Research/eval credibility

- Treat synthetic benchmarks as regression tests, not accuracy claims.
- Keep adding real-case/adjudicated datasets without private data.
- Track false positives and false negatives explicitly in CI-facing checks.

## Definition of solid

AdLint is “solid OSS” when a new contributor can clone it, run examples, trust
its privacy posture, understand policy scope, add a rule with tests, and see CI
catch both missed risky cases and noisy overtriggering.
Loading
Loading