Add Craigslist mirror

.gitignore

@@ -9,8 +9,11 @@ sites/*/static/external_cache/
 # =============================================================
 # Intermediate / volatile — never committed anywhere.
 # =============================================================
-sites/*/scraped_data/      # scrape pipeline intermediate; runtime data lives in instance_seed/*.db
-sites/*/instance/          # rebuilt at every container boot from instance_seed/
+# Scrape pipeline intermediate; runtime data lives in instance_seed/*.db.
+sites/*/scraped_data/
+
+# Rebuilt at every container boot from instance_seed/.
+sites/*/instance/
 sites/*/venv/
 
 # HF download metadata produced by `hf download`.
@@ -35,6 +38,7 @@ __pycache__/
 .mypy_cache/
 .ruff_cache/
 .tox/
+.playwright-cli/
 .coverage
 htmlcov/
 dist/
@@ -92,4 +96,4 @@ secrets.json
 # ============================================================
 # Agent demo results
 # =============================================================
-agent_demo/runs/
+agent_demo/runs/

Dockerfile

@@ -1,5 +1,5 @@
 # WebHarbor — slim, self-contained image.
-# 15 Flask mirror sites + control plane on :8101.
+# 16 Flask mirror sites + control plane on :8101.
 
 FROM python:3.12-slim-bookworm
 
@@ -33,6 +33,6 @@ COPY control_server.py  /opt/control_server.py
 COPY site_runner.py     /opt/site_runner.py
 RUN chmod +x /opt/websyn_start.sh
 
-EXPOSE 8101 40000-40014
+EXPOSE 8101 40000-40015
 
 CMD ["/opt/websyn_start.sh"]

README.md

@@ -36,17 +36,17 @@ WebHarbor takes a different approach. We leverage coding agent (e.g., Claude Cod
 - **Deep features unlocked** — carts, checkouts, accounts, all fully testable
 - **Evolving** — harder tasks drive richer mirrors; the environment grows with agents
 - **RL-ready** — sub-second database resets between rollouts
-- **Community-driven** — 15 sites today, scaling to 100+ together
+- **Community-driven** — 16 sites today, scaling to 100+ together
 
 ## 🚀 Quickstart
 
 One command to run all web environments:
 
 ```bash
-docker run -p 8101:8101 -p 40000-40014:40000-40014 battalion7244/webharbor:latest
+docker run -p 8101:8101 -p 40000-40015:40000-40015 battalion7244/webharbor:latest
 ```
 
-Then point your agent at `http://localhost:40000` through `http://localhost:40014` to explore 15 local mirrors of webvoyager sites: `Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, GitHub, Google Flights, Google Maps, Google Search, Hugging Face, Wolfram Alpha, Cambridge Dictionary, Coursera, and ESPN`.
+Then point your agent at `http://localhost:40000` through `http://localhost:40015` to explore 16 local mirrors of webvoyager sites: `Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, GitHub, Google Flights, Google Maps, Google Search, Hugging Face, Wolfram Alpha, Cambridge Dictionary, Coursera, ESPN, and Craigslist`.
 
 For sub-second reset between rollouts, expose the control plane and call `/reset/<site>`:
 
@@ -65,7 +65,7 @@ git clone https://github.com/aiming-lab/WebHarbor && cd WebHarbor
 
 ## 🤝 Contribute
 
-We have built 15 high-quality mirrors covering the [WebVoyager](https://github.com/MinorJerry/WebVoyager) benchmark. The next goal is **100+ sites**, covering everything in [Online-Mind2Web](https://huggingface.co/datasets/osunlp/Online-Mind2Web). We are inviting the community to build this together.
+We have built 16 high-quality mirrors covering the [WebVoyager](https://github.com/MinorJerry/WebVoyager) benchmark. The next goal is **100+ sites**, covering everything in [Online-Mind2Web](https://huggingface.co/datasets/osunlp/Online-Mind2Web). We are inviting the community to build this together.
 
 There are two ways to join the author list:
 
@@ -111,4 +111,4 @@ WebHarbor is initiated by UNC-Chapel Hill and Microsoft, with contributions from
   url          = {https://aiming-lab.github.io/webharbor.github.io},
   note         = {Project website.}
 }
-```
+```

control_server.py

@@ -26,7 +26,7 @@
     'allrecipes', 'amazon', 'apple', 'arxiv', 'bbc_news', 'booking',
     'github', 'google_flights', 'google_map', 'google_search',
     'huggingface', 'wolfram_alpha', 'cambridge_dictionary',
-    'coursera', 'espn',
+    'coursera', 'espn', 'craigslist',
 ]
 BASE_PORT = 40000
 WEBSYN_DIR = '/opt/WebSyn'

sites/craigslist/_health.py

@@ -0,0 +1,3 @@
+"""Per-site health probe (optional, called by control_server)."""
+def health():
+    return {"ok": True, "site": "craigslist"}