Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .github/workflows/scrape-feeds.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,19 @@ jobs:
with:
token: ${{ github.token }}

- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 9

- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: '20'
cache: 'npm'
cache: 'pnpm'

- name: Install dependencies
run: npm ci
run: pnpm install

- name: Run scraper
env:
Expand Down
12 changes: 6 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,12 @@ Update the `ACCESS_TOKEN_HASH` constant in `public/index.html` with the output.
```bash
cp .env.example .env
# Edit .env and add your actual API keys
npm install
pnpm install
```

2. **Run locally**:
```bash
npm start
pnpm start
# Server runs on http://localhost:3000
# Access the frontend and test API endpoints
```
Expand All @@ -116,7 +116,7 @@ Update the `ACCESS_TOKEN_HASH` constant in `public/index.html` with the output.

2. **Test locally** first:
```bash
npm start
pnpm start
node src/scraper-cli.js
```

Expand Down Expand Up @@ -261,7 +261,7 @@ schedule:

Before pushing changes:

- [ ] Run `npm start` and verify server starts
- [ ] Run `pnpm start` and verify server starts
- [ ] Visit `http://localhost:3000` and verify feed loads
- [ ] Test access token gate (clear localStorage and reload)
- [ ] Test all source filters (Twitter, Reddit, GitHub)
Expand All @@ -284,7 +284,7 @@ Currently no automated tests. To add:
3. Write unit tests for scrapers
4. Write integration tests for feed aggregation
5. Add to `package.json`: `"test": "jest"`
6. Run `npm test` before commits
6. Run `pnpm test` before commits

## Common Issues & Solutions

Expand Down Expand Up @@ -351,7 +351,7 @@ When deploying to a new environment:
5. Check for security alerts (Dependabot)

**Monthly maintenance**:
1. Update dependencies: `npm update`
1. Update dependencies: `pnpm update`
2. Review and clean old feed data if growing large
3. Audit access logs if needed
4. Rotate access tokens if compromised
Expand Down
12 changes: 6 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Thank you for your interest in contributing! This project is open source and we
```bash
cp .env.example .env
# Add your API keys to .env
npm install
pnpm install
```
4. **Create a branch:**
```bash
Expand All @@ -36,10 +36,10 @@ Thank you for your interest in contributing! This project is open source and we

```bash
# Run the local server
npm start
pnpm start

# Test the scraper
npm run scrape
pnpm run scrape

# Check the output
cat docs/data/feed.json
Expand All @@ -56,8 +56,8 @@ cat docs/data/feed.json

Before submitting:

1. **Test locally** - Run `npm start` and verify everything works
2. **Test scraping** - Run `npm run scrape` and check output
1. **Test locally** - Run `pnpm start` and verify everything works
2. **Test scraping** - Run `pnpm run scrape` and check output
3. **Check for secrets** - Run `git diff` and ensure no API keys are included
4. **Verify the frontend** - Open http://localhost:3000 and test the UI

Expand Down Expand Up @@ -161,7 +161,7 @@ Want to add a new social platform? Here's how:
- Verify token names match `.env.example`

### "Cannot find module"
- Run `npm install` to ensure all dependencies are installed
- Run `pnpm install` to ensure all dependencies are installed
- Check Node.js version (requires 18+)

### "GitHub Actions Failing"
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ A private social media feed aggregator for tracking mentions across Twitter, Red
### Local Development

```bash
npm install
pnpm install
cp .env.example .env # Add your API tokens
npm start # Visit http://localhost:3000
pnpm start # Visit http://localhost:3000
```

### GitHub Pages Deployment
Expand Down
Loading