AI-powered research paper analysis with evidence-based evaluation
An intelligent agent that analyzes research papers by extracting truth claims, evaluating evidence, identifying logical fallacies, and providing balanced assessments with quality ratings. Built on the Bindu Agent Framework for the Internet of Agents.
Key Capabilities:
- 🔍 Extracts and evaluates truth claims from research papers
- ✅ Provides supporting and refuting evidence with verifiable references
- 🚨 Identifies logical fallacies with examples
- 📊 Assigns quality ratings (A-F scale) to claims
- ⚖️ Generates balanced, centrist-oriented analysis
- Python 3.10+
- uv package manager
- API keys for OpenRouter and Mem0 (both have free tiers)
# Clone the repository
git clone https://github.com/raahulrahl/analyze-paper-agent.git
cd analyze-paper-agent
# Create virtual environment
uv venv --python 3.12.9
source .venv/bin/activate # On Windows: .venv\Scripts\activate
# Install dependencies
uv sync
# Configure environment
cp .env.example .envEdit .env and add your API keys:
| Key | Get It From | Required |
|---|---|---|
OPENROUTER_API_KEY |
OpenRouter | ✅ Yes |
MEM0_API_KEY |
Mem0 Dashboard | If you want to use Mem0 tools |
# Start the agent
uv run python -m analyze_paper_agent
# Agent will be available at http://localhost:3773# Analyze a research paper
"concise summary https://arxiv.org/abs/1706.03762"
# Evaluate arguments and evidence
"Evaluate the methodology and evidence in this medical study"
# Identify logical fallacies
"Identify logical fallacies in this economic policy paper"Plain Text:
Analyze this paper: [paste your research paper text here]
JSON:
{
"content": "Research paper text or argument",
"focus": "claims"
}The agent returns structured analysis with:
- Argument Summary: Brief overview (< 30 words)
- Truth Claims: Each claim with:
- Supporting evidence with references
- Refuting evidence with references
- Logical fallacies identified
- Quality rating (A-F)
- Characterization labels
- Overall Score: Lowest, highest, and average claim scores
- Overall Analysis: Summary with actionable recommendations
Input Query:
Identify logical fallacies in this paper https://arxiv.org/abs/1706.03762
Output Sample:
## ARGUMENT SUMMARY:
Self-attention alone can replace recurrence and convolution while achieving
superior translation performance with greater efficiency.
## TRUTH CLAIMS:
### CLAIM 1
#### CLAIM:
Self-attention architectures outperform recurrent and convolutional models
for machine translation.
#### CLAIM SUPPORT EVIDENCE:
- Higher BLEU scores on WMT 2014 English–German and English–French tasks
- Vaswani et al. (2017) report BLEU 28.4 (En-De) exceeding prior benchmarks
- Results reproduced and extended in subsequent work
**References:**
- Vaswani et al., 2017, *Attention Is All You Need*, arXiv:1706.03762
- Gehring et al., 2017, *Convolutional Sequence to Sequence Learning*
#### CLAIM REFUTATION EVIDENCE:
- Performance gains depend on hyperparameter tuning, dataset size, and compute
- Hybrid models and improved RNNs can match or exceed early Transformer results
- BLEU improvements were modest and within variance ranges
**References:**
- Britz et al., 2017, *Massive Exploration of Neural Machine Translation*
- Melis et al., 2018, *On the State of the Art of Evaluation in Neural Language Models*
#### LOGICAL FALLACIES:
- **Overgeneralization**: "We propose a new simple network architecture…
that relies solely on attention mechanisms."
- **Selection Bias**: Focus on benchmarks where Transformers perform best.
#### CLAIM RATING:
**B (High)**
#### LABELS:
Empirically supported, overgeneralized, benchmark-dependent
---
### CLAIM 2
#### CLAIM:
Recurrence and convolution are unnecessary for modeling sequence dependencies.
#### LOGICAL FALLACIES:
- **False Dichotomy**: "Without recurrence or convolution…" implies
exclusivity where hybrids exist.
- **Scope Overshoot**: Generalizing from translation to all sequence modeling.
#### CLAIM RATING:
**C (Medium)**
#### LABELS:
Reductionist, overextended, architectural absolutismKey Features Demonstrated:
- ✅ ArxivTools integration for automatic paper retrieval
- ✅ Both supporting AND refuting evidence with verifiable references
- ✅ Logical fallacy identification with quoted examples
- ✅ A-F quality ratings per claim
- ✅ Characterization labels for balanced assessment
The agent exposes a RESTful API when running. Default endpoint: http://localhost:3773
For complete API documentation, request/response formats, and examples, visit:
📚 Bindu API Reference - Send Message to Agent
Primary Capability:
- Analyzes truth claims and arguments with evidence-based evaluation
- Extracts verifiable claims from research papers
- Provides balanced perspectives with both supporting and refuting evidence
Features:
- Evidence verification with external sources
- Logical fallacy detection
- Quality scoring system (A-F scale)
- Comprehensive claim characterization
- Balanced, centrist-oriented analysis
Best Used For:
- Evaluating research papers for claim validity
- Fact-checking academic arguments
- Peer review assistance
- Getting balanced perspectives on controversial claims
Not Suitable For:
- Simple summarization (use a summarization skill instead)
- Creative writing or content generation
- Real-time fact-checking without verification time
Performance:
- Average processing time: ~5 seconds
- Max concurrent requests: 5
- Memory per request: 512MB
# Build and run with Docker Compose
docker-compose up --build
# Agent will be available at http://localhost:3773The agent runs on port 3773 and requires:
OPENROUTER_API_KEYenvironment variableMEM0_API_KEYenvironment variable
Configure these in your .env file before running.
# Use production compose file
docker-compose -f docker-compose.prod.yml up -dMake your agent discoverable worldwide and enable agent-to-agent collaboration.
# Authenticate with GitHub
gh auth login
# Set deployment secrets
gh secret set BINDU_API_TOKEN --body "<your-bindu-api-key>"
gh secret set DOCKERHUB_TOKEN --body "<your-dockerhub-token>"Get your keys:
- Bindu API Key: bindus.directory dashboard
- Docker Hub Token: Docker Hub Security Settings
# Push to trigger automatic deployment
git push origin mainGitHub Actions will automatically:
- Build your agent
- Create Docker container
- Push to Docker Hub
- Register on bindus.directory
analyze-paper-agent/
├── analyze_paper_agent/
│ ├── skills/
│ │ └── analyze-paper/
│ │ ├── skill.yaml # Skill configuration
│ │ └── __init__.py
│ ├── __init__.py
│ ├── __main__.py
│ ├── main.py # Agent entry point
│ └── agent_config.json # Agent configuration
├── tests/
│ └── test_main.py
├── .env.example
├── docker-compose.yml
├── Dockerfile.agent
└── pyproject.toml
make test # Run all tests
make test-cov # With coverage reportmake format # Format code with ruff
make lint # Run linters
make check # Format + lint + test# Install pre-commit hooks
uv run pre-commit install
# Run manually
uv run pre-commit run -aContributions are welcome! Please follow these steps:
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-feature - Commit your changes:
git commit -m 'Add amazing feature' - Push to the branch:
git push origin feature/amazing-feature - Open a Pull Request
See CONTRIBUTING.md for detailed guidelines.
This project is licensed under the MIT License - see the LICENSE file for details.
Built with the Bindu Agent Framework
Why Bindu?
- 🌐 Internet of Agents: A2A, AP2, X402 protocols for agent collaboration
- ⚡ Zero-config setup: From idea to production in minutes
- 🛠️ Production-ready: Built-in deployment, monitoring, and scaling
Build Your Own Agent:
uvx cookiecutter https://github.com/getbindu/create-bindu-agent.git- 📖 Full Documentation
- 💻 GitHub Repository
- 🐛 Report Issues
- 💬 Join Discord
- 🌐 Agent Directory
- 📚 Bindu Documentation
Built with 💛 by the team from Amsterdam 🌷