Skip to content

Create data provider integration examples (Snowflake, Exa, Tavily, Apify) #23

@r-marques

Description

@r-marques

Description

Create starter kit examples that integrate with real data providers, demonstrating how to monetize access to external data sources using Nevermined.

Data Providers to Consider

Provider Type Use Case Starter Kit
Snowflake Data Warehouse Enterprise data queries Kit B (Seller)
Exa AI Search Semantic web search Kit A (Buyer) / Kit B (Seller)
Tavily AI Search Real-time web research Kit A (Buyer) / Kit B (Seller)
Apify Web Scraping Structured data extraction Kit B (Seller)

Tasks

Research & Planning

  • Review each provider's API and pricing model
  • Identify which providers have free tiers for hackathon use
  • Determine best fit for each starter kit

Implementation (Pick 1-2 providers)

  • Create Exa/Tavily search agent with x402 monetization
  • Create Apify scraping agent with x402 monetization
  • Create Snowflake query agent with x402 monetization (if feasible)

Documentation

  • Document API key setup for each provider
  • Add provider-specific examples to relevant starter kits
  • Create integration patterns guide

Example: Tavily Search Agent

import { Payments } from "@nevermined-io/payments";
import { paymentMiddleware } from "@nevermined-io/payments/express";
import { TavilyClient } from "tavily";

const tavily = new TavilyClient({ apiKey: TAVILY_API_KEY });

app.use(paymentMiddleware(payments, {
  "POST /search": { planId: PLAN_ID, credits: 1 },
  "POST /research": { planId: PLAN_ID, credits: 5 },
}));

app.post("/search", async (req, res) => {
  const results = await tavily.search(req.body.query);
  res.json({ results });
});

Example: Apify Actor Agent

import { ApifyClient } from "apify-client";

app.post("/scrape", async (req, res) => {
  const run = await apify.actor("apify/web-scraper").call({
    startUrls: [{ url: req.body.url }],
  });
  const dataset = await apify.dataset(run.defaultDatasetId).listItems();
  res.json({ data: dataset.items });
});

Acceptance Criteria

  • At least one working data provider integration
  • Clear setup instructions for API keys
  • Demonstrates x402 payment flow with real data
  • Can be used as reference for hackathon participants

Resources

Related Issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions