AI agents and RAGFor AI engineers and agent builders

Live web search and clean Markdown for AI agents and RAG

Give your agent a search engine and a reader. Every Gridlode engine is available as a tool through our remote MCP server, and the page engines turn single URLs or whole sites into compact Markdown that costs far fewer tokens than raw HTML.

Start free with 1,000 credits See the code

The workflow

3 steps, one API key.

Run the steps in order, or call only the one you're missing. Every API links to its parameters, response fields and a live demo.

Step 1: Connect your agent
Add our remote MCP server to Claude, Cursor or any MCP client, and every engine becomes a tool.
Remote MCP server
Add this URL as a remote MCP server in Claude, Cursor or any MCP client and authenticate with your API key. Every live engine shows up as a tool.
https://api.gridlode.com/mcpSetup for Claude and Cursor
Step 2: Search and read
Search the web, then turn result pages or whole sites into compact Markdown.
Try it free: Bing Search Checker
Step 3: Extract structure
Turn pages into JSON with your own schema.
- AI Data ExtractionSoon/v1/ai/extractBy usageBy usage

What you get

Clean, normalised data you can use as-is.

Organic web results and news from Bing and DuckDuckGo
The main content of any page as Markdown, without menus, ads or cookie banners
Whole-site crawls to Markdown, one record per page with title and word count
Article text with author, publish date and reading time
YouTube captions as timed segments and full text
Structured JSON from any page, using your own schema

More APIs for this job

Every response includes

request_id, credits charged and your remaining balance
Cache age, so you know how fresh the data is
The normalised parameters we actually ran
One error format; errors never cost credits

Code

The whole workflow, ready to run.

Each step is one HTTPS request with your key in the Authorization header. The parameters are working examples; swap in your own niche, domains or IDs.

Base URL: https://api.gridlode.com
Auth header: Authorization: Bearer YOUR_API_KEY

Reference for each call

export GRIDLODE_KEY="your_api_key"

# 1. Connect your agent: add https://api.gridlode.com/mcp as a remote MCP server in Claude, Cursor
# or any MCP client (https://gridlode.com/docs/mcp). Its tools run the same calls as below.

# 2. Search and read: Bing Search
curl "https://api.gridlode.com/v1/search/bing?q=python%20programming%20language" \
  -H "Authorization: Bearer $GRIDLODE_KEY"

# then: URL to Markdown
curl "https://api.gridlode.com/v1/url/markdown?url=https%3A%2F%2Fexample.com%2F" \
  -H "Authorization: Bearer $GRIDLODE_KEY"

# 3. Extract structure: AI Data Extraction (launching soon)
curl -X POST "https://api.gridlode.com/v1/ai/extract" \
  -H "Authorization: Bearer $GRIDLODE_KEY" -H "Content-Type: application/json" \
  -d '{"url":"https://www.python.org/downloads/","fields":["latest_version","release_date","download_url"]}'

Prefer no code? Upload a list as a bulk job in the dashboard, or connect the MCP server to your AI assistant.

Have a list? Run it as a bulk job.

Send up to 10,000 inputs at once, for example every URL for step 2 (search and read), and download the results as CSV or JSON. Each item is billed like a single request, failed items are free, and the job stops cleanly at your spend cap.

Upload a CSV in the dashboard, or POST a list to the API
Signed webhook when the job finishes

Upload a list Bulk job docs

POST /v1/tasks · URL to Markdown

curl -X POST "https://api.gridlode.com/v1/tasks" \
  -H "Authorization: Bearer $GRIDLODE_KEY" -H "Content-Type: application/json" \
  -d '{
    "engine": "url.markdown",
    "field": "url",
    "inputs": [
      "https://stripe.com/",
      "https://www.shopify.com/",
      "https://www.notion.so/"
    ],
    "name": "AI agents and RAG: search and read"
  }'

Put any step on a schedule

Any engine can run as a custom monitor: choose the parameters, the fields to watch and the conditions that should alert you by email or signed webhook. Each check is billed like a normal request, and failed checks are free.

Create a monitor Monitor docs

Pricing example

1,000 items for $2.00.

You pay per result, in credits from one wallet that every engine shares. The estimate uses each engine's published price per unit; change the volume to see how it scales.

1,000 free credits when you sign up, no card
Packs never expire; plans only make credits cheaper
Hard monthly spend caps, so jobs stop where you say

Plans and credit packs

Items per step

Step 2 · Search and read
Bing Search
Live1 cr / call
1,000
credits for 1,000 calls
Step 2 · Search and read
URL to Markdown
Live1 cr / call
1,000
credits for 1,000 calls
Step 3 · Extract structure
AI Data Extraction
SoonBy usage
by usage

Total

2,000

credits

At list price

$2.00

$1.00 per 1,000 credits

Enterprise volume

$0.60

from $0.30 per 1,000 credits

Assumes 1,000 units at every step; real chains usually shrink (not every business lists an email). Failed requests are free and cached results cost half. AI Data Extraction is billed by actual model usage with a cap you set, so it is not in the total.

FAQ

AI agents and RAG: questions

Is there an MCP server?

Yes, at api.gridlode.com/mcp. See the MCP docs for setup in Claude and Cursor.

Does it respect robots.txt?

Yes, by default. Page engines refuse URLs a site's robots.txt disallows, and the crawler honours Crawl-delay. You can turn this off only for sites you own or have permission to process.

Which agents and frameworks work with it?

Anything that speaks MCP (Claude, Cursor and other MCP clients) through the remote server, and any framework that can make an HTTPS request through the REST API. The OpenAPI 3.1 spec and llms.txt help coding agents write the calls.

How much does AI agents and RAG with Gridlode cost?

Running the workflow on 1,000 items uses about 2,000 credits (Bing Search at 1 credits per call and URL to Markdown at 1 credits per call). That is $2.00 at list price and as little as $0.60 at enterprise volume. AI Data Extraction is billed by actual model usage on top, capped by a limit you set. Every new account gets 1,000 free credits, failed requests are free and cached results cost half.

Which of these APIs can I use today?

Bing Search, DuckDuckGo Search, Bing News Search, URL to Markdown, Website Crawler (Markdown for RAG), Article Extractor & Readability and Page HTML are live now. Google Search, Google AI Mode, AI Data Extraction and YouTube Transcript are built and launching soon; the status on each API page changes the day they switch on.

Run your first AI agents and RAG job on us.

1,000 free credits, no card. Your API key is ready the moment you sign up.

Create your free account Read the docs