LogoLogo
SupportDashboard
  • Community
  • Welcome to Hyperbrowser
  • Get Started
    • Quickstart
      • AI Agents
        • Browser Use
        • Claude Computer Use
        • OpenAI CUA
      • Web Scraping
        • Scrape
        • Crawl
        • Extract
      • Browser Automation
        • Puppeteer
        • Playwright
        • Selenium
  • Agents
    • Browser Use
    • Claude Computer Use
    • OpenAI CUA
  • HyperAgent
    • About HyperAgent
      • HyperAgent SDK
      • HyperAgent Types
  • Quickstart
  • Multi-Page actions
  • Custom Actions
  • MCP Support
    • Tutorial
  • Examples
    • Custom Actions
    • LLM support
    • Cloud Support
      • Setting Up
      • Proxies
      • Profiles
    • MCP Examples
      • Google Sheets
      • Weather
        • Weather Server
    • Output to Schema
  • Web Scraping
    • Scrape
    • Crawl
    • Extract
  • Sessions
    • Overview
      • Session Parameters
    • Advanced Privacy & Anti-Detection
      • Stealth Mode
      • Proxies
      • Static IPs
      • CAPTCHA Solving
      • Ad Blocking
    • Profiles
    • Recordings
    • Live View
    • Extensions
    • Downloads
  • Guides
    • Model Context Protocol
    • Scraping
    • AI Function Calling
    • Extract Information with an LLM
    • Using Hyperbrowser Session
    • CAPTCHA Solving
  • Integrations
    • ⛓️LangChain
    • 🦙LlamaIndex
  • reference
    • Pricing
    • SDKs
      • Node
        • Sessions
        • Profiles
        • Scrape
        • Crawl
        • Extensions
      • Python
        • Sessions
        • Profiles
        • Scrape
        • Crawl
        • Extensions
    • API Reference
      • Sessions
      • Scrape
      • Crawl
      • Extract
      • Agents
        • Browser Use
        • Claude Computer Use
        • OpenAI CUA
      • Profiles
      • Extensions
Powered by GitBook
On this page
Export as PDF
  1. reference
  2. API Reference

Scrape

PreviousSessionsNextCrawl

Last updated 1 month ago

Get scrape job status and result

get
Authorizations
Path parameters
idstring · uuidRequired
Responses
200
Scrape job details
application/json
404
Job not found
application/json
500
Server error
application/json
get
GET /api/scrape/{id} HTTP/1.1
Host: api.hyperbrowser.ai
x-api-key: YOUR_API_KEY
Accept: */*
{
  "jobId": "text",
  "status": "pending",
  "data": {
    "metadata": {
      "ANY_ADDITIONAL_PROPERTY": "text"
    },
    "markdown": "text",
    "html": "text",
    "links": [
      "text"
    ],
    "screenshot": "text"
  },
  "error": "text"
}

Get batch scrape job status

get
Authorizations
Path parameters
idstring · uuidRequired
Responses
200
Batch scrape job status
application/json
404
Batch scrape job not found
application/json
500
Server error
application/json
get
GET /api/scrape/batch/{id}/status HTTP/1.1
Host: api.hyperbrowser.ai
x-api-key: YOUR_API_KEY
Accept: */*
{
  "status": "pending"
}

Get batch scrape job status and results

get
Authorizations
Path parameters
idstringRequired
Responses
200
Batch scrape job details
application/json
400
Invalid request parameters
application/json
404
Batch scrape job not found
application/json
500
Server error
application/json
get
GET /api/scrape/batch/{id} HTTP/1.1
Host: api.hyperbrowser.ai
x-api-key: YOUR_API_KEY
Accept: */*
{
  "jobId": "text",
  "status": "pending",
  "data": [
    {
      "url": "text",
      "status": "pending",
      "error": "text",
      "metadata": {
        "ANY_ADDITIONAL_PROPERTY": "text"
      },
      "markdown": "text",
      "html": "text",
      "links": [
        "text"
      ],
      "screenshot": "text"
    }
  ],
  "error": "text",
  "totalScrapedPages": 1,
  "totalPageBatches": 1,
  "currentPageBatch": 1,
  "batchSize": 1
}
  • POSTCreate new scrape job
  • GETGet scrape job status
  • GETGet scrape job status and result
  • POSTStart a batch scrape job
  • GETGet batch scrape job status
  • GETGet batch scrape job status and results

Get scrape job status

get
Authorizations
Path parameters
idstring · uuidRequired
Responses
200
Scrape job status
application/json
404
Job not found
application/json
500
Server error
application/json
get
GET /api/scrape/{id}/status HTTP/1.1
Host: api.hyperbrowser.ai
x-api-key: YOUR_API_KEY
Accept: */*
{
  "status": "pending"
}

Create new scrape job

post
Authorizations
Body
urlstring · min: 1Required
Responses
200
Scrape job created
application/json
400
Invalid request parameters
application/json
500
Server error
application/json
post
POST /api/scrape HTTP/1.1
Host: api.hyperbrowser.ai
x-api-key: YOUR_API_KEY
Content-Type: application/json
Accept: */*
Content-Length: 937

{
  "url": "text",
  "sessionOptions": {
    "useStealth": false,
    "useProxy": false,
    "proxyServer": "text",
    "proxyServerPassword": "text",
    "proxyServerUsername": "text",
    "proxyCountry": "AD",
    "proxyState": "AL",
    "proxyCity": "new york",
    "operatingSystems": [
      "windows"
    ],
    "device": [
      "desktop"
    ],
    "platform": [
      "chrome"
    ],
    "locales": [
      "aa"
    ],
    "screen": {
      "width": 1280,
      "height": 720
    },
    "solveCaptchas": false,
    "adblock": false,
    "trackers": false,
    "annoyances": false,
    "enableWebRecording": true,
    "enableVideoWebRecording": false,
    "profile": {
      "id": "text",
      "persistChanges": true
    },
    "acceptCookies": true,
    "extensionIds": [
      "123e4567-e89b-12d3-a456-426614174000"
    ],
    "urlBlocklist": [
      "text"
    ],
    "browserArgs": [
      "text"
    ],
    "imageCaptchaParams": [
      {
        "imageSelector": "text",
        "inputSelector": "text"
      }
    ],
    "timeoutMinutes": 1
  },
  "scrapeOptions": {
    "formats": [
      "html"
    ],
    "includeTags": [
      "text"
    ],
    "excludeTags": [
      "text"
    ],
    "onlyMainContent": true,
    "waitFor": 0,
    "timeout": 30000,
    "waitUntil": "load",
    "screenshotOptions": {
      "fullPage": false,
      "format": "webp"
    }
  }
}
{
  "jobId": "text"
}

Start a batch scrape job

post
Authorizations
Body
urlsstring[]Required
Responses
200
Batch scrape job started successfully
application/json
400
Invalid request parameters
application/json
402
Insufficient plan
application/json
429
Too many concurrent batch scrape jobs
application/json
500
Server error
application/json
post
POST /api/scrape/batch HTTP/1.1
Host: api.hyperbrowser.ai
x-api-key: YOUR_API_KEY
Content-Type: application/json
Accept: */*
Content-Length: 940

{
  "urls": [
    "text"
  ],
  "sessionOptions": {
    "useStealth": false,
    "useProxy": false,
    "proxyServer": "text",
    "proxyServerPassword": "text",
    "proxyServerUsername": "text",
    "proxyCountry": "AD",
    "proxyState": "AL",
    "proxyCity": "new york",
    "operatingSystems": [
      "windows"
    ],
    "device": [
      "desktop"
    ],
    "platform": [
      "chrome"
    ],
    "locales": [
      "aa"
    ],
    "screen": {
      "width": 1280,
      "height": 720
    },
    "solveCaptchas": false,
    "adblock": false,
    "trackers": false,
    "annoyances": false,
    "enableWebRecording": true,
    "enableVideoWebRecording": false,
    "profile": {
      "id": "text",
      "persistChanges": true
    },
    "acceptCookies": true,
    "extensionIds": [
      "123e4567-e89b-12d3-a456-426614174000"
    ],
    "urlBlocklist": [
      "text"
    ],
    "browserArgs": [
      "text"
    ],
    "imageCaptchaParams": [
      {
        "imageSelector": "text",
        "inputSelector": "text"
      }
    ],
    "timeoutMinutes": 1
  },
  "scrapeOptions": {
    "formats": [
      "html"
    ],
    "includeTags": [
      "text"
    ],
    "excludeTags": [
      "text"
    ],
    "onlyMainContent": true,
    "waitFor": 0,
    "timeout": 30000,
    "waitUntil": "load",
    "screenshotOptions": {
      "fullPage": false,
      "format": "webp"
    }
  }
}
{
  "jobId": "text"
}