Appearance

What is the Scraping API? ​

BitFetcher Scraping API is a versatile tool designed to simplify and enhance the process of extracting data from websites. Whether you're dealing with static or dynamic content, our API provides a range of features to meet your scraping needs efficiently.

Just want to try it out? Skip to the Getting Started

Key Features ​

  • JavaScript Render: Render JavaScript on web pages using a headless browser to scrape dynamic content that traditional methods might miss.

    • Wait times and selectors: Specify wait times and CSS selectors to ensure elements are fully loaded before scraping.
    • Resource blocking: Block unnecessary resources from loading to speed up scraping.
    • Screenshot capabilities : Capture screenshots of the target page, including full-page and specific element screenshots.
    • JavaScript Scenario: Allowing you to interact with web pages dynamically.

  • Premium Proxies: Leverage a vast network of over 55 million residential IPs across 185+ countries, ensuring a 99.9% uptime for uninterrupted scraping.

  • Geolocation: Choose the geographical location of the IP address used for the request (for Premium Proxies).

  • Antibot bypass: The API is designed to overcome most of the antibot solutions like Cloudflare, Datadome, Akmai protection, making it easier to scrape protected websites.

  • Session management: Use a session ID to maintain the same IP address across multiple requests for up to 10 minutes.

  • Custom Headers/Cookies: Add custom HTTP headers to your requests for more control.

  • JSON response formatting: Receive responses in JSON format, including XHR or Fetch requests.

  • Language agnostic: While Python examples are provided, the API works with any programming language that can make HTTP requests.

  • Data Extraction: Multiple ways to extract exactly the data you need:

    • CSS Selectors: Extract data using CSS selectors to target specific elements
    • XPath Support: Use XPath expressions for complex data extraction patterns
    • AI Extraction: Leverage AI/LLM to extract data from web pages intelligently, adapting to different structures and formats.

BitFetcher Scraping API is particularly useful for extracting data from dynamic websites and handling complex scenarios like JavaScript content and web element interactions, without the need to manage proxies or handle dynamic user interactions manually.

Getting started ​

To get started with BitFetcher API, you need to sign up for an account and obtain an API key. You can then use this key to authenticate your requests and access the API's features.

  • API Key – Your unique key to authenticate requests.
  • URL – The target website you want to scrape.

That's it 🎉 With these two pieces of information, you're ready to start scraping. You can customize your requests further by adding parameters.

See API Overview for more details on the available parameters and their usage.