What is Web scraping?

Automatically collecting publicly available data from websites.

Responsible scraping means collecting public data only, respecting robots.txt and rate limits, never logging in or bypassing access controls, and handling any personal data lawfully.

Related terms

• Datacenter proxy: A proxy running on cloud or hosting servers.
• Headless browser: A real browser (like Chrome) running without a visible window, controlled by code.
• MCP (Model Context Protocol): An open protocol that lets AI assistants call external tools and data sources.