Develop and maintain scalable web scrapers, parsers, and backend solutions. Design resilient strategies to unblock websites, improve system observability, and support high-volume scraping infrastructure. The role involves Python development, asynchronous programming, browser automation, networking, and maintaining scraping-related features across systems processing substantial traffic and data volumes.
We’re a team of 500+ professionals who develop cutting-edge proxy and web data scraping solutions for thousands of the world’s best known businesses, including Fortune 500 companies.
What’s in store for you:
You’ll be solving complex challenges and maintaining our own infrastructure with 60PB+ monthly data traffic. Here are its scale and maturity in numbers:
- 6PB+ Ceph storage
- 60PB+ monthly data traffic through our systems
- 300k+ service requests/sec processed
- 500k+ Kafka messages/sec streamed
A word from the team:
We run one of the most advanced and largest scraping and parsing products in the world. We serve thousands of requests per second with a very high success rate. Our scrapers and parsers are used by leading e-commerce, market intelligence, and AI industry players making the work challenging and truly global. The team is a blend of different interesting personalities from different walks of life and nationalities. Here you can find people who are experts in gaming, playing guitar, riding bicycles, and other areas. We, as a team, will support you in learning how to build your own scrapers and will share all the tips, tricks and hacks we know to ensure that you are onboard in no time.
In this role, you’ll:
- Develop scalable scrapers.
- Define resilient scraping strategies, unblock websites for scrapingImprove observability in the system.
- Develop back-end solutions for scraping & parsing problems of various magnitudes.
- Maintain the current system and develop new features related to scraping & parsing.
Your skills & experience:
- Experience working with Python.
- Understanding of computer science, including data structures, algorithms, computability and complexity.
- Version Control skills using Git.
- Knowledge on how to unblock websites for scraping.
- Is able to use different scraping techniques & open-source tools to build scrapers.
- Is comfortable with using Dev Tools.
- Network (TLS/SSL) knowledge.
- Worked with browser automations.
- Knows their way around asynchronous programming.
Nice to have:
- Web development knowledge.
- Knows how to use CSS Selectors / XPaths for parsing.
- Experience working with Go & C++.
- Worked on browser source code.
- Knowledge of any front-end framework.
- Experience working with Pydantic, FastAPI, SQLAlchemy.
- Has experience working with Redis, MySQL, Docker, Kubernetes, Elasticsearch, Kibana and monitoring tools like Grafana, Prometheus.
- Experience with machine learning that is scraping domain-specific.
- Has experience in building scalable systems.
Salary:
- Gross salary: from 5000 eur/month. Keep in mind that we are open to discussing a different salary based on your skills and experience.
Up for the challenge? Let’s talk!
Similar Jobs
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Manages strategic, transformational, and cross-functional projects from planning through go-live. Responsibilities include coordinating teams and vendors, managing budgets, timelines, quality, testing, resources, risks, reporting, governance, and operational readiness. The role also mentors project managers, leads stakeholder meetings, and drives consistent project delivery across multiple regions.
Top Skills:
MS Office
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Build and operate backend-first, distributed platform services powering APIs, integrations, eventing, and analytical data systems to enable AI agents and enterprise-scale agentic workflows. Design for high throughput, reliability, security, and observability; apply AI-assisted engineering tools; mentor engineers and influence architecture, tooling, and developer experience.
Top Skills:
APIsClaude CodeCodexCursorEvent-Driven ArchitectureEvent-Streaming PlatformsGraphQLMcp ServersObservabilityRest
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Designs and develops resilient middleware, API integrations, and workflow-orchestration services for capital markets, trading, settlement, blockchain, and DLT systems. Responsibilities include FIX 4.4 message flows, transaction processing, event-driven integration, state management, security, auditability, testing, deployment, and production troubleshooting.
Top Skills:
.NetBlockchainC#Ci/CdDltFix 4.4GitJavaKafkaMessage QueuesNode.jsRelational DatabasesRest ApisSpring BootTypescript
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
