Senior Data Engineer - Web Scraping & Data Pipeline Development

Are you a Senior Data Engineer with web scraping experience looking to join one of the most prestigious quantitative hedge funds in the world here in New York City?

My client has a growing team supporting one of their trading teams that is urgently looking for a software engineer who is excited about data and has excellent web scraping experience.

This is a role I am exclusively recruiting for. If you are ready to move into the world of quantitative finance and are available to start soon, then this is the role for you.

Compensation

• $600,000 - $800,000 Total Compensation depending on level of experience.

• Fully covered Medical, Dental and Vision Insurance.

• Tuition and Student Loan reimbursement.

• Excellent 401(k) matching program.

• Relocation is fully covered.

What's the Job?

• Design and maintain scalable, production-quality data collection pipelines targeting dynamic, JavaScript-rich web applications, leveraging both browser automation and HTTP-based extraction while ensuring accuracy, throughput, and respectful crawling practices.

• Analyze network traffic and application behavior to extract data reliably and respond quickly to upstream site changes.

• Take full ownership of data integrity end-to-end, including validation, anomaly detection, and ongoing monitoring for freshness and completeness.

• Partner with quantitative researchers to scope dataset requirements — covering content, coverage, latency, and QA standards — and deliver well-documented, reusable deliverables.

• Stay current with emerging tools and frameworks, championing targeted improvements that demonstrably enhance reliability, data quality, and long-term maintainability.

Qualifications

• Proven track record of developing and maintaining production-grade web scrapers and crawlers using tools such as Playwright, Selenium, and/or direct HTTP extraction and parsing techniques.

• Excellent Python experience.

• Hands-on experience designing robust data pipelines, including orchestration, monitoring and alerting, and automated workflows for quality assurance and failure recovery.

Location

This role is located in New York City five days a week.

Who Are They?

My client is one of the most sought-after quantitative finance firms in the world to work for. They offer many opportunities for growth, learning and career advancement. They are also well-known for stability and consistent performance in the industry.

To learn more, apply here today or email me at: Matt@Stabilesearch.com.