Senior Backend/Infra Engineer for High-Scale Headless Browser Automation and Experimentation Framework
Remote
AI Summary
Design and develop a high-scale headless browser automation and experimentation framework. Develop a containerized, observable, and scalable platform. Work on a fully remote, async-friendly project.
Key Highlights
Design and develop a high-scale headless browser automation and experimentation framework
Develop a containerized, observable, and scalable platform
Work on a fully remote, async-friendly project
Technical Skills Required
Benefits & Perks
Fully remote work
Async-friendly work environment
Job Description
About the Project
I’m building a high-scale headless browser automation and experimentation framework that will:
- Run thousands of browser sessions in parallel for testing, data collection, and content experiments.
- Start life on a single VPS, and later move onto a dedicated on prem server
- Integrate with an existing Java/Appium codebase, but the new core will be containerized, observable, and easy to scale.
This is not a traditional CRUD app. It’s closer to building the backbone for a mini cloud: containers, queues, orchestration, metrics, and clean deployment.
I’m looking for a hands-on backend/infra engineer who is comfortable owning the base platform end-to-end.
Must-Have Skills
- Strong experience on Linux servers (Ubuntu).
- Deep hands-on with:
- Docker or Podman in production (multi-service setups).
- PostgreSQL (schema design, basic performance tuning).
- Redis (queues + caching).
- Experience setting up:
- Prometheus + Grafana (or equivalent) for metrics.
- Centralized logging for containerized apps.
- Comfortable designing service boundaries and deployment layouts from scratch.
- Can communicate clearly in English and explain trade-offs (e.g. Docker vs Podman, nginx vs Traefik, etc.).
Nice-to-Have
- Experience with distributed systems or high-throughput job queues.
- Familiarity with Playwright / Puppeteer / Selenium (for later milestones).
- Experience with Kubernetes (we may evolve there later).
- Previous work on load-testing / automation / scraping platforms.
- Fully remote, async-friendly.
- We’ll start with Milestone 1 only; if it goes well, there are follow-up milestones (orchestration engine, browser workers, campaign logic, etc.).
- I’ll provide design doc and be available as the “product/CTO” side, so you’re not guessing requirements.
1. A brief description of similar platforms you’ve built:
- e.g. “job queue + workers + metrics + dashboards on Docker/Kubernetes”.
2. A link to any repo or screenshot of:
- A docker-compose / infra repo you’ve set up.
- Or a Grafana dashboard you’ve created.