Join Brooksource's client's Real-Time Data Streaming team to build and maintain scalable data pipelines. Work with Scala and Spark Structured Streaming to ensure data accuracy and reliability. 1-2 years of professional experience required.
Key Highlights
Technical Skills Required
Benefits & Perks
Job Description
Software Engineer I – Real-Time Data Streaming (Scala + Spark)
Type: W2 long-term contract
Location: United States; Remote (potential conversion may require relocation to client hub)
Client Hubs: Denver, Charlotte, St. Louis, Stamford/NYC, Austin
Onsite Policy if Converted FTE: 4 days onsite / 1 day remote
Start Date: early January
Pay Rate: $38–$43/hr (DOE)
Overview
We are seeking a Software Engineer I to join our client’s Real-Time Data Streaming team. This group builds and maintains scalable data pipelines that ingest, transform, and deliver mission-critical data to internal and external stakeholders. You’ll work across real-time and batch processing systems, ensuring data accuracy, reliability, and integrity across billions of daily events.
This is an excellent role for an engineer with 1–2 years of professional experience, particularly in Scala and ideally Spark Structured Streaming in Scala, who wants to grow within a high-impact data engineering environment.
Key Responsibilities
• Maintain and enhance Spark-based streaming applications.
• Support BAU tasks, including privacy compliance updates and Spark version upgrades.
• Collaborate with stakeholders to customize data exports for internal teams and external partners.
• Troubleshoot, debug, and optimize Spark Structured Streaming jobs for performance and reliability.
Required Skills
• 1–2 years of professional software engineering experience, ideally in data engineering or streaming environments.
• Hands-on Scala experience (functional programming mindset preferred).
• Practical knowledge of AWS services such as S3, Athena, EC2, IAM, and EMR.
• Strong understanding of Apache Kafka (consumer groups, offset commits).
• Experience with Spark Structured Streaming in Scala.
• Proficiency with Git and comfort working in Mac development environments.
Nice-to-Have Skills
• Experience with Terraform, Kubernetes, or general containerization.
• SQL for data analysis or data exploration.
• Exposure to large-scale datasets or distributed data systems.