AI Summary
We are seeking a Senior Data Engineer to build a modern healthcare data platform. The ideal candidate will have strong Databricks and streaming experience. Key responsibilities include porting a high-volume event pipeline into Databricks, integrating with Kafka, and implementing complex patient matching and event handling logic.
Key Highlights
Port high-volume event pipeline into Databricks
Integrate with Kafka
Implement complex patient matching and event handling logic
Key Responsibilities
Port the core ingestion pipeline into a Databricks-native architecture conforming to standard ingestion layer patterns
Implement per-job success/failure tracking and metrics capture in alignment with platform engineering standards
Integrate with bulk patient matching libraries to accurately process patient update signals and lifecycle events
Build event handlers for patient merges, practice merges, and facility-level changes
Develop Databricks-to-Kafka (D2K) jobs for ingestion model outputs and downstream event streams
Technical Skills Required
Benefits & Perks
401(k) matching
Dental insurance
Health insurance
Nice to Have
Delta Live Tables (DLT)
Databricks Asset Bundles / CI/CD for Databricks
Confluent Kafka or AWS MSK
dbt on Databricks
Healthcare data or patient identity matching experience
Job Description
Benefits:
- 401(k) matching
- Dental insurance
- Health insurance
100% Remote
Skills:-
We are looking for a Senior Data Engineer with strong Databricks and streaming experience to build the core
ingestion layer of a modern healthcare data platform. You will port a high-volume event pipeline into
Databricks, integrate with Kafka, and implement complex patient matching and event handling logic.
Key Responsibilities
- Port the core ingestion pipeline into a Databricks-native architecture conforming to standard ingestion
Interested in remote work opportunities in Devops? Discover Devops Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.
- Implement per-job success/failure tracking and metrics capture in alignment with platform engineering
- Integrate with bulk patient matching libraries to accurately process patient update signals and lifecycle
- Build event handlers for patient merges, practice merges, and facility-level changes
- Develop Databricks-to-Kafka (D2K) jobs for ingestion model outputs and downstream event streams
- Ensure the solution is low-maintenance, well-documented, and observable in production
Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.
- Databricks — Delta Lake, Jobs, cluster management, Notebooks
- Apache Kafka — producer/consumer patterns, event-driven architecture
- PySpark / Spark Structured Streaming
- Python — advanced data engineering
- High-volume stateful event stream processing
- Experience porting or refactoring large-scale data pipelines
- Delta Live Tables (DLT)
- Databricks Asset Bundles / CI/CD for Databricks
- Confluent Kafka or AWS MSK
- dbt on Databricks
- Healthcare data or patient identity matching experience
Similar Jobs
Explore other opportunities that match your interests
Visa Sponsorship
Relocation
Remote
Job Type
Full-time
Experience Level
Entry level
medilinkers llc
United State
Visa Sponsorship
Relocation
Remote
Job Type
Part-time
Experience Level
Not Applicable
agentic ai: wir bauen apps und...
United State
Visa Sponsorship
Relocation
Remote
Job Type
Full-time
Experience Level
Entry level
medilinkers llc
United State