Senior Data Engineer (Databricks)

VanderHouwen • United State
Remote
Apply
AI Summary

Design and develop modern cloud-based data platforms including data lakes, warehouses, and lakehouse environments. Build scalable ETL/ELT pipelines using Databricks, PySpark, and Delta Lake to support enterprise analytics and decision-making. Requires 5+ years of data engineering experience with Databricks, Python, SQL, and cloud platforms.

Key Highlights
Expert-level Databricks experience with PySpark and Delta Lake
5+ years of data engineering and cloud platform development
Design Bronze, Silver, and Gold data layers for analytics
Remote role with potential project-based work model changes
Strong communication and mentorship skills
Key Responsibilities
Design, develop, and support modern cloud-based data platforms including data lakes, data warehouses, and lakehouse environments
Build and maintain enterprise-grade data engineering solutions utilizing Databricks, PySpark, Delta Lake, and related technologies
Develop scalable ETL/ELT pipelines that ingest, transform, validate, and deliver data from various enterprise systems and external sources
Create and maintain Bronze, Silver, and Gold data layers to support analytics, reporting, and downstream business applications
Design logical, physical, and dimensional data models for efficient data consumption and reporting
Integrate data from ERP systems, operational platforms, APIs, databases, flat files, and third-party applications
Develop and support REST API integrations and data services for enterprise data exchange
Partner with business analysts, architects, data consumers, and leadership teams to gather requirements and deliver data-driven solutions
Implement monitoring, alerting, and observability practices for production data pipelines
Troubleshoot and resolve data quality, integration, and performance issues across complex environments
Optimize data processing frameworks and cloud resources for scalability, efficiency, and cost management
Contribute to data architecture discussions, technical standards, and engineering best practices
Provide technical leadership, mentorship, and guidance to team members
Support continuous improvement of data engineering processes, tools, and development methodologies
Technical Skills Required
Databricks PySpark Delta Lake Python SQL Azure AWS REST API integrations Airflow Prefect Dagster SQL Server PostgreSQL Oracle MySQL DB2 Microsoft Fabric Power BI
Benefits & Perks
$55-$60/hr salary
Remote work
Nice to Have
Experience with streaming data platforms
CI/CD practices
Modern DevOps methodologies

Job Description


Status: Direct Hire

Job Title: Sr. Data Engineer

Location: Remote

Salary: $55-$60/hr


About the company:

We are a highly experienced team of architects, interface designers, and engineers who assist our clients with transforming data into beneficial information, delivering collaborative web experiences, and building custom business applications.


Our elegant software and data solutions help clients educate kids, transform healthcare, clean the environment, promote energy efficiency, and encourage healthy lifestyles. Organizations of all sizes like Nike, Northwest Energy Efficiency Alliance, Cambia Health, and the Beaverton School District rely on us.


Here it’s all about our people. We pride ourselves on hiring brilliant, empathetic, lifelong learners who bring the best results; reducing project risk and maximizing project success. We embrace agnostic solutions, knowing the right answer doesn’t always come from a single source. Because we architect from a diverse portfolio of modern technologies, we can ensure the delivery of innovative solutions our clients can successfully support and extend.


Beginning with the end customer, we apply a user-centered approach. Our seasoned team invests to understand the business problem, then creates sustainable, performant, and scalable architectures that are perfectly suited for their application.


Senior Data Engineer (Databricks):

Our client is seeking a Senior Data Engineer to join a growing data engineering team focused on building modern, cloud-based data platforms that drive enterprise analytics, reporting, and operational decision-making. This role is ideal for a highly skilled Databricks professional who enjoys designing scalable data solutions, solving complex integration challenges, and partnering with both technical and business stakeholders to deliver impactful results. The successful candidate will play a key role in developing robust data architectures, optimizing data pipelines, and helping shape the future of enterprise data capabilities.


This is a remote role with possibility of a changing work model dependent on project needs.


Senior Data Engineer Responsibilities:

  • Design, develop, and support modern cloud-based data platforms, including data lakes, data warehouses, and lakehouse environments.
  • Build and maintain enterprise-grade data engineering solutions utilizing Databricks, PySpark, Delta Lake, and related technologies.
  • Develop scalable ETL/ELT pipelines that ingest, transform, validate, and deliver data from a variety of enterprise systems and external sources.
  • Create and maintain Bronze, Silver, and Gold data layers to support analytics, reporting, and downstream business applications.
  • Design logical, physical, and dimensional data models that enable efficient data consumption and reporting.
  • Integrate data from ERP systems, operational platforms, APIs, databases, flat files, and third-party applications.
  • Develop and support REST API integrations and data services that facilitate data exchange across enterprise systems.
  • Partner with business analysts, architects, data consumers, and leadership teams to gather requirements and deliver data-driven solutions.
  • Implement monitoring, alerting, and observability practices to ensure reliability and performance of production data pipelines.
  • Troubleshoot and resolve data quality, integration, and performance issues across complex environments.
  • Optimize data processing frameworks and cloud resources for scalability, efficiency, and cost management.
  • Contribute to data architecture discussions, technical standards, and engineering best practices.
  • Provide technical leadership, mentorship, and guidance to team members as needed.
  • Support the continuous improvement of data engineering processes, tools, and development methodologies.


Senior Data Engineer Qualifications:

  • 5+ years of experience in Data Engineering, Data Platform Development, Software Engineering, or a related technical field.
  • Expert-level experience with Databricks, including PySpark, Delta Lake, Databricks Workflows, and Lakehouse Architecture.
  • Strong hands-on programming experience with Python and advanced SQL development.
  • Proven experience designing, building, and supporting large-scale ETL/ELT pipelines in production environments.
  • Experience developing cloud-based data solutions within Azure and/or AWS environments.
  • Strong understanding of modern data warehousing, data lake, and lakehouse design principles.
  • Experience working with structured, semi-structured, and large-scale enterprise data sets.
  • Hands-on experience with relational database platforms such as SQL Server, PostgreSQL, Oracle, MySQL, DB2, or similar technologies.
  • Experience integrating data from enterprise applications, financial systems, ERP platforms, and external data sources.
  • Knowledge of API-based integrations and data exchange methodologies.
  • Experience implementing data quality, governance, monitoring, and performance optimization strategies.
  • Familiarity with workflow orchestration and automation tools such as Airflow, Prefect, Dagster, or similar platforms.
  • Experience supporting business intelligence and analytics initiatives through delivery of curated, analytics-ready datasets.
  • Understanding of distributed data platforms, NoSQL databases, or big data technologies is preferred.
  • Experience with Microsoft Fabric, Power BI integration, streaming data platforms, CI/CD practices, or modern DevOps methodologies is a plus.
  • Strong communication skills with the ability to explain technical concepts to both technical and non-technical audiences.
  • Demonstrated ability to work independently, manage competing priorities, and thrive in evolving project environments.

Similar Jobs

Explore other opportunities that match your interests

Visa Sponsorship Relocation Remote
Job Type Contract
Experience Level Mid-Senior level

HMG AMERICA LLC

United State
Visa Sponsorship Relocation Remote
Job Type Contract
Experience Level Mid-Senior level

MSR Technology Group

United State

Senior Data Science Expert

Data Science
•
2h ago
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Not Applicable

Mercor

United State

Subscribe our newsletter

New Things Will Always Update Regularly