Senior Data Architect

Iris's Direct client, a leading Logistics solutions and services provider is looking to hire a Sr. Data Engineer. This role will be 100% remote in Canada and will be Full Time T4 with Iris Software.

MUST HAVE OVER 10 yrs of IT Experience.

Job Description / Requirements

• Create and Optimize data pipelines for performance and efficiency.
• Work with Databricks clusters and configuration management tools.
• Use appropriate tools in the cloud data lake development and deployment. Developing/implementing cloud infrastructure to support current and future business needs.
• Provide technical expertise and ownership in the diagnosis and resolution of issues.
• Ensure all cloud solutions exhibit a higher level of cost efficiency, performance, security, scalability, and reliability.
• Manage cloud data lake development and deployment on AWS /Databricks.

MUST HAVES:

• Experience & Proficiency with Databricks platform - Delta Lake storage, Spark (PySpark, Spark SQL). Must be well versed with Databricks Lakehouse, Unity Catalog concept and its implementation in enterprise environments.
• Familiarity of data design pattern - medallion architecture to organize data in a Lakehouse.
• Experience & Proficiency with AWS Data Services – S3, Glue, Athena, Redshift etc.| Airflow scheduling
• Proficiency in SQL and experience with relational databases.
• Proficiency in at least one programming language (e.g., Python, Spark, Pyspark, Java) for data processing and scripting.
• Good understanding of data principles, Cloud Data Lake design & development including data ingestion, data modeling and data distribution.

Regards,

Anurag Dang

www.irissoftware.com

https://www.linkedin.com/in/anuragdang/