Senior Data Engineer

Key Responsibilities:

• Design, develop, and optimise ETL/ELT processes using Azure Databricks.
• Build and maintain data pipelines and workflows to integrate structured and unstructured data sources.
• Develop and manage data warehouse solutions, ensuring scalability, performance, and security.
• Implement PySpark-based transformations in Databricks for efficient big data processing.
• Write complex SQL queries and stored procedures and optimize database performance.
• Automate data engineering workflows using DevOps and CI/CD pipelines for deployment.
• Develop Power BI datasets and dataflows and assist in report creation when required.
• Maintain and support SSRS reports and migrate to Power BI where necessary.
• Collaborate with cross-functional teams to ensure data solutions meet business needs.
• Implement best practices for data governance, security, and compliance.

Required Skills & Experience:

• 5+ years of experience in ETL, Data Warehousing, and BI development.
• Proficiency in Azure Databricks.
• Strong hands-on experience with PySpark and SQL.
• Experience in CI/CD, DevOps, and automation for data engineering workflows.
• Knowledge of Power BI and SSRS for reporting and visualisation.
• Solid understanding of data modelling, performance tuning, and optimisation.
• Familiarity with data governance, security, and compliance best practices.
• Experience working with cloud data solutions, preferably in Azure.
• Strong problem-solving skills and ability to work in an agile environment.