Data Engineer

Page 1
About Oliver Wyman

Oliver Wyman is a global leader in management consulting. With offices in 60 cities across 29 countries,

Oliver Wyman is a business of Marsh (NYSE: MRSH), a global leader in risk, reinsurance and capital, people

and investments, and management consulting, advising clients in 130 countries. With annual revenue of over $27 billion and more than 95,000 colleagues, Marsh helps build the confidence to thrive through the power of perspective.
Visit our website for more details about Oliver Wyman: www.oliverwyman.com

Job Overview:
The OWG Technology department is looking to hire a highly motivated Data Engineer to play a crucial role in our data transformation program. You will be instrumental in the migration of data sources to centralized cloud environment, with Databricks as our strategic standard on AWS. As a Data Engineer, you will be responsible for designing, developing, and maintaining our data infrastructure and pipelines, utilizing a variety of technologies and tools to ensure efficient data processing, integration, and management. You will collaborate closely with data scientists, analysts, and other stakeholders to ensure the availability, reliability, and scalability of our data systems, while promoting best practices in data governance and architecture.

Key Responsibilities:

• Design and implement processes to ingest data from various sources into the Databricks Lakehouse platform, ensuring alignment with architectural and engineering standards.
• Develop, maintain, and optimize data models and ETL pipelines that support the Medallion Architecture (Bronze, Silver, Gold layers) to enhance data processing efficiency and facilitate data transformation.
• Utilize Databricks to integrate, consolidate, and cleanse data, ensuring accuracy and readiness for analysis, while leveraging Delta Lake for versioned data management.
• Implement and manage Unity Catalog for centralized data governance, ensuring proper data access, security, and compliance with organizational policies and regulations.
• Collaborate with business analysts, data scientists, and stakeholders to understand their data requirements and deliver tailored solutions that leverage the capabilities of the Databricks Lakehouse platform.
• Migration of existing ETL processes from Informatica IICS and MS SSIS to cloud-based data pipelines
• Engage with clients to support data engineering requirements, address technical queries, and provide guidance on best practices for utilizing the Databricks Lakehouse platform.
• Stay updated on industry trends and emerging technologies in data engineering, particularly those related to Databricks and cloud data solutions, continuously enhancing your skills and knowledge.

Experience:

• Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related field.
• Minimum of 5+ years of experience in data engineering or a related data role.
• Proven experience in designing and implementing production-grade Spark-based solutions.
• Proficient in query tuning, performance tuning, troubleshooting, and debugging Spark or other big data solutions.
• Familiarity with big data technologies such as Spark/Delta, Hadoop, NoSQL, MPP, and OLAP.
• Experience with cloud architecture, systems, and principles, particularly in AWS.
• Proficient in programming languages such as Python, R, Scala, or Java.
• Expertise in scaling ETL pipelines for performance and cost-effectiveness, with a focus on tuning queries for big data.
• Experience in building and scaling streaming data pipelines.
• Strong understanding of DevOps tools and best practices for data engineering, including CI/CD, unit and integration testing, automation, and orchestration.
• Cloud certification is highly desirable.

Skills and Attributes:

• Candidates must possess full professional proficiency in both written and spoken English
• Strong problem-solving and troubleshooting skills.
• Strong communicator, in both verbal and written form, able to articulate concepts and ideas, break through barriers, engage people, and work effectively with others under pressure.
• Neutral toward technology, vendor, and product choices more interested in results than in personal preferences.
• Unflappable in the face of opposition to ideas