Summary
We are looking for a Senior Software Engineer to join our Data Engineering Infrastructure team, which builds and operates the foundational platforms that power data ingestion, transformation, and analytics across the organization. You will design and develop high-performance, reliable, and scalable systems that enable data engineers, analysts, and ML practitioners to move, process, and govern data efficiently and securely.
Description
As a Senior Software Engineer in the Data Engineering Infrastructure team, you will design and build distributed systems and frameworks that automate the lifecycle of data — from ingestion to transformation to serving. You’ll work at the intersection of software engineering, distributed data processing, and cloud infrastructure, helping to define the standards, abstractions, and tools that enable our data platform to operate at scale.
You will collaborate closely with teams across data engineering, analytics, ML, and platform engineering to deliver resilient infrastructure components such as data ingestion pipelines, metadata and schema management services, workflow orchestration, and monitoring frameworks. This is a hands-on role where you will influence architecture, write production-grade code, and drive engineering excellence across the data platform.
Minimum Qualifications
12+ years of experience in software engineering, with at least 5 years focused on data systems or platform infrastructure
Proven track record of leading design discussions, mentoring engineers, and driving cross-team technical initiatives
Strong programming skills in Java, Python; Scala - nice to have
Hands-on experience designing, developing, and operating high-performance backend services
Hands-on experience with distributed data frameworks such as Spark, Flink, or Kafka
Solid understanding of data modeling, storage formats (Parquet/Avro/ORC), and partitioning strategies
Experience with data governance, cataloging, and schema management systems (e.g., Hive Metastore, Glue, Iceberg, Delta Lake)
Experience working with cloud-based data platforms (AWS, GCP, or Azure)
Experience building data infrastructure frameworks, SDKs, or shared libraries used by multiple data teams
Familiarity with CI/CD, container orchestration (Kubernetes), and infrastructure-as-code tools (Terraform, CloudFormation)
Excellent problem-solving, debugging, and communication skills
Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent practical experience)
Preferred Qualifications
Deep expertise in optimizing and troubleshooting distributed data frameworks (e.g., Apache Spark internals, Flink stateful streaming, Kafka Connect)
Advanced experience with workflow orchestration tools (e.g., Airflow, Prefect, Dagster), including custom development
Proven track record in performance tuning, cost optimization, and designing comprehensive observability solutions for large-scale cloud data platforms
Familiarity with data security, privacy, and compliance principles within data infrastructure
Experience integrating data platforms with MLOps and machine learning workflows
Active participation or contributions to relevant open-source data technologies or industry communities
Master’s or Ph.D. degree in Computer Science, Data Engineering, or a related quantitative field
