Senior Data Engineer
The Data Engineering team builds and operates the analytical data platform that powers machine learning, data science, analytics, and reporting across Veriff. We are responsible for large-scale data ingestion, platform reliability, and enterprise data governance — ensuring Veriffians have access to accurate and timely data.
In this role, you will own and evolve our data lake and data warehouse infrastructure, driving platform-level data management, governance, and reliability at scale.
You'll help us protect honest people online by:
- Owning and evolving our data lake and data warehouse infrastructure using technologies such as Spark, Apache Iceberg, S3, Trino/Athena, and Redshift.
- Designing and maintaining platform-level data transformation pipelines in Python and SQL — focused on schema evolution, partitioning, compaction, and deduplication.
- Implementing optimized storage formats (Parquet, Avro, ORC), partitioning strategies, and indexing to improve query performance and reduce platform costs.
- Driving data governance initiatives — PII detection and classification, access control policies, data cataloging, lineage tracking, and data quality frameworks.
- Ensuring the availability, reliability, and cost efficiency of the data platform, including observability, monitoring, and alerting for pipeline and query engine health.
- Collaborating with ML, analytics, product, and engineering teams to define data contracts, maintain schema consistency, and provide clean, well-governed datasets.
- Contributing to disaster recovery strategy and multi-region reliability of the data platform.
You are the right future Veriffian for the job if you have:
- Strong experience with Python, SQL, and Apache Spark / PySpark for large-scale data processing.
- Deep knowledge of modern analytics platform architecture — object stores, columnar and row-based data formats (Parquet, Avro, ORC), orchestration tools, analytical query engines, schema registries, and data catalogs.
- Experience with data governance and data management at scale — PII handling, data cataloging, schema management, access control, and data quality frameworks.
- Experience designing and operating data lake and data warehouse infrastructure.
- Solid understanding of storage optimization — partitioning, compaction, and compression trade-offs.
- Experience building observability, monitoring, and alerting for data platforms.
- Strong problem-solving skills and comfort working with ambiguity — defining problems before solving them.
- A collaborative mindset — this role serves ML, analytics, and product teams as internal customers.
You're an especially awesome match if you have:
- Experience with Infrastructure as Code (IaC) and Terraform.
- Familiarity with containerization — Docker and Kubernetes.
- Experience with CI/CD pipelines for data platform deployments.
- Knowledge of data lake table formats beyond Iceberg (Delta Lake, Hudi).
- Familiarity with data catalog and metadata management tools (e.g., DataHub, Amundsen, AWS Glue Catalog).
- Understanding of data privacy regulations (GDPR) in the context of data engineering.
- Experience building streaming data pipelines.
- Experience with the AWS data stack.
Why Veriff?
We are the preferred identity verification platform partner for the world’s most innovative growth-driven organizations helping conveniently verify and safeguard users anywhere in the world. We support the broadest number of identity documents from nearly every country and territory in the world - and this is continually increasing! With a diverse team in the United States, United Kingdom, Spain, and Estonia, as well as robust backing and funding from investors including Accel, Alkeon, IVP, Tiger Capital, and Y Combinator, we’re dedicated to helping businesses and individuals build a safer and more secure world. We strive to be the benchmark for trust online, and we take pride in being a positive force.
When you join Team Veriff, we offer you the opportunity to have a real impact and advance your career, while looking after you along the way with a range of benefits designed with you in mind. Some of our favorites include...
- Flexibility to work from home
- Stock options that ensure your share in our success
- Extra recharge days on top of your annual vacation
- Comprehensive relocation support to Estonia or Spain
- Extensive medical, dental, and vision insurance to ensure you’re feeling great physically and mentally
- Learning and Development & Health and Sports budget that you are free to tailor to your own needs
- Four weeks of fully paid sabbatical leave after reaching your 5th work anniversary
We are an Equal Opportunities employer committed to a diverse and representative team. Different opinions, perspectives and personalities push us forward. We want to hear from people who are passionate about their work and align with our values. Regardless of race, religion, color, national origin, gender, sexual orientation, age, marital status or disability. We also know the confidence gap and imposter syndrome can get in the way of meeting spectacular candidates - so rest assured, whoever you are, we want to hear from you!
Please be informed that in the final stage of the recruitment process we may request you to go through identity verification. For more information, please see Recruitment Privacy Policy. #SPBTAL #LI-Remote
