Senior Software Engineer - Data Pipelines
About the Role - as a Senior Engineer at Obsidian, you’ll:
- Build or extend data pipelines that power Obsidian’s Agentic AI and supply chain risk products.
- Maintain, improve, and evolve existing systems to ensure performance, resilience, and scalability
- Design and implement APIs and backend services, including multithreaded applications
- Collaborate with product and engineering teams to support key product themes and ensure delivery of high-impact features
- Apply strong software engineering practices to requirements gathering, system design, and code reviews
- Contribute to a fast-moving, collaborative environment where adaptability and teamwork are essential.
- Identify and proliferate best practices in data pipeline design between both the UK and US teams.
What’s in It for You
- You will have direct impact on the core product used by enterprises worldwide
- Work alongside a talented, friendly team in a supportive and collaborative culture
- Grow your skills with opportunities to learn new technologies and engineering practices
- Be part of an innovative, fast-paced environment where your contributions are valued
- Enjoy a hybrid working, with supported remote working and great office spaces in Cheltenham and Manchester
Required Skills & Experience
- 5-7 years of experience in a software engineering role
- Proficiency in one or more modern programming languages such as Python, Go, or SQL
- Experience building backend services, APIs, and multithreaded applications
- Familiarity with containerization and orchestration technologies such as Docker and Kubernetes
- Strong knowledge of relational databases (e.g., Postgres)
- Experience collaborating in team environments and adapting to changing requirements
- Understanding of software design principles and engineering best practices
- Experience with cloud platforms (AWS, GCP), object storage (S3), or event/streaming systems (Kafka, Redis)
- Familiarity with Git for version control and deployment tooling, such as GitLab CI/CD
- Experience in influencing design patterns based on best practices.
- Experience assessing alternative data architecture designs to produce accurate and unbiased recommendations.
Desirable Experience
- Experience with system monitoring and observability tools such as Grafana, Prometheus, or similar platforms
- Understanding of quality engineering (QE) practices across development and testing lifecycles
- Exposure to large-scale distributed systems and performance optimisation
- Experience in forming strong stakeholder relationships to help promote joint working between teams.
- Experience of working alongside both UK and US development teams.
AI Skills & AI-Native Engineering Expectations: As an AI-forward engineering organization, we expect senior engineers to effectively leverage AI tools and understand foundational AI concepts to enhance development efficiency and build AI-ready systems.
AI Engineering Capabilities:
- Leverage AI tools effectively to improve development efficiency and build AI-ready systems.
- Proficient with AI-powered developer tools; able to critically evaluate and refine AI-generated outputs.
- Strong understanding of core AI/ML concepts (LLMs, embeddings, vector databases, inference, evaluation).
- Experience integrating AI/ML APIs and building AI-ready data infrastructure (e.g., for RAG).
- Ensure data quality, governance, observability, reliability, security, and performance in AI-driven systems.