AWS PySpark Databricks
Role & responsibilities
Design, develop, and optimize scalable data pipelines using PySpark on Databricks
Work with AWS services such as S3, Glue, Lambda, Redshift, and EMR
Develop ETL/ELT workflows for structured and unstructured data
Collaborate with data architects and stakeholders to define data solutions
Implement data quality checks and ensure data integrity
Optimize Spark jobs for performance and cost efficiency
Manage and monitor Databricks workflows and clusters
Ensure best practices in data governance, security, and compliance
Troubleshoot production issues and provide timely resolutions
Must-Have:
Strong hands-on experience with PySpark
Expertise in Databricks platform
Experience working with AWS Cloud services (S3, Glue, EMR, Lambda, Redshift)
Solid understanding of distributed data processing
Experience in building data lakes and data pipelines
Strong SQL skills
If you're looking for your next career move or know someone who fits, feel free to connect or share!
I would greatly appreciate any referrals or leads. Please feel free to connect or message me directly.
Apply Now: [email protected]
Only Whatsapp: 8977020346
Preferred candidate profile