Data Engineer – Spark & Scala

About the role

We are seeking a Data Engineer with strong expertise in Apache Spark and Scala to design, build, and maintain high‑performance data pipelines. The role involves working with large‑scale datasets, integrating data from multiple sources, and supporting our data‑driven initiatives.

Key responsibilities

• Design, develop, and maintain robust data pipelines using Apache Spark and Scala for both batch and real‑time processing.
• Integrate and consolidate data from diverse sources into a unified data warehouse, ensuring data quality and reliability.
• Collaborate with data scientists, analysts, and other stakeholders to translate business requirements into technical solutions.
• Optimize Spark jobs and Scala applications for performance, scalability, and cost efficiency.
• Troubleshoot and resolve complex data engineering issues.
• Monitor system performance and implement improvements to meet business needs.

Required profile

• 3+ years of experience in data engineering or a related field.
• Proven experience handling large datasets and building end‑to‑end data pipelines.
• Strong problem‑solving abilities and attention to detail.
• Excellent communication skills and ability to work both independently and within a team.

Required skills

• Apache Spark
• Scala
• Hadoop/HDFS
• NoSQL databases
• SQL (MySQL, PostgreSQL)
• Cloud platforms (AWS, Google Cloud, Azure)
• Python or Java (optional)
• Docker
• Kubernetes
• Data warehousing solutions (Redshift, BigQuery, Snowflake)