Data Engineer – Spark & Scala

About the role

RED TIC is seeking a skilled Data Engineer with strong expertise in Apache Spark and Scala. You will be responsible for designing, building, and maintaining high‑performance data pipelines that process large volumes of data for our client’s analytics platform.

Key responsibilities

• Design, develop, and maintain robust data pipelines using Apache Spark and Scala.
• Integrate data from multiple sources into a unified data warehouse, ensuring data quality.
• Build and optimise both real‑time and batch processing workflows.
• Collaborate with data scientists, analysts, and other stakeholders to translate business requirements into technical solutions.
• Troubleshoot and debug complex data engineering issues to maintain system reliability.
• Monitor and optimise Spark jobs and Scala applications for performance, scalability, and cost efficiency.

Required profile

• 3+ years of experience in data engineering or a related field.
• Proven track record of working with large datasets and building end‑to‑end pipelines.
• Strong problem‑solving abilities and attention to detail.
• Excellent communication skills and the ability to work both independently and within a team.

Required skills

• Apache Spark (large‑scale data processing)
• Scala programming language
• Hadoop ecosystem, HDFS, or NoSQL databases
• SQL and relational databases (MySQL, PostgreSQL)
• Cloud platforms (AWS, Google Cloud, Azure)
• Familiarity with containerisation (Docker) and orchestration (Kubernetes)
• Knowledge of data‑warehousing solutions (Redshift, BigQuery, Snowflake)