Data Automation Engineer

About the role

The Data Automation Engineer will design, build, and maintain automated data pipelines that support both real‑time and batch processing across the organization. You will collaborate closely with data analysts, machine learning teams, and security specialists to ensure reliable, secure, and high‑quality data flows.

Key responsibilities

• Develop and implement automated pipelines for ingesting, transforming, and storing structured and unstructured data.
• Integrate data from multiple sources using real‑time and batch mechanisms, ensuring consistency and reliability.
• Implement data quality checks, monitoring, and alerting to proactively address issues.
• Write automation scripts in Python, Java, or Scala and leverage ETL tools such as Apache Spark and AWS Glue.
• Collaborate with intelligent data teams to support machine‑learning and analytics applications.
• Document data processes, system architecture, and codebase for knowledge sharing.

Required profile

• At least 1 year of experience in data engineering or automation.
• Strong understanding of data pipeline lifecycle and data quality methodologies.
• Excellent analytical, problem‑solving, and communication skills.

Required skills

• Python, Shell scripting
• Java or Scala
• Apache Spark, AWS Glue
• Airflow or Luigi (preferred)
• Cloud platforms: AWS, Azure, GCP