Full-stack Data Engineer II

About the role

We are looking for a forward‑thinking Full‑Stack Data Engineer II to join our team in Egypt. This is not a traditional ETL role; you will work at the forefront of the Data + AI era, using Databricks to build intelligent data products and agentic AI workflows that enable autonomous business decisions.

Key responsibilities

• Design, create and optimise Delta Lake tables in Databricks using Unity Catalog for data governance.
• Develop and deploy LLM‑based agentic workflows with LangChain, AutoGen or Databricks Mosaic AI, including Retrieval‑Augmented Generation (RAG) patterns.
• Build Python backend services (FastAPI) to expose AI‑driven insights.
• Manage end‑to‑end data pipelines with Databricks Workflows and Delta Live Tables (DLT).
• Implement monitoring frameworks to ensure agent accuracy, safety and performance.

Required profile

• 2–4 years of professional experience in data engineering.
• Strong hands‑on experience with PySpark, SQL and Databricks (Unity Catalog, DLT, Notebooks).
• Solid understanding of Lakehouse architecture and data localisation requirements in Egypt.
• Experience building AI agents, prompt engineering and working with LLMs.
• Backend development expertise in Python and Docker.

Required skills

• PySpark
• SQL
• Databricks (Unity Catalog, Delta Live Tables, Workflows, Notebooks)
• Lakehouse architecture
• Large Language Models (LLMs) and prompt engineering
• LangChain, AutoGen, Databricks Mosaic AI
• Python
• FastAPI
• Docker
• MLflow (model tracking and deployment)
• Vector databases (Pinecone, Weaviate, Databricks Vector Search)

What we offer

• Opportunity to work on cutting‑edge agentic AI projects.
• Competitive compensation in EGP with performance bonuses.
• Hybrid work environment based in New Cairo.