Advanced Software Engr

We are looking for a Senior Data Engineer with 6+ years of experience to design, build, and scale cloud‑native data and AI platforms on Azure using Databricks. The role requires strong hands‑on expertise in data engineering, lakehouse architecture, and AI/ML data pipelines to support advanced analytics, machine learning, and business intelligence use cases.

The ideal candidate will lead complex data initiatives, collaborate closely with data scientists and ML engineers, and play a key role in shaping the organization’s data and AI strategy.

• Architect and develop end‑to‑end data pipelines on Azure using Databricks (Spark / PySpark)

• Design and maintain lakehouse architectures using Azure Data Lake + Delta Lake

• Build and optimize batch and streaming pipelines for large‑scale datasets

• Create and manage feature pipelines and curated datasets for AI/ML model training and inference

• Collaborate with data scientists and ML engineers to enable scalable ML workflows

• Support MLOps pipelines, including data versioning, feature stores, and model deployment readiness

• Optimize Databricks workloads for performance, scalability, and cost efficiency

• Implement data quality, validation, monitoring, and observability frameworks

• Ensure data security, governance, and compliance using Azure and Databricks best practices

• Review code, define standards, and mentor junior and mid‑level data engineers

• Lead architectural decisions and contribute to data platform roadmap planning