Azure Databricks Consultant

Infosys is seeking a highly skilled Azure Databricks Engineer with 6 to 8 years of experience in designing, developing, and optimizing cloud-based data engineering solutions. The ideal candidate will possess strong expertise in Azure Databricks, PySpark, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), SQL, and Data Engineering practices.

Design, develop, and maintain scalable Azure Databricks solutions for enterprise data platforms. Build and optimize end-to-end ETL/ELT pipelines using PySpark, Spark SQL, and Azure Data Factory (ADF). Develop robust data ingestion frameworks from multiple structured and unstructured data sources. Implement and manage Delta Lake and Lakehouse Architecture for high-volume data processing. Create and optimize data models, transformations, and workflows to support analytical and business requirements. Perform performance tuning and troubleshooting of Databricks workloads and Apache Spark jobs. Work with Azure services including ADLS Gen2, Azure Synapse Analytics, Azure Key Vault, and Azure Data Factory. Collaborate with business stakeholders, architects, and cross-functional teams to deliver data solutions. Ensure adherence to data governance, security, and best practices. Participate in code reviews, CI/CD implementation, testing, deployment, and production support activities.