Data Engineer

Career Category

Information Systems

Job Description

Data Engineer (GCF 4)

What you will do

Let’s do this. Let’s change the world. We are looking for a talented Data Engineer, who is curious to learn and able to develop data engineering and data analytics solution in a fast-moving environment. Candidate will work closely with senior data engineer and product owner/business analyst to understand the requirement.

Role Summary

• Build and operate large-scale healthcare data pipelines across batch workflows, metadata-driven ingestion, and data service publishing.

• Own end-to-end engineering from source ingestion to conformed data products, with strong focus on reliability, data quality, and operational observability.

• Partner with analytics, business, and platform teams to deliver trusted datasets for sales, claims, activity, patient, and rare disease use cases.

Key Responsibilities

• Design and maintain PySpark/SQL pipelines in Databricks for landing, unified, unstitched, and published data layers.

• Build and support Airflow DAGs for scheduling, dependencies, retries, and production operations.

• Implement metadata/config-driven frameworks for ingestion, transformation, and rule-based processing.

• Develop robust data quality controls, DQ summaries, failure handling, and alerting workflows.

• Manage batch/process audit logs, run status tracking, release flags, and operational reporting.