Senior Data Engineer

Designs and develops data models and data pipelines to enable efficient data collection and processing from diverse data sources. Implements data governance policies and procedures for data handling to manage data consistency, integrity, accuracy, security, and reliability. Implements rigorous data validation and integrity checks to support data pipeline and model performance. Designs, develops, and optimizes automated and scalable data pipeline architectures to build data products, independently. Collaborates in an agile development environment to develop, maintain, and debug data solutions that are scalable, efficient, and reliable.

Key Responsibilities

Data Processing & Pipelining – Data Requirements, Collection, and Infrastructure:

- Analyzes business requirements

- Analyzes data sources to ensure successful data extraction and pipeline development.

- Designs data models for optimal data processing and reporting performance.

- Designs and implements data pipelines for optimal data processing.

- Conducts comprehensive testing of deliverables to ensure quality, accuracy, and reliability.

- Identifies, analyzes, and troubleshoots data issues, such as mismatches, refresh errors, and inconsistencies.

- Analyzes deployment statistics, customer feedback, and platform enhancements to identify and drive product improvements.

Data Processing & Pipelining – Data Governance:

- Implements data governance policies and procedures for data handling (e.g., data retention) to manage data consistency, integrity, accuracy, and reliability throughout the data lifecycle.

- Redacts Personally Identifiable Information (PII) and Protected Health Information (PHI) data to ensure compliance with data privacy and security standards.

- Follows data security measures to protect data from unauthorized access, use, disclosure, alteration, or destruction.

- Ensures data compliance with relevant laws, regulation, and industry standards.

Data Processing & Pipelining – Data Validation & Quality Assurance:

- Implements rigorous data validation and integrity checks, identifying and addressing any data quality issues that could impact data pipeline and model performance.