Python Data Engineering with AI

Global Controllers Technology is looking for a hands-on data engineer who is passionate about solving business problems through innovation and engineering practices. In this role, you will leverage your deep technical knowledge to drive the creation of a scalable and high-quality data intensive platform leveraging some of the latest methodologies such as Data Virtualization and Data Domains. The ideal candidate should have a strong foundation in data engineering principles, understanding of different data pipelines and the ability to optimize and scale data processing and analytics solutions.

Qualifications:

Core Data Engineering Skills

- 5+ years overall experience in Big Data or Enterprise large-scale application development using scalable tools like Databricks, Scala, Java, and the Python ecosystem.

- Mandatory skills - data engineering, Python

- Deep understanding of data modeling, data warehousing concepts, methodologies, and best practices.

- Strong experience with data processing (e.g., Spark, Starburst, Snowflake, Redshift), storage (e.g., Hadoop, MongoDB, Oracle), and ETL (e.g., Airflow, Ab Initio, Talend) solutions.

- Experience with data engineering techniques such as building data lakes and data warehouses, data mesh architectures, data pipelines, and ETL vs. ELT patterns.

- Experience with real-time data processing and data virtualization technologies such as Starburst, Snowflake, and Flink (preferred).

- Experience with designing and developing distributed systems, handling structured and unstructured data to store, analyze, and report.

- Experience with developing and maintaining large and complex database systems using relational databases like Oracle, and other big data technologies like Hadoop, Spark, etc.

- Proficient in PL/SQL and Unix Shell scripts. Strong SQL, PL/SQL development, and database tuning experience.

- Experience in designing Online Transaction Processing (OLTP), Operational Data Store (ODS), and Data Warehouse applications.

- Experience with Data Quality and data lineage.

AI Agents, Orchestration & Applied AI

- Demonstrable experience with AI Agents, including their design, development, and integration into data engineering workflows and software systems.