Data Engineer | Aws | Glue | Spark @ Best infosystems

Job Description

: Mandatory skills* : Data Engineer with AWS using Glue and Spark, Apache Airflow, NiFi, S3,Lambda, MySQL + Scrum master as nice to have Design and Implement end-to-end ETL solutions on AWS using Glue and Spark for data preparation and feature engineering of data science projects Be comfortable to convert SQL, Pandas code that data scientists use in experiment notebooks to proper pyspark applications Develop and maintain data orchestration pipelines using Apache Airflow and data ingestion using NiFi Improve on existing software, enhance code quality and organize code structure Implementation of unit and integration tests Test Driven Development

Employement Category:

Employement Type: Full time
Industry: Others
Role Category: Application Programming / Maintenance
Functional Area: Not Applicable
Role/Responsibilies: Data Engineer | Aws | Glue | Spark

+ View Contact

Keyskills:   AWS GLUE SPARK MySQL Scrum master Data Engineer Apache Airflow NiFi S3 Lambda