Minimum Big Data experience 3 years.
Minimum hands on with DataIku- 6 months
Minimum Overall Experience 6 years
Responsibilities for Big Data Engineer
Design and develop complex programs using Big Data Hive, PySpark and Python
Should have some experience on Dataiku platform for data preparation, data integration, and data analysis.
Understand complex data system by working closely with business & application teams
Perform data cleaning, integration, validation and analysis
Extensive experience of developing(must) , deploying (must) and maintaining(must) the Big Data Ecosystem
Assemble large, complex data sets that meet functional / non-functional business requirements
Create complex data processing jobs in PySpark to load data from RDBMS (Oracle) & process based on complex business rules (must)
Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues (must) and support their data infrastructure needs.
MUST have Experience with below big data tools:
Hadoop Ecosytem
Dataiku
Hive
Python
PySpark
Keyskills: level design data cleaning design big data unit testing high level design
IRIS SOFTWARE Inc Iris is a professional software services organization offering high-quality, cost-effective solutions to businesses. It has helped meet the IT requirements of companies ranging from those among the Fortune 100 to medium-sized firms by utilizing best-of-breed technologies, rapid...