Involved in building and designing of new data pipelines for ingestion using Sqoop and transformation using Apache Spark with Java/PySpark.
Experience in Big Data ecosystem related technologies like Storm, development of Spark streaming with Kafka, NiFi, HBase, Phoenix and Hive.
Experience in Apache spark(core structured streaming ) using scala
Experience in Administration, configuration management, monitoring, debugging and performance tuning of Hadoop applications, YARN and HDFS.
Exploring with the Spark on improving the performance and optimization of the existing algorithms in Hadoop using Spark Context, SparkSQL, Data Frames.
Managing and monitoring data using Hive/ HQL.
experience in Apache Spark
experience in java Hive
Python experience
HDFS
Bigdata
SQL
DB2
Data Engineering
Keyskills: Performance tuning Db2 Configuration management Debugging SCALA sqoop Monitoring SQL Python HBase
As VITO wants to accelerate the transition to a sustainable world, we work in India, where fast transitions are taking place and where we can have an impact. We have undertaken research projects in collaboration with India since 2007, starting with bio-economy and green chemistry.