NiFi Data Engineer @ Nexwave Talent

Job Description

We are looking for a Senior NiFi Data Engineer who is able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building a data standardization and curation needs on Hadoop cluster using Apache NiFi as primary tool for integration and engineering. This is a high visibility, fast-paced key initiative will integrate data across internal and external sources, provide analytical insights, and integrate with the customer s critical systems. Requirements

• Bachelor s degree with a minimum of 5 years of IT experience, OR in lieu of Degree, a High School Diploma/GED
• Minimum 3 years hands-on experience in NiFi, HDFS, Kafka, Hive, Sqoop, shell scripting
• Good knowledge of the data transformation, service handling, error handling and processors available in NiFi.
• Able to transform heterogeneous data formats (Json, Xml, Avro, Parquet, Orc)
• Good knowledge on Batch and Streaming data processing
• Knowledge on improving Flow standards with exception and error handling with zero data loss
• Excellent communication skill
• Solution oriented and problem-solving skills
• Strong teamwork and interpersonal skills

Key Responsibilities

• Design, build and unit test applications on APACHE NIFI either on Hortonworks Data Flow ( HDF ) or on Cloudera Flow Management ( CFM ) platforms.
• Build data pipelines for both batch and streaming data requirements, which will require in- depth knowledge on Hive, Kafka, HDFS, Sqoop and NoSQL databases as well.
• Parse hierarchical XML and JSON documents using NiFi processors and flatten them as per data modelling requirements.
• Build complex transformations and expressions on FlowFile attributes and content within NiFi processors
• Design & Build REST-ful service pipelines on NiFi with appropriate error handling
• Also build NiFi pipelines which can perform GET-POST-PUT-DELETE method calls on those service pipelines
• Build NiFi controller services to interact with other databases (RDMBS+NoSQL) and for credentials & kerberos tickets retrieval purposes
• Publish NiFi flows into NiFi Registry and promote to higher environments
• Build orchestration flows for coordination and interaction among various NiFi data flows.
• Execute CLI commands to perform flow executions and promotions respectively
• Extensively use variables and context variables which hierarchically falls-through from parent to child NiFi flows
• Optimize performance of the built pipelines in Hadoop using configurations of different processors
• Optimize performance for data access requirements by choosing the appropriate native Hadoop file formats ( Avro, Parquet, ORC etc ) and compression codec, while writing into HDFS.
• Analyzing and processing large amounts of structured and unstructured data, including integrating data from multiple heterogeneous sources.
• Create and maintain integration and regression testing pipelines on Jenkins integrated with BitBucket and/or GIT repositories.
• Participate in the agile development process, and document and communicate issues and bugs relative to data standards in scrum meetings
• Work collaboratively with onsite and offshore team model.
• Develop & review technical documentation for artifacts delivered.
• Ability to solve complex data-driven scenarios and triage towards defects and production issues
• Ability to learn-unlearn-relearn concepts with an open and analytical mindset
• Participate in code release and production deployment.
• Challenge, inspire & mentor team members to achieve business results in a fast paced and quickly changing environment

Employement Category:

Employement Type: Full time
Industry: Recruitment Services
Role Category: General / Other Software
Functional Area: Not Applicable
Role/Responsibilies: NiFi Data Engineer

Contact Details:

Company: Techno Wise
Location(s): Hyderabad

+ View Contact

×

Login

Candidates can login here to view contacts and apply.

Sign In Sign Up

Email:

Password:

Password too short

To create your profile, apply for a job or make a registration

Sign In Close

Your name (*)

Email (*)

Mobile (*)

Preferred City (* max. 2 w/comma)

Designation / Expected Role

Current / Recent Company (*)

Experience (*)

0 - 1 Yrs 1 - 2 Yrs 2 - 3 Yrs 3 - 4 Yrs 4 - 5 Yrs 5 - 6 Yrs 6 - 7 Yrs 7 - 8 Yrs 8 - 9 Yrs 9 - 10 Yrs 10 - 11 Yrs 11 - 12 Yrs 12 - 13 Yrs 13 - 14 Yrs 14 - 15 Yrs 15 - 16 Yrs 16 - 17 Yrs 17 - 18 Yrs 18 - 19 Yrs 19 - 20 Yrs 20 - 21 Yrs 21 - 22 Yrs 22 - 23 Yrs 23 - 24 Yrs 24 - 25 Yrs 25 - 26 Yrs 26 - 27 Yrs 27 - 28 Yrs 28 - 29 Yrs 29 - 30 Yrs 30+ Yrs

Expected Salary (*)

Rs 0 - 50K PM Rs 50K - 1 LPA Rs 1.0 - 1.5 LPA Rs 1.5 - 2.0 LPA Rs 2.0 - 2.5 LPA Rs 2.5 - 3.0 LPA Rs 3.0 - 3.5 LPA Rs 3.5 - 4.0 LPA Rs 4.0 - 4.5 LPA Rs 4.5 - 5.0 LPA Rs 5 - 6 LPA Rs 6 - 7 LPA Rs 7 - 8 LPA Rs 8 - 9 LPA Rs 9 - 10 LPA Rs 10 - 12 LPA Rs 12 - 14 LPA Rs 14 - 16 LPA Rs 16 - 18 LPA Rs 18 - 20 LPA Rs 20 - 22 LPA Rs 22 - 24 LPA Rs 24 - 26 LPA Rs 26 - 28 LPA Rs 28 - 30 LPA Rs 30 - 32 LPA Rs 32 - 34 LPA Rs 34 - 36 LPA Rs 36 - 38 LPA Rs 38 - 40 LPA Rs 40 - 42 LPA Rs 42 - 44 LPA Rs 44 - 46 LPA Rs 46 - 48 LPA Rs 48 - 50 LPA Rs 50 - 75 LPA Rs 75 Lakh - 1C PA Rs 1 Crore+ PA

Desired Industry (*):

Accounting / Finance Advertising / MR / PR / Events Agriculture / Dairy Animation / Multimedia Architecture / Interior Design Astrology Automobile / Auto Ancillaries / Auto Components Aviation / Airline / Aerospace / Aeronautical Banking / Financial Services / Broking BPO / Call Center / ITeS Brewery / Distillery Broadcasting Cement / Building Material Chemical / Petro / Plastic / Rubber / Glass Consumer Electronics / Appliances / Durables Courier / Transportation / Freight Defence / Government Education / Teaching / Training Electricals / Switchgears Engineering / Construction / Cement / Metals Environment / Waste Management Export / Import / Trading Facility Management Fertilizers / Pesticides FMCG / Foods / Beverage Food Processing Furnishings / Ceramics / Sanitaryware / Electricals Gems / Jewellery Gifts / Toys / Stationary Government / Departmental Heat Ventilation / Air Conditioning Hotel / Restaurant Industrial Design Industrial Products / Heavy Machinery Insurance Internet / E-Commerce IT - Hardware / Networking IT - Software / Services KPO / Research / Analytics Leather / Leather Products Legal / Courts Logistics / Courier / Transportation Management Consulting / Strategy Manufacturing / Production Matrimony / Matchmaking Media / Entertainment Medical / Healthcare / Hospital Merchant Navy Metal / Iron / Steel MFI - Micro Finance Military / Police / Arms Ammunition Mining / Quarrying NBFC-Non Banking Financial Services NGO / Social Work Office Equipment / Automation Not Mentioned Paint / Art / Sculpture / Craft Paper / Wood Personal Care / Beauty Pharma / Biotech / Clinical Research Politics Power / Energy / Oil / Gas / Petroleum Printing / Publishing / Packaging Quality Certification Real Estate / Property Recruitment Services / RPO Religion / Spirituality Retail Security / Detective Services Semiconductors / Electronics Shipping / Marine Telecom / ISP Textile / Garments / Fashion Travel / Tourism / Hotels / Airlines / Railways Unskilled Labor / Domestic Help Veterinary Science / Pet Care Sports / Fitness / Beauty

Functional area / Department (*):

Accounts / Finance / Tax / CS / Audit Agent / Agency Analytics & Business Intelligence Architecture / Interior Design Banking / Insurance Beauty / Fitness / Spa Services Content / Journalism Corporate Planning / Consulting CSR & Sustainability Engineering Design / R&D Export / Import / Merchandising Fashion / Garments / Merchandising Guards / Security Services Hotels / Restaurants HR / Administration / IR IT- Hardware / Telecom / Technical Staff / Support IT Software - Application Programming / Maintenance IT Software - Client Server IT Software - DBA / Datawarehousing IT Software - E-Commerce / Internet Technologies IT Software - Embedded /EDA /VLSI /ASIC /Chip Des. IT Software - ERP / CRM IT Software - Mainframe IT Software - Middleware IT Software - Mobile IT Software - Network Administration / Security IT Software - Other IT Software - QA & Testing IT Software - System Programming IT Software - Systems / EDP / MIS IT Software - Telecom Software ITES / BPO / KPO / Customer Service / Operations Legal Marketing / Advertising / MR / PR Pharma / Biotech / Healthcare / Medical / R&D Packaging Production / Maintenance / Quality Purchase / Logistics / Supply Chain Sales / BD Secretary / Front Office / Data Entry Self Employed / Consultants Shipping Site Engineering / Project Management Teaching / Education Top Management Ticketing / Travel / Airlines TV / Films / Production Web / Graphic Design / Visualiser Other Areas

Enter Skills (key skills, subjects, technologies & roles to use in search)

Write briefly about yourself, your experience and education (*)

Attach Resume  Max 2.38 MB (RTF, PDF, DOC, DOCX formats only parsed)

Please, check the file size and type.

Add social media [ + ]

Create password

I agree with website service terms and conditions

Sign Up Close

Candidates are expected to provide most recent and accurate profile information, inappropriate content is strictly prohibited!

Keyskills:   data warehousing informatica python java it data flow sql