Tech Lead (Sr. Data Engineer) Job in
- Kpr sugar apperals
- 5 to 9 Yrs
- Other Maharashtra
- 1 month ago
- Email to a friend
- Report this job
Job Description
- Job Title: Tech Lead (Sr. Data Engineer with 9+ Years of Experience) Location: Pune Job Type: Full-time, Onsite Edgmatics Technologies is a leading global consulting and solutions integrator in the field of data management, business intelligence, and analytics. We specialize in delivering end-to-end data management solutions to help organizations make data-driven decisions and achieve their business objectives. To know more about Edgematics visit our website - www.edgematics.ai About This Role We are looking for a talented and experienced Data Engineer with Tech Lead with hands-on expertise in any ETL Tool with full knowledge about CI/CD practices with leading a team technically more than 5 and client facing and create Data Engineering, Data Quality frameworks. As a tech lead must ensure to build ETL jobs, Data Quality Jobs, Big Data Jobs performed performance optimization by understanding the requirements, create re-usable assets and able to perform production deployment and preferably worked in DWH appliances Snowflake / redshift / Synapse Responsibilities Work with a team of engineers in designing, developing, and maintaining scalable and efficient data solutions using Any Data Integration (any ETL tool like Talend / Informatica) and any Big Data technologies. Design, develop, and maintain end-to-end data pipelines using Any ETL Data Integration (any ETL tool like Talend / Informatica) to ingest, process, and transform large volumes of data from heterogeneous sources. Have good experience in designing cloud pipelines using Azure Data Factory or AWS Glues/Lambda. Implemented Data Integration end to end with any ETL technologies. Implement database solutions for storing, processing, and querying large volumes of structured and unstructured and semi-structured data Implement Job Migrations of ETL Jobs from Older versions to New versions. Implement and write advanced SQL scripts in SQL Database at medium to expert level. Work with technical team with client and provide guidance during technical challenges. Integrate and optimize data flows between various databases, data warehouses, and Big Data platforms. Collaborate with cross-functional teams to gather data requirements and translate them into scalable and efficient data solutions. Optimize ETL, Data Load performance, scalability, and cost-effectiveness through optimization techniques. Interact with Client on a daily basis and provide technical progress and respond to technical questions. Implement best practices for data integration. Implement complex ETL data pipelines or similar frameworks to process and analyze massive datasets. Ensure data quality, reliability, and security across all stages of the data pipeline. Troubleshoot and debug data-related issues in production systems and provide timely resolution. Stay current with emerging technologies and industry trends in data engineering technologies, CI/CD, and incorporate them into our data architecture and processes. Optimize data processing workflows and infrastructure for performance, scalability, and cost-effectiveness. Provide technical guidance and foster a culture of continuous learning and improvement. Implement and automate CI/CD pipelines for data engineering workflows, including testing, deployment, and monitoring. Perform migration to production deployment from lower environments, test & validate Must Have Skills Must be certified in any ETL tools, Database, Cloud.(Snowflake certified is more preferred) Must have implemented at least 3 end-to-end projects in Data Engineering. Must have worked on performance management optimization and tuning for data loads, data processes, data transformation in big data Must be flexible to write code using JAVA/Scala/Python etc. as required Must have implemented CI/CD pipelines using tools like Jenkins, GitLab CI, or AWS CodePipeline. Must have managed a team technically of min 5 members and guided the team technically. Must have the Technical Ownership capability of Data Engineering delivery. Strong communication capabilities with client facing. Bachelor's or Master's degree in Computer Science, Engineering, or a related field. 5 years of experience in software engineering or a related role, with a strong focus on Any ETL Tool, database, integration. Proficiency in Any ETL tools like Talend , Informatica etc for Data Integration for building and orchestrating data pipelines. Hands-on experience with relational databases such as MySQL, PostgreSQL, or Oracle, and NoSQL databases such as MongoDB, Cassandra, or Redis. Solid understanding of database design principles, data modeling, and SQL query optimization. Experience with data warehousing, Data Lake , Delta Lake concepts and technologies, data modeling, and relational databases. Familiarity with tools and platforms for CI/CD, such as Jenkins, GitLab CI/CD, or CircleCI. Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform. Strong understanding of software development lifecycle (SDLC) and CI/CD best practices. Excellent problem-solving skills and the ability to work independently as well as part of a team. Strong programming skills in languages such as Java, Python, or Scala. Preferred Skills Understands Data Quality process. Understands Data Governance. Understands full DEVOPS implementation. Understands Data Security Understands Master Data / reference Data Data Engineering in Cloud based technologies (AWS/Azure Stack),
Employement Category:
Employement Type: Full timeIndustry: IT Services & Consulting
Role Category: Not Specified
Functional Area: Not Specified
Role/Responsibilies: Tech Lead (Sr. Data Engineer) Job in
Contact Details:
Company: Edgematics GroupLocation(s): Other Maharashtra
+ View Contact
Keyskills: ETL Tool Data Engineering Big Data SQL Snowflake Java Scala Python Jenkins MySQL PostgreSQL Oracle MongoDB Cassandra Redis Data Warehousing Data Modeling Relational Databases AWS Azure Google Cloud Platform SDLC Azure Data Factory Redshift Problemsolving
Fraud Alert to job seekers!
₹ Not Specified
Similar positions
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in