Lead Data Engineer (Snowflake) @ Dell International

Job Description

Role Objective: We are seeking a Lead Data Engineer to join our team of data experts. The ideal candidate will have a passion for designing and implementing end-to-end data solutions, from data ingestion and processing to analytics and reporting. As a Lead Data Engineer, you will work closely with the Data Architect and other stakeholders to understand their data needs and provide solutions to meet those needs. The ideal candidate will have a strong background in designing, developing, and maintaining data pipelines and ETL processes using technologies such as Snowflake, DBT, Matallion in Data warehousing. As a Lead Data Engineer, you will work closely with the Data Architect and other stakeholders to understand their data needs and provide solutions to meet those needs. You will also be responsible for leading a team of data engineers and ensuring that all data engineering projects are completed on time and within budget. Roles & Responsibilities : Collaborate with the Data. Architect and other stakeholders to understand their data needs and provide solutions to meet those needs Design and implement end-to-end data solutions using technologies such as Snowflake, Apache Spark, and Hadoop Lead a team of data engineers and ensure that all data engineering projects are completed on time and within budget Implement novel query optimization, major security competencies with encryption. Solve performance issues and scalability issues in the system. Transaction management with distributed data processing algorithms Possess ownership right from start to finish. Build, monitor, and optimize ETL and ELT processes with data models Migrate solutions from on-premises setup to cloud-based platforms. Understand and implement the latest delivery approaches based on data architecture. Project documentation and tracking based on understanding user requirements. Perform data integration with third-party tools including architecting, designing, coding, and testing phases. Manage documentation of data models, architecture, and maintenance processes Continually review and audit data models for enhancement Maintenance of ideal data pipeline based on ETL tools. Coordination with BI experts and analysts for customized data models and integration Code updates, new code development, and reverse engineering Performance tuning, user acceptance training, application support Maintain confidentiality of data Risk assessment, management, and mitigation plans Regular engagement with teams for status reporting and routine activities Migration activities from one database to another or on-premises to cloud Skills & Experience Required: 8+ years of experience in data engineering and data architecture Strong experience in data engineering fundamentals (SQL, RDBMS, Data Models, Data Structures, orchestration, Devops etc.) Knowledge of SQL language and cloud-based technologies Good working knowledge of leveraging DBT (SQL and Python models) to perform transformations in Snowflake Experience in working with AWS S3/ Azure ADLS Storage Accounts and Snowflake. Strong experience building data pipelines with Spark and Python/Scala Strong experience building ELT pipelines (batch and streaming) in Snowflake cloud warehouse Able to write structured and efficient queries on large data sets using Statistical Aggregate functions and Analytical functions and reporting datamarts. Experience in working with Snowflake concepts like Snowpipe, Streams, Tasks, Cloning, TimeTravel, Data Sharing, Data Replication e.t.c. Handling large and complex datasets like JSON, ORC, PARQUET,CSV filesfrom various sources like AWS S3,Azure DataLake Gen2. Understanding customer requirements, analysis, design, development and implementation into the system, gather and define business requirements and enhancing business processes. Knowledge in Snowflake tools like Snowsight and SnowSQL and any partner connects. Performance tuning and setting up resource monitors Snowflake modeling roles, databases, schemas SQL performance measuring, query tuning, and database tuning ETL tools with cloud-driven skills SQL-based databases like Oracle SQL Server, Teradata, etc. Snowflake warehousing, architecture, processing, administration Data ingestion into Snowflake Enterprise-level technical exposure to Snowflake applications Experience with data modelling is a plus. Excellent problem-solving and analytical skills Ability to work independently and as part of a team. Experience working in an Agile environment. Skilled in building relationships with clients and in practice development activities. Excellent written and oral communication skills; Ability to communicate effectively with technical and non-technical staff. Education: BE/B.Tech, MCA, MSC, M.Tech

Employement Category:

Employement Type: Full time
Industry: IT Services & Consulting
Role Category: General / Other Software
Functional Area: Not Applicable
Role/Responsibilies: Lead Data Engineer (Snowflake)

+ View Contact

Keyskills:   SQL RDBMS Data Models Data Structures Devops DBT Python Scala Spark Snowflake Streams Cloning Data Sharing JSON ORC Performance tuning Query tuning Database tuning ETL tools Analytical skills Agile Communication skills SQL language AWS S3 Azure ADLS Snowpipe Tasks TimeTravel Data Replication PARQUET CSV files Snowsight SnowSQL Resource monitors Snowflake modeling SQL performance measuring Oracle SQL Server Teradata Data ingestion Data modelling Problemsolving Client relationship building