Tech Lead - Bengaluru/Bangalore - Tech @ Tech Mahindra

Required skills:

• Strong Experience in design and development in Azure Datalake solutions
• Strong experience in SQL analytic queries
• Strong experience in designing data distributions strategy
• Design a partition strategy for efficient analytical workloads
• Designing incremental data load
• Designing Meta stores in Azure SQL/Synapse and Azure Databricks
• Strong experience in Data transformations using Apache Spark, T-SQL, Data factory, Encode and decode data, Data Exploratory analysis
• Strong experience in batch loads using Data Factory, Data lake, PySpark, Function Apps
• Strong experience in Python, Java
• Strong experience in Data streaming components like Azure Event hubs, Kafka etc.
• Strong experience in developing application in Azure Databricks

Tools:

• Azure Data Factory

• Azure Databricks

• Azure Storage Accounts

• Azure Function Apps

• Azure SQL and Azure Synapse

• Azure Event Hubs, Stream Analytics and Kafka

• Java or Python

• Apache Spark and Spark Streaming

Roles & Responsibilities:

• Design and develop data pipelines using Azure Data Factory to various components like Azure SQL/Synapse or storage accounts.
• Design and develop full load and incremental loads using ADF to Azure SQL/Synapse
• Develop real time data processing applications using Azure Event Hubs, Kafka and Azure Stream analytics
• Connect to different source systems and collect the data in scheduled time and process them using different components like Azure Data bricks, Function Apps etc., and distribute it to the downstream applications or downstream locations like Azure SQL/Synapse, Amazon S3/RedShift etc.
• Data cleansing and pruning with python libraries like Pandas, Numpy etc.
• Develop cost effective, serverless applications using Azure Function apps (Java, Python or any supported language) to process and transform data

Interested candidates please share your cv to bp*******5@te*********a.com