Lead Data Engineer - BigQuery, Machine
- Synopsys
- 7 to 11 Yrs
- Other Kerala
- 2 months ago
- Email to a friend
- Report this job
Job Description
- 7 - 9 Years 1 Opening Trivandrum Role description As a Data Engineer, you will be responsible for designing, building and maintaining data pipelines and infrastructure that support the client Data Science team. You will work with Google Cloud Platform (GCP) tools and technologies to collect, store and process data from various sources and formats. You will also write scripts to transform and load data explicitly for use in the machine learning (ML) pipelines process. You will optimize data access and performance for ML applications and ensure data quality and availability for analytics/ML teams. You will collaborate with data scientists, analysts, engineers and other stakeholders to deliver data solutions that enable data-driven decision making and innovation. Key Responsibilities As a Data Engineer for the Data Science team, you will perform the following tasks: Architect, build and maintain scalable, reliable and secure data pipelines and infrastructure using GCP tools such as BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, etc. Write scripts to transform and load data for use in ML pipelines using languages such as Python, SQL, etc. Optimize data access and performance for ML applications using techniques such as partitioning, indexing, caching, etc. Ensure data quality and availability for analytics/ML teams using tools such as Data Catalog, Data Studio, Data Quality, etc. Monitor, troubleshoot and debug data issues and errors using tools such as Stackdriver, Cloud Logging, Cloud Monitoring, etc. Document and maintain data pipelines and infrastructure specifications, standards and best practices using tools such as Git, Jupyter, etc. Collaborate with data scientists, analysts, engineers, and other stakeholders to understand data requirements, provide data solutions and support data-driven projects and initiatives. Support generative AI projects by providing data for training, testing and evaluation of generative models such as Gemini, Claude 3, etc. Implement data augmentation, data anonymization and data synthesis techniques to enhance data quality and diversity for generative AI projects. Qualifications To be successful in this role, you will need to have the following qualifications: Bachelor's degree in Computer Science, Engineering, Mathematics, Statistics or related field. experience in data engineering, data warehousing, data integration or related field. Proficient in GCP tools and technologies for data engineering, such as BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, etc., or equivalents in other cloud platforms Proficient in scripting languages such as Python, SQL, etc. for data transformation and loading. Knowledge of data modeling, data quality, data governance and data security principles and practices. Knowledge of ML concepts, frameworks and tools such as TensorFlow, Keras, Scikit-learn, etc. Knowledge of generative AI concepts, frameworks and tools. Excellent communication, collaboration and problem-solving skills. Skills BigQuery, Machine Learning, Python/SQL About UST UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the worlds best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients organizations. With over 30,000 employees in 30 countries, UST builds for boundless impacttouching billions of lives in the process.,
Employement Category:
Employement Type: Full timeIndustry: IT Services & Consulting
Role Category: Not Specified
Functional Area: Not Specified
Role/Responsibilies: Lead Data Engineer - BigQuery, Machine
Contact Details:
Company: UST GlobalLocation(s): Other Kerala
+ View Contact
Keyskills: Machine Learning Python SQL BigQuery
Fraud Alert to job seekers!
₹ Not Specified
Similar positions
Synopsys
Synopsys, Inc. (Nasdaq:SNPS) is a world leader in delivering semiconductor design software, intellectual property (IP), design for manufacturing (DFM) solutions and professional services that companies use to design systems-on-chips (SoCs) and electronic systems. The company?s products enable semico...
Plugin template missing! Fix or contact support.
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in