TQuanta Technologies - Spark/Scala Developer
- VDart Technologies
- 5 to 10 Yrs
- All India
- 1 month ago
- Email to a friend
- Report this job
Job Description
- Job Description We are looking for an experienced Spark Scala Developer to join our dynamic team. The ideal candidate will have strong technical expertise in Apache Spark and Scala, along with hands-on experience in building, managing, and optimising large-scale data pipelines. This role offers an opportunity to work in a fast-paced environment, tackling complex data challenges, and contributing to the development of cutting-edge solutions. Key Responsibilities Design, develop, and maintain scalable data pipelines using Apache Spark and Scala. Optimise and transform large datasets into structured formats for analytics and machine learning models. Integrate data from various sources such as APIs, relational databases, and No SQL databases. Implement data validation and cleansing procedures to ensure data quality and consistency. Optimise Spark jobs for performance, ensuring efficient use of resources and cost-effective solutions. Analyse and resolve performance bottlenecks in distributed data processing systems Work closely with data engineers, architects, analysts, and other stakeholders to understand business requirements and deliver data-driven solutions. Provide technical guidance to junior developers and contribute to code reviews and best practices. Write and maintain unit tests to ensure code reliability and functionality. Document system designs, workflows, and processes for knowledge sharing and future reference. Monitor and troubleshoot production issues related to data processing pipelines. Implement robust error-handling mechanisms to ensure data pipeline resilience. Required Skills And Qualifications Experience: 5-10 years in software development with a focus on big data technologies. Proficiency in Scala programming language and functional programming concepts. Hands-on experience with Apache Spark (core, SQL, streaming, and MLlib modules). Strong understanding of distributed computing principles and architectures. Familiarity with data formats such as Avro, Parquet, JSON, and ORC. Experience with Hadoop, Hive, HDFS, and related tools is a plus. Familiarity with Kafka or similar message queue systems for real-time data processing. Proficient in SQL and working knowledge of relational databases (e.g., MySQL, PostgreSQL) and No SQL databases (e.g., Cassandra, MongoDB). Exposure to cloud-based data services like AWS EMR, Azure Data bricks, or GCP Data flow is highly desirable. Familiarity with Git and CI/CD pipelines for automated deployments. Soft Skills Strong problem-solving abilities and analytical mindset. Excellent communication skills to collaborate with technical and non-technical stakeholders. Ability to work independently and as part of a team in a dynamic, fast-paced environment. Preferred Qualifications Education : Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. (ref:hirist.tech),
Employement Category:
Employement Type: Full timeIndustry: IT Services & Consulting
Role Category: Not Specified
Functional Area: Not Specified
Role/Responsibilies: TQuanta Technologies - Spark/Scala Developer
Contact Details:
Company: TQuanta TechnologiesLocation(s): All India
+ View Contact
Keyskills: Apache Spark Scala functional programming distributed computing Avro JSON ORC Hadoop Hive HDFS Kafka SQL MySQL PostgreSQL Cassandra MongoDB Git
Fraud Alert to job seekers!
₹ Not Specified
Similar positions
VDart Technologies
VDart Technologies Pvt Ltd. supports its $100 Million Client VDart Inc which operates across Major GEO's, Leading IT Clients and Emerging Technologies
Plugin template missing! Fix or contact support.
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in