Location: Infopark Phase 2, Kochi
Experience: 3+ Years
Company: CHISQUARE LABS
We are looking for a highly skilled Data Engineer with expertise in PySpark and Databricks to design, build, and optimize scalable data pipelines for processing massive datasets.
Key Responsibilities:
- Build & Optimize Pipelines: Develop high-throughput ETL workflows using PySpark on Databricks.
- Data Architecture & Engineering: Work on distributed computing solutions, optimize Spark jobs, and build efficient data models.
- Performance & Cost Optimization: Fine-tune Spark configurations, optimize Databricks clusters, and reduce compute/storage costs.
- Collaboration: Work closely with Data Scientists, Analysts, and DevOps teams to ensure data reliability.
- ETL & Data Warehousing: Implement scalable ETL processes for structured & unstructured data.
- Monitoring & Automation: Implement logging, monitoring, and alerting mechanisms for data pipeline health and fault tolerance.
Qualifications:
- Bachelor’s/Master’s in Computer Science, Data Engineering, or related field
- 3+ years of experience in Big Data Engineering with PySpark and Databricks
- Proficient in SQL, and data modelling
- Strong command over Python and Linux environments
- Bonus: Experience with Kafka or Healthcare data
- Familiarity with AWS, GCP, Azure, and storage solutions
Tech Stack:
- Python | PySpark | Databricks
- SQL | Delta Lake | Data Warehousing
- Linux | Shell Scripting
- Kafka (Optional)
- AWS / GCP / Azure
- Data Modelling
Why Join Us?
- Work on innovative projects with emerging technologies
- Collaborative and learning-oriented work culture
- Opportunities for growth and upskilling
If you’re passionate about automation and excited to grow in a future-ready tech environment, we’d love to hear from you!
