Sr. Data Engineer Ai India Innovations

About Ai India Innovations

Job Description

You are a talented Data Engineer with a strong background in data engineering, and we are seeking your expertise to design, build, and maintain data pipelines using various technologies, focusing on the Microsoft Azure cloud platform. Your responsibilities will include designing, developing, and implementing data pipelines using Azure Data Factory (ADF) or other orchestration tools. You will be required to write efficient SQL queries for data extraction, transformation, and loading (ETL) into Azure Synapse Analytics. Utilizing PySpark and Python, you will handle complex data processing tasks on large datasets within Azure Databricks. Collaboration with data analysts to understand data requirements and ensure data quality is a key aspect of your role. You will also be responsible for designing and developing Datalakes and Warehouses, implementing data governance practices for security and compliance, monitoring and maintaining data pipelines for optimal performance, and developing unit tests for data pipeline code. Working collaboratively with other engineers and data professionals in an Agile development environment is essential. Preferred Skills & Experience: - Good knowledge of PySpark & working knowledge of Python - Full stack Azure Data Engineering skills (Azure Data Factory, DataBricks, and Synapse Analytics) - Experience with large dataset handling - Hands-on experience in designing and developing Datalakes and Warehouses Job Types: Full-time, Permanent Schedule: - Day shift - Monday to Friday Application Question(s): - When can you join Mention in days. - Are you serving Notice Period (Yes/No) - What is your current and expected CTC Education: - Bachelor's (Preferred) Experience: - Total work: 6 years (Required) - Data engineering-Azure: 6 years (Required) Location: - Pune, Maharashtra (Required) Work Location: In person Only immediate joiners are preferred.,