Job Description

A            Position Overview

Position Title

Junior Data Engineer - Azure / JDE(A)

Department

IT

       Level  / Band

Assistant Manager/ Manager

Role Summary: ' Junior Data Engineer - Azure' or SDE(A) will focus on developing data engineering solutions leveraging Azure platform and associated technologies following the Tata AIA architecture, development guidelines and principles. He/She will closely work with Data Lake Architects, Senior Data Engineers and other stakeholders involved in data lake design and development utilizing core DE practices and principles.   The Senior Data Engineer is expected to develop the required competency to lead and deliver successful outcomes in data projects and maintain the core system documentation. An ideal candidate for the position of SDE(A) should:

• Have excellent problem-solving and troubleshooting skills. This also includes the ability to search and find the optimal solutions for a Data Engineering problem.
• Have an innovative mindset to challenge existing solutions on their inefficiency and provide better alternatives.
• Be a team player having decent inter-personal capabilities to liaise with other Data Engineers, Data Scientists and Analysis, ML Engineers and other technical counterparts in a data-driven organization.
• Be able to follow the instructions for deploying artifacts in production environment.
• Be able to work in a fast-paced Agile environment with a Fail-Early / Fail-Fast mindset.
• Be having excellent verbal and written communication skills.
• Be flexible in providing extended support occasionally to meet project timelines
• Be able to flag issues at an early stage to avoid delivery risks.

B            Organizational Relationships

Reports to                                              

Data Lake Architect, Senior Data Engineers, VP - Data Initiatives

Supervises

NA

C            Job Dimensions

Geographic Area Covered

India

Stakeholders                                       Internal


                                                              External

N/A

D            Key Result Areas

Project / Delivery Management

·        Ensure the solution meets overall business requirements and adheres to modern Data Lake patterns.

·        Collaborate with project managers to estimate, plan and execute sprints.

·        Liaise with Data Lake Architect to understand core technical and functional requirements.

·        Develop extensible pipelines with a "Reuse Later" mindset to develop artifacts (code, libraries, patterns, consumable frameworks etc.) which can be utilized later with customizations.

·        Identify potential areas of improvement in order to optimize the pipelines

·        Follow quality assurance initiatives

·        DevOps  tools implementation for CI/CD.

·        Contribute to overall system documentation

Misc

·        Ensure compliance to AIA and TALIC rules and standards

·        Constantly align with the IT team, Vendor  team and business stakeholder

E            Skills Required

Technical

·        5+ yrs of experience in developing cloud-based analytics solutions with a minimum of 2 yrs core experience in MS Azure.

·        Strong development and problem-solving skills.

·        A fair understanding of commonly used data offerings on Azure.

·        Working knowledge in Azure Data Factory, Azure Synapse, Azure Data lake, Azure SQLand App Services.

·        Development of data workflows and pipelines using ETL or ELT approach, API based ingestions and Steaming solutions.

·        Knowledge of Dev-Ops processes and hands-on experience in implementing pipelines using CI/CD methodology

·        Working Knowledge of Python, Spark and Powershell

·        Good to have Pyspark, Scala.

·        Comfortable in adopting security-first development principles.

·        Knowledge of Snowflake, Tableau and Microsoft BI Stack is a plus.

Behavioral

Essential

Desired

Interpersonal skills

·         


Communication skills

·         


Creative thinking skills

·         


Supervising/Leadership skills

·         


Teamwork Skills

·         


Influencing skills

·         


Relationship Building skills

·         


Decision making skills

·         


F             Incumbent Characteristics

Essential

Desired

Qualification

B.E / B.Tech

· Azure Data Engineer

· Other Azure certifications

Experience

·         5+ yrs of overall experience developing Data Engineering solutions on cloud platforms

·         2+ yrs of experience with MS Azure.

·         Working knowledge in Azure Data Factory, Azure Synapse, Azure Data lake, Azure SQLand App Services.

·         DevOps  tools implementation for CI/CD.

·         Good interpersonal and communication skills with ability to build productive relationships across the participants in the ecosystem