Data Engineer(NCS/Job/ 886)

Data Engineer

Technical Skills:

Responsibilities

• Designs and establishes secure and performant data architectures, enhancements, updates, and programming changes for portions and subsystems of data pipelines, repositories or models for structured/unstructured data.
• Analyzes design and determines coding, programming, and integration activities required based on general objectives and knowledge of overall architecture of product or solution.
• Writes and executes complete testing plans, protocols, and documentation for assigned portion of data system or component; identifies and debugs, and creates solutions for issues with code and integration into data system architecture.
• Ability to manage and execute on an E2E data project from architecture inputs, to laying pipelines, data processing and dashboard building
• Leads a project team of other data engineers to develop reliable, cost effective and high-quality solutions for assigned data system, model, or component.
• Collaborates and communicates with project team regarding project progress and issue resolution.
• Represents the data engineering team for all phases of larger and more-complex development projects.
• Provides guidance and mentoring to less experienced staff members.

Knowledge and Skills

• Using data engineering tools, languages, frameworks to mine, cleanse and explore data.
• Fluent in NoSQL & relational based systems.
• Fluent in Databricks, Py spark and NodeJS
• Fluent in complex, distributed and massively parallel systems.
• Strong analytical and problem-solving skills with ability to represent complex algorithms in software.
• Designing data systems/solutions to manage complex data.
• Strong understanding of database technologies and management systems.
• At least 1 year of exposure to working on AWS systems (Redshift and S3 must, Good to have EC2, lambda)
• Fluency in Python and Linux shell scripting and small-scale server maintenance
• Strong understanding of cloud-based systems/services.
• Database architecture testing methodology, including execution of test plans, debugging, and testing scripts and tools.
• Ability to lay pipelines across multiple systems is a key requirement
• Familiarity with BI visualization tools, e.g. Looker, Power BI, Tableau

Behavioral Skills:

• Ability to use a wide variety of open source technologies and tools
• Excellent written and verbal communication skills; mastery in English and local language.
• Ability to effectively communicate product architectures, design proposals and negotiate options at management levels
• Collaborates with peers, junior engineers, data scientists and project team.
• Typically interacts with high-level Individual Contributors, Managers and Program Teams.
• Leads a project requiring data engineering solutions development

Qualification:

• Bachelor’s or master’s degree in computer science, Information Systems, Engineering or equivalent.
• 4 to 6 years of experience