Data Engineer:(NCS/Job/ 1849)

Must have Skills:

• 3+ yrs. Development experience with Spark (PySpark), Python and SQL.

• Extensive knowledge building data pipelines

• Hands on experience with Databricks Devlopment

• Strong experience with

• Strong experience developing on Linux OS.

• Experience with scheduling and orchestration (e.g. Databricks Workflows,airflow, prefect, control-m).

Good to have skills:

• Solid understanding of distributed systems, data structures, design principles.

• Agile Development Methodologies (e.g. SAFe, Kanban, Scrum).

• Comfortable communicating with teams via showcases/demos.

• Play key role in establishing and implementing migration patterns for the Data Lake Modernization project.

• Actively migrate use cases from our on premises Data Lake to Databricks on GCP.

• Collaborate with Product Management and business partners to understand use case requirements and reporting.

• Adhere to internal development best practices/lifecycle (e.g. Testing, Code Reviews, CI/CD, Documentation) .

• Document and showcase feature designs/workflows.

• Participate in team meetings and discussions around product development.

• Stay up to date on industry latest industry trends and design patterns.

• 3+ years experience with GIT.

• 3+ years experience with CI/CD (e.g. Azure Pipelines).

• Experience with streaming technologies, such as Kafka, Spark.

• Experience building applications on Docker and Kubernetes.

• Cloud experience (e.g. Azure, Google).