Principal Data Engineer
Horizontal Talent

Projects the candidate will be working on:

• Principle Data Engineer will support the  Digital KPI dashboard project. The project has just started and wee adding more capacity to achieve end of the year goals.

Ideal Background:

• Experience Azure developer. Expert in SQL Queries. Experience in Adobe Click stream. Experience in Azure data bricks in current project is must.

Experience:

• Experience in Azure and data bricks, CICD, Jenkins, Scala or Python. Excellent in writing/understanding complicated queries.
• Must know containerization of data.
• Experience working on Interconnectivity between Azure and AWS in data sharing.
• Experience in Azure and data bricks, CICD, Jenkins, Scala or Python. Excellent in writing/understanding complicated queries.
• Must know containerization of data. Experience working on Interconnectivity between Azure and AWS in data sharin
• 8+ years of experience in distributed systems, data engineering, software engineering, or similar fields
• 5+ years of solid coding skills in a language such as Scala, Java, or Python
• 4+ years of experience with CI/CD development environment
• 2+ years of experience with Azure Cloud with Databricks technologies
• 1+ years of Experience with building APIs

Team and Team size:

• One Scrum team with 9 team members: 1 SM, 1 Product Owner, 1 Tech Lead, 4 Data Engineers (Onshore/Offshore) 2 Data Analyst.

Responsibilities:

• Build and deploy performance and scalable solutions by applying data engineering concepts and development best practices
• Build scalable and end to end Spark pipelines for large-scale healthcare data
• Ensure high quality solutions by design and build unit tests, integration test, performance test and user acceptance tests
• Optimize data pipelines to handle growing data volumes
• Collaborate on designs for data pipelines and data tooling that meet performance and data quality standards
• Build methods and tools to automate development activities
• Ability to prioritize tasks and work concurrently on multiple tasks
• Participate in business analysis and data discovery to gather data requirements
• Ability to build complex but compact well performing SQL queries.

Required Qualifications:

• 8+ years of experience in distributed systems, data engineering, software engineering, or similar fields
• 5+ years of solid coding skills in a language such as Scala, Java, or Python
• 4+ years of experience with CI/CD development environment
• 2+ years of experience with Azure Cloud with Databricks technologies
• 1+ years of Experience with building APIs

Preferred Qualifications:

• Experience with DevOps, Continuous Integration, and Continuous Delivery
• Experience developing Java RESTful Services using Spring Boot
• Experience building and deploying applications to the Microsoft Azure cloud using Infrastructure as Code tools, such as Terraform
• Experience working in an Agile environment
• Background in the healthcare industry
• Expertise with modern programming languages, systems, and architectures
• Develop analysis and design of transactional systems and/or programs