Principal Data Engineer

• Lead the discovery phases and technical designs for medium to large data and analytical solutions across multiple teams. Carry out effective technical design reviews to ensure that the right architecture patterns are used by the team. Drive proof of concepts and prototypes to validate ideas. 


• Complete data engineering coding and design tasks on problems of high scope and complexity, in particular for building frameworks that promote automation and reusability. Set up and configure new technology platforms, tools, and libraries. Demonstrate good coding principles and exercise good judgment in designing and building solutions. Conduct solution design reviews for peers. Ensure solutions adhere to published data privacy and cybersecurity principles. 


• Implement and practice fit-for-purpose estimation techniques, to promote iterative delivery, both at the sprint/story level and the initiative/epic level. Help the program team in refining estimates, with optimal assumptions and inter-dependency understanding. Mentor and coach engineers on effective estimation techniques. 


• Operate with a data-driven mindset. Work with team members to envision solutions as a set of data products and data pipelines. Ensure teams are set up to continuously enhance analytical solutions based on new business requirements, without the need for re-training and re-work. 


• Ensure solutions being built are stable, scalable, and maintainable. Enable test automation and ensure CI/CD pipelines are in good health. Implement monitoring of data applications and track product quality, performance, and stability. Drive corrective, adaptive, preventative, and perfective actions to maintain solution reliability and quality. 


• Provide technical leadership to members of the cross-functional team to address areas of inefficiency and resolve technical issues. Identify activities resulting in optimal resource utilization, cost reduction, technical debt remediation, service improvement, and reuse value. Partner with architects and product owners to prioritize and implement such activities. 


• Ensure teams keep data inventories and registries updated, following agreed Data Governance standards, guidelines, and principles. Help shape Data Governance guidelines, engineering standards, data analysis templates, and knowledge base. 


• Champion development of best engineering practices and modernization techniques. Partner with Enterprise Data & Analytics leadership team and Enterprise Architects for developing and implementing said practices, as well as promoting solution reusability, process automation, built-in-quality, test-driven development, agile delivery, timely root cause analysis, and blameless incident post-mortems. Key contributor in building and adopting data engineering playbooks for the relevant technologies, and ensuring adherence to said playbooks, as well as other published coding standards data technology blueprints.


• Mentor and coach data engineers on writing efficient code, and automating test suites and other development frameworks. Manage some less senior data engineers and guide them through a career development framework. Build fit-for-purpose onboarding guides and robust feedback mechanisms to ensure an optimal experience for data engineering talent and continuous improvement. Help identify any skills gaps and support staff movements and rotations.


• Manage multiple assigned teams, with accountability for hiring top talent, defining development goals, and measuring performance. Contribute to and support building a scalable data engineering team model. Identify any skills or resource gaps and assist chapter leads and other data program leaders in closing those gaps. Work with partner and/or offshore teams to accelerate delivery.

Qualifications:

• 8+ Years of experience or Degree or Honours (12+3 or equivalent), Degree in a relevant field such as Computer Science, Computational Mathematics, Computer Engineering or Software Engineering. 


• Specialization or electives in a Data & Analytics field (e.g. Data Warehousing, Data Science, Business Intelligence) a nice-to-have Experience in Data Engineering (Fewer years's experience will be considered for Masters degree holders) 

Key Requirements:

• Minimum 8+ years of leading development efforts and providing support for enterprise analytic applications such as Data Lake and Data Warehouse (including using the Big Data stack and Microsoft Azure cloud infrastructure)  


• Experience with batch and real-time data ingestion, experience with designing and coding pipelines that handle massive quantities of data (structured and unstructured), securely and in a timely fashion  


• Deep understanding of data architecture concepts such as data modeling, Big Data storage, Lambda architecture, data vault and dimensional modeling 


• Deep understanding of system integration architecture, and demonstrated experience building integration with various types of source systems, able to design and develop loads of operational systems data into a single data platform using variety of data integration tools  


• Experience in building reusable components - Experience in introducing new technologies and practices


• Experience designing monitoring and alerting solutions for data pipelines and data repositories  


• Experience and knowledge of a wide variety of testing methods and tools covering functional, performance, regression, system integration, smoke testing of Big Data, ETL (Extract-Transform-Load) Business Intelligence applications and pipelines  


• Operates with efficiency and automation in mind, experience building reliable, re-usable automation frameworks (e.g. CI/CD)  


• Strong SQL querying skills required - Experience working with partner and/or offshore teams is preferred  


• Experience coaching and mentoring less senior talent, experience directly managing staff is preferred  


• Airline industry experience a nice-to-have Knowledge/Skills: - Experience participating in data/analytics product evaluations and proof-of-concepts  


• Operates with a "You Code It, You Own It" mindset (i.e. supports the products they build)  


• Demonstrated problem-solver, able to design and document complex solutions independently  


• Team player, able to collaborate with other senior resources to remove blockers, solve complex design problems, and debug/resolve issues  


• Good communicator, strong influencing skills for participating in decision-making forums, stakeholder management or socializing data/analytics strategies 


• Able to deliver solutions (and associated value) interactively  


• Is accountable and displays a positive attitude  


• Self-starter and has a passion for exploring and learning new technologies, especially those in the Enterprise Data & Analytics space Technology Domain