- MidniteLead Data EngineerTECHApril 2022 - Today (2 years and 10 months)FranceLead Data Engineer responsible for the full design and implementation of greenfield data infrastructure
- BidnamicData EngineerJanuary 2021 - Today (4 years and 1 month)Data Engineer working with Python, Airflow, Postgres and AWS to develop a market leading Google Shopping bid optimisation solution.• Redesigned the entire source data ingestion process and migrated legacy code to new Google Ads API.• Researched, recommended and implemented best practice of continuous data quality monitoring in pipelines.• Designed and implemented a standardised library of reusable and fully unit tested data cleaning utilities.• Designed and implemented company parquet lake to provide analytics capabilities to the wider business, utilising AWS Athena as the query engine.• Designed internal tooling and components to standardise data lake and parquet lake file paths to provide integrity to the lake structures and underlying data.• Independently identified performance bottlenecks and delays, recommended and implemented best practice solutions, including SQL query optimisation and implementation of indexes and other best practices.• Identified Airflow anti-patterns causing performance bottlenecks and system reliability issues, recommended and implemented best practice solutions.• Migrated legacy exchange rates codebase to a simple, but powerful internally managed system.• Provided constructive feedback on PR reviews of colleagues and peers.• Ensured all work was thoroughly unit tested by using a TDD approach.
- graze.comData EngineerMarch 2020 - December 2020 (9 months)London, UKLead Data Engineer responsible for the design, implementation and maintenance of the Unilever UK Insights Hub, an Azure-based big data platform, enabling a shift to a data-driven decision-making process in the UK.• Researched, designed and implemented best-practices for key technologies in the architecture including Azure Data Lake Storage, Databricks Delta Lake, Spark, Data Factory and Azure SQL Data Warehouse• Identified limitations of Azure services and designed appropriate solutions to achieve business objectives.• Designed, implemented and maintained over 24 data pipelines, delivering over 100TB of data to data analysts, data scientists and business-oriented end users.• Worked with the Data Science team to provide clean and timely data, enabling the timely delivery of business objectives.• Assisted the CTO in key decisions of the overall architecture of the UK Insights Hub.• Trained, mentored and assisted with junior members of the data engineering and wider data team.
- Post-Graduate Diploma of Science, Data ScienceUniversity of Southern Queensland2019Post-Graduate Diploma of Science, Data Science
- Graduate Diploma Chartered Accountancy, Accounting and Business/ ManagementChartered Accountants Australia and New ZealandGraduate Diploma Chartered Accountancy, Accounting and Business/ Management
- Bachelor of Commerce (B.Com.), Accounting and ManagementUniversity of TasmaniaBachelor of Commerce (B.Com.), Accounting and Management