Leighton Obrien
As a member of data engineering team, I palyed crucial role in improving data ingestion, modernlising ETL pipeline, creating report portal and infrastructure provisioning. I introduced numerous best practices and standards to the team that increased the effectiveness and reliability of our services delivery .
Data Warehouse
- Data ingestion
- Data lake
- Wetstock analysis
- Data warehouse
- Reporting and alarm
Python2 to 3
- Upgrade Python2 to Python3
- Merged 6 copies of client-featured scripts set into one git repo, extracted the difference as configurations
Move to AWS/Azure
- Serverless Application
- Lambda/AWS Batch
- RDS/Redshift
- SQS/SNS/Kinesis
- Router53/APIGateway
- Glue/Authena
- Cloudwatch/AWS Batch/SSM
POC move Kernel IP into Python
Worked with primary data scientist and finished a POC project to migrate IP intelligence to a modern Python implementation
- Jupyter
- Pandas/Polar
- Numpy
- Matplotlib/Seaborn/Plotly
CI/CD pipelines
- Jenkins
- Github Actions
- AWS SSM
- Shell/Powershell script
Linux Maintenance
- Managed 80+ Linux servers
- Upgrade 40+ Ubuntu 14/18 to newest LTS 22.04
- Security/Vulnerability fix
Technical support for whole team
- Python packages
- Linux
- Docker
- Security/Cryptography
- Open Source Community
