Ntsako Ngobeni

Overview: Worked at Capitec Bank Business, focusing on designing and implementing ETL processes and leading the development of credit data products for data science and the credit data warehouse.

Responsibilities:

• Designed and implemented high-volume data pipelines using AWS Glue, Lambda, Step Functions, Terraform, Airflow, and EMR.
• Utilized Apache Spark for data processing within EMR.
• Employed Redshift for data warehousing and S3 for the data lake.
• Managed job scheduling with Control M.
• Collaborated with data scientists in Credit Analytics, handling data acquisition, ingestion, and transformation.
• Led the development of the credit data warehouse, translating data models into SQL queries for building dimensions and facts.
• Provided training and support for junior data engineers.
• Automated data loads with Apache Airflow and Control M, monitored via PowerBI.
• Planned, guided, and led the data engineering team to ensure high performance and successful project delivery.
• Collaborated with various teams to design solutions, ensuring alignment and consensus, with individual decisions made primarily for ad hoc requests.

Skills:

• ETL Tools: AWS Glue, Lambda, Step Functions, Terraform, Apache Airflow, EMR
• Data Processing: Apache Spark
• Data Warehousing: Redshift, S3
• Job Scheduling: Control M
• Programming: Python, SQL
• CI/CD: GitLab
• Data Visualization: PowerBI

Overview: Cenfri is Part of the Mastercard Foundation team in Rwanda, focusing on digitizing Rwanda's economy by building and optimizing secure data infrastructures for government and private sector institutions.

Responsibilities:

• Developed cost-efficient data lakes and analytical environments in AWS using open-source tools.
• Utilized EC2 instances for:Apache Airflow (orchestrating Docker containers)
  Apache Spark (data processing cluster used by Airflow to get data from source)
  Metabase (reporting)
  OpenMetadata (data documentation)
  VPN (secure data access)
• Hosted data lake on RDS Postgres.
• Built reusable Python data pipelines for data ingestion.
• Collaborated with data analysts and stakeholders to structure and optimize data for analysis.
• Consulted with private and public sectors to implement and leverage data infrastructure for decision-making.

Skills:

• Cloud Platforms: AWS (EC2, RDS)
• Data Engineering: Python, Docker, Apache Airflow, Apache Spark
• Data Analysis and Reporting: Metabase
• Data Documentation: OpenMetadata
• Database Management: PostgreSQL
• Security: VPN configuration and management
• Consultation: Stakeholder engagement, capacity building, infrastructure optimization

Overview: Cash Connect is a cash management and payments solution company that secures cash, provides risk cover, and facilitates supplier payments from retail cash vaults.

Responsibilities:

• Built in-house data infrastructure on AWS for a data lake, integrating data from various business sources for historic reporting and BI.
• Part of the software engineering team, contributing to backend system development.
• Managed data sources including SAP, sales data, in-house applications, and data synchronization between systems.
• Provided ongoing support and maintenance for parts of the systems as needed.
• Utilized AWS tools and solutions similar to those used at Cenfri.

Skills:

• Cloud Platforms: AWS
• Data Engineering: Python, Docker, Apache Airflow, Apache Spark
• Data Warehousing: Redshift, S3
• ETL Tools: AWS Glue, Lambda, Step Functions, Terraform
• Job Scheduling: Control M
• Programming: Python, SQL
• CI/CD: GitLab
• Data Visualization: Metabase, PowerBI
• Software Development: Backend system development

Overview: Yoco provides card machines, online payments, and business tools to help businesses start, run, and grow.

Responsibilities:

• Built data pipelines into an existing data warehouse.
• Worked on data requests and reporting using Tableau/Periscope.
• Improved and enhanced the current infrastructure using AWS, Redshift, Python, and Apache Airflow.
• Worked with large transaction databases from card payments, focusing on optimizing SQL query performance.

Skills:

• Cloud Platforms: AWS
• Data Engineering: Python, Apache Airflow
• Data Warehousing: Redshift
• Data Visualization: Tableau, Periscope
• Programming: Python, SQL
• Database Management: Handling large transaction databases and query optimization

Overview: Statistics South Africa is the national statistical service of South Africa, dedicated to producing timely, accurate, and official statistics to advance economic growth, development, and democracy.

Responsibilities:

• Maintained applications and databases, implementing changes through change control and management.
• Worked with C# applications and ASP.NET applications.
• Managed SQL Server databases.
• Created datasets and updated queries for analysts.
• Performed application testing using automated testing tools.

Skills:

• Programming: C#, ASP.NET
• Database Management: SQL Server
• Data Preparation: Dataset creation, query optimization
• Application Testing: Automated testing tools
• Change Management: Implementing application changes through controlled processes

Overview: The Agincourt HDSS at the MRC/Wits Rural Public Health and Health Transitions Research Unit strengthens and extends a high-functioning health and socio-demographic surveillance system, serving as the scientific foundation for advanced research and intervention studies.

Responsibilities:

• Developed and updated software for data entry systems.
• Managed databases, including quality control, administration, security, and backups.
• Extracted high-quality data for researchers and contributed to scientific publications.
• Trained and managed junior team members.
• Developed and managed GIS systems.
• Created and maintained structures for GIS data storage and integrated the GIS database with the HDSS database.

Skills:

• Software Development: Data entry systems
• Database Management: Quality control, administration, security, backups
• Data Extraction: Preparing data for research and publications
• Team Management: Training and managing junior members
• GIS Systems: Development, management, and data integration

Overview: Hoedspruit Independent College, where I tutored maths for grade 8 and taught Computer Application Technology (CAT), focusing on computer applications, MS products, and basic computer usage.

Responsibilities:

• Tutored grade 8 maths.
• Taught Computer Application Technology subject to Grade 10-12 learners, covering computer applications and MS products.
• Developed a C# Windows application program for recording learner marks, which generated mark sheets, mark schedules, and learner reports used by all teachers.
• Managed SQL Server for storing and managing data related to learner marks.

Skills:

• Teaching: Maths, Computer Application Technology
• Software Development: C# Windows applications
• Database Management: SQL Server
• Educational Tools: Development of programs for academic record-keeping and reporting

• Data Collection, as a fieldworker for Municipal Mass property valuation
• Promoted to Data capturing and Data auditing
• Team leader for data entry clerks, and compiling reports for project manager.