Senior GCP Data Engineer

Our Scrums.com Story

At Scrums.com (formerly SovTech), we believe the future of software lies in the talent, innovation, and creativity found on the African continent. For over a decade, we have been pioneering world-class software development for a global audience. We are the leading custom software development company from Africa that, like any great tech company, was founded in a garage. Scrums.com has now grown to more than 350 of the best AI-powered software engineering experts, building world-class platforms for growing businesses across four cities on three continents.

‍Looking ahead, we remain committed to growing Africa's tech ecosystems. Africa is home to the world's youngest population, with over 50% of Africans under the age of 25. Recent estimates indicate there are over 700,000 professional software developers across Africa today. That number is projected to double by 2025. We believe in the future of African tech talent and the impact it will have on scaling global companies.

Accelerating business growth with expert tech talent

Trusted by 400+ companies, Scrums.com helps businesses scale with our cloud-based engineering subscription platform, offering consistent, cost-effective AI-powered experts, teams, and reliable developer analytics. 

Our Culture

Our culture is underpinned by 8 core values. 

Earth’s leading software company from Africa- that's where we're going.

Why Join Us?

Be part of a dynamic, people-first team that’s showcasing and advancing Africa’s tech landscape on the global scale. At Scrums.com, we celebrate your individuality, empower your decision-making, and create meaningful experiences. Together, we’re driving innovation and building  exceptional teams 🚀 Take a look at some of the highlights:

Embrace Our African Talent Footprint, Flexibility, and Autonomy 🌍

• At Scrums.com, we champion an empowered workplace where you shape the culture, influence decisions on tech stacks, and take charge of user testing—all while working from wherever you thrive best across our interconnected Guilds.

• Our distributed teams span South Africa, Nigeria, Kenya, Zimbabwe, and other parts of the continent, showcasing the diversity and brilliance of African talent.

The Scrums.com Vibes 💙

• Annual Foos-Fest ⚽️ : Showcase your skills at our legendary Foosball Festival and take home quirky, sought-after awards.

• AI-Powered Hackathons 💻 : Collaborate with your team to create innovative, market-ready products in just 24 hours—all while enjoying live music and family-friendly fun.

• Year-End Celebrations 🏞️ : Reflect and connect at our unforgettable year-end events, a decade-old tradition we continue to cherish.

• Popcorn Fridays, Dev Chats, and Braais🍿 : Bond with colleagues over insightful discussions and sizzling good vibes.

• Fluid Clubs🥾 : Join like-minded peers in activities like hiking, running, and more to foster shared interests beyond work.

• Weekly All-Hands 🙌 : Stay inspired and in the loop with our weekly All-Hands sessions that bring us together as one team.

• Explore Week 🐘 : Our annual tradition encourages you to work from anywhere around the globe for one exciting week—freedom meets inspiration!

Scrums.com Rewards & Recognition 🎉

• Stacos (our company currency) : Reward and recognise your peers with Stacos and exchange them for cool vouchers to celebrate accomplishments big and small.

• Commission Incentives : Unlock attractive rewards for successful business development leads and employee referrals.

Continued Career Growth & Support 🔑

• Internal AI Platforms : We empower our teams with access to both closed and open source LLM’s that make work easier and empower your professional growth. We love AI! 

• High-Performance Culture : Stay aligned and on track with OKRs (Objectives and Key Results) and structured annual reviews.

• Personalized Mentorship : Receive dedicated guidance from leadership and the Talent SuccessTeam through one-on-one sessions and quarterly check-ins tailored to your career aspirations.

Our teams are powered by AI—Transforming How We Work 🤖

• Our teams embrace cutting-edge AI platforms, agents and tools, creating smarter workflows and accelerating career development with AI. 

• OpenAI, Anthropic, Llama - we use them all so choose what suits you best!

• You don’t need your own licenses - we cover the costs and have our own internal AI platforms that gives you access to them all in a secure environment. 

Outstanding Employee Benefits* ➕

• Discovery Medical Aid : Prioritizing your health with the best care.

• Half-Day Birthday Leave : Celebrate your special day with an afternoon off!

• Family responsibility & Parental Benefits : Providing full support for a meaningful work-life balance.

*Please note: Benefits may vary depending on employment type

Join us and be part of a dynamic, supportive team that’s making waves across Africa.

We are seeking a highly skilled and self-sufficient Data Engineer contractor to design and build their core, end-to-end data ingestion and processing workflow on Google Cloud Platform (GCP). This is a greenfield project, meaning all components will be developed from the ground up. The solution must adhere to modern data engineering best practices, with a strong focus on modularity, scalability, and maintainability to ensure smooth future ownership by the internal team.

Key Skills

Google Cloud Platform (GCP) | Terraform | Python | SQL

Responsibilities

• Architect a comprehensive Google Cloud Platform (GCP) data solution, including networking, security, and storage strategy.

• Use only GCP-native services and implement the Medallion Architecture pattern (Raw, Bronze, Silver, Gold layers).

• Provision all required GCP infrastructure using Terraform for repeatability and maintainability.

• Build robust, end-to-end ETL/ELT pipelines using Cloud Functions and Dataflow for processing.

• Use Dataform for in-BigQuery data transformations.

• Orchestrate Workflows and Implement pipeline scheduling, logging, and monitoring using Cloud Composer (Apache Airflow).

• Complete the full end-to-end data pipeline for the first company’s SFTP data source, making data ready for business reporting.

• Create high-quality documentation, architecture diagrams, and runbooks.

• Conduct workshops and sessions to transfer knowledge to the in-house Analytics Team, ensuring they can operate and extend the pipelines independently.

Requirements

• GCP Data Engineering Expertise

• BigQuery: Data modeling, partitioning, clustering, cost optimisation, complex SQL.

• Cloud Storage (GCS): Folder structures, security, lifecycle management.

• Cloud Dataflow (Apache Beam): Scalable, serverless batch and stream processing (preferably Python SDK).

• Secure, reliable data ingestion using Cloud Functions or Cloud Run (APIs, SFTP , etc.).

• Strong proficiency in Python and advanced SQL.

• Production workflow management with Cloud Composer (Apache Airflow).

• Hands-on experience with Dataform (or dbt on GCP) for version-controlled, testable, automated data transformations in BigQuery.

• Mandatory experience with Terraform for provisioning all GCP resources.

• Deep knowledge of GCP IAM, service accounts, and network access controls (least privilege principle).

• Familiarity with Cloud Data Catalog/Dataplex for data governance and asset discovery.

• Ability to produce clear, concise architecture diagrams and documentation for non-data engineers.

• Proven mentorship and knowledge transfer skills.

• Ability to operate independently, drive decisions, and define best practices from scratch.