Solutions
Solution
Industry spotlight
.jpg)
Watch our latest video case study!
Check out how Colibri's partnership with Nomo Fintech has transformed their approach to data
Learn more
Success stories
Insights
Mana, a London-based ed-tech startup, was stepping into the next phase of its growth journey. The client's platform provides a curated directory of experts, which mainly includes tech leaders, career advisors, lifestyle gurus, and teachers. It allows users to bring content from YouTube, Instagram, Udemy, TikTok, and other platforms into a portfolio for their followers.
Led by visionaries and backed by a passionate team of technologists, the company's main focus remains on boosting the quality of user engagement and keeping up with evolving trends. Behind a neat-looking user interface, a massive amount of data is generated, holding significant potential to provide actionable insights for both users and the business.
The data analysts were struggling with data quality. At the time, even the aggregated data in RDS contained usernames that could be changed an infinite number of times, unlike static user IDs, which resulted in incorrect data for analysis.
The client needed a more efficient way to analyse and gain insights from its log data. The business needed to have an accurate count of profile views, content/session views, and referral sources. Colibri was approached by the client for an architecture review and data lake implementation.
After a few meetings and a discovery call with the CEO and Director of Engineering, our architects performed a thorough assessment of the existing stack and made recommendations to revamp the architecture and build a smart data pipeline leveraging DataOps.
Working closely with the client's developers and ML engineers, we created a list of data points captured from the web app. Our lead cloud engineer carefully crafted the data schema and data capture design for collection in DynamoDB and provisioned an S3 data lake bucket connected to AWS Lambda functions, Amazon SQS, and AWS Kinesis. After receiving client approval to use Terraform for infrastructure deployment, Python Lambdas were implemented to consume data from the app. Each implementation task included unit testing, CI/CD, logging, and alerting.
At the point of data ingestion, we directly joined the click event data with the client’s RDS database to retrieve the user ID, which was then stored in the raw layer of the S3 data lake, enabling aggregation based on the static user ID, rather than a changeable username.
Meilisearch was also productionised, leveraging ECS, Fargate, and EFS, and configured to pick up data for incremental loads on RDS to enhance search capabilities.
The new data architecture has boosted innovation, giving the client an agile setup that supports evolving business and technical needs. Data analysts now have accurate data at their disposal for prescriptive and predictive analysis, providing a precise view of the user's journey.
Even if a user changes their name, click data is retained and carried over. They're able to track time spent on each card, tap interactions, likes, top searches, and build custom metrics, giving users clear insights that help them enhance their profile engagement and make learning addictive for their followers.
With the IaC approach, the data engineering team is now nimbler and better equipped to adapt to changes quickly and securely, using Terraform to deploy workloads. This enables them to explore new possibilities, not only keeping the platform functional but also introducing new features that contribute to a beautiful user experience.
The all-new search service now enables a dramatically enhanced user experience, with features like type-ahead completion and multi-faceted queries. This new, decoupled solution allows search indexes and search data to scale and grow independently of other application databases.
Retaining their creative edge for continuous product development, the Mana team is now ready to onboard more users and their followers.
Reduced to within 6-second lead time
Faster predictions on customers' needs and capturing new opportunities
+ 20 headcount in the data team
Unlock growth, efficiency and innovation through data and AI
Turning raw data into actionable, competitive insight
Modernise with purpose. Transform with data. Scale with AI
Intelligent solutions built for real-world business impact
Empowering cloud-first operations with confidence
Can’t find the answer you’re looking for? Please chat to our friendly team.