Scaling with Canva, Faster iOS Builds at Asana, and AI Testing in Java!

Did someone forward you this email? Be one of the first to join an exclusive newsletter which unlocks curated actionable tech insights straight to your inbox—subscribe here! 

In today's email:

• Canva's Counting Conundrum: How does Canva handle billions of events?

• Asana's Swift Success: Why are iOS builds faster now?

• Java's GenAI Jumpstart: How to test AI applications effectively?

CANVA’S LEAP

From MySQL to Millions: How Canva Scales Content Usage Counting

Imagine counting billions of events every month – that's the challenge Canva faces in tracking content usage to pay creators. In this article, we break down how they built a system to handle this massive data influx, offering valuable insights for anyone wrestling with data scalability.

The Problem: Counting Counts Accurately at Scale

Canva's creator program thrives on accurate usage tracking. But with billions of content interactions each month, ensuring every count is correct becomes a monumental task.

Here's what they needed to address:
- Accuracy: Every count must be precise, as creator income and trust depend on it. No room for overcounting or undercounting!
- Scalability: The system needs to handle explosive data growth and adapt to future increases.
- Operational Efficiency: Maintaining and troubleshooting a complex system with rising data volume can be a nightmare.

The Journey: From Simple to Scalable

Canva's solution involved a fascinating evolution:

- Starting Simple: They began with MySQL, a familiar technology, but it struggled with scalability as data volume grew.
- Exploring Alternatives: Migrating raw data to DynamoDB offered some storage relief, but processing scalability remained an issue.

The Breakthrough: Embracing Simplicity with OLAP and ELT

The winning approach involved a complete overhaul:

- Leveraging OLAP: They switched to an OLAP (Online Analytical Processing) database like Snowflake, excelling at large-scale data analysis.
- Adopting ELT (Extract, Load, Transform): Data is extracted from source databases, loaded into the warehouse, and transformed within using SQL queries. This simplified the codebase and eliminated the need for intermediary data storage.

Read More
^{by Sangzhuoyang Yu ⸱ Canva ⸱ 11 min read}

IOS BUILD TIME

Splitting an iOS Codebase for Blazing-Fast Builds: Asana’s Transition to Tuist

Asana’s Engineering Team has achieved a notable milestone in optimizing the development process for their iOS app. By switching to Tuist, a tool for managing Xcode projects, and restructuring their code into modular components, they’ve slashed build times dramatically—from 45 seconds to just 15 seconds. This transformation not only speeds up the development cycle but also enhances the overall developer experience.

Actionable Insights:

1. Modular Codebase for Faster Builds: Asana divided their Swift codebase into several modules. This modular approach ensures that unchanged components don’t need to be rebuilt, thereby reducing build times significantly.

2. Utilizing Tuist for Project Management: Tuist automates the creation of Xcode projects and workspaces, streamlining the setup process. This tool has been crucial in managing the complexity of the codebase, allowing for rapid adjustments and iterations.

3. Migration Challenges Overcome: Transitioning to Tuist involved moving away from CocoaPods to other dependency managers like Swift Package Manager and Carthage, as well as adjusting build settings. Despite these hurdles, the long-term benefits of faster and more reliable builds have been well worth the effort.

This strategy is ideal for projects with growing codebases experiencing long build times. The benefits might not outweigh the initial setup effort if you're a solo developer or have a small, stable codebase.

Read More
^{by Engineering Team ⸱ Asana ⸱ 9 min read}

JAVA GENAI

Revolutionize Java AI Development: A New Methodology for GenAI Application Testing

The integration of Generative AI (GenAI) in Java has brought groundbreaking advancements and new challenges, particularly in the realm of application testing. Tools like LangChain4j and Spring AI have streamlined the creation and interaction with large language models (LLMs), making GenAI more accessible to Java developers. However, ensuring these applications perform correctly remains a significant hurdle due to the inherent complexity and non-deterministic nature of AI responses.

Actionable Insights:

1. Simplifying Interaction with LLMs: Using LangChain4j, developers can quickly set up and interact with LLMs by configuring a few parameters. This makes it possible to start generating AI-powered responses swiftly, a boon for rapid development cycles.

2. Testing Challenges: Traditional testing methods, such as keyword verification, often fall short when assessing the relevance and coherence of AI-generated text. Instead, consider using cosine similarity measures to compare embeddings of the expected and actual responses. This approach provides a more nuanced and semantic assessment.

3. Innovative Testing Approach: A novel method involves using one LLM to evaluate the output of another. This "Validator Agent" can assess if responses meet detailed predefined criteria, providing a more tailored and rigorous validation process.

4. Implementation Strategy: Define clear validation criteria and implement a structured Few-Shot Prompting technique to guide the Validator Agent. This ensures the responses are not only accurate but also justifiable, enhancing reliability in testing outcomes.

This approach opens doors for more robust testing of AI-powered applications. As AI evolves, so will our testing methods. How can we leverage AI itself to ensure the reliability and effectiveness of these new technologies?

Read More
^{by Ignasi Lopez Luna ⸱ Docker ⸱ 15 min read}