App Thinning: Syncing Localized Strings to Outlook for iOS

This post has been republished via RSS; it originally appeared at: Microsoft Mobile Engineering - Medium.

Sync localized strings to your app over the air.

Outlook for iOS has seen tremendous growth over the past few years. As with any mobile app used across the world, a key area where we strive to deliver is internationalization and localization. With dozens of major languages spoken around the world, it is imperative that every user be met in whichever language they are most comfortable with.

Outlook ships with support for a variety of languages. However, as the app grows and we continue to support a plethora of languages, an interesting challenge starts to appear:

How do we add features in all supported languages without having the app size balloon?

Unlike other aspects of app thinning, Apple does not supply any out of the box tooling or guidance when it comes to localization driven app size increase.

As a result, we arrived at our own solution to offload all the strings in the app and deliver them dynamically over the air.

Below, we will walk through how we arrived at this solution, the technical challenges solved along the way, and how this results in little to no developer impact.

Key Goals

Before getting started with any level of the implementation, we wanted to be clear eyed and focused as to what our key goals were. We were able to narrow our goals to the following three.

1. Resilient, Fast, and Reliable

Due to the critical nature of strings in the user facing experience, there was no doubt that the solution must be reliable & fast. Given the nature of moving resources to a remote server, in which a variety of things can go wrong on sync or fetch, it required us to think long and hard about resiliency. Particularly, what would happen if a given language sync failed? We also wanted to be aware of bandwidth consumption for both our users and infrastructure. We will explore both themes in the latter technical deep dive.

2. Dynamic & Up to Date

In the spirit of meeting our users where they were in terms of preferred system language, we needed to ensure our solution was dynamic and up to date. Moreover, translated strings should sync intelligently on a per language basis. In addition to this, language changes at the system level should not impair the user’s ability to use the app — the solution should remain up to date in terms of user facing strings that are synced.

3. Minimum Process Changes + Automation:

We also recognized early that this type of change impacts processes and contributing developers directly. The solution needed to be seamless in that, when new strings are checked in by a developer, they become “offloadable” for free. The developer should not even have to think about how sync works or if they should put their strings in a particular location.

With these fundamental principles in mind, let us look at how localized strings, string tables, and bundles are generated before we talk about the sync system.

Localized String Generation

The first step in being able to offload all string resources to a remote server was to generate those resources.

LocalizedStringKit

Outlook leverages our open sourced LocalizedStringKit client library and python package to generate string tables for all supported languages. From there, we use an automated recurring pipeline to populate translated text for a given target language.

Prior to any changes, LocalizedStringKit only supported a single aggregate strings bundle that held all language string tables:

We realized early on that we wanted the ability to split Packaged strings and Offloaded strings. The reasoning behind this was to deliver the must unobtrusive user experience possible. More specifically, when a new user launched the app for the first time, we did not want to introduce a string sync dependency for onboarding. Rather, we wanted to be able to cherry-pick strings that we ship with the app to the App Store in all languages and offload the rest.

Respectively, we now needed to support multiple bundles in the host application, as well as be able to override the file system URL to that bundle — and we did just that.

Now that multiple bundles were supported, we needed to make another tweak. The host app needed the ability to specify from which bundle a localized string should be returned. Respectively, we added this functionality as a settable property.

With these client library changes in place, the python package which generates string tables needed to be extended. With multiple bundles being supported, the string tables needed to correctly de-duplicate a string and which bundle it belonged to create a string table for each language.

With the LocalizedStringKit python package and client library changes in place. It was time to think about the user-facing sync UI and experience.

User Interface & Inline UI Updates

As a result of strings changing between different app releases, we wanted to ensure we did not show a cumbersome spinner each time a sync was needed. However, we did want to make sure that the first sync — which would be larger than thereafter syncs — did show a respective UI. The reasoning here was to allocate a suitable amount of time for the strings to sync and not keep the user wondering as to what was happening.

For existing users, when they updated the app to a version supporting Over the Air String Sync, they saw a simple sync UI as follows.

Similarly, when a new user downloaded the app, we wanted to ensure the best possible communication to the user if sync was still in progress. Unlike existing users who updated their app, new onboarding users were handled differently. Because we packaged onboarding strings into the app for all languages, we were able to begin a sync for first time app launches behind the scenes.

For most use cases, the string sync would be complete and cached prior to the user setting up their first email account. Should that not be the case, upon initial set up, we would show a similar UI to what is figured above for existing users — pictured below.

Next up, we made changes to the client app to allow for re-layout of key User Interface elements upon string sync success. This ensured that things like Right to Left, Safe Areas, Margins, and Padding were respected once the new language was delivered to the device.

We accomplished this re-layout with a new label base class that is capable of updating its text upon receiving a notification & calls alayout method on its super view.

Moreover, the refreshText() function was extended to call setNeedsLayout() and layoutIfNeeded() on the label’s super view.

Infrastructure & Automation

Now that we had string generation and user experience figured out, it was time to think about getting the strings onboarded to a remote server; more specifically a Content Delivery Network (CDN).

With one of the key goals being the Avoidance of Developer Process Impact, we decided Continuous Integration (CI) Pipelines would be the best place to introduce these changes.

We needed to accomplish the following:

Compress string files on a language basis & upload.
Version each of those files to match the application version.
Optimize bandwidth needs between version changes.

Compression & Upload

This step was straightforward. It encompassed archiving each string table file — on a per language basis — and uploading it to Azure Storage on every Release pipeline run. This would ensure that there are string assets on the remote side prior to shipping that respective application version. We compress these assets prior to uploading them to reduce bandwidth needs and latency on the client side when it is time to fetch.

Versioning

This was a critical aspect to figure out early as it set the stage for many other decisions. Because strings may change between application versions, the app needed a way to figure out which version to sync.

We went back to the basics to solve this. We incorporated the app version that we shipped to the App Store into the remote sync URL. This not only reduced complexity but gave way to core optimizations that we were looking to make for bandwidth needs.

Optimizing Bandwidth

As described above, we had a plan to version strings between every app for the sake of string churn. This posed the following, and quite interesting, question.

Why should the app have to download each version of strings if the difference is only a single string change?

For some context, each string table has an approximate size of 255 Kilobytes. Assuming a weekly app update schedule and 100 Million users, that comes out to about 1.3 Petabytes — 1,300 Terabytes — of bandwidth needs each year.

With this significantly high number and the above question standing, we introduced the concept of String Diffing.

The notion is simple. If the app has the prior version of a given language’s strings synced already, the app should only sync and merge the difference of the next version. You can think of this as a patch operation. Conversely, if the app does not have the prior version of strings, the latest version’s full string asset is fetched.

We updated our Release Pipeline to generate diffs by examining current string tables and the prior uploaded ones. Strings could either have been added, removed, or updated. Respectively, a diff file is generated for each language in the new version and uploaded as well.

The average size of a string diff file is about 10 Kilobytes. Taking the same calculation as above, excluding the first sync bandwidth needs, the numbers come out to the following.

10 Kilobytes x 100M Users * 52 App Updates/Year = 52 Terabytes

This optimization resulted in a 96% decrease in bandwidth needs, figured below.

Resiliency & Error Handling

With the groundwork for string generation, CDN onboarding, and sync in place, it was time to double down on Resiliency and Error Handling.

Keeping with the theme of strings being up to date despite needing a sync, we introduced Notification listeners for system language changes. Based on that, and prior synced asset availability on disk, we can keep application strings up to date in a seamless way.

When it came to error handling, there were two things to consider: User facing error paths and resilient sync upon failure.

In the case where we show the above-mentioned sync UI to the user, and sync fails, we simply allow the user to Retry or escalate the issue to a Help/Feedback path.

In the case of behind-the-scenes sync, things are a bit different. Rather than showing any failure UI, the sync incorporates retry tiers.

Because most of the syncs will target String Diff files, we wanted to ensure that our optimization didn’t come at the expense of User Experience. As a result, the sync process falls back to syncing the current version’s full language asset if the diff fails for any given reason.

Moreover, if the full language sync fails, we show a simple Failure UI with a Retry option & Help/Feedback user path.

Takeaways & Summary

This all began with the simple goal of reducing app size as we increase the number of strings we show to users in a variety of languages. We ended up with a solution that is not only seamless, resilient, and automated, but one that is based on the foundations of our Open-Sourced Projects.

Revisiting our Key Goals, we were able to build a system that scales with fault tolerance and speed in mind. The solution also stays up to date with Notifications & Listeners to meet the user in whatever language they are operating in. Finally, as a result of incorporating automation into our Pipelines and String Segmentation into bundles, we were able to avoid developer disruption.

Placing a special emphasis on optimizations, we were additionally able to reduce bandwidth costs and user facing latency as well.

This project, approach, and system was viewed from a lens of Fundamentals and Basics. Keeping it simple, automated, and organized helped pave the way for tying it all together.

Lastly, we kept a close eye on Apple’s own position when it comes to decreasing app size. App size reduction concretely improves an app’s ability to be distributed over cellular networks, reduces storage needs on an end user’s device, and helps your app succeed.

Resources

App Thinning: Syncing Localized Strings to Outlook for iOS was originally published in Microsoft Mobile Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.