Behind the scenes of 1Password for Linux
We officially launched 1Password for Linux today! 🎉
What makes this release even more amazing is it was created from scratch and developed using new languages and techniques most of our team never used before.
Almost everything you can name we did in a new way: from team organization, programming languages and toolkits, design language and processes, all the way through to new deployment and packaging and testing procedures. In fact more than half of the team that contributed to 1Password for Linux joined our family within the last 18 months.
I thought it would be fun to go through a high-level overview of how 1Password for Linux was architected, how we came to fall in love with Rust, and highlight some of the incredible things our team has been doing to make our new Linux app a reality.
A hybrid app powered by Rust 🦀
1Password for Linux is our first hybrid app.
The backend is written in Rust, a true systems programming language known for its safety and performance. Rust compiles directly to native code and avoids the overhead associated with runtimes or garbage collection.
On the frontend side of things we used web technologies to allow us to create an entirely new design language for 1Password. Not only does it look great but it allows us to be incredibly agile and iterate quickly.
We had a lot of success with TypeScript and React when developing the web interface for 1Password.com as well as our browser extension so these were natural choices here.
To tie everything together we used Neon along with a home-grown TypeShare tool for generating types on both sides of the FFI layer. Finally we bundled everything using Electron to allow us to integrate deeply with the operating system.
Here’s a quick sketch on how everything fits together.
A quick trip down memory lane
It took us a while to realize how great Rust is and how it could solve many of the problems we had. To understand how we got here we first need to take a step back to see how we developed apps over the last 15 years.
Before our new approach for building hybrid applications existed all of our apps were written with a completely different backend on every platform. While they all connected to the same 1Password.com service, they each did so with their own hand-written code.
Every platform was responsible for syncing, conflict resolution, data storage, creating view models for the UI, and everything else you can imagine.
This may seem surprising in today’s day and age when Rust exists and has mostly matured, but these apps were developed over the course of 15 years and back then there weren’t any languages that we could use to write software across operating systems in a safe and secure fashion.
Of course there was C and C++ but neither Roustem nor myself were fans of C++ and even if we were, the thought of writing a multi-platform library in an unsafe language scared the hell out of us. And in the case of crypto we just flat out refused.
So each team used the best languages and libraries that were available on each platform at the time they were written.
As you can imagine we did our best to keep feature parity (and bug parity!) consistent across every platform but it was incredibly difficult to do with so many code bases.
We all wanted to find a way to write the the backend code in a safe and performant way and share across all the platforms that we supported. This was a noble and exciting endeavour, but how?
How we came to fall in love with Rust 🥰
Our Windows team was the first to experiment with Rust back in 2017. They quickly fell in love and on May 2018 1Password for Windows 7.0.558 was released with a lot of the heavy lifting performed using Rust.
The number of crashes and bug reports on Windows quickly fell.
As great as this was, it took the rest of the team longer to see the brilliance of using Rust. At the time a lot of the team was using Go lang to create a shared backend to be used by 1Password.Next (our code name at the time for 1Password core) and they were seeing enough success that they weren’t eager to switch languages.
Along with GopherJS to compile to JavaScript, we had support for everything we needed with Go. At least for a while.
Over time we outgrew GopherJS. When we started experimenting with WASM we discovered that Rust compiles to WASM incredibly well and has a solid toolchain we had to try it out.
So we started an experiment porting the brain from Go to Rust at NSNorth 2019 and then things really took off during a conference at Roustem’s house a few weeks later.
Very quickly we went from not knowing Rust to a prototype of filling within the browser and this marked a huge turning point for 1Password development as a whole.
At this point we realized what our Windows team had been trying to tell us all along: Rust was the real deal. It was fast, safe, and able to compile to every platform including WASM.
From baby steps to a full Linux app
To make sure our enthusiasm of Rust would survive the real world we made a small library for deriving time-based one-time passwords (TOTP) and shipped it in all our desktop and mobile apps. We also shipped the new WASM brain in our browser extensions.
The results were wonderfully uneventful. There were no surprises other than how smooth the rollout was.
Rust had proven itself so it was time to see if we could create a full blown 1Password core for all our apps to use. By the end of the fall we had seen enough internal successes that we wanted to find a way to start sharing our progress with our users.
Linux was the perfect playground so in late 2019 at a conference at my house we started creating a Linux app to focus our efforts. Disney had just premiered The Mandalorian so naturally this was our choice for movie time and Project Mando was born. 🤩
It was a fun proof of concept and since most of our code wasn’t in Rust yet it used the 1Password CLI as the backend (Secrets Automation didn’t exist at the time).
Things went so well that we decided to pull in more of the team and in January 2020 we held a larger conference at Roustem’s house to build it out. One month later an early prototype of 1Password for Linux was demoed to the whole company on Navigator of the Seas during our annual conference.
From there things continued to heat up as we got more comfortable with Rust and the team continued to grow. By August 2020 we soft-launched our first development preview and then in October we opened beta testing.
That brings us to today’s official launch of 1Password for Linux! 🎉
Ironically we now have yet another implementation for handling all the backend heavy lifting. It’s like XKCD 927 all over again. 😂 Thankfully this is temporary as we have gone to great lengths to create a common 1Password core that can be used on every platform in the future.
The beauty of the 1Password core 😍
“By creed, it is in your care. This is the way” — The Mandalorian Armorer
One of the primary objectives of our Linux release was to create a common core that could be used for interacting with 1Password.com and perform as much of the heavy lifting as possible in one place.
We wanted a single code base for every client to allow us to develop things once and fix bugs all in one place. We also wanted to architect things in such a way that this common core couldn’t be used wrong.
After some iteration we decide that core should basically be a full client app with no UI. A headless client if you will. The core would have it’s own runtime loop that would handle every aspect of what it means to be a 1Password client and provide a safe and hard-to-misuse API for each UI to use.
There’s a lot to love in this diagram. First and foremost the core is completely self-contained and has a clearly defined API for client apps to use.
The op-appand op-uicrates stitch together all the other crates to provide everything each app needs. The majority of state is kept completely internal to ensure keys and other secrets are handled properly, as well as enabling each client UI to focus on their strengths rather than business logic.
Each platform has their own strengths as well so the foundationcrate enables the core to reach out to platform-specific services such as the kernel keyring and biometrics.
My favourite part is the Build Verification Test. Our BVT is written in Rust and automates testing by using the API and view models directly. Our GitLab CI has jobs to run these tests during Merge Requests and takes a lot of weight off our QA team.
And since we can spin up as many headless clients as needed we’re able to test concurrency scenarios or other situations that are hard to setup manually.
Our dream is to have this entire runtime working within WASM as well so we can replace our TypeScript implementation. This should be possible but proved quite difficult as the web and WASM environments are significantly different so for now we’re limiting our use of WASM for common libraries.
Overall this new 1Password Core approach has been very successful. Going forward we will be using it for our next generation of 1Password apps that we’ll be rolling out over the coming year. 🙌🏼
1Password for Linux by the numbers
No behind the scenes developer post is complete without some numbers. So here’s some fun numbers about 1Password for Linux and our development process:
- Number of repos: 1. Monorepo ftw! 🤘🏽 Well, not everyone celebrates this point but I personally love it. 🙂
- File stats: 7,528 files, 1,038,929 lines total, 813,514 lines of code, 161,095 comments
- Start date: ~18 months ago for Linux; Rust experiments started earlier
- Date of first commit that mentions Linux: Dec 17, 2019
- Total commits since Dec 17: 40,575
- Merge requests: 7,365
- Issues created: 7,529 (assume each and every one was fixed and closed 😉)
- CI/CD: 183,905 pipelines, 1,741,045 jobs (65% success ratio)
- How many Node package managers does it take to create an app? 3.5! During development we switched from npm to yarn to yarn 2 and then to pnpm. 😂
- Usability testing: 39 1-on-1 sessions with both new and long-time users
- User surveys: 3
- Releases: 12 development previews w/ 109 changes, 27 betas w/ 460 changes, 83 attempted nightly builds with about 50 successfully published
- On the design side we experimented a lot as well, going from Sketch to Zeplin to Abstract and then Figma
- Design highlights: 2,802 components in Figma, 7 designers supporting a team of 60+ developers, engineers and project managers
- Team members: this is harder to count than you might expect as we have many teams outside of development. Our #core Slack channel has 143 people and has more than doubled since we started. Here’s a screenshot from our Linux launch celebration Zoom call.
Every bullet point above deserves a blog post of its own and hopefully we can make that a reality someday. Please let me know in the comments which you’d enjoy most and help encourage these post to be written. 🙂
I hope you enjoyed this behind the scenes look at our development process and enjoy our latest addition to the 1Password family. 😘
