A journey that began in the late summer of 2022, has finally come to a joyful end. Third-party Zcash wallets are working again, and Electric Coin Co. (ECC) is exiting Emergency Mode.
During this time, ECC released multiple updates to zcashd and lightwalletd, plus new mobile SDKs that, together, introduced new innovations (and learning) in the world of cryptography and decentralized money. These releases provide massive upgrades to privacy, scalability, and user experience in Zcash, and they have implications for all privacy-focused crypto projects.
Before going further, we need to give the hugest THANK YOU to our wallet partners Edge, Nighthawk, and Unstoppable who worked with us to test releases and submit bugs, then were quick to implement SDKs 2.0 when they were ready.
The ECC engineers deserve special recognition — and possibly a vacation — after putting in long hours over many months on complicated discovery and development. Working through the ECC restructure and battling tough deadlines, while tackling other crucial emergencies like the Halborn disclosure, they delivered groundbreaking technology that elevates ZEC user experience.
Background
Starting in June 2022, the Zcash network began experiencing a huge increase in shielded transaction volume. This additional network load caused a “data pileup” that prevented Edge, Nighthawk, and Unstoppable wallets from being able to sync in a reasonable amount of time. These three wallets rely on ECC’s mobile SDKs.
Wallet syncing processes required scanning the entire blockchain before funds were available to be spent. And when the wallets wouldn’t sync, users couldn’t access or spend their funds.
In October 2022, ECC went into Emergency Mode and put aside all other roadmap priorities. From our blog in March 2023:
It’s a complicated challenge to address, and at minimum, it requires developing and implementing (1) a faster algorithm that does not require a linear sync of all blocks on chain and (2) tooling modifications that will give users the ability to spend funds without having a fully synced chain. As mentioned, the solution comprises changes to every component in the shielded mobile wallet stack: zcashd, lightwalletd, the ECC wallet SDKs, and the ECC prototype wallet.
Emergency Mode exit criteria
Our top priority was ensuring users could access and spend their ZEC (Zcash coins). This is fundamental to our mission of economic freedom and a requirement for real-world private digital cash.
Speed and reliability are also critical to Zcash user experience, and these were our criteria for exiting Emergency Mode:
- Users of Edge, Nighthawk, and Unstoppable can spend their current funds (funds that are already synced when they open their wallet).
- Users of those wallets can receive and become able to spend new incoming funds at a rate of a month’s worth of transactions in 1 hour.
- Users of those wallets see sync updates which are minimally confusing about progress.
- None of those wallets are impacted by frequent crashes or inconsistent behavior (such as failing to display some already synced transactions), nor do they require work-around behaviors due to the ECC SDK.
With the release of new mobile wallet SDKs 2.0 in September, ECC delivered the final piece in our efforts to mitigate third-party wallet performance issues. Edge, Nighthawk, and Unstoppable quickly implemented the SDKs, and as of today, we are pleased to report that the criteria above have been met.
What is Spend Before Sync and why is it important?
Available now in Edge, Nighthawk, and Unstoppable, Spend Before Sync (SBS) introduces non-linear sync, which allows wallets to scan chunks of the blockchain out of order and/or in parallel. This makes it possible for users to spend their existing funds without requiring the wallet to fully sync the entire blockchain.
Separately, but also important, the data structures that enable SBS substantially improve the performance of note commitment tree updates. As an example, Zingo! didn’t adopt SBS, but just by using the updated data structure that wallet got a better than 10x improvement in linear scanning speed.
Aditya Bharadwaj, better known as Adi, founder of Nighthawk, called SBS “a groundbreaking advancement that promises to revolutionize Zcash’s peer-to-peer usage for payments and shielded app interactions.”
What we learned during Emergency Mode
- Our mobile SDK code did not meet the needs of a typical Zcash user. The syncing UX was confusing and the API design precluded wallets from spending funds until the wallet had fully synced the chain.
- Before Emergency Mode, and to some extent before the ECC restructure, our team was spread too thin and trying to focus on too many things, which allowed issues like transaction load impact to surface.
- We knew long ago that a transaction-load issue was possible, but we prioritized other Zcash features and improvements. This challenge of prioritizing new feature improvements versus making the existing system more resilient or robust is always present. In this case we underestimated the likelihood of this problem and the amount of effort it would take to mitigate it.
- We chose to address the issue in the most thorough manner by making wallets function during high transaction load, rather than the expedient of short-term interventions to lessen the transaction load. We believed those short-term interventions would lead to more complexity without fully resolving the issue in the long run. We still believe our approach was best for the long-term robustness of the Zcash ecosystem, yet we underestimated the time and effort and the overall disruption to Zcash users in the interim.
- We also learned about comms. It took us too long to define Emergency Mode and communicate a concise explanation and plan, both internally and externally.
- Having our own wallet in development (Zashi), which uses our SDKs, was key to speeding up the process because it allowed us to test our solutions before deploying and/or sharing with third-party wallets.
With Emergency Mode complete, ECC is turning its attention to our main focus areas: proof-of-stake Zcash, development of the Zashi wallet, continued improvements to SDKs and Zcash Core Libraries, and US Policy work.
