BlueMatt's Blog On Building a Bitcoin for Everyone

Why Noncustodial Bitcoin Still Matters

This post originally appeared as a part of Spiral’s newsletter.

If I’ve learned anything in my nearly 15 years of working on bitcoin, it’s that building trustless stuff on bitcoin is Hard (with a capital H). For the last 14 years, I’ve spent probably tens of thousands of hours trying to create the best user experiences for people who want to use bitcoin in a truly noncustodial way. Yet, every day feels like more of an uphill battle than the last.

In bitcoin’s early days, this meant transacting directly on-chain. As with every cryptocurrency, this works great for a while, but at some point, people realize that the utility in a censorship-resistant (or basically censorship-resistant) decentralized database devours more capacity than is available, even if you trade lots of censorship-resistance for centralization and scalability.

Of course, this isn’t news to anyone who’s been in bitcoin for a while or has at least done their homework. Bitcoin fought a long civil war, the Blocksize Wars, when it reached this point. But long before even that calamity, we were already exploring better ways to make payments on bitcoin. All the way back in 2013, I worked on payment channels as a step in that direction. After all, the ability to transact instantly is an infinitely better user experience than a few seconds, let alone ten minutes.

By the time the Blocksize Wars concluded and the dust gradually settled, Lightning had advanced payment channels and became the state-of-the-art option for noncustodial, scalable user experiences on bitcoin.

While all this was happening, custodial services came and went. Many exit scammed, taking customer funds with them. However, the pain from those experiences never fully overcame the generally poor user experience of noncustodial bitcoin, driving many consumers to continue to use custodians in spite of the risks.

Of course, that isn’t to say these bitcoiners are “wrong” or that there aren’t problems with self-custody UX beyond scalability. Private key management and security challenges abound, but so do a general lack of places to spend your bitcoin. A lot of us, most of us, just stack and hold. Add ten to thirty-minute confirmation times to this and highly variable fees, and it’s a wonder that anyone even bothers with self-custodial bitcoin.

The Lightning ecosystem is hardly better (I say this as someone whose life is working on Lightning). For many years, Lightning was considered a thing to be run on an always-online machine carefully managed by an attentive operator, trading liquidity with others and ensuring their channels are carefully maintained with only the best of peers. The time commitment alone made Lightning borderline inaccessible to everyone but the most committed pro-bitcoin masochists.

When Lightning did come to mobile apps with user experiences designed for humans, the protocol’s corner cases often made it janky and failure-prone. By the time many of these were resolved (with some likely to be resolved only this year), much of bitcoin had already made up their mind on Lightning, deciding that a good user experience simply cannot be built on its back. Worse, the high cost of developing noncustodial bitcoin solutions drove up fees on some Lightning wallets as vendors sought to cover costs, further tarnishing the community’s view of Lightning.

Instead, the bitcoin community has become enamored with newer custodial solutions, from Liquid (Blockstream’s multi-sig, custodial private blockchain) to ecash (blinded single or multi-sig custodians), Spark (Lightspark’s Statechains extension, which requires near-full trust in the operator(s)), even modern Ark variations (which rely heavily on the Statechain trusted-operator model to work around its substantial scalability constraints). Tons of engineering resources have gone into exploring what can be built with custodial solutions, with many of these systems providing excellent user experiences and privacy but at the cost of much of the censorship resistance that’s at the core of bitcoin’s soul.

Building bitcoin solely on the backs of these solutions will not provide a more robust future for bitcoin. Instead, it will force all bitcoin transactions into custodial solutions that make it impossible to imagine the kind of freedom that is bitcoin’s promise.

To be clear, custodial solutions will always have a place in cryptocurrency. If there’s one thing that the entire cryptocurrency space has learned over the past 16 years, from bitcoin to terrible memecoin, it’s Cryptocurrencies Do Not Scale. No matter how high you push your block size, how often you claim to have “solved scalability,” or how your system’s transactions will always be free, there will be fees, and those fees will dictate how “noncustodial” something is. Anything less than the cost of a transaction and the fee required to exit a trustful layer two system unilaterally will dominate the amount you seek to recover.

But this doesn’t mean we should give up on noncustodial or write off Lightning because attempts to run it on someone’s phone years ago were unusable.

If we care about the principles of bitcoin, we must bridge that gap. Mutiny started down a path (before their pivot away from bitcoin wallet software) that seemed compelling. Wallet software should seamlessly transition between the best option at each balance. For the class of bitcoiners with a balance too small to reasonably provide a self-custodial option, store their funds in one of the custodial solutions bitcoiners have spent so much energy building lately. For those with considerably more wealth in bitcoin, where a noncustodial option makes sense, migrate their balance to something non-custodial. Given the options today, Lightning, directly on a bitcoiner’s device, is the only viable option here and ultimately must play an important role in any wallet wishing to consider itself a standard-bearer for Bitcoin.

Lightning Routing Replayer

In Lightning, whenever you send a payment, you must first pick a path over which to send it. If you pick that path well, the payment will complete immediately and for (hopefully) a low fee. If you pick poorly, a node in the path may not have enough funds available for your payment and will have to fail the payment back to you, at which point your node will need to pick a new path and try the payment again, resulting in slow payments. If you pick poorly several times, the payment may time out and ultimately fail, even if there may have been sufficient capacity for your payment over some path.

In practice, most pathfinding algorithms in Lightning lean heavily on memory of past payment or probe attempts in order to select paths which are likely to succeed. This allows a Lightning node to learn which channels in the network are usually saturated (and in which direction) and allow it to avoid common logjams in the network. Pathfinding algorithms without memory of past attempts appear to perform quite poor, especially in paying a handful of hard-to-pay nodes which operate as “sinks” - generally receiving more than they send.

Testing new Lightning pathfinding algorithms is quite easy when you aren’t concerned with memory - you simply write your algorithm and try to pay many nodes on the network. The more nodes you manage to pay, the better your algorithm is. However, when you’re dealing with memory you have to test an algorithm over a long time horizon, testing lots of payments and comparing the success over time. This makes evaluating pathfinding changes difficult.

Today, I’m announcing a simple pathfinding evaluator (and accompanying dataset) which allows researchers and developers to test a pathfinding algorithm by replaying many historical probes from the perspective of my Lightning node, comparing the real results with the path probabilities their pathfinding model predicts.

While the use of paths LDK’s router picked biases the framework somewhat, it should be general enough with enough data to still provide a good indication of whether a model change is positive or negative.

You can find the replayer and can learn more from its README in its git repo.