Historical study of cryptocurrency exchange defaults
Quick Summary
- Exchange credit risk is a huge, neglected risk that all crypto traders face
- An exchange’s past track record of hacks is predictive of future hacks
- Exchange profitability is a predictor of whether customers get repaid
- Other major factors to consider: cybersecurity, regulations, and banking
- We will be releasing monthly credit ratings for the top 150 exchanges using the research outlined in this article. Sign up below for free access:
However, the latter one, credit risk, is a mostly unknown and invisible type of risk that crypto traders unwillingly take on. Yet, it is a highly asymmetric, negatively skewed risk that has directly caused many of the biggest one-day losses ever faced by crypto traders.
The main source of credit risk in the ecosystem has been the risk of centralized exchanges defaulting on its depositors. This happens when hackers drain the wallets of the exchange, the founders of the exchange steal the funds, or authorities shut down the exchange.
The goal of this article is to provide research on how to price credit risk so the broader crypto trading community can understand, quantify, and mitigate this constant threat.
Conceptual Model for Measuring Credit Risk
Credit risk is the risk of loss associated with “credit events.” Credit events are broadly classified as instances when a person or organization (“reference entity”) defaults on a significant obligation.
A customer’s relationship with an exchange is similar to a traditional IOU. Customers custody assets at an exchange, and in return receive the right to withdraw assets at a moment’s notice.
Therefore, if an exchange refuses to allow customers to withdraw assets for an extended period of time, they have effectively defaulted on this obligation. We thus define credit events as instances where a crypto exchange disables withdrawals for Bitcoin for a minimum of five days across all customers.
However, not every crypto exchange that suffered from a credit event transferred losses onto their customers. It is therefore imperative to measure the losses, if any, that customers suffer from exchange credit events.
Calculating overall credit risk therefore consists of taking the product of these two metrics:
- Credit Risk = Probability of a credit event * Expected loss from credit event
Calculating the Probability of a Credit Event
The first step in measuring credit risk for exchanges is predicting the probability of an exchange experiencing a credit event.
A pair of researchers previously tackled this problem in early 2013, releasing a paper called “Beware the Middleman: Empirical Analysis of Bitcoin-Exchange Risk”.
In their paper, they studied the 40 different crypto exchanges that had existed up to January 2013. They discovered that an exchange’s volume was predictive of whether or not it would close i.e. obscure exchanges were more likely to be shut down than popular ones. They also found that exchanges that had been hacked in the past were more likely to be hacked in the future.
To test these two propositions, we first need a survivorship bias-free dataset of exchanges to test. To do this, we look at snapshots of the top 25 exchanges by volume in August of 2014, 2015, 2016, and 2017. We then record their name, volume, and whether they had previously suffered a hacking-related incident.
Next, we catalog whether each exchange faced a credit event in the subsequent twelve months from the snapshot date, ignoring instances where the exchange refunded deposits before shutting down. We’re solely looking for events where customer assets were stranded a minimum of 5 days.
Armed with this dataset of 100 different data points (top 25 exchanges for four years), we can compare the link between trading volume and past hack occurrence versus the rate of subsequent credit events.
i) Summary Statistics & Trends
Before jumping into an analysis of these relationships, we find it helpful to show the broad summary statistics of this dataset. The graph below shows the historical occurrence of credit events for the top 25 exchanges by year:
The recent hack of the Zaif exchange was mere days after the August to August measuring period. Taking the Zaif hack into account would put 2017 in-line with 2014 and 2015 at 12%, and make 2016 seem more like a outlier rather than the start of a downtrend.
In total, nine credit events occurred across 100 exchange data points, indicating a credit event base rate of 9.00% across the 4 years.
ii) Using Volume to Predict Credit Event Occurrence
The first relationship we investigate is between daily exchange trading volume and credit occurrence. Following the results of the prior paper, larger exchanges are expected to suffer more credit events as they are more attractive targets for hackers. A logical counter-argument to this would be that larger exchanges would have better security practices.
Graphing these two variables on a scatterplot is shown below:
Running a regression on the data above shows that this relationship is essentially non-existent.
Exchanges with daily volumes below $500,000 (such as Gatecoin) have been just as likely to suffer credit events as exchanges with over $500,000,000 of volume (such as Bithumb).
In short, volume is an unreliable predictor of credit event occurrence.
iii) Using Past Hack History to Predict Credit Event Occurrence
Next, we investigate the second proposition outlined in the paper: exchanges that have publicly disclosed a hacking-related incident in the past are more likely to suffer credit events in the future.
The logic behind this is simple. Exchanges that have already suffered a major hack are more likely to become targets for other hackers as the past hack reflects inadequate internal security procedures. Both of which expose them to higher subsequent hacking risk and thus higher credit event occurrence relative to “clean” exchanges. This is reflected in the data below:
Exchanges that have been previously hacked historically suffered credit events at a rate of 22.2% per year. In contrast, “clean” exchanges that have never been hacked before suffered credit events at a rate of 6.1% per year.
That means exchanges that have been previously hacked are almost four times more likely to suffer a subsequent credit event relative to “clean” exchanges.
Predicting Credit Event Loss
The next step in pricing credit risk is calculating the expected loss upon default.
This relationship is much easier to model. Following the logic of the paper, larger exchanges are expected to be more likely to pay back customers in the event of a credit event as they are more profitable. This makes it more likely they can cover any losses with internal funds, or be able to sell themselves to another exchange to make customers whole.
To analyze this relationship, we recorded every publicly disclosed credit event in crypto exchange history to analyze the relationship between daily average trading volume and repayment rates. This data is graphed below:
The data is plotted on a logarithmic scale scatterplot using a logarithmic regression (hence why the trend line looks linear). The relationship while somewhat weak, clearly illustrates a positive trend between an exchange’s average daily trading volume and credit event repayment rates.
Exchanges with higher volumes are more likely to repay their customers. The halfway point comes around $700,000, which is where exchanges would be expected to pay out $0.50 of every $1 of customer deposits after a credit event.
Adjusting for Subjective Factors
The model up to this point has relied exclusively on historical data in its design. While this allows us to be relatively objective and gives us historical base rates to anchor our expectations, there’s a number of subjective risk factors that have no historical precedent that need consideration as well.
Because these risk factors are subjective by their very nature, it’s difficult to quantify their impact and test them against historical data.
There are three subjective risk factors to consider for each particular exchange:
- Cybersecurity procedures
- Regulatory risk
- Banking risk
We propose a simple methodology for taking in each of these factors.
i) Cybersecurity procedures
Cybersecurity is a particularly challenging factor to quantify. A recent report came out which details a methodology for measuring different exchange security procedures.
We’re interested in overall exchange risk rather than individual user security, so two facets of this methodology are important to focus on: registrar and domain security & web protocols security. Fortunately, there are websites that provide quick quantitative assessments for every website on the internet.
The main problem with these ratings is that they provide surface level insight and therefore can’t be relied on exclusively given their crudeness. They do provide a way to objectively rank exchanges though, and provide value by flagging exchanges with particularly poor security procedures.
ii) Regulatory Risk
Regulatory risk is another factor that is challenging to quantify. However, given the recent uptick in regulator-driven shutdowns and predictions by some in the space that these will continue to increase in the future, it is clearly a tail risk that traders can’t afford to neglect.
The clearest example of regulatory risk was the shutdown and seizure of the BTC-e exchange in July 2017.
Bad way to start the day if you’re a BTC-e customer
The US Ministry of Justice, with the help of the Greek police, arrested the alleged founder of the exchange and seized the website. BTC-e customers in return lost access to their assets. Regulators acted because the exchange was openly breaking US anti-money laundering (AML) laws and didn’t require users to do KYC to access the exchange.
Another example of regulatory risk was the recent shutdown of crypto CFD exchange 1Broker. Particularly important in the 1Broker example was the fact that they were primarily shut down for breaking US Securities Laws by offering access to derivatives and securities. This same argument can be made against a number of other current crypto exchanges that offer either derivatives or trading of tokens that are very clearly securities.
Therefore, a quick way to gauge regulatory risk is presented below:
- Does the exchange require KYC to access the platform?
- Does it offer trading on tokens likely to be classified as securities?
- Does the exchange offer margin and/or derivatives?
- Did the exchange do a potentially illegal ICO to fund operations?
- Is the exchange part of a formal regulatory environment?
By answering these five questions and weighting them by their importance, traders can get a sense of each exchange’s relative regulatory risk.
iii) Banking Risk
The final subjective risk factor relates to the risk that exchanges that enable fiat-denominated deposits and transactions will lose access to their banking partners. This has historically been the least severe of the aforementioned risk factors, as it at least enables the exchange to find new banking partners.
It is somewhat easy to track this risk as well, because traders that get worried about this risk will buy BTC at above market prices in order to withdraw assets, leading to a premium BTC/USD spot price on the exchange.
A good recent example of this is the increasing BTC/USD premium on the Bitfinex exchange relative to other exchanges after it announced earlier that it had temporarily lost access to its banking partner:
It should be mentioned that this risk also applies for any exchanges that use banking-dependent stablecoins (such as Tether) as a trading pair currency for their exchange. This is because if the stablecoin permanently loses access to the banking system, it’s value would plummet and the exchanges and their customers that rely on it would suffer severe losses.
Conclusion
It’s estimated that around a quarter of all crypto asset value is currently being held on a centralized cryptocurrency exchanges. With the current crypto asset market capitalization around $200 billion, that’s $50 billion of uninsured assets sitting in exchanges with relatively poor track records of safeguarding customer funds.
This article represents the first step in our mission of providing greater clarity around quantifying credit risk in the crypto ecosystem.
Moving forward, we will be publishing monthly credit ratings for the largest 150 crypto exchanges using the ideas and methodology articulated in this article.
However, the only way to truly drive transparency into the actual magnitude of credit risk in the crypto ecosystem is through the creation of tradable credit risk markets. By allowing traders to specifically bet on and profit from credit risk, we can get market-driven pricing of crypto credit risk.
Our mission as founders of CDx is to create these credit risk markets. Join the community and help us make the crypto ecosystem safer for everyone.
- Telegram: CDx
- Reddit: r/CDxProject
- Twitter: @CDx_Project