Data Signals vs. Noise: Misleading Metrics and Misconceptions About Crypto-Asset Analytics

The steady growth in the crypto-asset space has increased the need and popularity of market intelligence/analytics products. However, like any other new asset class, the methodologies and techniques to extract meaningful intelligence about crypto-assets are going to take some time to mature. Fortunately, the crypto market was born in the golden age of data science and machine learning so it has a shot at building the most sophisticated generation of market intelligence products ever seen for an asset class. Paradoxically, it seems that we prefer to remain lazy and come up with half-baked analytics that have the mathematical rigor of a fifth grade class.

The current wave of analytic products for crypto-assets are still very nascent and experimenting with all sorts of new ideas. However, there is a difference between experimentation and lack of rigor. Sadly, the crypto-market is constantly bombarded with outrageous claims from analytics providers that don’t require a PH.D in statistics to know they are flawed. Today, I would like to deep dive through some of the most common “flawed analytics” you might have read on research materials about crypto-assets.

Analytics are hard and its construction is hard to understand by most people. As a result, it is easy to misinterpret a semi-complex analytics by robust ones. If to that, we add the fact that the behavior of crypto-assets remains an enigma to most investors, we have the perfect storm to produce fantasy analytics and ridiculous explanations about the crypto-markets. While the are no silver bullets to determine whether a specific metric or analysis for crypto-assets is relevant, there are a few tips that might help dissect signal from noise in this area:

1) Crypto-Asset Specific: The most effective metrics to evaluate the behavior of crypto-assets are, surprise-surprise, those that factor in elements that are specific to crypto and that have no equivalent in other asset classes.

2) Regular Data Validation: Look for metrics/signals that offer regular data validations proofs about their impact of correlation with price(if any). You will be surprised what you find 😊.

3) Human Relatable: Crypto remains largely a retail investor market in which data is mostly free and available. Analytics and market metrics should mimic the demographic of the investor population and, in crypto, that means to have simple signals that normal people can understand. Complicated patterns in a market that we don’t quite understand yet is a recipe for disaster. Simple doesn’t only mean easy to understand but flexible to change and adapt to market changes.

Like any other asset class in history, crypto-markets are full of speculation and propense to formulate theories without basics on real data. Without citing any specifics, I listed some of my favorite examples of misleading analytics for crypto asset. You might recognize some of those from different tools 😊

We’ve all heard this before “our score combines 3–4 factors and is the perfect measure of the performance of a crypto-asset….”. When you dig a bit deeper into those market scores you realize that are simple weighted of a number of basic factors. The idea of analyzing a complex financial market with a single score is preposterous. Markets are based on complex, non-linear dynamics that involves tens of thousands of factors many of them unknown to us. Analyzing those complex dynamics requires combining factors in many ways for different periods of time. No score works for all assets throughout all market conditions. That fact is even more relevant in the case of crypto-markets that are completely irrational and immature. Next time a crypto-analytics vendor tries to sell you a score run for the door 😊.

Sentiment analysis of social feeds like Twitter or Telegram is another common metric in the crypto space. Certainly, intelligence from social data can reveal trends in specific crypto-assets but you really need to dig deeper. Using basic sentiment analysis APIs such as Watson Assistant for analyzing crypto-assets is just being lazy. News are different than Twitter, discussions about the SEC, China or a listing in Coinbase are different from a token sale. Not to mention the level of noise introduced in mainstream social channels such as Twitter or Telegram. In other to extract any meaningful intelligence about crypto-assets using social fees, we need custom deep learning models that are really tailored to the specifics of the market.

Technical traders are an important element of any financial market and crypto is not the exception. However, regardless of what your favorite chartists say, technical has never been enough to understand any financial market and certainly not a nascent one like crypto. Crypto is a new asset class that requires new fundamentals Any technical analysis needs to be a complement of models based on the native characteristics of a crypto-asset such as network or ownership behavior.

The wisdom of the crowds is playing a significant role on how we predict the outcome of specific events in society. In financial markets, crowdsourced human predictions are typically used as a complementary data point rather than as the core analysis. The reason is that crowdsourced predictions tend to be more effective when applied to isolated, statics events such as a political election than when used to predict constantly changing dynamics such as the ones seen in financial markets. Movements in financial markets are often describe by complex, non-linear relationships between large number of variables, not by subjective human opinions. In the case of crypto-assets, the idea of basing market predictions on crowdsourced human opinions is even more questionable given that all the information is relatively public and the market is so irrational that escapes the understanding of most of us.

One of the cornerstones of factor investing models is that some factors will serve as predictors to other factors. Its very strange, almost illogical, to try to predict a factor using recursive formula based on the same factor. Well, not in the case of crypto. The market is certainly floated with ideas about prediction the price of Bitcoin using the price of Bitcoin. If you would like to determine the rigor behind those predictions ask your provider how much money are they investing using that strategy. The answer, almost certainly, will be zero.

These are some of the most misleading metrics you will find in the current generation of crypto-asset analytics. Extracting meaningful intelligence from crypto-assets requires digging deeper into factors that are specific to this asset class.