How to Build Telegram Chats with a Crypto-trading Bot

Perhaps a sign of the times: my most active Telegram chat is with a crypto-trading bot that constantly listens for opportunities to trade on my behalf. I used an open-source library to develop some strategies and configure the bot to execute them using my Binance account. The bot communicates all of its trades through Telegram and can reply to my requests to take action or share live updates.

15-second demo:

The Freqtrade library lets you code, backtest and perform optimization on a trading strategy the same way you would a machine learning model. This post will walk through some essential features of the bot and how to fine-tune strategies, without going into too much detail about every configuration/implementation step — there are plenty of tutorials that do a great job, and they're linked in the Resources section at the bottom.

Freqtrade uses TA-Lib (Technical Analysis Library), which is also an open-source library used by trading software and professionals to perform technical analysis on financial market data (not specific to cryptocurrencies). It includes about 200 indicators such as Bollinger Bands, RSI, MACD and more.

THE RECIPE

Freqtrade's documentation contains instructions on how to configure your Telegram bot and connect to your exchange of choice (Binance or Bittrex), but the core of it all is the strategy itself. The process can be summarized in 10 steps:

Implement a strategy
Dry-run the strategy with simulated trades
Backtest the strategy with historical price data
Optional: Plot and visualize the strategy with indicators
Hyperparameter Optimization, a.k.a "hyperopt"
Update your strategy to implement recommendation from hyperopt
Dry-run the new strategy
Backtest the new strategy
Optional: Walk-forward analysis on new strategy
Turn on live trading

Let's dive into each step:

1. Implement a strategy

There are many out-of-the-box strategies you can use, but you can choose to have them as a starting point and then do your own tuning, polishing and optimization. For my bot, I started out with an existing strategy and iterated on it by selecting my own technical analysis indicators, configuring buy and sell rules and running the optimizer while tweaking a small number of variables at a time.

Every strategy has a skeleton with the functions

populate_indicators

populate_buy_trend

and

populate_sell_trend

This is what the

populate_indicators

function looks like for the strategy I used, Strategy002 (from the freqtrade/strategies repository)

    def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
        """
        Adds several different TA indicators to the given DataFrame
        Performance Note: For the best performance be frugal on the number of indicators
        you are using. Let uncomment only the indicator you are using in your strategies
        or your hyperopt configuration, otherwise you will waste your memory and CPU usage.
        """

        # Stoch
        stoch = ta.STOCH(dataframe)
        dataframe['slowk'] = stoch['slowk']

        # RSI
        dataframe['rsi'] = ta.RSI(dataframe)

        # Inverse Fisher transform on RSI, values [-1.0, 1.0] (https://goo.gl/2JGGoy)
        rsi = 0.1 * (dataframe['rsi'] - 50)
        dataframe['fisher_rsi'] = (numpy.exp(2 * rsi) - 1) / (numpy.exp(2 * rsi) + 1)

        # Bollinger bands
        bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=4)
        dataframe['bb_lowerband'] = bollinger['lower']

        # SAR Parabol
        dataframe['sar'] = ta.SAR(dataframe)

        # Hammer: values [0, 100]
        dataframe['CDLHAMMER'] = ta.CDLHAMMER(dataframe)

        return dataframe

Out of the box, the strategy uses the following indicators:

Stochastic Oscillator Slow (Stoch): This is the Slow Stochastic Oscillator, a momentum indicator that shows the location of a closing price relative to the asset’s price range across a period of time. Its value is between 1 and 100.
RSI: Also a momentum indicator, the Relative Strength Index (RSI) measures the speed and change of price movements. An asset is considered overbought when its RSI is above 70, and oversold when it’s below 30. The RSI is used to identify general trends, and its value also ranges from 1 to 100.
Inverse Fisher transform on RSI: This is a transformation of the normal RSI indicator, adjusted so that its values are centered around zero. This one will range from -1 to 1.
Bollinger Bands: Bollinger bands have a center line and two price bands above and below it. The center line tracks the exponential moving average, and the price bands above and below are the standard deviations of the stock price.
SAR Parabol: The parabolic SAR can indicate the direction in which an asset is moving. The indicator appears as dots on a chart placed either above or below the brice bars. A dot below means a bullish signal. A dot above is a bearish signal indicating that the momentum will go down.
Hammer: The Hammer candlestick is a price pattern that occurs after a price decline. It indicates that sellers came into the market during the period, but selling was absorbed and buyers pushed the price back near the open. So an asset might have traded significantly lower than its opening, but rallied between the open and close and finished near its opening price again.

Read more about the TA-Lib indicators here: http://mrjbq7.github.io/ta-lib/doc_index.html

2. Dry-run the initial strategy
Dry-runs are very effective in testing the code itself as well as the soundness of a strategy. All one has to do to dry-run a strategy is to set

"dry_run": false

in their config.json file.

A bot in dry-run mode behaves just like it would in production, but all the trades will be simulated. You will still be able to communicate with your bot and monitor all its activity.

3. Backtest the initial strategy

Backtesting is very easy once you have the data you want. All the steps are specified in the Freqtrade documentation:

Create a directory under user_data/data/<exchange> and copy the freqtrade/tests/testdata/pairs.json file into that directory
Populate the pairs.json file with the pairs you're interested in getting historical data for

Download the data with

freqtrade download-data --exchange-binance

Run a backtest with

freqtrade backtesting --strategy <StrategyName>

4. Plot the strategy

Plotting can help you visualize the bot's buy/sell activity, price movement, as well as indicators against an asset pair.

The follownig plot contains ETC/BTC prices along with the bollinger bands and fisher RSI.

To plot this, we run:

freqtrade --strategy <StrategyName> plot-dataframe -p ETH/BTC --indicators1 bb_lowerband --indicators2 fisher_rsi

5. Hyperparameter Optimization

This is a key step in systematically improving any strategy, pre-packaged or not. Similar to tuning a machine learning model, we will be running a process to optimize on a loss function such as the Sharpe ratio or pure profit. The Freqtrade library uses algorithms from the scikit-optimize package to accomplish this.

The strategy itself is defined in one Python module, and the optimizer is in another. A hyperopt process consists of the following parts:

Indicators: There are two different types of indicators: guards and triggers. Guards are conditions like "never buy if ADX < 10" or "never buy if current price is over EMA10". Triggers are conditions that actually trigger a buy in specific moment, like "buy when EMA5 crosses over EMA10" or "buy when the close price touches the lower Bollinger band"
Buy strategy hyperopt: Fill in the
```
indicator_space()
```
and
```
populate_buy_trend()
```
based on buy strategy
Sell strategy hyperopt: Fill in the
```
sell_indicator_space()
```
and
```
populate_sell_trend()
```
based on our sell strategy

Determine a loss function, such as

DefaultHyperOptLoss

SharpeHyperOptLoss

OnlyProfitHyperOptLoss

, or write your own

The

populate_indicators()

in the hyperopt file will the be same as the actual strategy's, as will

populate_buy_trend()

and

populate_sell_trend()

When it comes to indicators, we're asking for the optimizer to randomly combine to find the best combination of them. Here's an example definition of

indicator_space()

(there will be one for buy and one for sell):

  def indicator_space() -> List[Dimension]:
       """
       Define your Hyperopt space for searching buy strategy parameters.
       """
       return [
           Integer(10, 25, name='mfi-value'),
           Integer(20, 40, name='rsi-value'),
           Categorical([True, False], name='mfi-enabled'),
           Categorical([True, False], name='rsi-enabled'),
           Categorical(['bb_lower'], name='trigger')
       ]

Two of them are

Integer

values (

mfi-value

rsi-value

), which means that the optimizer will give us the best values to use for those.

Three

Categorical

variables: First two are either True or False: Use these to enable or disable the MFI and RSI guards. The last one is for triggers, which will decide which buy trigger we want to use. We do the same for the sell indicator space.

Run the optimizer, specifying the hyperopt class, loss function class, number of epochs, and optionally the time range. The

--spaces all

is asking the optimizer to look at all possible parameters. But you can actually limit the set, as the optimizer is quite customizable. Let it run for many epochs, i.e. 1000.

freqtrade hyperopt --customhyperopt SorosHyperopt --hyperopt-loss OnlyProfitHyperOptLoss -e 1000 --spaces all

After running the hyperopt script for 1000 epochs, I got the following results:

Buy hyperspace params:
{   'mfi-enabled': False,
    'mfi-value': 30,
    'rsi-enabled': True,
    'rsi-value': 17,
    'slowk-enabled': False,
    'slowk-value': 10,
    'trigger': 'bb_lower3'}
Sell hyperspace params:
{   'sell-fisher-rsi-enabled': True,
    'sell-fisher-rsi-value': -0.37801505839606786,
    'sell-sar-enabled': True,
    'sell-sar-value': -0.8827367296210875}
ROI table:
{0: 0.07223, 29: 0.0288, 82: 0.01703, 159: 0}
Stoploss: -0.0865

Interpretation:

Strategy for buying: Do not use MFI. Use RSI, and buy when its value is < 17. Do not enable Stoch. Buy when the closing price is less than the bollinger band 3 standard deviations below the typical price
Strategy for selling: Use the inverse fisher transform on RSI value, and sell when it’s greater than -0.378. Use the SAR value, and sell when it’s less than -0.883
Use the ROI table to set the
```
minimal_roi
```
value in your strategy
Set the stoploss to -0.0865

6. Implement results from hyperopt

Once you have the recommendations from hyperopt, you want to update the populate_buy_trend and populate_sell_trend in your actual strategy (not the hyperopt file) so that next time you run

freqtrade --strategy <StrategyName>

, it will execute the optimized strategy.

The hyperopt results above would translate to the following buy and sell strategies:

   def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
       """
       Based on TA indicators, populates the buy signal for the given dataframe
       :param dataframe: DataFrame
       :return: DataFrame with buy column
       """
       dataframe.loc[
           (
               (dataframe['rsi'] < 17) &
               (dataframe['bb_lowerband'] > dataframe['close']) # where dataframe['bb_lowerband'] is 3 stdev
           ),
           'buy'] = 1
 
       return dataframe
 
   def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
       """
       Based on TA indicators, populates the sell signal for the given dataframe
       :param dataframe: DataFrame
       :return: DataFrame with buy column
       """
       dataframe.loc[
           (
               (dataframe['sar'] > -0.8827367296210875) &
               (dataframe['fisher_rsi'] > -0.37801505839606786)
           ),
           'sell'] = 1
 
       return dataframe

7. Dry-run the new strategy and interpret the performance

Just like we did with the default strategy in the beginning, dry-run your new strategy to see how it performs!

8. Backtest the new strategy

Just as you did with the initial strategy, run

freqtrade backtesting --strategy <StrategyName>

The last line shows us how the bot did overall. Looks like we have a total profit of 3.55%

For more on how to interpret the backtesting results, read this page in the Freqtrade documentation.

9. Walk-forward analysis on new strategy

There are a few ways to avoid overfitting:

Have a generally sound strategy
Large enough sample size
Walk-forward analysis: Say you have data from Jan - March. Instead of using all of the data to optimize, just use a fraction of the data (Jan, Feb). This will be the in-sample data. Whatever optimal parameters we get out of this, we use those parameters on the following month (March). This is the out of sample data. Do we get the same results and performance? If so, we can be more confident about the quality of our strategy

From the user_data/data/<exchange>/ folder, get the latest data from a data file (the last candle).

For example, the last line in user_data/data/binance/LTC_BTC-1m.json is:

 [1567380600000,0.006791,0.006791,0.006791,0.006791,1.06]

Converted to human date, it's Sept 1 2019

We can take from July and August — about 40 days to optimize on. After that, run the strategy on all the dates between that August end date and Sept 1. Generally we can do 2/3 for in-sample, 1/3 for out-of-sample.

Run the hyperparameter optimizer (hyperopt) on a specific timerange, by specifying two dates with the --timerange parameter

 --timerange=20190701-20190815

The full command would look like:

 $ freqtrade hyperopt --customhyperopt <HyperoptName> --hyperopt-loss SharpeHyperOptLoss -e 50 --spaces all --timerange=20190701-20190815

If you get an error about missing data, make sure you actually downloaded the data for the ranges you specified.

10. Enable live trading

All you have to do is flip a switch: Set

dry_run

false

You may also choose to tune some parameters like `bid_strategy`(strategy when buying) which are less relevant in simulations but could apply in live scenarios:

For example, you can set

use_order_book: true

. This allows buying of pair using rates in order book bids. The ask price is how much someone is willing to sell an asset for, and the bid price is how much someone is willing to buy for. Turning this flag on allows you to buy at the bid price instead of ask. Sometimes the bid price will be lower than the ask, as opposed to ask = bid.

We don't do that in simulation because in a simulation you can't mock that live environment properly. When you're trading live, you want to buy at bid so you can make more money.

Ready to run? Trigger freqtrade with your optimized and tested strategy, and go to your Binance account to monitor the trades!

VARIABLES THAT AFFECT OUTCOMES

Here is a non-exhaustive list of independent variables — values that when changed, can influence the profit/loss:

Trading pairs
TA indicators deployed by the strategy
Indicator search space during hyperoptimization
Loss function used during hyperoptimization

RESOURCES

Freqtrade homepage: https://www.freqtrade.io/
Udemy course on Freqtrade (some content is slightly outdated, but active community of students and lecturers): https://www.udemy.com/algorithmic-trading-in-python/
Documentation about indicators: http://www.tadoc.org/indicator
Documentation for TA Lib: http://mrjbq7.github.io/ta-lib/doc_index.html