Using Python, Selenium and Telegram
Think filling out forms on the web is boring and tedious? I cannot agree with you more. I’m writing this post to document the process of creating a Telegram Bot to automate a web process and demonstrate how we could save time and effort by using programming to automate simple tasks. Let’s get started!
Motivation and Idea💡
Created by Katerina Limpitsouni
I often find myself sending articles to my friends and family when I stumble upon something interesting. Then, we have a lot to share and discuss and we often think that it would be great if we can highlight and comment upon the article. After googling for a solution, I found about Outline.com, which is a tool that enhances readability of online articles by removing the clutter and also allows the reader to make annotations. The process is simple:
- Copy the link of the article you want (E.g. https://towardsdatascience.com/an-introduction-to-json-c9acb464f43e).
- Go to Outline.com and paste it in the box and hit “Create Outline” (An “outline” is basically what Outline.com creates of your article — a simple, ease to read and nice to see version)
- Within a few seconds, Outline will redirect you to your “outline” with a customised link that you can share with anyone. Your highlights and annotations can be seen by anyone with the link.
However, I quickly realised that it would be quite tedious each time to copy the link, go on Outline.com and then fetch another link (of the outline) to share. Thus, I decided to automate this process. Since I mostly use Telegram to share these links, I decided to write a Telegram bot that can automate this process of obtaining an Outline link for me.
Tech Stack 📚
Created by Katerina Limpitsouni
To build a Telegram Bot, let’s use Python (as it is less verbose, easy to use) version 3.7.x and has an excellent wrapper for the API. For automating the process, let’s use Selenium, the most widely used open source tool for such purposes. We’ll also use logging (to log our program better) and time (to make it sleep). Thus, we’ll be using the following python libraries:
- Telegram Python Wrapper
- Logging (should be in Python by default)
- Selenium Chrome WebDriver (For this post, we use Chrome on macOS, but you can choose any OS and browser as Selenium supports most of them).
- Time (should be in Python by default)
Process 👨🏻💻
Created by Katerina Limpitsouni
With the concept and the tech stack clear, let’s dive into writing some code. The following section is divided into sub-sections. Please feel free to skip if you’re familiar it.
Setup logging
Let’s use the logging module in Python to setup logging. This is an important step as it allows us to debug the program better and monitor the flow of the program. You can read more about it here.
After importing the logging module, we can set it up:
logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',level=logging.INFO)
This sets out the basics.
Obtain authorisation key from Bot father.
Every telegram bot requires an authentication key from Telegram. It is a complex string of alphanumeric character that is required to authenticate and send requests to the Bot API. It helps Telegram recognise the owner of a bot.
In Telegram, head to BotFather — a bot by Telegram to handle all bot related stuff. Next, we can send the /newbot command to create a new bot. BotFather requires a username and name for the same.
We can copy it and store it in a safe place. You can read more about BotFather and its various capabilities here.
Enter Auth-ID, start Updater
At first, let’s import all the necessary libraries:
from telegram.ext import Updater, CommandHandler, MessageHandler, Filters
The updater object receives updates from Telegram and sends them to the dispatcher. You can read more about it here. It basically represents the Telegram.Bot object and acts as a front-end to it. We can setup the Updater this way and connected it to a dispatcher:
updater = Updater(token='TOKEN', use_context=True)
dispatcher = updater.dispatcher
Note: use_context=true is a special argument only needed for version 12 of the library.
Setup updater, dispatcher and start/stop
It is recommended that all telegram bots have the global commands: /start,/help , and /settings. /start is especially very important as it begins interaction with the bot.
Let’s define a function to handle the /start command (when the user starts the bot)
def start(update, context):
context.bot.send_message(chat_id=update.message.chat_id, text="I'm a bot, please talk to me!")
Now, we need a CommandHandler (remember the import above?) to handle the /start command from the user. We also need to register it in the dispatcher:
dispatcher.add_handler(CommandHandler('start', start))
Next, let’s add the /read command — the main feature of our bot. First, let’s import the necessary:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
Let’s define a read function to handle the /read command
def read(update, context):
and setup a handler and add it to the dispatcher:
dispatcher.add_handler(CommandHandler('read', read))
Next, we need to use the Selenium driver downloaded earlier. Let’s add import the necessary items:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
Now, let’s setup a browser object. A browser object is basically an automated browser that can be controlled by the Python commands we write. You’ll need to specify the directory of the chrome driver you downloaded earlier.
browser = webdriver.Chrome('**directory of chromedriver**')
Let’s make it visit Outline.com:
browser.get('https://www.outline.com')
Now comes the interesting part. How do we make it submit the link from the user to the form on the outline website? We can use the Chrome dev tools to inspect the form and copy its ID to enable the selenium browser to find it. We can also extract the link (the only argument of the /read command) and passed it on:
linkbar = browser.find_element_by_id('source')
linkbar.send_keys(context.args)
linkbar.send_keys(Keys.ENTER)
We will have to stop the program for 10 seconds, as it takes sometime for the URL to change from just outline.com to outline.com/something — this is the link we want. Finally, we send the obtained link back to the user using the browser’s current_url attribute:
time.sleep(10)
context.bot.send_message(chat_id=update.message.chat_id, text=browser.current_url)
At last, we clean up our workbook and put the key info in a main function:
def main():
# Create updater and pass in Bot's API token.
updater = Updater(token='your_key_here', use_context=True)
# Get dispatcher to register handlers
dispatcher = updater.dispatcher
# answer commands
dispatcher.add_handler(CommandHandler('start', start))
dispatcher.add_handler(CommandHandler('read', read))
# start the bot
updater.start_polling()
# stop
updater.idle()
if __name__ == '__main__':
main()
And that’s how we built this telegram bot. Here’s how our final code looks like:
<a href="https://medium.com/media/9ab07641d147d22910c2e8c5fca5531b/href">https://medium.com/media/9ab07641d147d22910c2e8c5fca5531b/href</a>
Feel free to use this as a template! To get started, you can fork my repository here or copy the code from the above gist!
Have fun coding! Here are some resources that you may find useful:
<a href="https://medium.com/media/4fb30b4e5f4a7f77008f0dc21e5b2bfd/href">https://medium.com/media/4fb30b4e5f4a7f77008f0dc21e5b2bfd/href</a>