The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death.
-Guido van Rossum,Creator of python
Python powers a wide range of technologies, from smartphone apps to spaceships and robots in industrial plants. The language's versatility and the fact that developers can do whatever they want with it have made it popular.
Why Python with DevOps?
As an approach to culture, platform, and automation, Python is an important language for DevOps teams working to automate repetitive tasks, infrastructure management, provisioning, Api driven deployment.
Since the advent, we now have MLOps, NetDevOps, SecOps, and GitOps…
Knowing the basics of Python is precious to developers.
Let’s discover the 5 important modules to shine in your DevOps journey:
- Requests
In the past few years, more and more companies have embraced the REST paradigm to interact with smart devices and servers. This module makes HTTP requests easy and consumes REST APIs in Python.
DevOps is built on APIs, by using this module developers can achieve tons of tasks!
How to use it? First of all, requests is not a built-in Python library, it’s available on PyPi with more than 30 million downloads per week according to GitHub.
To install on Linux, the command installs requests in the Python environment.
$ python -m pip install requests
It offers rich features such as:
- cookies support
- a wide range of HTTP Methods
- support a variety of authentication such as basic auth, Digest, OAuth …
- proxy supports
Here is an example of how to send a simple request:
import requests
apiUrl = 'https://webexapis.com/v1/rooms'
access_token ='MTIwZDJmNGItOTFiNi00YmI5LTk4NzQtN2MyMzJhNDkxMDkzMGQzZDdiZDUtOWY4_PF84_3f179706-f11a-4ab7-ba3e-57a4a96b0892340s4ff'
httpHeaders = { 'Content-Type': 'application/json', 'Authorization': 'Bearer ' + access_token }
body = { 'toPersonEmail':'[email protected]' , 'text': 'welcome to the DevOps room ' }
response = requests.post( url = apiUrl, json = body, headers = httpHeaders )
print( response.status_code )
print( response.text )
The snippet shows how to import the requests module to send a message in a room using Webex APIs.
Daily there is an interaction between clients and servers on the internet and different private networks around the globe, Api is the new CLI, and developers moving to DevOps should learn this module to work easily with their peers in testing and production environments, Apis are everywhere now.
- OS
It provides functions for interacting with an operating system using local, remote, and cloud files. In the realm of DevOps/System admin, it is usual to interact with operating systems daily.
In a nutshell, it uses the underlying operating system. By doing so, we can automate various tasks such as creating and removing directories, checking if a file or directory exists, managing environment variables, and process management.
It offers rich features such as :
- Error handling
- Path manipulation
- File system access
- Process management
here is an example of how to create a file :
import os
with open ("DataCenterLogs.py", "xt") as DT :
DT.write("Hello DevOps admins, this is a new file,just created ")
DT.close()
The script creates a new file called DataCenterLogs.py in the current working directory. The x mode in the open() function tells Python to create a new file if it does not already exist. The t mode indicates that the file is a text file.
One of our daily routines is to work with operating systems such as Mac OS, Windows, and Linux with its various distributions, Mastering the basics of the OS module and its functions is a must for DevOps engineers.
The OS module is built-in, no need to install it in Python
- Sys
The sys module in Python provides a plethora of functions and variables that are used to manipulate different parts of the Python runtime environment, a great way for DevOps engineers to retrieve important information about the system they are working on, such as where the code is running on, the python version they are using, the interpreter version and so much more …
Some of the well-known features are :
- input and output
- command line arguments
- raising exceptions and controlling the flow of execution
Here is an example of how to get information about the Python interpreter we use:
import sys
print(sys.version)
The output should look like this:
3.7.16 (default, Dec 7 2022, 01:11:59)
[GCC 7.5.0]
As one of the most important modules in Python, developers moving to DevOps should learn because every day we work in a testing environment before moving to production, they have to know the development environment they are working on, to work easily with the rest of their teams.
The sys module is built-in, no need to install it in Python
- Beautiful soup
This module helps us to extract information from websites, a process known as Web Scraping.
The world behind the internet seems strange, network protocols and data formats are hidden in the transport and exchange of data while we enjoy browsing the web and using apps on our smartphones.
It is useful for extracting data on web pages from HTML and XML documents, scraping websites, and automating tasks, in the DevOps realm It is also valuable for debugging and testing in a CI/CD pipeline.
It offers rich features such as :
- Parse HTML and XML documents
- Extracting data from documents
- Modifying documents
To install the module:
pip3 install beautifulsoup4
After installing, we can test to import the module Beautiful soup as bs4 and the requests module :
import bs4
import requests
def get_network_device_info(url, device_name):
# Get the HTML content of the webpage.
response = requests.get(url)
html_content = response.content
# Create a Beautiful Soup object from the HTML content.
soup = bs4.BeautifulSoup(html_content, "html.parser")
# Find the network device information.
network_device_info = soup.find("table", class_="network-device-info")
return network_device_info
def main():
# Get the URL of the webpage.
url = "https://www.TalesOfTechnology.com/network-devices"
# Get the network devices.
network_devices = get_network_devices(url)
#iterates over the list of network devices
for network_device in network_devices:
print(network_device)
if __name__ == "__main__":
main()
The code imports beautiful soup and then gets the content of a webpage using requests, which is useful for consuming REST APIs by sending HTTP requests then it creates a beautiful soup object from the HTML content and finds the network devices information.
By writing this type of code we can automate the process of getting information about a specific network device from a web page and there are various tasks we can achieve beyond what we think with beautiful soup.
In the world of testing and Automation, the goal is to reduce some manual tasks by automating a process, Developers should beautiful soup!
- Platform
We live in a world of hardware and software, One of the usual tasks for developers is to work with operating systems of different flavors, with the module they can gather precious information about the OS and hardware they are using.
It provides valuable information by identifying data such as the Python version, OS version, and CPU architecture.
This module plays a major role for developers when they want to test if the hardware specification meets the requirement to use a program and if it is compatible with a Python version installed on a particular system.
It offers rich features such as:
- Os information
- Hardware information
- Python version
The Platform module is built-in, no need to install it in Python
Here is a snippet of how to get the OS name :
import platform
#get the os name
Os_Name = platform.system()
print(Os_Name)
The output should look like this:
Linux
The code imports the platform module and then uses the platform.system() function to get the operating system name, it returns the OS name Linux.
how useful is the platform module? developers moving to DevOps should learn to simplify their work.
- Where do the modules fit in the DevOps Pipeline?
Some engineers say DevOps is boring! 😀
The fact is it’s around processes and automation, In a pipeline, we have eight stages that combine the roles of two teams: Development and Operation, which is known as DevOps.
In the Development phase, we have:
- Plan
- Build
- Code
- Test
and in the Operation phase:
- Release
- Deploy
- Operate
- Monitor
The modules requests, os, sys, beautiful soup, and platform are versatile and can be used in different phases of the pipeline, They are precious tools to help engineers gather information on systems, automate complex tasks, troubleshoot problems, and save time for more efficiency.
- Conclusion
DevOps has changed the way we work now in the tech industry. It is not a technology but a philosophy, a culture, and a new way of doing things quickly by automating some processes, Python is the language of automation, and developers moving to the new culture should learn some modules to start their journey.
To climb to the DevOps pyramid of success, it is always better to start with the basics; it is not possible to learn and master all the modules! It takes days and years to become a seasoned developer.
The modules shown are not exhaustive, there is a plethora, The goal of this article was to show the most used modules you can encounter in your daily tasks once you start your journey. As soon as you grow, you will discover more.
Go step by step with the basics for a successful journey, Rome was not built in a day!