Network Scanning or network recon is a fingerprinting method used by hackers to find open ports, services, service versions, and possible vulnerabilities. But the question is how these scanners work and how can we implement them using a few open-source modules or build one.
In this blog, I will be discussing various things a network scanner does, and for this explanation, I will be explaining using nmap (network mapper) as a medium of scans and we will be implementing a simple network scanner using python and Python-Nmap module.
Table of content
- Intro to network scanners
- Network scanner working
- Needs for a scanner
- Types of scans
- Os fingerprinting
- Service fingerprinting
- Build a scanner in python
- Conclusion
Intro to network scanners
Network scanners, as the name suggests, scan an IP or a HOSTNAME for open ports, services, service version, and also vulnerability assessment. Scanners are used in information on a target this provides the security personnel crucial insight into the networking side of the target.
If information is found by an attacker, it can cause serious damage as it can lead to a successful breach of the system and cause damage to data and its integrity such networking related are safely maintained by firewalls and incident response personnel and occasionally tested by white hat hackers and testers to ensure its security is intact and do possible patches to enhance the security.
Network scanners such as nmap use the TCP protocol to communicate with the ports and check whether the service is active, closed, or if not defined filtered. There are different methods and different scan types used to determine ports and which are open and which are closed but mainly the filters also work as bypass tools to escape detection.
Network scanner working
For this explanation, we will be looking into NMAP or Network Mapper one of the most widely used and accurate network scanners we will be looking into its basic working mechanism and look into how they are used by the scanner to decide its outputs.
The main functions Nmap or any network mapper performs are as follows:
- Identifying filtering systems between two active hosts on a network.
- Running UDP and TCP network services.
- Detect the TCP Sequence Number of both the hosts.
For example, when I perform a scan on my website morpheuslord.tk I run this command
$ nmap -sV -A morpheuslord.tk
The output is as follows.
NMAP uses the conversation and the various SYN ACK requests to determine whether the port is open, closed, or filtered this also has a few added benefits of also gaining more knowledge on the service, service version, and the software specification running behind the open port which can help in determining if the service is vulnerable or not.
It also closely examines the scan fingerprints and segments over several packets to analyze the OS or the Service running as each Service or OS has its different segment values for the packet sent, we will be looking into OS and Service fingerprinting in a bit.
The background process running:
There is a constant request and response between the client and server and the NMAP tool will analyze the packets and assess the data accordingly.
The above output is a simple scan on the target website. There is a detailed output on the service and if the service is open closed or filtered. By checking the packets, the service can also check for the Linux OS version and possibly the kernel version.
This is a vital part of the scan outputs that can be used to further analyze the server for deciding the best exploit.
Needs for a network scanner
Scanners of such fashion are used in many areas other than just hacking some use them for defense and some use them to debug and test specific services.
These are the major needs of a network scanner:
- Debug and test a network configuration
- Find vital information within a network
- Conduct recon for vulnerability analysis
- Do a comprehensive analysis of Network architecture.
When in a network environment we will have to find many processes and services and keeping track of such information is tedious checking whether they have proper functionality also plays a vital role. Network scanners check for the connectivity and the configuration structure by performing connection requests and also by conducting basic configuration tests to make sure that the configuration is good enough for further procedures.
In a network environment where data is considered a priority maintaining security is vital that is where hackers and Pentester use scanners and other related tools to scan the network internally and externally for crucial information such as vulnerabilities loopholes information leaks miss configs etc and help in rectifying them to avoid future threats.
Types of Scans
Typical 3-Way Handshake:
Typical UDP conversation:
Several types of scans can be used they are:
- TCP Scan (connect() scan)
- SYN Scan (Half Handshake scan)
- ARP Scan (ARP requests)
- UDP Scan
- Traceroute Scan
- Fast Scan
These Scans are the bases of all the other scans in the system each scanned work mainly on the 3-Way Handshake that the Client-Server system makes to determine whether the IP is active, PORT is OPEN or to find the OS information and service information.
TCP SCAN:
The TCP scan or connect() scan is a type of scan that, unlike SYN scans, will send connection requests on the port specified for example instead of checking whether the port is open via a simple hello. It checks if the port is open by trying to connect it and if there is an open port there will be an exchange of banner data signalling an open port.
This scan uses a complete 3-Way Handshake unlike SYN scan or FIN scan. This scan has a high chance of getting caught but can provide accurate results when used with appropriate measures.
SYN SCAN:
The SYN scan or stealth scan is a scan done where a half handshake sequence is conducted. This happens when there is a conversation where the client sends an SYN request and when the server responds with an SYN-ACK the client will end the conversation without sending the ACK request. This leads to the server resetting the conversation with an RST signal. This is most commonly used to conduct DoS attacks on services and can also mitigate detection to a certain extent.
The scan is used by security testers to check the target information without getting detected that includes sending requests and receiving data the logging system will record the incident to prevent getting logged the program will gain as much information as possible with just half a conversation and ends the conversation without sending any further requests after the timeout period the server ends the conversation with an RST signal stating the end of the conversation to prevent any DoS attacks this also stops the logging to a certain point but it is still logged as an incomplete connection for future reference.
ARP SCAN:
ARP or Address Resolution Protocol is used for local device discovery over IPV4 addresses and can be used to discover a large number of devices on a given subnet this can be used to discover various other devices connected to a local network.
This works by sending ARP requests over the local subnet addresses and based on how the IP addresses respond to the request whether they are active or inactive this is a versatile utility as this can be used in conjunction with other scans to resolve and scan through an extensive range of targets.
UDP SCAN:
UDP just like TCP is widely used in almost all systems and they have support for every single networking device kernel, therefore, is effective for scanning as it does not need a 3-Way Handshake like the TCP scans but instead moves to the TLS connection.
These scans are relatively slow compared to the 3-Way Handshake method but are also highly effective scanning methods for the newer QUIC method introduced in HTTP3 Methods which works on UDP packet transfer.
These scan outputs are simple if the probs return with data it signifies that the port is open if the probs have no returning data that signifies that the port is closed or filtered and if there is an error then it signifies the port is closed.
Traceroute Scan:
Traceroute is a system command and is used in networking to map the data transfer from the Server to Client systems. Commonly used to diagnose the network latency and to also discover the various hops the data makes before it reaches its destination.
Traceroute is a default system command and also has a lot of features to list the intermediary devices and also to reverse engineer proxies.
Fast Scan:
Fast Scan is a TCP scan that is done only for the first 400-500 ports and excludes the rest of them.
This is a simple and fast method to check for fewer ports and needs. This is used when you know that there is a vulnerability in one of the 0-400/500 ports and also when you don’t have enough bandwidth to scan all 65535 ports if you have no clue how many are open and how many are closed.
Os fingerprinting
Os fingerprinting works by checking each packet’s fingerprint over 5-6 packets and based on the matches in each box we can determine which Os it is as each Os has a unique sequence of matches.
In nmap specifically, they use a database to verify the Os. They have a nmap-os-DB with many matches and sequences listed for a huge array of operating systems and using this the nmap code can guess what kernel is used and also narrow the search by guessing the closes Os to the sequence.
Finding out which Os is running in the backend is very vital to determine any exploits mainly kernel-based exploits.
You can view this via this link.
Service fingerprinting
Service fingerprinting works in the same way as Os fingerprinting does by analyzing. Nmap does it like this. Service fingerprinting works by first scanning for the port each port has a dedicated service for example 22 is for SSH, 21 telnets, etc. Using this information, the services can narrow down to a few ports to be fingerprinted.
After getting to know the open ports the scanner can check the contents of the packets with the database the TCP data of the packets are analyzed over a sequence of consecutive packets and then the service is determined.
Nmap is also based on the same concept and using the same technique the algorithm determines the services.
Services and service versions are used to detect cve for specific services and their versions if you know the service and its respective version you can search the vulnerability and exploits over the internet in resources such as exploit-DB or any cve database.
Exploit nmap database: https://svn.nmap.org/nmap/nmap-services
Build a scanner in python
Now let’s get to the interesting part: let’s build a Network Mapper implementation. In this example we will be using 2 python modules. One is subprocess and the second is python-nmap we will be implementing the nmap in two ways one is using the os installed nmap which is fast and reliable and the second is python nmap which is also fast but the output retravel is not that convenient.
Python-nmap
Installing
$ pip3 install python-nmap
In python-nmap, all the scan features are predetermined the most important functions we can use are:
- nmap.PortScanner()
- The actual scanner module that contains all the other scan functions
- nmap.csv()
- Used to show a CSV output of scans
- nmap.analysis_nmap_xml_scan()
- Used to show a complete JSON output of the scan.
- nmap.scan()
- The scan function to scan the target
- nmap.scan(‘’,arguments=’’)
- The scan function with the first argument is the target and the arguments variable is used to declare special arguments into the scan to improve scans.
- The scan function with the first argument is the target and the arguments variable is used to declare special arguments into the scan to improve scans.
So now that all are taken care of let’s implement a scan.
**Simple demo code:
This will scan the localhost and show a CSV output from this we can understand how we can declare the nmap module.
Let’s increase the productivity of the code.
Let’s take the IP/Hostname from the user also with arguments that he wants to scan specifically and see a JSON output.
Now instead of showing a console output, let’s convert JSON output into an HTML report.
This was a simple implementation via a few command-line inputs and an HTML report generation but we can use this to achieve much more for example implementing this on a flask webpage for a static web page with such a crazy implementation.
**Subprocess
Subprocess is a python module that is used to run system commands and using this we can run nmap directly on the system and redirect the CSV output into JSON format and then to an HTML format
We will not implement all the things step by step but write a complete code and explain the code.
For this implementation you will need:
Subprocess: which is an inbuilt module
LXML: module to convert XML data to HTML using an XSL stylesheet
Install LXML
$ pip3 install lxml
Code:
The XSL file mentioned above can be copied from the following locations:
LINUX : /usr/share/nmap/nmap.xsl
WINDOWS : C:\Program Files (x86)\Nmap\nmap.xsl
From the above code we are:
- Taking the IP or HOST and ARGS from the user.
- Defining the command and adding data using formation and executing it
- Reading and storing output XML data
- Using XML tree to declare a spreadsheet.
- Using the spreadsheet to convert the XML data to HTML data.
- Write the HTML data to a file to be viewed.
Advantages:
- More reliable.
- Better output.
- Well-defined report.
Conclusion
In conclusion, we can see how a network mapper works and also see various other technologies involved using a few python programs and a few lines of code we can implement a console wrapper code to run nmap and create a simple port scanner.
Sources
The main source for all the data is personal experience and research documents from nmap.
Note
If you want to get in touch with me you can do that via LinkedIn using the below link.