Why reading RFCs doesn’t have to be scary
IRC has always been a big part of my life. Some of my strongest friendships grew from chatting in IRC channels I visited every day since I was a teen. As I walked along the path to a career in computer science, it was only natural that I’d wonder how it all worked under the hood. But where do you even start?
Turns out people from the Internet Engineering Task Force (IETF) write these things called Request for Comments (RFCs) detailing the exact specification of how different systems and protocols work. Many of these set the standard by which people build their implementations. One of these is for the IRC protocol. Sounds kinda intimidating, right? Let’s say it again. A bunch of geniuses from the Internet Engineering Task Force write RFCs that everyone on the planet considers as standards.
It doesn’t have to be scary, though. I was initially scared off of reading RFC 1459, the spec followed by almost all IRC servers and clients, because I thought it would be too complicated. I thought they were written by real engineers, for real engineers, and not computer science students like myself. But a couple years later I dove in, and found out the water was a lot shallower than I imagined.
The IRC spec really is a simple protocol, and you don’t need to be a genius to figure it out. To prove it to you, I’ll go over some excerpts of RFC 1459 to gain an understanding of how the protocol works. Then we’ll use what we learned to implement our own simple IRC client in Node.js. Hopefully by the end it should be clear that RFCs are approachable, and that anyone can write their own implementations of standard protocols. Knowing that is an important skill to have in your toolbox, as you will have the freedom to implement a protocol in your language of choice. You don’t have to be stuck waiting for someone else to implement it or use a bad existing implementation.
What is IRC?
If you got this far without knowing what IRC is, thanks for sticking around! Instead of coming up with an explanation of my own, lets jump right into the abstract of RFC 1459:
The IRC protocol was developed over the last 4 years since it was first implemented as a means for users on a BBS to chat amongst themselves. Now it supports a world-wide network of servers and clients, and is stringing to cope with growth. Over the past 2 years, the average number of users connected to the main IRC network has grown by a factor of 10.
The IRC protocol is a text-based protocol, with the simplest client being any socket program capable of connecting to the server.
It’s chat rooms! You take a client, connect it to a server, and you can chat with other people in different channels. It’s not unlike a decentralized version of Slack, if you are familiar with that. There are many different server and client applications. Some clients run entirely in the terminal. Some have GUIs. The IRC client I use every day, Textual, looks like this:
Me, asking a question in the #choo Freenode IRC channel
Let’s break down the data being displayed into questions about the protocol we can answer using RFC 1459.
- On the left, there is a list of servers I’m connected to. How do you connect to and communicate with a server?
- Under each server, there is a list of channels I’ve joined. How do you join a channel?
- On the right, there is a list of people in the focused channel, #choo. How do you know who is in a channel?
- In the middle, there is a log of messages coming from the channel. How do you know what is said in a channel?
- At the bottom, I can enter and send my own messages to the channel. How can you send your own message?
There’s a lot more that IRC clients can usually do, but this is a good amount of functionality for a first reading.
How do you connect to and communicate with a server?
Let’s see what the RFC has to say about how to the server as a client:
1.2 Clients
A client is anything connecting to a server that is not another server. Each client is distinguished from other clients by a unique nickname having a maximum length of nine (9) characters. See the protocol grammar rules for what may and may not be used in a nickname. In addition to the nickname, all servers must have the following information about all clients: the real name of the host that the client is running on, the username of the client on that host, and the server to which the client is connected..
So all we need to do is connect to the server, and boom, we’re a client… unless we’re another server. We’ll just have to make sure to not do server-y things while trying to do client-y things as we keep reading.
We also learn that clients have are identified by a unique nickname, and also have a real name and username. That’s a lot of names. Let’s keep that in mind as we continue.
Once we connect to a server, what do we send it, and what do we get back in return? If we keep reading, we’ll find the chapter on messages.
2.3 Messages
Servers and clients send eachother messages which may or may notgenerate a reply. If the message contains a valid command, asdescribed in later sections, the client should expect a reply asspecified but it is not advised to wait forever for the reply; clientto server and server to server communication is essentiallyasynchronous in nature.
Okay, this is telling us that not all messages will immediately receive replies. Maybe we can fire off messages and react to server replies separately. More importantly, what is a message?
Each IRC message may consist of up to three main parts: the prefix(optional), the command, and the command parameters (of which theremay be up to 15). The prefix, command, and all parameters areseparated by one (or more) ASCII space character(s) (0x20).
A message has three parts, separated by spaces. What are those parts?
The presence of a prefix is indicated with a single leading ASCIIcolon character (‘:’, 0x3b), which must be the first character of themessage itself. There must be no gap (whitespace) between the colonand the prefix. The prefix is used by servers to indicate the trueorigin of the message. If the prefix is missing from the message, itis assumed to have originated from the connection from which it wasreceived. Clients should not use prefix when sending a message fromthemselves; if they use a prefix, the only valid prefix is theregistered nickname associated with the client. If the sourceidentified by the prefix cannot be found from the server’s internaldatabase, or if the source is registered from a different link thanfrom which the message arrived, the server must ignore the messagesilently.
We’re getting a lot of information here, let’s list what we learned:
- Messages can start with a prefix
- The prefix starts with an ASCII colon (
:) and then some text without a space inbetween. It looks kinda like:abcd. - Clients should not use prefixes. Sweet, that’s easier for us! Remember, we want to do client-y things and not server-y things, and this is one of them.
The command must either be a valid IRC command or a three (3) digitnumber represented in ASCII text.
We don’t know what a valid IRC command is yet, but now we know it’s just text (or a 3 digit number in ASCII). Maybe it could be COOL or 123.
IRC messages are always lines of characters terminated with a CR-LF(Carriage Return — Line Feed) pair, and these messages shall notexceed 512 characters in length, counting all characters includingthe trailing CR-LF. Thus, there are 510 characters maximum allowedfor the command and its parameters. There is no provision forcontinuation message lines. See section 7 for more details aboutcurrent implementations.
CR-LF is the same as \r\n in JavaScript and most languages. Now we have an small idea of what the messages sent between clients and servers send look like. They might look something like these:
WHATSUP these are params\r\n
NOTMUCH i have params also\r\n
:somekindaprefix 123 i have a prefix now\r\n
If you’re reading along, you probably noticed that we’ve skipped a few paragraphs and details. That’s perfectly fine, we don’t need to have a complete understanding the first time through. You can always read it again later to fill in more gaps in knowledge. The important part is to realize that RFCs are written in plain English, and that they’re supposed to be easy to understand.
As soon as I say that, the next chapter has some weird looking notation called BNF.
2.3.1 Message format in ‘pseudo’ BNF
The protocol messages must be extracted from the contiguous stream ofoctets. The current solution is to designate two characters, CR andLF, as message separators. Empty messages are silently ignored,which permits use of the sequence CR-LF between messageswithout extra problems.
The extracted message is parsed into the components <prefix>, and list of parameters matched either by <middle> or <trailing> components.
The BNF representation for this is:
<message> ::= [':' <prefix> <SPACE> ] <command> <params> <crlf><prefix> ::= <servername> | <nick> [ '!' <user> ] [ '@' <host> ]<command> ::= <letter> { <letter> } | <number> <number> <number><SPACE> ::= ' ' { ' ' }<params> ::= <SPACE> [ ':' <trailing> | <middle> <params> ]
<middle> ::= <Any *non-empty* sequence of octets not including SPACE or NUL or CR or LF, the first of which may not be ':'>
<trailing> ::= <Any, possibly *empty*, sequence of octets not including NUL or CR or LF>
<crlf> ::= CR LF
If you haven’t gone through a university Programming Languages course, you might be thinking “aw jeez, what the hell?”. Don’t worry, this syntax is easy to understand once you know what it is. BNF stands for Backus-Naur form, and it just a way to describe a formal grammar, or a set of rules for a certain language. The protocol that IRC clients and servers speak are a kind of language, so this is a very short way to describe everything we read in the paragraphs before this.
BNF lists a bunch of grammar rules where each element is made up of other elements or text. You can read it like
<crlf> ::= CR LF
Which says “wherever you see <crlf> in the rules, that is a CR LF or \r\n. If you see something in square brackets ([]), that means it’s optional.
<message> ::= [':' <prefix> <SPACE> ] <command> <params> <crlf>
This says a message might start with colon, a <prefix>, and a <SPACE>, but it also might not. That’s just like what we learned in the description of the prefix earlier in the RFC!
A pipe symbol (|) separates multiple rules for the same symbol. You can kind of read it like an “or”.
<prefix> ::= <servername> | <nick> [ '!' <user> ] [ '@' <host> ]
A <prefix> is either a <servername> or it’s a <nick> with some optional other stuff.
Finally, curly brackets ({}) mean “zero or more” of what’s inside.
<SPACE> ::= ' ' { ' ' }
This means a <SPACE> symbol is at least one space, followed by zero or more extra spaces.
Even you still don’t fully get the BNF here, that’s okay. We already have a good enough understanding of what messages look like from all the stuff we read before it. Now that we know what a message looks like, we can read Chapter 4, which is basically a giant list with all the different kinds of messages we can use. Specifically, lets see what we need to send to register a connection.
4.1 Connection Registration
The commands described here are used to register a connection with an IRC server as either a user or a server as well as correctly disconnect.
A “PASS” command is not required for either client or server connection to be registered, but it must precede the server message or the latter of the NICK/USER combination. It is strongly recommended that all server connections have a password in order to give some level of security to the actual connections. The recommended order for a client to register is as follows:
1. Pass message
2. Nick message
3. User message
Ooh, we’re finally getting into real IRC commands! Since it says PASS commands are optional, we’ll skip that (but you can always read it yourself if you want to connect to a server with a password). Lets focus on the NICK and USER messages.
4.1.2 Nick message
Command: NICKParameters: <nickname> [ <hopcount> ]
NICK message is used to give user a nickname or change the previousone. The <hopcount> parameter is only used by servers to indicatehow far away a nick is from its home server. A local connection hasa hopcount of 0. If supplied by a client, it must be ignored.
…
Example:
NICK Wiz; Introducing new nick “Wiz”.
:WiZ NICK Kilroy; WiZ changed his nickname to Kilroy.
I’ve cut out most of the description, since it was information only needed for people implementing servers. For our client, the NICK command is what sets our nickname, one of the many names a client has that we read about before. The list of parameters is also given in BNF format, but it’s very simple here so it’s easy to understand. The <hopcount> isn’t used by clients, so all we have to send is NICK mycoolnickname.
4.1.3 User message
Command: USERParameters: <username> <hostname> <servername> <realname>
The USER message is used at the beginning of connection to specify the username, hostname, servername and realname of s new user. It is also used in communication between servers to indicate new user arriving on IRC, since only after both USER and NICK have been received from a client does a user become registered.
Between servers USER must to be prefixed with client’s NICKname. Note that hostname and servername are normally ignored by the IRC server when the USER command comes from a directly connected client (for security reasons), but they are used in server to server communication. This means that a NICK must always be sent to a remote server when a new user is being introduced to the rest of the network before the accompanying USER is sent.
It must be noted that realname parameter must be the last parameter, because it may contain space characters and must be prefixed with a colon (‘:’) to make sure this is recognised as such.
…
Examples:
USER guest tolmoon tolsun :Ronnie Reagan
Ah, there’s the other two names, username and real name, with a couple more bonus names! It does say the hostname and servername are ignored from clients, so we can put whatever we want in there. We also learn something new about the message format: the last parameter can have spaces if it is prefixed with a colon. If you look back at the BNF message format rules, you’ll see this is also described there.
Okay, phew. That was a lot of reading. But now we know enough to answer our first question: How do you connect to and communicate with a server? To register a client with a server, we connect to it and and it messages. The first messages we send are NICK and USER messages. They look like this:
NICK rahat\r\n
USER rahat_ahmed whatever whatever :Rahat Ahmed\r\n
That took a long time, but the next questions will be much faster to answer since we know what the message format already looks like.
How do you join a channel?
Now that we know how to connect, we have to find out how to join a channel so that we can chat in it. All the types of messages are listed in Chapter 4, and we can scan the table of contents to find something that sounds like what we want. The section “4.2.1 Join message” under “4.2 Channel operations” sounds promising.
4.2.1 Join message
Command: JOINParameters: <channel>{,<channel>} [<key>{,<key>}]
The JOIN command is used by client to start listening a specific channel…
Once a user has joined a channel, they receive notice about all commands their server receives which affect the channel. This includes MODE, KICK, PART, QUIT and of course PRIVMSG/NOTICE…
If a JOIN is successful, the user is then sent the channel’s topic (using RPL_TOPIC) and the list of users who are on the channel (using RPL_NAMREPLY), which must include the user joining.
Examples:
JOIN #foobar; join channel #foobar.
JOIN &foo fubar; join channel &foo using key “fubar”.
JOIN #foo,&bar fubar; join channel #foo using key “fubar” and &bar using no key.
That seems easy! To join a channel we just send JOIN #channelname and then we start receiving all the messages for that channel. And this also helps lead us to the answer for our next question:
How do you know who is in a channel?
The description for the JOIN command said that after we join a channel, we automatically get sent a RPL_NAMREPLY message with a list of users who are in the channel. If you search the RFC for “RPL_NAMREPLY” with Ctrl+F you’ll find it in Chapter 6 on Replies.
6. REPLIES
The following is a list of numeric replies which are generated in response to the commands given above. Each numeric is given with its number, name and reply string.
…
353 RPL_NAMREPLY
<channel> :[[@|+]<nick> [[@|+]<nick> [...]]]
It’s not strictly BNF but we can tell the 353 reply will have a channel argument followed by a final parameter of space separated nicknames, which is exactly what we want! The optional@’s and +'s in front of nicknames are described in another part of the RFC, and they’re not very important right now. I’ll leave it up to you as an exercise to find out what they mean.
How do you know what is said in a channel?
The last set of information we want to receive from the server are the message sent in a specific channel. It’s probably a good idea to check the list of commands listed in the JOIN command that are automatically received once we join a channel. The one we want is PRIVMSG:
4.4.1 Private messages
Command: PRIVMSGParameters: <receiver>{,<receiver>} <text to be sent>
PRIVMSG is used to send private messages between users. <receiver> is the nickname of the receiver of the message. <receiver> can also be a list of names or channels separated with commas.
Examples:
:Angel PRIVMSG Wiz :Hello are you receiving this message ?; Message from Angel to Wiz.
PRIVMSG Angel :yes I'm receiving it!; Message to Angel.
Perfect. To check who sent incoming PRIVMSGs, we read the prefix of the message. To see who or where the message was sent to, we check the first parameter. The examples show direct messages between users, but the description says it can also be a channel name.
How can you send your own message?
This one is a freebie. As described in the PRIVMSG section, we send our own PRIVMSG to the server, with the destination and message as parameters.
Okay, let’s make a client!
Enough reading. Now we have enough of an understanding of the basics of the protocol, we can start our implementation.
First, we need to connect to the server. Before we do that, we need a server to connect to. I’ll use the popular Freenode (where the official #node.js channel is hosted). Since we’re using node.js, we can use the net module to open a connection.
var net = require('net')
var client = net.connect({host: 'irc.freenode.net',port: '6667'})
Now we want to see what the server is sending the client, so lets pipe the connection into stdout so it prints to the screen.
client.pipe(process.stdout)
Finally, we want to be able to send commands to the server. The easiest way to do this is to pipe stdin to the connection, with one caveat. Terminal inputs use \n for new lines, while IRC messages are separated by \r\n. No problem, we’ll just grab a stream replace package from npm with npm install stream-replace.
var replace = require('stream-replace')
process.stdin.pipe(replace('\n', '\r\n')).pipe(client)
Okay, all together that looks like this.
var net = require('net')var replace = require('stream-replace')
var client = net.connect({host: 'irc.freenode.net',port: '6667'})
client.pipe(process.stdout)process.stdin.pipe(replace('\n', '\r\n')).pipe(client)
That’s it. We’re done!
What?
You’re probably thinking that I made you read this whole article for nothing right about now. Don’t close that tab, hear me out.
The point of going through all this wasn’t to show you how to write an IRC client. The real purpose was to show that anyone can read an RFC, and hopefully by now they should seem a little less intimidating than before. That program up there isn’t the real client, you see. The IRC client was inside you the whole time! The code is only there so you yourself can communicate with the server, in the language you just learned. Lets try it out, shall we?
Save that script to a file and run it with node script.js. You should immediately see some text on the screen coming from the server.
:sinisalo.freenode.net NOTICE * :*** Looking up your hostname...:sinisalo.freenode.net NOTICE * :*** Checking Ident:sinisalo.freenode.net NOTICE * :*** Found your hostname
It’s a bunch of NOTICE messages. We haven’t seen that before, but what you don’t know can’t hurt you, and you can always read about it later. Let’s stick to what we know.
If you leave the script running for a while, eventually you’ll get this error:
ERROR :Closing Link: 127.0.0.1 (Connection timed out)
This means we took too long doing nothing, and the server closed the connection. We can try again, but this time we’ll try registering a connection like we read about in Chapter 4. The commands we need to send are NICK and USER. The parameters for these commands set your nickname, username, and real name so pick whatever you like (feel free to scroll back up for a refresher on which param is which).
NICK rahatUSER rahat_ahmed these_params dont_matter :Rahat Ahmed
Hit enter after each of the above lines, and you’ll receive a flood of data from the server. Put on your indoor hacker sunglasses, because we’re in.
:verne.freenode.net 001 rahat2 :Welcome to the freenode Internet Relay Chat Network rahat2:verne.freenode.net 002 rahat2 :Your host is verne.freenode.net[185.30.166.37/6667], running version ircd-seven-1.1.4:verne.freenode.net 003 rahat2 :This server was created Thu Sep 22 2016 at 20:50:37 UTC:verne.freenode.net 004 rahat2 verne.freenode.net ircd-seven-1.1.4 DOQRSZaghilopswz CFILMPQSbcefgijklmnopqrstvz bkloveqjfI:verne.freenode.net 005 rahat2 CHANTYPES=# EXCEPTS INVEX CHANMODES=eIbq,k,flj,CFLMPQScgimnprstz CHANLIMIT=#:120 PREFIX=(ov)@+ MAXLIST=bqeI:100 MODES=4 NETWORK=freenode KNOCK STATUSMSG=@+ CALLERID=g :are supported by this server:verne.freenode.net 005 rahat2 CASEMAPPING=rfc1459 CHARSET=ascii NICKLEN=16 CHANNELLEN=50 TOPICLEN=390 ETRACE CPRIVMSG CNOTICE DEAF=D MONITOR=100 FNC TARGMAX=NAMES:1,LIST:1,KICK:1,WHOIS:1,PRIVMSG:4,NOTICE:4,ACCEPT:,MONITOR: :are supported by this server:verne.freenode.net 005 rahat2 EXTBAN=$,ajrxz WHOX CLIENTVER=3.0 SAFELIST ELIST=CTU :are supported by this server:verne.freenode.net 251 rahat2 :There are 144 users and 82040 invisible on 29 servers:verne.freenode.net 252 rahat2 33 :IRC Operators online:verne.freenode.net 253 rahat2 19 :unknown connection(s):verne.freenode.net 254 rahat2 53193 :channels formed:verne.freenode.net 255 rahat2 :I have 4729 clients and 2 servers:verne.freenode.net 265 rahat2 4729 5738 :Current local users 4729, max 5738:verne.freenode.net 266 rahat2 82184 92341 :Current global users 82184, max 92341:verne.freenode.net 250 rahat2 :Highest connection count: 5740 (5738 clients) (579387 connections received)
Remembering what we read about the message format, these seem to be a bunch of numeric server replies with miscellaneous server data. Some of these are easy to read, others are a bit tougher, but nothing is beyond a quick search of the list of replies in Chapter 6 of the RFC for their true meaning. Now that we’re in, we can try joining a channel and chatting in it. Lets try joining #node.js
JOIN #node.js
(If you don’t immediately get a response, press Enter a bunch of times. Node will probably buffer your input before sending it). You should again receive some data, this time we get a whole lot of 353 (RPL_NAMREPLY) replies containing a bunch of nicknames who are in the channel and a confirmation JOIN of us entering #node.js. If anyone speaks in the channel you’ll see a PRIVMSG from them. You can try sending a message to the channel, but you’ll get a 404 reply because #node.js has restricted sending messages only for people who have registered their nickname with Freenode. You can always join another channel or join your own channel and play with it in there, just make sure you’re not bothering any other users. A cool thing to do is use another IRC client to join the same channel and chat with yourself. You can see the raw protocol messages for everything you do!
It goes without saying that there’s a whole lot more that you can do on an IRC server, and there’s more that you can read about and try to understand in RFC 1459 as well. I hope this quick dive into the inner workings of IRC gives you the confidence to keep reading. Even if you aren’t particularly interested in IRC, if there is other protocol or system you wanted to understand, seek out the RFC for that!
Happy reading!
Hacker Noon is how hackers start their afternoons. We’re a part of the @AMIfamily. We are now accepting submissions and happy to discuss advertising &sponsorship opportunities.
To learn more, read our about page, like/message us on Facebook, or simply, tweet/DM @HackerNoon.
If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!