This article is part 6 of the Blockchain train journal, start reading here: Catching the Blockchain Train.
IPFS human-friendly naming; is it possible?
Did you see the traffic-magnet-title I came up with? It is a bit over the top; we’re just looking at how the addressing of content can be made a bit easier than ipfs cat /ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV.
As mentioned in the previous post, adding some mutable “label” to point to (immutable) IPFS content is sometimes desired.
An example is the content of this blog: the original address (hash) for the home page /ipfs/QmcPx9ZQboyHw8T7Afe4DbWFcJYocef5Pe4H3u7eK1osnQ/ is different from the current (ah no, already expired!) home page /ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV/.
Mutable links are essential, but not enough for us humans. They are hard to read, let alone remember.
What we’ll do in this episode is to explore different ways to:
- resolve this problem of adding mutable links, and
- to make the links more human-friendly, while
- trying to stay decentralized where possible.
We’ll see the following methods in action:
- The IPFS gateway
- IPNS
- DNS TXT IPNS Records
- Browser extensions
- Namecoin
- Blockstack ID
- Ethereum Name Service
- Proquint Pronounceable Identifiers
- Name shorteners
- Filecoin
TL;DR To make IPFS more human-friendly - in the context of a decentralized IPFS powered website - there is currently no other way than sacrificing a part of decentralization using HTTP-to-IPFS gateways and old school DNS. Naming systems in the blockchain sphere (Blockstack, Namecoin, EthNames) are possible candidates to improve the UX while keeping it 100% decentralized, but at this point of time none of these work out of the box. Filecoin might be the way to go, but that is still far away.
The IPFS gateway
The most obvious step to make content available on the old skool internet with a regular browser is to make use of the IPFS gateway. These gateways speak HTTP and access content through their IPFS node.
Any gateway will do, but the gateway where content is pinned is of course faster. Examples of accessing the previous (before this post) home page of this blog:
- http://decentralized.blog:8080/ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV/
- https://gateway.ipfs.io/ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV/
- http://localhost:8080/ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV/ (if you have a local IPFS node running)
So while that works, it has a couple of problems:
- when the content of the home page changes these URL’s still point to old content
- the hash is not very human-friendly (a.k.a. ugly)
- it introduced an element of centralization (sort of, there is no dependence on a particular single gateway)
IPNS
To solve the first of these three problems we can, of course, use IPNS. Any time the content changes we publish the latest content hash like:
ipfs name publish QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeVPublished to QmRf4ERGvYpVo6HRa2VueZT8pWi8YvyLS3rW6ad2y83tdN: /ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV
Now we can access the home page through any gateway but with the mutable IPNS link: http://decentralized.blog:8080/ipns/QmRf4ERGvYpVo6HRa2VueZT8pWi8YvyLS3rW6ad2y83tdN/
Much better, but still ugly…
Note: IPNS is still a bit shaky and forgets published names after about 12 hours. You might want to run a cron job to republish every 8 hours or so.
DNS TXT IPNS Records
Now, let’s get rid of the hash. The white paper mentions the use of a DNS TXT record. This is what Wikipedia has to say about it:
A TXT record (short for text record) is a type of resource record in the Domain Name System (DNS) used to provide the ability to associate some arbitrary and unformatted text with a host or other name, such as human readable information about a server, network, data center, and other accounting information.
So the owner of a domain can associate any data to this domain by adding a TXT record. It is like a key-value store where only the owner of the domain can write, and the world can read.
This is used by IPNS to look up the /ipns/<peerID> path if you insert a valid domain as the path.
This is how IPNS normally resolves:
# Publishipfs name publish QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeVPublished to QmRf4ERGvYpVo6HRa2VueZT8pWi8YvyLS3rW6ad2y83tdN: /ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV
# Normally IPFS resolves the peerID hash$ ipfs name resolve /ipns/QmRf4ERGvYpVo6HRa2VueZT8pWi8YvyLS3rW6ad2y83tdN/ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV
To be able to resolve a domain we need to add a dnslink to the TXT record for that domain.
After doing so for the domain name of this blog (decentralized.blog) pointing to the /ipfs link
$ dig txt decentralized.blog..;; ANSWER SECTION:decentralized.blog. 900 IN TXT "dnslink=/ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV"
we can resolve by domain name
$ ipfs name resolve decentralized.blog/ipfs/QmcJTRZuGVdqUoNS1414G2mim3mt39RU14JsTmbh4KJYeV
which allows us to use a much nicer address:
This works, but every time I change the content, I have to change the DNS TXT record and wait for propagation. The cool thing is that I can make the DNS TXT record point to a /ipns link, and simply do a ipfs name publish when content has changed:
$ dig TXT decentralized.blog..;; ANSWER SECTION:decentralized.blog. 900 IN TXT "dnslink=/ipns/QmRf4ERGvYpVo6HRa2VueZT8pWi8YvyLS3rW6ad2y83tdN"
# points to /ipns path now
Note, I should have done this in the first place, but I was led on the wrong track by this example in the white paper:
# this DNS TXT record ipfs.benet.ai. TXT “ipfs=XLF2ipQ4jD3U …”
# behaves as symlink ln -s /ipns/XLF2ipQ4jD3U /ipns/fs.benet.ai
That is probably a typo, although the ipfs.io domain also points to an immutable /ipfs path for some reason:
$ dig TXT ipfs.io..;; ANSWER SECTION:ipfs.io. 60 IN TXT "dnslink=/ipfs/QmPCawMTd7csXKf7QVr2B1QRDZxdPeWxtE4EpkDRYtJWty"
More about how IPFS resolves paths here.
So now we have a human-friendly URL but at the cost of adding more centralization (the DNS).
Browser extensions
I was hoping there was a browser extension that allowed me to browse a web of /ipfs and /ipns paths, but the extensions that do exist are not made for that purpose.
Because it is still interesting, I’ll mention it anyway. The most up to date extension is ipfs-companion. There are browser extensions available for Firefox and Chrome.
One of the cool things it does is helping to distribute data by redirecting requests to the public gateway to your local node.
Another exciting project is the Beaker browser that used to support IPFS, but not anymore. It is a browser for distributed content based on Dat.
Namecoin
What is Namecoin?
Namecoin is a very interesting protocol based on blockchain technology. I won’t go into it here in much depth, because I want to start exploring the mother of all blockchains first (i.e., Bitcoin, but you knew that).
I just realized, this is episode 6 of Catching the Blockchain Train, and we haven’t seen a single blockchain yet! And worst, we are not going into that here either… Please bear with me, and we will get there eventually.
So, Namecoin. This is what they say about themselves on the website:
Namecoin is an experimental open-source technology which improves decentralization, security, censorship resistance, privacy, and speed of certain components of the Internet infrastructure such as DNS and identities.
(For the technically minded, Namecoin is a key/value pair registration and transfer system based on the Bitcoin technology.)
Bitcoin frees money — Namecoin frees DNS, identities, and other technologies.
The part about DNS might be able to help us here. Namecoin facilitates the .bit TLD, which has its own portal.
Getting a .bit domain
To get started we need to get hold of a .bit name. I rented one a while ago at an exchange, but I now see that there is a better way: using a Namecoin client. If you go the exchange way, realize that you don’t have the private keys to control the domain (change value in the KV pair), so you don’t own it.
To get your own .bit domain name, follow the steps at Register and Configure .bit Domains.
You’ll see that you need a bit of namecoin (NMC) to get your .bit domain, but mining is not really an option (it is done alongside Bitcoin mining and hard), but you can also buy some. Here is a list of exchanges where you can buy NMC, Namecoin Markets, but I’d recommend ShapeShift if you already own some bitcoin or altcoin. You need 0.02 NMC which is about nothing. You can use this namecoin explorer to see if your desired name is still available.
How a .bit lookup works
.bit domains have a unix-path equivalent like this: d/somedotbitname. These paths are the key of a record stored in the Namecoin's blockchain. So to see what the value is you have to have access to a Namecoin node, and ask for the value of the record with key d/somedotbitname.
That value can be anything, but in practice, it is either a reference to nameservers (standard DNS) or directly to an IP address, or to a Bitmessage address. Others add their email and say the .bit address is for sale.
We need access to a namecoin node to query it, but there are online REST API’s (introducing a central point of failure) we can use, like webbtc:
# Bitmessage address value$ curl http://namecoin.webbtc.com/name/d/mysite.json{"name": "d/mysite","value": "BM-2cUyrUNq91XqdPSKvcHSytuED9nnTazf5r","txid": "78e212c47b41b3ae80413a7064c7bee044f8f2bc27362e9c37d5474b68bfd9e3","address": "NFE3ED8C3BGRXuzjBRD3YGJV6jrmbS5h3Q","expires_in": 23292}
# IP address value$ curl http://namecoin.webbtc.com/name/d/dappersoftware.json{"name": "d/dappersoftware","value": "{\"ip\":\"207.111.216.146\",\"map\":{\"*\":{\"ip\":\"207.111.216.146\"}}}","txid": "da72c6b0cf85c94d7b2d4f2f8534d1cdafa47ca94fd00489bf205f00c3fc1cb2","address": "N2hrunEcP3PNxdKvihMKCoEnvLUcCcBhKw","expires_in": 4344}
How browsing .bit websites works
Obviously the normal DNS system doesn’t know what to do with a .bit domain, so in a browser that won't resolve to anything. The Namecoin team provides a layer on top op the namecoin node to be able to browse .bit domains: ncdns. I'll leave it at that since we are diverging from our goal...
How do we resolve a .bit domain to an IPFS hash?
What are we trying to do again? So much exciting stuff is going on that we almost got lost in the shapeshiftbitmessagewebbtcncdnschain. Focus!
The goal of this exploration is to be able to pass around a human-friendly address like dweb.bit to bring up an IPFS-powered website. That would be possible if the IPNS system knew how to read the value of dweb.bit in the Namecoin blockchain and interpret it and resolve it to an IPFS address.
It’s just like we saw with the DNS TXT record trick, but now the IPFS node would need to have access to the Namecoin blockchain (or a centralized public API like webbtc we saw before).
So: /ipns/dweb.bit needs to resolve to /ipfs/<some multihash>. The value field of a .bit record can be anything, so that is easy to solve. The other part would be that the IPNS resolver implementation needs to recognize .bit domains and access the Namecoin network somehow. This was not implemented as can be easily shown:
$ ipfs name resolve dweb.bitError: Could not resolve name.
The IPFS-Namecoin integration is discussed here. Personally, I would start by providing a central .bit lookup service on ipfs.io, and extend that later with a Namecoin resolver implementation in the ipfs code. It doesn’t look like it has priority now in the IPFS project though.
In any case, I’m ready: dweb.bit points to { "ipns": "/ipns/QmRf4ERGvYpVo6HRa2VueZT8pWi8YvyLS3rW6ad2y83tdN" }.
Blockstack ID
Another contestant for the naming layer that is mentioned in Juan Benet’s presentations is Blockstack.
What is Blockstack?
This is what Wikipedia has to say about it:
Blockstack is the first implementation of a decentralized DNS system on top of the Bitcoin blockchain. It combines DNS functionality with public key infrastructure and is primarily meant to be used by new blockchain applications.
That sounds very similar to what Namecoin does, but it uses the Bitcoin blockchain instead of its own fork. In the past, it used the Namecoin blockchain, but the idea is that it should be blockchain agnostic.
On the homepage of Blockstack a bigger goal is stated:
A New Internet for Decentralized Apps
Blockstack is a new decentralized internet where users own their data and apps run locally. A browser portal is all that’s needed to get started.
But for now, let’s limit our focus to the decentralized DNS system.
How to register a Blockstack ID
The naming system is called Blockstack ID, and the TLD is .id. You can get your own .id at onename or use the terminal as described here: Blockstack CLI.
Let’s try that:
# First install blockstack with pip (Python2 only, booh!)
# Create a wallet; not mentioned in the documentation, but required to do first$ blockstack setup
# Check price for the desired name, the shorter the name, the more expensive$ blockstack price dweb.id{"name_price": {"btc": 0.016,"satoshis": 1600000},"preorder_tx_fee": {"btc": 0.00219904,"satoshis": 219904},"register_tx_fee": {"btc": 0.0020583,"satoshis": 205830},"total_estimated_cost": {"btc": 0.02213352,"satoshis": 2213352},"total_tx_fees": 613352,"update_tx_fee": {"btc": 0.00187618,"satoshis": 187618}}
# Get a bitcoin address to pay the fee to$ blockstack deposit{"address": "3BmAHjCLgELuRdg3jM1MMgm5twXPWWxr7s","message": "Send bitcoins to the address specified."}
# Start the API server; also not documented, but the error on register is helpful$ blockstack api start{"status": true}
# Now register a name$ blockstack register dweb.idCalculating total registration costs for dweb.id...Registering dweb.id will cost about 0.02213352 BTC.Use `blockstack price dweb.id` for a cost breakdown
The entire process takes 48 confirmations, or about 5 hours.You need to have Internet access during this time period, sothis program can send the right transactions at the righttimes.
Continue? (y/N): y{"message": "Name queued for registration. The process takes several hours. You can check the status with `blockstack info`.","success": true,"transaction_hash": "c4fc237f4b13d9925f8180fbe9e131cd0a1ee54d9b8c7455c4b2e2464fbe1315"}
# Check the status$ blockstack info{"cli_version": "0.14.4.2","consensus_hash": "47f015a3959a80e7ce9c5fb425833860","last_block_processed": 482064,"last_block_seen": 482070,"queues": {"preorder": [{"confirmations": 0,"name": "dweb.id","tx_hash": "c4fc237f4b13d9925f8180fbe9e131cd0a1ee54d9b8c7455c4b2e2464fbe1315"}]},"server_alive": true,"server_host": "node.blockstack.org","server_port": 6264,"server_version": "0.14.4.0"}
This is a Bitcoin transaction which can be inspected here c4fc237f4b13d9925f8180fbe9e131cd0a1ee54d9b8c7455c4b2e2464fbe1315.
About 5 hours later the registration was completed:
# Check if we own the dweb.id name now$ blockstack names{"addresses": [{"address": "3PR8Js7pLt5Bagk5MZHffXoR43EpQrJHxT","names_owned": ["pors.id","dweb.id"]}],"names_owned": ["error"]}# Hmm, sort of I guess :)
All in all, it seems more like an identity thing, but the .id domains can also be used as a DNS record (BNS in this case). Blockstack also introduced namespaces, the bit after the dot (now id), which is obviously expensive.
How a .id lookup works
There is a command for lookups in the client:
$ blockstack lookup timblee.id{"profile": {"@type": "Person","account": [{"@type": "Account","identifier": "timbl","proofType": "http","proofUrl": "https://gist.github.com/timbl/04e8ac7c81cd2dee2f51a5e8c672188d","service": "github"},{"@type": "Account","identifier": "timberners_lee","proofType": "http","proofUrl": "https://twitter.com/timberners_lee/status/740677355950080001","service": "twitter"}],"image": [{"@type": "ImageObject","contentUrl": "https://s3.amazonaws.com/97p/lUU.jpeg","name": "cover"}]},"zonefile": "$ORIGIN timblee.id\n$TTL 3600\n_http._tcp URI 10 1 \"https://blockstack.s3.amazonaws.com/timblee.id\"\n"}
The zonefile bit is the interesting part in our use case; we can store routing information in there.
Also, there is an API (it seems a bit shaky, but it works for some accounts): https://core.blockstack.org/v2/users/werner. For pors.id the API doesn’t work, but the blockstack explorer does return data for this account.
So how can we use all this?
How do we resolve a .id domain to an IPFS hash?
As said, the zone file entry seems to be the best spot to store routing information. This can be done with the CLI client with the blockstack update command. But to what value do we need to set it? That all depends on what the Blockstack and IPFS teams have agreed on.
A bit of Googling and digging in the Blockstack and IPFS forums didn’t give me any answers and my hope was set on this duo-presentation Muneeb Ali & Juan Benet: Blockstack IPFS “CTO Briefing” at CONSTRUCT 2017, but not a word about integration there (at least not the addressing bit).
There is some work about the integration between Blockstack and IPFS as a data storage layer. But that’s not what we are looking for here.
So just as with Namecoin, there is no way to use Blockstack as the name layer for IPFS.
But again, I want to be ready when they are so I updated my zonefile:
$ echo '{"ipns":"ipns/QmRf4ERGvYpVo6HRa2VueZT8pWi8YvyLS3rW6ad2y83tdN"}' > new_zone_file.txt$ blockstack update dweb.id new_zone_file.txt{"message": "Name queued for update. The process takes ~1 hour. You can check the status with `blockstack info`.","success": true,"transaction_hash": "026c739761ec6ed60119b4d80da2ccf81f274d558f53f377d6862a6020d767b8","zonefile_hash": "4db474dd1dda7502c6152d248b7a24302f6e104a"}
See the resulting Blockstackified zone file here.
I will revisit Blockstack in depth in a future post, as soon as we got a better understanding of blockchain technology in general.
Ethereum Name Service
We start to see a trend here: very cool blockchain powered name systems, but no support in IPFS. Let’s see how ENS fares.
What is ENS?
From the documentation:
ENS is the Ethereum Name Service, a distributed, open, and extensible naming system based on the Ethereum blockchain.
ENS can be used to resolve a wide variety of resources. The initial standard for ENS defines resolution for Ethereum addresses, but the system is extensible by design, allowing more resource types to be resolved in future without the core components of ENS requiring upgrades.
We haven’t looked at Ethereum yet, but that will, of course, be the case in an upcoming post (multiple posts most likely). It is next to Bitcoin the most important blockchain around.
In short: Ethereum is building a decentralized virtual machine where you can run applications. From the Ethereum website:
Ethereum is a decentralized platform that runs smart contracts: applications that run exactly as programmed without any possibility of downtime, censorship, fraud or third party interference.
These apps run on a custom built blockchain, an enormously powerful shared global infrastructure that can move value around and represent the ownership of property. This enables developers to create markets, store registries of debts or promises, move funds in accordance with instructions given long in the past (like a will or a futures contract) and many other things that have not been invented yet, all without a middle man or counterparty risk.
As said, more on that later, for now we just look at the naming system.
To understand the ENS basics, I recommend to:
- read ENS documentation introduction and
- watch ENS Ethereum Domain Name System, slides here.
The currently available TLD is .eth. Let's try to get one...
How to register a .eth name
The registration for a .eth name is set up as an auction. The details are not really relevant, but you can read more about it here: registrar.ens.domains.
This is also the officail app, but it didn’t work for me for some reason. This one works much nicer: myetherwallet. It operates in combination with an Ethereum browser or this browser extension: metamask.
You need some Ether (the Ethereum currency) and a bit patience, and you will be the owner of a .eth address.
Progress can be followed here etherscan.io.
How a .eth lookup works
There is a javascript library that allows you to hook into the ENS system. This code needs to run on the Ethereum virtual machine. Currently, there seems no other way to interact with it.
How do we resolve a .eth domain to an IPFS hash?
Well, we don’t. Although IPFS is mentioned once in the ENS documentation (in the intro) there is no code that makes integration possible.
Intermission: now what?
As we have seen so far, there is no (easy) way to use blockchain-based names to address IPFS content. We want to be able to pass around .eth, .id, and .bit addresses and know it will resolve in a trusted way to the correct piece of content. I.e., the content that the owner of such an address wants it to point to.
One way forward that will certainly happen for Ethereum and Blockstack is that IPFS will be used as one of the supported file systems. In that case, the user lives in the Ethereum or Blockstack world and will be able to use .eth or .id addresses.
The alternative, where IPFS itself resolves these addresses is harder because IPFS nodes need trusted access to nodes of the other network. For example, this proof of concept relies on having access to a Namecoin node on the same machine where the IPFS node runs. This is obviously not feasible (who wants to run a node for every naming system that comes out next to its IPFS node?).
A proposal to solve this that I like is pluggable ipns resolvers. I have added my own 2 cents to it, let’s see where it leads. But till there is a solution, we still have a couple of contestants, starting with…
Proquint Pronounceable Identifiers
Another round another chance!
From the white paper:
There have always been schemes to encode binary into pronounceable words. IPNS supports Proquint. Thus:
# this proquint phrase /ipns/dahih-dolij-sozuk-vosah-luvar-fuluh
# will resolve to corresponding /ipns/KhAwNprxYVxKqpDZ
I’m not sure if this is an improvement, especially because the definition of “pronounceable” is stretched a bit here.
As an aside, this remembers me of my first open source contribution Sort-of-pronounceable password generator back in 2000. There was no github yet, so the source code is gone, but someone put it in a pastebin 11 years later (I found it when I Googled for my own name, ahem). To make sure this historic piece of code never gets lost I added it to IPFS.
Back to Proquint! This is actually implemented in IPFS. So let’s give it a try:
$ ipfs name resolve -r /ipns/dahih-dolij-sozuk-vosah-luvar-fuluhEU�o�C.��M�# yuk
So yeah that works, but /ipns/KhAwNprxYVxKqpDZ is not something valid.
You can replay it with these online tools:
To encode a valid IPNS peerId is a bit more work to figure out. I’m not interested in that now and also lazy, so I’ll leave that to anyone who likes to dive into that :)
Another solution mentioned in the white paper is…
Name shorteners
From the white paper:
Services are bound to spring up that will provide name shortening as a service, offering up their namespaces to users. This is similar to what we see today with DNS and Web URLs:
# User can get a link from /ipns/shorten.er/foobar
# To her own namespace /ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm
So this is based on the ipns feature it can resolve domains via a DNS TXT record.
Resolving goes like this:
- A request for
/ipns/shorten.er/foobaris resolved to/ipns/<hash-of-shortener-service-provider>/foobar. - This file
/ipns/<hash-of-shortener-service-provider>/foobarpoints to the peerID address offoobar, like:/ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm
Now, this pointing is not as easy as it sounds. This was probably the intended use of what is described in the white paper as Peer Links:
As encouraged by SFS, users can link other users’ Objects directly into their own Objects (namespace, home, etc). This has the benefit of also creating a web of trust (and supports the old Certificate Authority model):
# Alice links to bob Bob ipfs link /<alice-pk-hash>/friends/bob /<bob-pk-hash>
# Eve links to Alice ipfs link /<eve-pk-hash/friends/alice /<alice-pk-hash>
# Eve also has access to Bob /<eve-pk-hash/friends/alice/friends/bob
# access Verisign certified domains /<verisign-pk-hash>/foo.com
Unfortunately, the ipfs link command is not implemented yet which probably explains why no-one implemented an IPFS name shortener.
There is a way to do something similar to peer links with ipfs files, but it is not suitable to build a name resolver. See here for my search for a solution: Are Peer Links supported?.
Again no success here, but there might be hope for the future…
Filecoin
As mentioned in a previous post, Filecoin will be the cryptocurrency to help incentivize the file storage marketplace. This means that there will be a blockchain available in the IPFS network. The primary function of this blockchain is to pay and get paid for storage, but since it’s there, it can also be used for other functionality. Like identity or a name system.
In other words, just like Namecoin, Blockstack and ENS provide blockchain enabled identity/naming, Filecoin can do the same. We’ll have to see how this all plays out in the next months (years?) but this seems like an attractive option since the IPFS nodes and the Filecoin nodes will most likely be one and the same or at least tightly integrated.
The Filecoin white paper doesn’t mention anything about naming, but it is still very early to say anything definite about it.
This is an fascinating introduction Filecoin | White Paper Breakdown and Token Sale Analysis although he gets into a hate speech for a couple of minutes about the ICO (which I think is not completely undeserved).
OK, more than enough about friendly naming I’d say! Time to move on to the next step in our journey to decentralizing this blog (more soon in the next episode…).
Originally published at decentralized.blog.