I love the open source community. The node.js ecosystem is a fantastic place and you can find packages for literally everything on npm. The problem is, you can find packages for literally everything, and sometimes more than one package that accomplishes the same thing.
From the most abstract standpoint, this is one of the best things about open source code. Anyone can publish anything, and if you don’t like a package or you want something else, you can fork it and modify it however you want to suit your own purposes. This has proven useful to me in the past. In a previous post, I talked about how the existing gulp-pdflatex package didn’t satisfy my needs, so I wrote my own version and released it as gulp-pdflatex2. This is great, but it fractures the ecosystem. If you search up gulp pdflatex on Google, you’ll find three packages that match.
For beginners, this can be very annoying since the best solution is unclear. In addition to the packages having slightly different usages, each package may or may not also have bugs or be in a broken/outdated state.
This week, I was working on one of my open source projects (getnews.tech) and I needed a node.js library to handle geolocation by IP address, commonly referred to as geoip.
Searching ‘node geoip’ on Google
Crap.
Let’s go through them:
- geoip: A thin wrapper around libgeoip C library that — by the owner’s admission — has memory leaks and should not be chosen over node-geoip.
- geoip-native: A native JavaScript geoip library that uses the free MaxMind databases. I used this for a while until I realized that the MaxMind database provided by the package was outdated.
- node-geoip: A native JavaScript library that uses a prepackaged CSV file to lookup IP addresses, facing the same problem as geoip-native.
- geoip-lite-country: “Stripped down node-geoip-lite”. Oh now there’s also node-geoip-lite?
- node-service-geoip: “a RabbitMQ microservice wrapping the node-geoip package (geoip-lite on npm).” There’s a geoip-lite package as well?
Here are at least 7 packages that handle geolocation, and a quick search on the npm package directory shows a handful more.
Packages with the keyword ‘geoip’ on npmjs.
Did I say handful? I meant a FUCKLOAD more. This proved to be a massive headache for me. Not all the packages had good documentation on the output either, so instead of immediately ruling one out, I had to install it, try it out, and then delete it if it didn’t suit my purposes.
Since I wanted to be able to determine a country and a timezone from an IP address, I eventually settled for the maxmind package so that I could provide my own MaxMind binary database and update it independently from the package. (This was for detecting a user’s timezone for getnews.tech)
This is just a small illustration of a problem that plagues the ecosystem. It’s not just limited to npm and node.js, a quick search for “pip geoip” also shows a handful of different packages for Python (although the problem isn’t as bad as npm). While npm does contain a large repository of packages to do anything you could possibly imagine, sorting through them can be a mess.
I don’t know of a good way to solve this problem, but perhaps showing this to the community would allow us to discuss a way to constructively approach this. Should we encourage developers to unpublish unused/broken packages that they no longer maintain? At the very least, we should encourage them to clearly mark outdated or deprecated packages so that they are ranked lower on search results. npm already allows ranking packages by popularity and quality when searching, but this didn’t help me too much in my quest for a suitable geoip package.
If you like hearing me complain or talk about my code, please consider following me on Twitter. Thanks for reading!