I Wrote a Go Script to Generate Gists, Here’s What I Learned

That’s not a gopher, its a badger. Photo by Hans Veth on Unsplash

There is a lot of busywork that comes with writing blogs and posting them to different platforms.

For example, I usually start writing an article by creating a markdown file within my code editor. As I write the post, I insert code snippets and images to make the content a little more digestible.

However, problems arise when I need to post this same markdown to Medium. Although their text editor is pretty, it is not perfectly safe to copy and paste my markdown files there.

Here are some issues:

• code words such as fields or someFunction wrapped in backticks are not formatted properly on paste
• larger code blocks do not currently support syntax highlighting

function doSomething() {return function(name) {return 'Hello ' + name}}

• words that are bold or italicized maintain their styling, but are not separated from the *s that wrap them in my markdown file

And here is a screenshot of those 3 bullet points pasted:

Call me lazy, but after writing a 2500+ word post, I must:

• go through each line and make sure my grammar makes sense.
• assure my code examples work since some of my articles are tutorials.
• make my code look pretty through Github Gists, thus bringing along a whole new issue of tab sizing.
• clean up every markdown-style hyperlink and remove excess asterisks or backticks from highlighted words and phrases.

This becomes draining. It becomes a lot of work to try to produce content that can be cross-posted to multiple sites. It event takes a lot of work to post content on one site.

Why don’t I just cave in and post exclusively to dev.to or my own site where I literally just need to paste my markdown file?

Because that’s not my nindō. I must become omnipresent throughout the internet.

Cleaning Code Snippets

Let’s start by grabbing the lowest hanging fruit, that is, clean up those code snippets. How can we do this?

Well, the brute force way would be to copy and paste as normal, remove the backticks denoting code blocks, press “CMD+option+6” to create a Medium Codeblock™, and then struggle with the awkward newlines and tabs within the codeblock. Don’t forget, tabs in the Medium editor don’t work like they do in a code editor.

Nope. I’m not gonna do that again.

The next option is to manually create Github gists, which delegates the code formatting to Github. We even get syntax highlighting out of it.

However, in my most recent toils, the tab size for gists seems to default to 8. Here’s an example I just pasted into a gist, specifically setting spaces of size 2 in the editor.

It seems that the only way to fix this issue is by replacing all instances of tabs with spaces (2 spaces, in my case). In this example, I could do this manually. Not so bad if there’s only one snippet to clean up.

The drawback is that every tab must be replaced by spaces. My previous article had 18 JavaScript code snippets. It was a miserable editing experience.

Introducing the Gist Generator

*I hope I come up with a cooler name sometime

I decided to take advantage of my dev chops, utilizing the Github API in order to create gists. This script, written in Go, will proceed as follows:

1. Read in the markdown file
2. Parse language snippets like “go” or “javascript”
3. For each snippet, create a gist
4. Keep an object with references to the snippet and the gist
5. Replace the snippets with their respective gist URLs
6. Write to a new markdown file, with code snippets replaced by gists for Medium

If you interested in checking out the source code, you can check out the repo here. I’d like to put together some tests before I decide to sell anybody on it, but it’ll be open source in any case.

Currently the usage goes like this:

./gist-generator -f example.md -token <GITHUB_ACCESS_TOKEN>

After running this script, we end up with a file called example.medium.md, meaning this is the file we’d ideally post on medium, with gist snippets and all.

If I make a web client from which this script would be consumed, the user would sign with their Github account, eliminating the step where they have to manually generate their personal access token and paste it into the CLI.

That’s something that would be nice to have but isn’t in my cross-hairs just yet — let’s check out some more pressing issues.

Problems Encountered

There were a couple of places in writing this script where I ran into trouble.

One such problem was handling the formatting of the gist that would be created. What looks good in my markdown code snippets may not necessarily look good in the gist.

In my first pass, I had excess whitespace due to newlines and oversized tabs. By iterating over the bytes in the code snippet, I could remove unnecessary newlines and replace tabs with spaces. Here’s an example:

Now out of all of the Go code I wrote for this script, why did I show the trimming of the newlines and tabs?

It is because the whitespace issue was the easiest problem that yielded the biggest results.

There were also some lower level issues that dealt with how to best parse files or build strings using byte buffers. I may tackle this in a future post since it deserves its own — Go has a lot of powerful built-in methods that the docs alone don’t do enough justice.

Lessons Learned and Future Projects

I loved this project for several reasons:

• It was small enough that it wasn’t too intimidating
• The end goal was clear and practical: to improve the quality of my blog posts
• I had the opportunity to practice developing a Go command line tool
• It can easily fit into an ecosystem of similar blog-productivity tools

By that last point, I mean that the gist generation step may be one of several.

I could write a similar script that would eliminate the excess markdown characters from the blog text. I discovered one way to partially do this step via this post.

Medium allows one to import a markdown file if its pasted in a gist (since it requires a URL). This causes it to handle backtick and asterisk wrapped words properly. It even handles the hyperlinks within your text.

To handle grammatical mistakes I could add a step that perhaps calls a simple Grammarly_ish_ API. If an open source solution doesn’t already exist, that might be another future project idea.

I could even create a dashboard that would allow me to easily delete gists if a mistake in my script cause faults in any of them. The current official gist dashboard is rather inefficient for editing or deleting gists.

Finally, I’ve been floating around the idea of a cloud-based markdown editor that would be accessible from mobile. You know, when I’m waiting in line at Disneyland and want to edit one of my articles. Times like these.

Wrapping Up

So much to do, so little time it seems. I suppose my next step is to cover those other aesthetic issues regarding importing stories from markdown.

At this point, any project that reduces the friction involved in writing technical content is valuable.

This project arose from my desire to eliminate some of the pain associated with tons of redundant editing. Some pains are cause by lack of familiarity with platforms, others are caused by an actual limitation in the platform.

Writing articles like this one and embarking on similar projects solves both problems: you learn the capabilities of that platform all the while improving the parts where it falls short.

A great learning experience, no doubt, in the name of productivity.

Again, here is the link to the project’s repo if you’d like to investigate how it works.

Curious for more posts or witty remarks? Give me some claps and follow me on Medium, Github and Twitter!