“Lets sit down for a month and install some aliases and learn how to make effective terminal scripts.” ~said no one ever
gog is a developer community platform aiming to improve the quality of terminal scripts — along with gog CLI, it allows users to quickly find, manage and execute terminal scripts directly from the terminal.
Why I decided to start gog
In mid 2015, while working as a senior full-stack developer for a startup, I came across a build process problem that kept coming back in more and more cumbersome ways. Over a span of 2 months, I had accumulated so much code that I sometimes forgot its order of execution or the optional parameters that pertained to specific symptoms. Sometimes it even required me to manually reinstall complete stacks of code in very specific ways. I finally decided to start scripting the solution to the problem.
While scripting made it less clunky, I couldn’t permanently fix the build process because our project needed the new features and with every update it felt like someone was pulling the carpet from underneath me. This led me to an even more optimal solution. I decided to improve my scripts by organizing them in a CLI.
Going global
My first task was to figure out a way to consistently execute my scripts in the same exact way every time. Since my scripts could fail spectacularly depending on which directory I executed them from, I needed to ensure that I couldn’t accidentally run them from root instead of the proper project folder. Hence, gog was born (go-global). I created a CLI that would live in the local/bin directory and execute scripts from local/lib directory, so that no matter where I was in the terminal I wouldn’t have to spend hours fixing a hasty mistake. This also allowed me to execute scripts across users — handy!
Structure
In addition to proper execution I also wanted to track changes to my scripts as I periodically updated them, but needed a way back when we sometimes reverted changes or applied hot-fixes to our project. I could have created a repo in GitHub and tracked my changes there. But who wants to deal with that mess? Especially when it’s 1am, you’re half asleep, and pushing your 10th update before you fall asleep.
I also added descriptions that would ship with the code and the ability to quickly list all of my scripts. Since gog already executed scripts by name it was very simple to version everything and quickly see what was being executed without jumping around repos. Enter:
Listing all scripts with descriptions with gog CLI
Executing script with gog CLI
That was a pretty decent solution to the problems at hand so I left gog alone for a while and only updated it from time to time for system support.
Creating a platform
Years later I discovered that I was not the only one experiencing problems with build processes, as hybrid app development had grown in popularity. Also, almost every single developer I ever talked to has told me that they still use terminal scripts to some capacity. Either for their own build process, some legacy code, platform they still have to support, or to fix something that broke on their system. Ironically, the ones that had to execute their scripts less frequently had more trouble remembering how to do it and wished that they could spend more time making them a little nicer. But, they wouldn’t.
Sharing gog CLI with others wasn’t enough. Sharing scripts across machines was problematic and people still had to look for solutions and sift through pages of pseudo-code on StackOverflow or difficult examples on sites like tldp.org. I knew that I had stumbled onto something that other coders could use and benefit from. I created gog.codes — a web portal — that connected to gog CLI and would allow users to quickly search for scripts from the terminal. It also created a place where people could add their own scripts and have them improved upon by others on actual. executable. code. I also added some social features to promote better scripts and mark problems in scripts on specific lines of code.
Gog.codes user profile with multiple scripts.
Script in gog.codes with additional line annotation
Benefits
There are endless benefits in using terminal scripts alone that include:
- Build processes
- System information/monitoring
- System configuration
- Application Repair
- Storage cleanup
- Network configuration
… and more.
To help users take full advantage of these benefits, gog aims to remove all of the scary aspects of script development and execution. The platform is built for Terminal scripts, so only features supporting terminal use are considered. There are already great platforms for project management like GitHub or npm, and gog is not trying to replace them.
The most common fears surrounding terminal use are total corruption of the systems by executing improper commands, total system failure by executing commands improperly, and data corruption by applying a file transformation to the entire hard-drive. These problems, of course are all addressed in gog by executing every command from a consistent location. In addition, having complete working scripts available instead of pseudo-code examples greatly reduces the chance of something going horribly wrong.
In future plans, machine learning will further safeguard against accidentally executing commands like rm -rf *or :(){:|:&};: and hopefully one day become the way people share scripts, instead of typing out commands in a blog post and hoping for the best.
(Note: Do not execute the above commands. They will definitely damage your system)
Security
The biggest negative feedback I’ve ever received about gog was, “you are downloading scripts from the internet and executing them on your machine, that’s dangerous!” As this concern is completely valid and I would never recommend blindly executing random code on your terminal without knowing what it does, I do have to point out that you do it all the time. As a developer, if you have ever worked with dependencies, you have installed code, including bash scripts, without even realizing that it’s there. As a consumer, you trust the applications you install not to install malicious scripts on your computer.
gog was specifically designed to never automatically execute code you download from the platform. This allows users to review the code they download before it’s executed. Eventually, I plan on adding machine learning to automatically detect malicious scripts and flag ill-formed code. For now, I advise reading comments and editing the code you execute.
Future improvements
There are many additions already planned for gog that include:
Wider system support for Linux platforms or terminals with aliases
Different Linux systems have slightly different Bash support that sometimes breaks the CLI. Also, terminal aliases can cause problems by having different profile strategies that also break the CLI. Unfortunately, full system support will take some time but the CLI is currently written entirely in Bash and it runs without any special dependencies. That should expand the system support, especially for legacy systems.
Autostart and cron jobs
Script automation is an important part of any system ecosystem and gog will eventually allow for an easy way to add installed scripts to either auto-start or cron tasks.
Syntax support in the web portal and machine learning
To help ease the process of writing good scripts gog will support syntax for Bash, JavaScript and Python including machine learning that will detect obvious problems or malicious code.
Virtual functions
To stimulate a quick, productive scripting experience, gog will contain a set of modular script fragments that can take parameters and return parameters. Each fragment, or virtual function, will be versioned and easily pluggable into scripts to perform some functionality. Each version of the function will be preserved to prevent breaking current scripts that use a particular version.
Code annotation
In addition to syntax support code annotation will provide information about the code you are writing, including hints and small examples as users create their scripts.
Conclusion
My hope is that other developers don’t have to go through the same hair-pulling routine that I did for years. You can use gog for free by visiting the site. I’m always looking for feedback so that I can make it even more efficient and secure. Feel free to email me at: kpx.dev@gmail.com.