In recent years, since the arrival of D3.js, whenever there is a task involving charts or graph visualisation I tend to gravitate towards JavaScript libraries. This is mainly because D3.js provides an easy way to bring data to life using HTML, SVG, and CSS, with a data-driven approach. It’s easy to customise and integrate for your own use case, and there are plethora of beautiful interactive chart examples out there.
However, having experience in creating C++ Qt user interfaces, sometimes I miss a more native approach. There is something about compiled small-sized binary executables, or something without having to deal with different browser involvements. So as a side project, I thought it was time to explore something new, and go on an adventure to find an alternative.
Always be curious, have an adventure
Looking around within my work environment (something I’m familiar with) for inspiration, I decided that a suitable subject for this project would be a tool to visualise operations in MongoDB log files. The tool would use a scatter plot to map out log entries based on the timestamp and duration of the operation.
Example of MongoDB log entry (v3.4.9):
2017-11-13T10:22:19.039+1100 I COMMAND [conn3] command charts.golang appName: "MongoDB Shell" command: find { find: "golang", filter: { a: { $regex: /^d.*/ } } } planSummary: COLLSCAN keysExamined:0 docsExamined:1000 cursorExhausted:1 numYields:7 nreturned:42 reslen:2638 locks:{ Global: { acquireCount: { r: 16 } }, Database: { acquireCount: { r: 8 } }, Collection: { acquireCount: { r: 8 } } } protocol:op_command 1ms
As a newbie in the Golang programming language, this would also be a perfect opportunity to increase my fluency in the language. Because you know what they say: if you don’t use it, you lose it. With those decisions made, it was now time to look for suitable Golang dependencies.
Keep Looking
There were (probably) two libraries needed to start the project: a library to render scatter plots and a library to parse MongoDB log files.
Generally, the Golang plotting libraries that I know just generate static, albeit beautiful, images; to name a couple:
At this point I decided to go down the path of writing my own custom interactive widget. After a quick search I found golang-ui/nuklear. It’s a package that provides Go bindings for a small ANSI C GUI library. See also the related github.com/vurtun/nuklear.
One of the examples of GUI from the Github repo.
Nuklear has some desirable features that caught my eye:
- Small codebase (~18k LOC)
- No dependencies (not even the standard library if not wanted)
- Low memory footprint with total memory control if needed or wanted
- Optional font baker and vertex buffer output
First, I did a quick test to create entry plots:
c1 := nk.NkRect(float32(fx), float32(fy), 5.0, 5.0) nk.NkFillCircle(canvas, c1, nk.NkRgb(171, 239, 29))
if nk.NkInputHasMouseClickDownInRect(input, nk.ButtonLeft, c1, 1) > 0 {log.Println("Receive a click on coordinate: ", fx, fy)}
Full source code for the simple example above can be found on gist://test_scatter.go
The size of this binary executable was only ~5MB. Without any code optimisation I could render ~75,000 points before there’s a bit of lag.
With another quick search I found a Golang library to parse MongoDB log files, github.com/honeycombio/mongodbtools, written by honeycomb.io. With a few extra lines of Golang code, I could parse log files and plot them into the interactive widget:
The size of this binary executable was ~10MB.
At this point I could finish fixing up the X and Y axis information that was missing, scaling the input timestamp and the duration to fit within the size of the application window and calculate the marker points.
Another path that I explored was using one of the many Golang plotting libraries to generate an SVG or PNG image and render it as a texture/bitmap on the GL widget, refreshing the texture generation as new data arrives. I quickly tried implementing this path using github.com/wcharczuk/go-chart to generate the static image:
The challenge here was to map a mouse click received on the widget and correlate it back to an item on the texture/bitmap to find out which entry point was clicked.
Always learning…
Okay, so what did I learn from this little adventure:
- There are plenty of (useful) Golang libraries out there.
- Writing Golang plotter applications using only GL is doable, you just have to write the X and Y axis scaling.
- Using existing Golang plotting libraries that produce SVG output and mapping that as GL textures is doable as well, you just have to map the click on the GL back to the textures.
- One advantage of only using GL is that the plot entry points can be rendered without having to read all of the data first.
Perhaps it’s useful to start a Golang library to render interactive graphs, perhaps there’s already one out there that I’m not aware of. Let me know what you think.