I saw this link on Hackernews this morning.It’s a small experiment on the speed of writing from memory using the yes command.
$ yesyyyyyyyyy
That’s the entirety of the the command. It prints the letter y, forever.
From the article, we can use the pv command to test the bandwidth used to write to /dev/null. From the pv man page..
pv allows a user to see the progress of data through a pipeline, by giving information such as time elapsed, percentage completed (with progress bar), current throughput rate, total data transferred, and ETA.
Results on my old mac mini running Ubuntu
$ yes | pv > /dev/null... [3.59GiB/s] [
Results on my macbook pro running OSX
$ yes | pv > /dev/null... [26.2MiB/s]
Well.. that’s a bit of a difference. It looks like the yes command is OSX is shit. Who knew? Let’s see if we can make it faster.
GOPHERS ASSEMBLE!
Yes, we’re going to try and make a faster yes in Go. How fast can we get?
Round 1
package main
import ("fmt")
func main() {
for {fmt.Println("y")}
}
erm.. slow on OSX
./yes | pv > /dev/null... [1.69MiB/s]
and Linux
$ ./yes | pv > /dev/null... [1.99MiB/s]
What if we use a more direct method of writing to stdout?
y := []byte("y\n")
for {os.Stdout.Write(y)}
a bit better on OSX
./yes | pv > /dev/null... [1.99MiB/s]
and a bit better on Linux
$ ./yes | pv > /dev/null... [3.05MiB/s]
Round 2
What happens if we use the same trick as the article. The main way it speeds things up is to create a buffer of output in an array in memory. This can then be very quickly written to Stdout.
Something like this.
package main
import ("os")
func main() {
// create a buffer that has space equal to the page sizebuf := make([]byte, 4096)
y := []byte("y\n")used := 0
for {
buf\[used\] = y\[0\]
buf\[(used)+1\] = y\[1\]
used += 2
if used == 4096 {
break
}
}
// Print out the entire buffer at once - writing direct to// stdout is the fastest way of doing this.// It's the size of a standard linux pagefile.for {os.Stdout.Write(buf)}
}
and the results?
On OSX
$ ./yes | pv > /dev/null... [1.88GiB/s]
On Linux
$ ./yes | pv > /dev/null... [2.75GiB/s]
Well that looks ok! On OSX we’ve managed nearly 105x speed increase. On Linux, slightly less good results but I’m not surprised there — the Linux version seems to be the the most efficient anyway.
Can we do better?
One more thing to try, what if we use a bigger buffer?
buf := make([]byte, 16384)
OSX
$ ./yes | pv > /dev/null... [2.98GiB/s]
Linux
$ ./yes | pv > /dev/null... [ 3.19GiB/s]
I think i’ll leave it there :)
If you have any questions please feel free to comment here or tweet me — matzhouse
Hit the heard if you enjoyed reading this!