Building good end to end tests is hard. Having good end to end tests is wonderful. How do you get from the former to the latter?
Last week I introduced a simple chat server and client written in Dart. Half way through writing that code I decided to stop and add end to end tests — so they could help me make faster progress. It worked, and saved time overall.
Testing can be a big time sink; but I was sure it would work, because I’d used the same pattern before in similar code. Now I’m here to share that pattern with you.
For those following along at home, here’s how to get and run the tests. You’ll need git and dart, then:
git clone https://github.com/google/built_value.dart.gitcd built_value.dartgit checkout tags/v0.4.3-articlecd chat_examplepub getpub run test
If you run this you’ll see it’s ridiculously fast_._ On my machine it takes around one second — total — to run all 13 end to end tests. Further, if you examine one of the tests, you’ll find they’re incredibly easy to write. Here’s a test that private messages are private:
group('tell', () {test('goes to a single user', () async {var alice = environment.newUser()..type('/login Alice letmein');var bob = environment.newUser()..type('/login Bob letmein');var eve = environment.newUser();
alice.type('/tell Bob Hi there.');
bob.expectMatch(r'Alice \\(private\\): Hi there.');
eve.expectNoMatch('Hi there.');
});});
It’s testing both the client code — the code that runs in your browser — and the server. We create three users: Alice, Bob and Eve. Alice and Bob log in, then Alice sends a “tell”, a direct message, to Bob. The test passes if Bob can see the message and Eve can’t.
Dart Everywhere
The key to how the chat example is tested is that both the client code and the server code are written in Dart. This means that it’s perfectly possible to run the client code on the server, or the server code on the client — with the exception of VM-only libraries like “dart:io” and browser-only libraries like “dart:html”.
So what I did was to fake out client-only code. That means that “dart:html”-backed classes are split into an interface and separate implementation. For example, here’s the HTML display:
/// Chat window main text display.abstract class Display {/// Adds [text] to the display, coloured to indicate a local/// command.void addLocal(String text);
/// Adds [text] to the display.void add(String text);}
Then, to load the client code in a VM, I supply a fake implementation:
/// Fake [Display] that stores added text.class FakeDisplay implements Display {List<String> text = <String>[];
@overridevoid add(String text) {this.text.add(text);}
@overridevoid addLocal(String text) {}}
Now I can run the server and client code in the same VM. So far so good. But they need to be able to talk to each other.
The same approach is used; connection classes are split out into interfaces:
/// Two-way connection between client and server; the client.abstract class ClientConnection {Stream<String> get dataFromServer;
void sendToServer(String string);}
And then for testing, fake implementations are used that plug the whole thing together. A test environment maintains an instance of the server code and creates test users connected to it.
And that’s it. Now, you might argue that isn’t really end to end testing. To do that we’d need to use something like WebDriver. Such tests have their place, particularly when integrating with complex systems. They’re necessary for shipping rock solid software. But they’re not the best way to boost developer velocity.
For that you need fast, easy to write tests that, like these, test just the code you’re working on. For some code, unit tests are ideal; but to really understand things at a high level you want tests that interact with the whole application as a user. They’re “end to end” — but only for the parts you care most about.
A Bit More Testing
While working on this article I noticed a bug in the chat code: not all commands are correctly echoed locally. That is, some commands just “disappear” when typed. It’s not surprising: I don’t have any tests for this! So I decided to add some.
First, the “FakeDisplay” needs to record locally echoed text too:
class FakeDisplay implements Display {List<String> text = <String>[];List<String> localText = <String>[];
@overridevoid add(String text) {this.text.add(text);}
@overridevoid addLocal(String text) {this.localText.add(text);}}
Then we need to be able to assert on it, so “TestUser” gets a new method:
/// Checks local text for this user.void expectLocalMatch(Pattern pattern) {expect(_display.localText, anyElement(matches(pattern)));}
And that’s it, I’m ready to add tests like this one:
test('away echoes locally', () async {environment.newUser()..type('/away Not here.')..expectLocalMatch(r'/away Not here\.');});
Bug caught, and fixed. Here’s the full pull request. I added seven new end to end tests, bringing the total to 20, and fixed two bugs as a result.
The tests now all run in about 1.1 seconds. I find that acceptable.
Happy Holidays
I’ve now written in depth about everything from my talk at the Dart summit (video). Fortunately, it’s a good time to take a break. You can expect some more articles from me in 2017 — as soon as I have something new to write about. In the meantime, if there’s something you’d particularly like to me to cover, just let me know.
Hacker Noon is how hackers start their afternoons. We’re a part of the @AMIfamily. We are now accepting submissions and happy to discuss advertising &sponsorship opportunities.
To learn more, read our about page, like/message us on Facebook, or simply, tweet/DM @HackerNoon.
If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!