In different post in the past, we have explorer why is testing so important to us. Also, we took a look at ScalaCheck, a Property Based Testing Framework for the Scala programming language.
Because we are explorers, we have started a new adventure, creating a small, yet functional and easy to use Property Based Library to be used in .NET based languages.
Motivation
We want a library that works with our current testing framework, in our case, xUnit. By doing a quick look at NuGet, we can find some frameworks that target the same space that ours, but they required learning the entire framework. Also, you might need to give up your favorite testing framework in order to use it. Many other problems rise from this such as having a test runner for your IDE, etc….
Our library is not intended to change anything that we do today, but instead, allows us to use property based testing along our regular tests.
Core Concepts
The most import piece is what we call Gen<T> and it represents a way to generate bounded data of type T.
The most simple generator can be created in the following way:
var gen = Gen<int>.Empty();
Assert.Empty(gen.Generate());
Notice that we are using xUnit.Assert so we are not introducing any assertion API at all.
Now, let’s look at some other ways to create generators.
A single item generator
var once = Gen<int>.Once(5)
Assert.Single(Gen<int>.Once(5).Generate());
Repeat generator
var repeat = Gen<int>.Repeat(5, 20);
Assert.True(repeat.Generate().Count() == 20);Assert.True(repeat.Generate().ToList().TrueForAll(x => x == 5));
In here, we are saying that we want to generate the value 5 as many as 20 times.
Important to notice is that calling .Generate()will always return the same dataset.
Enumerable generator
A more interesting generator is the one created from IEnumerable<T>.
var enumerableGen = Gen<int>.FromEnumerable(Enumerable.Range(1, 100));
Assert.True(enumerableGen.Generate().Count() == 100);
A simple version can be generated by doing:
var enumerableGen = Enumerable.Range(1, 100).ToGen();
Assert.True(enumerableGen.Generate().Count() == 100);
Generators by itself cannot do much. However, the idea is to use them as value generators in order to test some properties on them.
Producers
Many times we need a more robust way to generate data. A Producer’s intend is exactly that.
public interface IProducer<out T> : IEnumerable<T>{
}
As you can see, IProducer is only a way to generate a sequence of values of the same type T.
A simple Producer for generating **int**s could be:
public class IntProducer : IProducer<int>{public IEnumerator<int> GetEnumerator(){while (true){yield return new Random().Next();}}
IEnumerator IEnumerable.GetEnumerator()
{
**return** GetEnumerator();
}
}
It is important to notice that IntProducer generates a unbounded collection of ints. We need to obtain a generator (Gen<T>) from the Producer so we can force bounds on it.
We can use the following to do so:
var intGenerator = (new IntProducer()).ChooseFrom()
By applying .ChooseFrom(howMany) to any Producer we create a Gen<T> which forces bounds on the Producer.
Another example could be StringProducer:
public class StringProducer : IProducer<string>{public IEnumerator<string> GetEnumerator(){while (true){yield return new Random().Next().ToString();}}
IEnumerator IEnumerable.GetEnumerator()
{
**return** GetEnumerator();
}
}
Again, we could get a Gen<T> from it by doing:
var stringGenerator = (new StringProducer()).ChooseFrom(20)
We also included a way to generate tuples of the form (T, K) by using TupleProducer<T, K>.
public class TupleProducer<T, K> : IProducer<(T, K)>{private readonly IProducer<T> _tProducer;private readonly IProducer<K> _kproducer;
**public** TupleProducer(IProducer<T> tProducer, IProducer<K> kproducer)
{
**\_tProducer** \= tProducer;
**\_kproducer** \= kproducer;
}
**public** IEnumerator<(T, K)> GetEnumerator()
{
**var** tenum = **\_tProducer**.GetEnumerator();
**var** kenum = **\_kproducer**.GetEnumerator();
**while** (tenum.MoveNext() && kenum.MoveNext())
{
**yield return** (tenum.**Current**, kenum.**Current**);
}
}
IEnumerator IEnumerable.GetEnumerator()
{
**return** GetEnumerator();
}
}
Once more, we can get a generator from it using .ChooseFrom.
ForAll & Any
Once we have created a generator we can use .ForAll and .Any as existential quantification to proof that a property holds for the values being generated.
Let’s look at some examples by testing the following functions:
T id<T>(T x) => x;
int sqrt(int x) => x * x;
string concat(string a, string b) => a + b;
Notice this is perfectly valid C# code, here we are only using method body syntax, the equivalent in a more traditional C# will be:
T id<T>(T x) {
return x;}...
Let’s move forward to test these functions.
[Fact]public void Identity(){Assert.True((new IntProducer()).ChooseFrom().ForAll(x => id(x) == x));}
.ForAll receives the statement we want to state about the values.
In the case of Identity, we want to proof that for all x, id(x) == x.
The property will be evaluated for every single value on the generator.
Now, let’s test sqrt:
[Fact]public void SomeSquares(){Assert.True(Gen<int>.FromEnumerable(Enumerable.Range(1, 100)).Any(x => sqrt(x) == x));}
In this case, the property sqrt(x) == x is only valid for the value 1. Since we are using .Any the test is also a passing test. It is enough that the property holds for at least one value when we are using .Any.
Finally, we can test concat.
[Fact]public void Concat(){var stringProducer = new StringProducer();var producer = new TupleProducer<string, string>(stringProducer, stringProducer);
producer.ChooseFrom().ForAll(t =>concat(t.Item1, t.Item2).StartsWith(t.Item1)&&concat(t.Item1, t.Item2).EndsWith(t.Item2)).Should().BeTrue();}
This one is very interesting since we are using the StringProducer and TupleProducer together.
In here, we generate random strings, concat them together and proof the property:for all a, b that are string, a + b == a* && a + b == *b. Our library will choose 100 strings (bounded collections or Gen<T>) produced by the StringProducer and test that the property holds for all of them.
Notice we are doing the assertions through the FluentAssertion API instead of simply Assert. However, it is just to show that our library is not limited to one testing framework or assertion style.
Endings
Even though this is a very small library, it does its job well and allows us to bring a simplified property based testing to .net.
you can start using it today by getting it from the NuGet Gallery.
PM> Install-Package NetChecker -Version 1.0.0
Contributions are always welcome on the GitHub NetChecker Project.
If you want to learn more about Property Based Testing you can read Exploring Property-Based Testing with ScalaCheck (simple examples)