Problem - How many times have we faced this problem of worrying about cleaning up after running test cases. When some of our test cases run, they might add data to our database or add files to our directory which we don’t want to worry about every time.
Solution -We can easily solve this using Docker. Imagine spawning a Docker container just to run test cases and killing it after it’s purpose is solved. Poof! our problem is solved, no need to worry about the collateral damage done by our test cases since killing that container handles that for us.
What is docker?
There are many resources out there which will explain what is docker and containerization. To explain you in the context of this in layman terms, it can be considered a free computer that we get to do some stuff and you’re given to liberty to do whatever you want with it because once you’re done it will be destroyed and next time you’ll get a brand new one.
Step 1: Creating our app
I have created a simple todo app that basically allows a user to save, fetch, and delete the item. I will be exposing the REST endpoint for all these functionalities and will be using MySQL as my database.
func main() {
// Initialize a connextion to DB. Sourcename contains the auth information, host information
// obtained from the env.
db, err := sql.Open("mysql", sourceName)
if err != nil {
panic(err)
}
Init(db)
}
// Init initializes the dependecies and boots up the server.
func Init(db *sql.DB) {
// Initialize our model
repo := todo.NewListRepository(db)
err := repo.Init()
if err != nil {
panic(err)
}
// Pass our model to our service which will handle business logic
listService := todo.NewListService(repo)
// start server
http.HandleFunc("/save", listService.AddItem)
http.HandleFunc("/delete", listService.DeleteItem)
http.HandleFunc("/find", listService.FindItem)
http.ListenAndServe(":8080", nil)
}The main file initializes services and boots up the server
As we see we connect to MySQL and pass the connection to our model/repository which is a dependency of our service which handles our business logic.
type ListService struct {
repo ListRepository
}
func NewListService(repo ListRepository) (service ListService) {
return ListService{
repo: repo,
}
}
func (service ListService) AddItem(w http.ResponseWriter, r *http.Request) {
param1 := r.URL.Query().Get("item")
if param1 != "" {
ID := uuid.New().String()
err := service.repo.SaveItem(ID, param1)
if err != nil {
w.Write([]byte(err.Error()))
w.WriteHeader(400)
return
}
w.Write([]byte(ID))
} else {
w.WriteHeader(500)
}
}
func (service ListService) DeleteItem(w http.ResponseWriter, r *http.Request) {
ID := r.URL.Query().Get("ID")
if ID != "" {
err := service.repo.DeleteItem(ID)
if err != nil {
w.Write([]byte(err.Error()))
w.WriteHeader(400)
return
}
w.Write([]byte("delete success"))
} else {
w.WriteHeader(500)
}
}
...Above is our ListService which is essentially our controller, having logic for 3 endpoints.
// ListRepository is an interface
type ListRepository interface {
SaveItem(ID string, name string) (err error)
FindItem(ID string) (res string, err error)
DeleteItem(ID string) (err error)
Init() (err error)
}
// MySQLListRepository impl of Listrepo
type MySQLListRepository struct {
// conn sql.Conn
db *sql.DB
}
// NewListRepository is a constructor
func NewListRepository(db *sql.DB) (repo ListRepository) {
repo = MySQLListRepository{db: db}
return repo
}
func (repo MySQLListRepository) SaveItem(ID string, name string) (err error) {
_, err = repo.db.Exec("Insert into `items` (`ID`, `name`) values (?, ?)", ID, name)
return
}
func (repo MySQLListRepository) FindItem(ID string) (res string, err error) {
var row *sql.Row
row = repo.db.QueryRow("SELECT `name` FROM `items` WHERE `ID` = ?; ", ID)
err = row.Scan(&res)
return
}
func (repo MySQLListRepository) DeleteItem(ID string) (err error) {
var res sql.Result
var affected int64
res, err = repo.db.Exec("DELETE FROM `items` WHERE `ID` = ?; ", ID)
affected, err = res.RowsAffected()
if err != nil {
return
}
if affected == 0 {
return errors.New("invalid ID")
}
return
}
func (repo MySQLListRepository) Init() (err error) {
// var result sql.Result
_, err = repo.db.Exec(`CREATE TABLE IF NOT EXISTS items (ID VARCHAR(255), name VARCHAR(255)) ENGINE=INNODB;`)
return err
}This is our model handling database operations.
Step 2: Start testing it using Docker
Unit tests
First, we will start unit testing. We can directly test our models. This is where the importance of cleanup comes in. As we can see we are modifying essentially our database after every operation. Let’s say sometimes there are 100s of inserts running in each test. After every test run unnecessarily after DB will start filling.
We will use a MySQL docker container to spawn a database and pass that connection to our ListRepository. After unit testing our repo methods, we will purge the container thus avoiding the hassle of cleanup.
Starting and stopping the docker containers via code
There are essentially docker clients available for each language
We will be using a Go client to handle docker operations.
func TestMain(m *testing.M) {
// Create a new pool of connections
pool, err := dockertest.NewPool("myPool")
// Connect to docker on local machine
if err != nil {
log.Fatalf("Could not connect to docker: %s", err)
}
// Spawning a new MySQL docker container. We pass desired credentials for accessing it.
resource, err := pool.Run("mysql", "5.7", []string{"MYSQL_ROOT_PASSWORD=secret"})
if err != nil {
log.Fatalf("Could not start resource: %s", err)
}
connected := false
// Try connecting for 200secs
for i := 0; i < 20; i++ {
// Try establishing MySQL connection.
Conn, err = sql.Open("mysql", fmt.Sprintf("root:secret@(localhost:%s)/mysql?parseTime=true", resource.GetPort("3306/tcp")))
if err != nil {
panic(err)
}
err = Conn.Ping()
if err != nil {
// connection established success
connected = true
break
}
// Sleep for 10 sec and try again.
}
if !connected {
fmt.Println("Couldnt connect to SQL")
pool.Purge(resource)
}
// Run our unit test cases
code := m.Run()
// Purge our docker containers
if err := pool.Purge(resource); err != nil {
log.Fatalf("Could not purge resource: %s", err)
}
os.Exit(code)
}I’ll explain what’s happening
We create a new connection pool to docker and try connecting it.After the docker connection is successful we spawn a new MySQL container hosting our database.We try connecting to that container in a loop with a given timeout since spawning the container will take some time. If a timeout occurs then purge the container and exit.If the Conn.ping() doesn't return an error, congrats we have successfully connected to SQL container.Run the test cases and after that's done stop the SQL container.Voila! no hassle of cleanup our db since purging the container will take care of it.
Below is the example of our test case
func TestRepo(t *testing.T) {
assert := assert.New(t)
listRepo := todo.NewListRepository(Conn)
err := listRepo.Init()
assert.NoError(err, "error while initializing repo")
ID := uuid.New().String()
name := "itemName 1"
// Save Item
err = listRepo.SaveItem(ID, name)
assert.NoError(err, "error while saving item")
// Find Item
foundName, err := listRepo.FindItem(ID)
assert.NoError(err, "error while saving item")
assert.Equal(foundName, name)
// Delete Item
err = listRepo.DeleteItem(ID)
assert.NoError(err, "error while saving item")
foundName, err = listRepo.FindItem(ID)
assert.Error(err, "delete unsuccessful")
}Hence we have successfully tested our repository using Docker MySQL.
E2E test using Docker
Above, we are only testing our repository. What if we want to test our business logic and server as well with docker. Say if our business makes some file changes on our system. On running it’s test cases, we will also need to do the cleanup.
What we essentially need to do is start our app server in a separate docker container and run our e2e tests.
Setup
FROM golang:latest
WORKDIR $GOPATH/src/todo
ENV GO111MODULE=on
COPY . .
RUN go build -o main .
EXPOSE 8080
CMD ./mainAbove is a simple docker file for our application. But this docker container will not have MySQL installed with it. We need to start SQL in a separate container and bridge the network. We can do this using Docker Compose.
services:
mysql_db:
image: mysql:5.7
environment: # Set up mysql database name and password
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: todo
MYSQL_USER: user
MYSQL_PASSWORD: password
ports:
- 3306:3306
networks:
- my-network
todo_app:
image: todo_app
container_name: todo_app
environment: # Set up mysql database name and password
host: 0.0.0.0:3306 # Passing MySQL host
build:
context: .
dockerfile: Dockerfile
depends_on:
- mysql_db
ports:
- "8080:8080"
networks:
- my-network
networks:
my-network:
driver: bridgeThis Docker compose files can help us running both the service(SQL, todo_app) together and connecting them via a network bridge
We start our service using:
docker-compose up -dThis will ensure that our server is up and running and we can do our e2e tests below.
Our server will be up on localhost:8080 and we can directly hit it.
func TestE2E(t *testing.T) {
assert := assert.New(t)
client := &http.Client{}
req, err := http.NewRequest("GET", "localhost:8080/save?item=test122", nil)
assert.NoError(err, "error while initializing request")
res, err := client.Do(req)
assert.NoError(err, "error while making request")
assert.Equal(res.StatusCode, 200)
... // Other E2E requests
}E2E test
We can test it by making HTTP requests to docker server and asserting responses.
After we have completed the test we will have to manually shutdown todo server and SQL containers using `docker-compose down`. In this case, we have to manually start and stop the docker service but this can be automated as well by using commands via code to start and stop the containers.
Hence by doing this, we ensure that if our app makes changes in our system or our database those are not persisted and we don't have to clean those up.
Conclusion:
Hence we saw using docker we can run e2e and unit test without creating a mess of our system rendered by our application. Full code available here.
Previously published at https://medium.com/@mohdgadi52/testing-application-without-creating-a-mess-af1a26e9c2c4