Automation Approach
Test automation is one of the most important parts in the modern software development process and it is required to be placed in order to implement modern DevOps best practices. With an automation in your development life cycle, developing your application continuously becomes more structural, free of huge manual testing concerns and matching some level of quality standard everytime.
Today, I am going to tell you how to integrate your selenium test in your application pipeline and run them as part of your CI/CD process. Therefore, I will create one demo react application, one selenium framework with demo test, dockerize the app and tests, prepare demo Github Actions pipeline, docker hub for keeping images and many more interesting things. And of course it will be kind of MVP and many more things can be added/removed. So if you are ready, let’s get started to build this.
Note: Tools are chosen randomly. You can choose any other tools for any operation, i.e. cypress for automation, CircleCI for CI/CD. I choose selenium because generally people tend to run selenium in a virtual host machine and not try to dockerize it so I just wanted to show how you can apply this with selenium.
Installation:
You need to install these tools in order to build the project.
Required tools:
- Java 8+
- Maven
- Docker desktop
- Nodejs
- Github account
- Intellij Idea
- VS Code
First MVP — Create React App
I will create a react app first and then run selenium tests against this application. I will use the create react app command to create a react application and it creates a ready application in a couple of seconds.
Open terminal and navigate where you want to create the app folder.
Run:
npx create-react-app my-app
this will create a react application in my-app folder. You can change directory to this folder and run react app
cd my-app
npm start
Open the browser and navigate to localhost:3000
That’s it. You have one application running now. But as you notice, it is running in local machine so, before moving to selenium test, let’s quickly dockerize the application and run it in a docker container.
Dockerize React app
First of all, it would be good to keep the npm start command in a separate file so you can just quickly call it with the sh command. So let’s create a bin folder in the root and create a run.sh file in this bin folder. And put react app start command there
#!bin/bash
npm start
Secondly you need to create a dockerfile. Let’s keep it in the .docker/frontend folder to keep the project more organized. Create .docker/frontend in the root of your project and create a Dockerfile.dev file in it.
FROM node:16-alpine
WORKDIR /app
COPY . /app/
RUN npm install
ENTRYPOINT [ "sh", "bin/run.sh" ]
and since you will use it for only development and test purposes, name it Dockerfile.dev.
You can test your setup with these command:
docker build -f .docker/frontend/dockerfile.dev -t frontend .
docker run -p 3000:3000 frontend
Open the localhost:3000 and verify everything is correct.
Now, you are good to go with the selenium test.
Selenium Project setup
I won’t tell you every detail of the selenium tests since it is not our topic today but just share the general structure of the test framework. You can check the test repo from this link. https://github.com/selcuktemizsoy/demo-ui-selenium-cucumber
Here is the general project structure
I will pass my test config as .env file so that in the future I can use different values and run my test in a different environment with another file.
Here is the how .env file looks like:
Note: 172.17.0.1 is the docker host ip address. I will run all the things on docker so I am configuring my project to run with docker. But for futher test development, it would be better to run test in local machine without dealing with docker build so in order to achieve this you can create one moe .env file and name it as .env.local and use it for local test development purpose.
host=react app host address.
grid=selenium grid hub.
browser=browser type (I will run test in selenium grid so chose remote webdriver)
I have adjusted my driver class to take the browser and grid address here.
It is also supporting chrome and firefox. But for now I will continue with the remote firefox driver. (Note: I have faced some errors for remote chrome so I continued with firefox. I couldn’t find time to investigate the problem but if I could find a solution, I will push it to the test repo.
My test written in cucumber:
Feature: My first feature
Scenario: my first scenario
When I go to homepage
Then homepage should openAnd here is the step implementation.
As you see, it is simply going to host and check if the text is correct or not.
To test the initial setup, you can choose the local config to run your test. It doesn’t require to have grid up and running so you can just run them:
env $(cat .env.local) mvn test
Note: Here I am giving my config to test with the env command. And then running the test with the mvn command.
Dockerize Selenium Test suite
Okay, you have a demo app running in docker container and you also have one selenium test running against react app. Since you are using a separate repo, and you may want to run your test in any environment, it is better to dockerize your test and keep it in a separate registry. Of course there are lots of other options but dockerizing the test and run them as part of the pipeline makes your job easier and you can deploy your tests to wherever you want if you have docker version of your test. So let’s create .docker folder inside of the selenium repo and create a dockerfile:
FROM maven:3.6-jdk-11
WORKDIR /tests
COPY ../pom.xml .
RUN mvn install
COPY .. .
ENTRYPOINT [ "tail", "-f", "/dev/null" ]
Lastly, create a bin folder again and put a test script inside of it. run.sh:
#!/bin/bash
env $(cat .env) mvn test
I gave an unnecessary command for entry point since in pipeline, you will trigger your test as a separate step, as a result you need to start your container first and then give this run.sh.
Now the automation framework is almost done. You didn’t test it because you haven’t completed selenium grid installation so that it will give an error when you try to start your test container. So just postpone it for now and you will get back there later.
Github actions for your tests
Now, since you have a docker container version of your tests, you need to push this docker image to somewhere else to reuse them. In a real development environment you will use your organization docker container registry and it will be private. Since this is not a real project I will use the public registry in docker hub. To be able to push test images to docker hub, you need to have some workflow for your repository. As I said before, this project will use github actions, now it is time to configure the github workflow to build the test container and push it to the docker hub registry.
Let’s create a .github/workflow folder and put github-actions.yaml in this folder.
This is the file that tells github to run some workflow after you push some changes to your test suite.
Here is the github actions file:
name: Selenium test CI pipeline
on:
push:
branches: [ master ]
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v3
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_PASSWORD }}
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Build images
uses: docker/build-push-action@v3
with:
context: .
file: .docker/Dockerfile
push: true
tags: selcuktemizsoy/selenium-demo:latest
It starts with name to name it workflow. And you can state when this workflow runs. In this case, it is only running when somebody pushes to master branch.
In the jobs section, it is defining which job will be running. First, you need to checkout the code and then you have to make docker login to be able to push your images to docker hub. And then setting up docker buildx to build and push docker images. Finally, it is time to build and push docker images to docker hub.
You can find more information related to github action here. https://docs.github.com/en/actions
Now, you are good to go but as you notice you have to put your secrets to your github repo so that github actions can take username and password there and use it. So you have to go to the repository page(create one if you haven’t created one before) Settings>Secrets>Actions>New Repository Secret and your secrets from there.
Now you are ready to push your test repo and see how your github actions work. After pushing you can go to repository page and navigate the actions tab and see something like that:
After it finished:
While it is building, you can click any job to see how it is working.
Now, your test image is ready. it is deployed on your docker hub and now you can fetch anywhere you want. So let’s get back to the frontend project and complete the setup with grid and other required tools.
Docker-Compose for test environment
Now, you need to configure your test environment and run tests. You can use docker-compose to have a test environment and run this anywhere you want. As you remember, I told you that using selenium grid, docker compose setup will also include selenium grid.
In the root folder, create docker-compose.ci.yml file and put the required image in it.
version: '3.9'
services:
frontend:
build:
context: .
dockerfile: .docker/frontend/Dockerfile.dev
ports:
- '3000:3000'
selenium-hub:
image: selenium/hub:4.6.0-20221104
ports:
- '4442:4442'
- '4443:4443'
- '4444:4444'
firefox:
image: selenium/node-firefox:4.6.0-20221104
depends_on:
- selenium-hub
volumes:
- /dev/shm:/dev/shm
shm_size: 2gb
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_NODE_SESSION_TIMEOUT=25
- SE_NODE_MAX_INSTANCE=1
- SE_NODE_MAX_SESSIONS=1
selenium-host:
image: selcuktemizsoy/selenium-demo:latest
As you see, it is quite easy. Selenium hub and node images coming from selenium, you don’t need to do anything. Selenium host is your test suite image and it includes your tests. You have already pushed it to docker hub and it is coming from there. The only thing that you need to do is add your app image. But since you have already created this dockerfile, it is enough to pass the dockerfile path to compose and that’s it.
Now you can test your setup with those commands:
docker-compose -f docker-compose.ci.yml build
docker-compose -f docker-compose.ci.yml up -d
after everything ready (app and selenium grid)
docker-compose -f docker-compose.ci.yml exec -T selenium-host sh bin/run.sh
after test finishes, you can stop env:
docker-compose -f docker-compose.ci.yml down
Before moving github action for the frontend app, it would be good to add some script to check if your selenium hub is ready or not. In bin folder, create one wait.sh file and put this command inside of it.
#!bin/bash
until $(curl --output /dev/null --silent --head --fail http://localhost:4444 && curl --output /dev/null --silent --head --fail http://localhost:3000 ); do
echo "waiting for selenium hub and react app being started"
sleep 1
done
it will wait until the app and selenium hub is ready.
Makefile to consolidate commands
One last thing before moving to github actions, it would be good to add a makefile to keep the app, docker and test command.
To achieve this, create a file in the root and name it as Makefile. Inside of it, prepare your commands. I prepared mines as:
# building docker images
build-ci-env:
docker-compose -f docker-compose.ci.yml build
# starting docker-compose
start-ci-env:
docker-compose -f docker-compose.ci.yml up -d
# wait for app and hub
wait-ci-env:
sh bin/wait.sh
# start selenium test inside of the container
start-ci-test:
docker-compose -f docker-compose.ci.yml exec -T selenium-host sh bin/run.sh
# stop environment
stop-ci-env:
docker-compose -f docker-compose.ci.yml down
Now, you can easily use commands to start, stop, build etc.
Github actions for react app
It is time to create github actions and let’s create .github/workflows folder and create file github-actions.yml
In this time, you need to build your environment in CI and run tests. And also you can add more steps to build your application code and deploy it in any environment. But since it is not our topic today, it will include steps only related to tests.
And here is the general workflow:
name: react app CI
on:
push:
branches: [master]
jobs:
build-and-test-frontend:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v3
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Setup CI environment
run: make build-ci-env
- name: Start CI environment
run: make start-ci-env
- name: Wait for selenium hub to be ready
run: make wait-ci-env
- name: Start tests
run: make start-ci-test
- name: Deploy
run: echo "you can add deployment steps from now on. Remaining steps will be running after the previous steps have passed."
As you see, it starts with checking out the code again. Secondly, setting up docker buildx to build docker images. After adding buildx step, actually, it is same with your local computer. First, build-env and then start this environment. Next wait until setup is ready. Finally, you can just run the test. I put an example deploy step here but now it is just empty.
Test the pipeline
Okay, the project setup is finished. Let’s push our app to the github repo to see if everything is working or not.
And we have all the green light. Congrats 🎉 You are done!.
Now, let’s take closer to test step:
Now, you can see your test output like in your machine. So until now everything is green but your aim was to measure application quality and block the development if something is wrong. So let’s evaluate this by changing some source code in the frontend app. Open the src/app.js file and change the text inside of the p tag. I just removed some part of the text and am expecting to fail in the pipeline.
Let’s push it and see
You see, the deployment step didn’t work and your assertion failed as expected. So, all the setups are working perfectly.
Summary
In conclusion, I explained how to prepare a pipeline for your selenium test and block the development if your tests failed. I have shown you how to dockerize your test suite, using docker compose for running the test environment, using selenium grid with docker compose and run your test against your application as part of your pipeline. It is a demo setup and many more things can be added/removed. i.e you can choose having one repo, another automation/test tool, triggering another pipeline from your app pipeline etc. Today I chose this kind of setup and I hope it will give you an idea about having a fully automation flow in your software development life cycle. I hope you like it and stay tuned for upcoming articles.
Projects:
Test suite:
React app:
You can follow me on Linkedin