Two labs, two students, two weeks, too many test. Nah, you can never have too many tests, right? This lab had us installing a testing suite into our link checker repository. We also included some continuations integration into our project using GitHub Actions.
Step 1 – Choose and set up a framework
The first step in this process is to select what testing framework to use and install it. After doing a bit of searching and reading on the three main frameworks I decided to go with pytest. I chose this because it seemed to be the most popular framework as it still allows you to use features of the other ones like using unittest‘s Mock class in order to mock things up which I’ll talk about in a bit.
Installing pytest was a breeze, all I had to do was use the following pip command
$ pip install pytest.
After it was installed I copied the basic example they have in their docs to test things out and all was good, got the green light on my first test of the test!
Step 2 – My First Test
Well, not my first first test, but my first python test. I did some Junit stuff a few semesters back so I’m familiar with the assert concept.
In order to get the ball rolling I decided to write a test for… Hmmm, what’s an “easy” place to start? I spent a decent amount of time looking at my code trying to figure out where to start but the way I had built my code didn’t really lend itself to being tested. Well, it was built good for testing, but just not easy to start. My main logic was all encapsulated in a class that took a parameter to construct. While it was all done in smaller functions, I called all those functions in the classes constructor so the the constructor did everything.
This was not optimal so I did a bit of refactoring and took the print statements out and called them in the main file out side of the class. I also took out the call to my checkFile() function that did most of the leg work. This gave me a little more flexibility with testing but I still had to figure out how to pass some arguments to the constructor.
I decided a test to check if the file exists would be the easiest to check because it is a required argument so I will need to get it working before anything else can work. I tried making a dictionary, and then a tuple, and then a class. Finally, a class was working! After some slight adjustments I was able to pass my first test!
def test_no_file_exception():
args.file = "wrong/file/path"
with pytest.raises(FileNotFoundError):
checkFile.checkFile(args)
Step 3 – Write a test for 200 and 404
This part was a big challenge for me. I read, and re-read, and re-read the docs and looked at multiple repos in order to try and wrap my head around what was needed to accomplish this step. I knew I needed to mock up a HTTP request but doing so was confusing. It didn’t help that the only example I could find were all using request to do there requests where I was using urllib3. After many iterations I came up with this
@mock.patch("src.checkFile.checkFile.headRequest")
def test_headRequest_200(mock_headRequest):
link = "http://google.com"
mock_headRequest.return_value = {"url": link, "status": 200, "secured": False}
cF = checkFile(args)
assert cF.headRequest(link) == {
"url": "http://google.com",
"status": 200,
"secured": False,
}
Now it looks ok, and it passes, but it turns out it’s still wrong. More on that later…
Step 4 – Do you have coverage?
OK, I got a few tests written, time to figure out how much of my program is being covered by them. After looking around a bit it was pretty apparent that pytest-cov was the coverage tool I should use with pytest. After an install with pip I was only a few wrong bash commands away from being able to run my coverage.
*Insert picture of initial coverage results that I forgot to take.*
Here’s a pic of my final coverage results.
Step 5 – Adding CI
Continuous integration will allow us to ensure that when a contributor submits a PR that there are no alterations that break a test. If something does break a test it will not be merged into the master branch.
To accomplish this we use the handy GitHub Actions feature. This allows us to set up CI from the GitHub website into any repository of our choosing. Since I’m using Python I naturally decided to install their pre-made Python Workflow. After I clicked on the button a folder was created in the root of the repository that had a file in it. The default file seemed so I made a PR to add it into master. As I checked out the PR I noticed that the CI checks were already happening, and a couple failed. I changed the defaults a bit to not include older versions of python and that made it work.
Step 6 – Test the CI with some real tests
Ok, now the real test to test the tests. I wanted to add some coverage to a file that had 0% on it so I figured that would be a good spot to add another test to make a PR with. The file in question was the file that called the class that ran everything. It sat in the root directory while I had the other files located in root/src/. This turned out to be a major issue that caused me a lot of head scratching and frustration. The coles notes version of it is that I’m relatively new to Python and still learning about it as I go. File structure is more important then I though in Python and you need __init__ files in order to import things. pytest will also look around for the first directory `without` and init file and use that as the root. After I figured it out I helped someone else with the issue, as well as fix the same problem my partner for this lab had.
Still not sure why we needed to wrap the imports in a try statement but hey, it works…
Now that I can finally import checkThatLink into a test file I’m off to the races. I wanted to test out the argparser portion of the file to ensure all the flags where storing the variables properly. I came across an example of how to do it eralier from this really cool repo that I just gotta share their gif of.
Ok, back to business. I started off by testing it out with only one set of parameters. After getting that to work I added the rest of the parameters and made some adjustments to get them all to pass.
@pytest.mark.parametrize(
"argv, result",
[
([path, "-s"], _args(secureHttp=True)),
([path, "-j"], _args(json=True)),
([path, "-a"], _args(all=True)),
([path, "-g"], _args(good=True)),
([path, "-b"], _args(bad=True)),
([path, "-i", igpath], _args(ignoreFile=igpath)),
([path, "-t"], _args(telescope=True)),
],
)
def test_setupArgs(argv, result):
assert vars(cl.setupArgs(argv)) == result
Time to test out the CI. I did this my making sure my test originally passes, then break it, and then fix the break.
I the updated my contributing documentation to include how to run the tests.
Step 7 – Partner up
The last step of this lab is to find a partner and add some tests to each others repo. I updated the submission file with the link the my GitHub Actions page and then indicated that I was looking for a partner in the part where you are to link a PR. An hour later Roger messaged me on Slack indicating he would like to work with me.
Roger did a great job and even fixed how I was using Mock to now work properly. I now understand what I was doing wrong earlier! I needed to mock the `manager` from urllib3 instead of mocking my entire function. The only thing I requested he change was the names of the functions to better represent what they tested.
I asked Roger if I could refactor his argparser so that I could write some tests for it. After getting the thumbs up I had to re format his file structure like I did with mine because there was the same issue of having a file in a higher level than the tests. I then refactored the parser function out and wrote the tests for it. All the changes can be found here.
Tim's Blog for DPS 909 