Site icon TheWindowsUpdate.com

Ensuring Code Changes do not Break Microservice REST APIs, as Part of Pull Request Validation

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Tech Community.

Typically Integrated microservice APIs are tested quite late in the Application lifecycle, where the application code which has already been merged into the main branch, is then deployed to an environment, after which the API tests are executed. Detection of failures at this stage means that your main branch is already in an unclean / un deployable state

In this post we look at a Webvalidate based test harness which in many cases would enable us to validate that the changes introduced in a Pull Request (PR) do not break the Microservice API.

 

What we will be building?

 

Our objective is to shift the microservice API tests left, and execute them as a part of PR validation itself, using Azure Pipelines.  To enable this for a sample RESTful Microservice (backed by a document database) we need to build a PR pipeline / test harness which allows us to achieve these requirements:

 

Sample Microservice

For this post we will be looking at the Books service as the Microservice for which the APIs need to be tested during PR validation. The initial code for Books API has been taken from the dotnet samples and tweaked for purpose of this post.

 

It is a simple API supporting basic CRUD operations for Books (Add, Delete , Put, Get one, List etc). The datastore which this microservice relies on is a Mongodb collection.

 

Application configurations like mongodb connection string, database name and collection name are injected using environment variables. We later look at how these are injected as part of the PR Validation pipeline.

 

With a few minor modifications (which will be covered later in this post), the test harness can also work if the application fetches configurations using Default Azure Credentials, Azure App Config and Azure Key Vault.

 

What is webvalidate?

The microsoft/webvalidate tool enables us to define tests cases and validations as simple JSON objects as shown below:

 

 

{ "path": "/api/books/60e304aac7b8d60001b2d3cd", "verb": "Get", "tag": "GetNonExistentBook", "failOnValidationError": true, "validation": { "statusCode": 404, "contentType": "application/problem+json;" } }

 

 

In this test case with tag "GetNonExistentBook" we assert that a request to get a book with id "60e304aac7b8d60001b2d3cd" would return a HTTP status code of 404 as book with that Id is not expected to exist. Complex validations as shown in the webvalidate samples can also be easily achieved. 

 

Code supporting this Post

The working code referenced in this post is available at the api-test-harness-webv github Repo This includes the tweaked api application code as well as the dockerfile, docker-compose file, test harness scripts, test data, and webvalidate test cases.

 

Overview of the key folder and files in this repository:

 

Digging Deeper into the key files

 

Mongodb collection test data - /TestFiles/ApiTests/Data/BooksTestData.json

 

[ { "_id": { "$oid": "60e2e8fe7ed72f0001bf3a41" }, "Name": "The Go Programming Language", "Price": 20, "Category": "Computer Programming", "Author": "Alan Donovan" }, { "_id": { "$oid": "60e2e8fe7ed72f0001bf3a46" }, "Name": "Design Patterns", "Price": 54.93, "Category": "Computers", "Author": "Ralph Johnson" } ]

 

 

Webvalidate Test Cases - /TestFiles/ApiTests/TestCases/BooksTestCases.json

 

{ "requests": [ { "path": "/api/books", "verb": "POST", "tag": "CreateBookValidRequest", "failOnValidationError": true, "body": "{\"Name\":\"Kubernetes Up and Running\",\r\n\"Price\":25,\r\n\"Category\":\"Computer Programming\",\r\n\"Author\":\"Adam Barr\"\r\n}", "contentMediaType": "application/json-patch+json", "validation": { "statusCode": 201, "contentType": "application/json", "jsonObject": [ { "field": "Id" }, { "field": "Name", "value": "Kubernetes Up and Running" }, { "field": "Price", "value": 25 } ] } }, { "path": "/api/books", "verb": "POST", "tag": "CreateBookInvalidPrice", "failOnValidationError": true, "body": "{\"Name\":\"Kubernetes Up and Running\",\r\n\"Price\":\"twenty five\",\r\n\"Category\":\"Computer Programming\",\r\n\"Author\":\"Adam Barr\"\r\n}", "contentMediaType": "application/json-patch+json", "validation": { "statusCode": 400, "contentType": "application/problem+json;", "jsonObject": [ { "field": "errors", "validation": { "jsonObject": [ { "field": "Price" } ] } } ] } }, { "path": "/api/books/60e304aac7b8d60001b2d3cd", "verb": "Get", "tag": "GetNonExistentBook", "failOnValidationError": true, "validation": { "statusCode": 404, "contentType": "application/problem+json;" } }, { "path": "/api/books/60e2e8fe7ed72f0001bf3a41", "verb": "Get", "tag": "GetExistingBook", "failOnValidationError": true, "validation": { "statusCode": 200, "contentType": "application/json", "exactMatch": "{\"Id\":\"60e2e8fe7ed72f0001bf3a41\",\"Name\":\"The Go Programming Language\",\"Price\":20.0,\"Category\":\"Computer Programming\",\"Author\":\"Alan Donovan\"}" } } ] }

 

 

PR Validation Azure Pipeline Yaml file - /ApiTestsAzurePipelines.yaml

 

. . steps: - script: | cd $(System.DefaultWorkingDirectory) docker-compose -f BooksApi/dockerComposeBooksApiTest.yml build --no-cache --build-arg ENVIRONMENT=local docker-compose -f BooksApi/dockerComposeBooksApiTest.yml up --exit-code-from webv | tee $(System.DefaultWorkingDirectory)/dc.log displayName: "Execute API Tests" - script: | # Pass parameters: path to docker-compose log file, path to output Junit file, and path to scripts directory bash $(System.DefaultWorkingDirectory)/TestFiles/scripts/webvToJunit.sh $(System.DefaultWorkingDirectory)/dc.log $(System.DefaultWorkingDirectory)/junit.xml $(System.DefaultWorkingDirectory)/TestFiles/scripts displayName: "Convert Test Execution Log output to JUnit Format" - task: PublishTestResults@2 displayName: 'Validate and Publish Component Test Results' inputs: testResultsFormat: JUnit testResultsFiles: 'junit.xml' searchFolder: $(System.DefaultWorkingDirectory) testRunTitle: 'webapitestrestults' failTaskOnFailedTests: true

 

There are 3 main steps in this PR validation Azure Pipeline file:

 

Docker Compose API Test Harness - /BooksApi/dockerComposeBooksApiTest.yml

 

version: '3' services: booksapi: build: context: . dockerfile: ./Dockerfile ports: - '5011:80' networks: - books # volumes: # - ${HOME}/.azure:/root/.azure environment: - BookstoreDatabaseSettings__ConnectionString=mongodb://mongo:27017 - BookstoreDatabaseSettings__DatabaseName=BookstoreDb - BookstoreDatabaseSettings__BooksCollectionName=Books mongo: container_name: books.mongo image: mongo:4.4 networks: - books mongo-import: image: mongo:4.4 depends_on: - mongo volumes: - ../TestFiles:/testFiles/ - ../TestFiles/scripts/import.sh:/command/import.sh - ../TestFiles/scripts/index.js:/command/index.js networks: - books environment: - MONGO_URI=mongodb://mongo:27017 - MONGO_DB=BookstoreDb - MONGO_COLL=Books - TEST_TYPE=ApiTests entrypoint: /command/import.sh webv: image: retaildevcrew/webvalidate@sha256:183228cb62915e7ecac72fa0746fed4f7127a546428229291e6eeb202b2a5973 depends_on: - mongo - booksapi volumes: - ../TestFiles:/testFiles/ - ../TestFiles/scripts/executeTests.sh:/command/executeTests.sh networks: - books environment: - TEST_TYPE=ApiTests - TEST_SVC_ENDPOINT=http://booksapi - TEST_DATA_LOAD_DELAY=25 entrypoint: ["/bin/sh","/command/executeTests.sh"] networks: books:

 

 

The Test harness in action

 

Abstract from execution stage log

 

. Successfully built b312635a0c59 Successfully tagged booksapi_booksapi:latest . . Creating books.mongo ... Creating booksapi_booksapi_1 ... Creating books.mongo ... done Creating booksapi_mongo-import_1 ... Creating booksapi_booksapi_1 ... done Creating booksapi_webv_1 ... Creating booksapi_mongo-import_1 ... done Creating booksapi_webv_1 ... done Attaching to books.mongo, booksapi_booksapi_1, booksapi_mongo-import_1, booksapi_webv_1 . . mongo-import_1 | 2021-07-12T07:23:29.882+0000 2 document(s) imported successfully. 0 document(s) failed to import. . . webv_1 | {"date":"2021-07-12T07:23:55.3355122Z","verb":"POST","server":"http://booksapi","statusCode":201,"failed":false,"validated":true,"correlationVector":"MtKp\u002BU7CpE6VAJLRF/kGlQ.0","errorCount":0,"duration":550,"contentLength":136,"category":"","tag":"CreateBookValidRequest","path":"/api/books","errors":[]} . . Stopping booksapi_mongo-import_1 ... Stopping booksapi_booksapi_1 ... Stopping books.mongo ... Stopping booksapi_booksapi_1 ... done Stopping booksapi_mongo-import_1 ... done Stopping books.mongo ... done Aborting on container exit...

 

 

Failure Scenario

Before testing the harness, we need to create an Azure Pipeline using out PR Validation Pipeline yaml file, and then configure this pipeline to execute as part of main branch build validation. For simplicity to cause the API test to fail let us modify the CreateBookValidRequest webvalidate test case to expect a Price of 26 in the response instead of 25, as shown in the commit

 

Once a Pull request is created, we should see the PR validation build kicking in, and then failing in a couple of minutes as shown:

 

Next we drill down into the pipeline and choose the Tests tab, which gives us an overview of the test execution. Where we see that 1 of the 4 tests has failed:

 

On clicking the failed test, the error details window shows us the reason of the error (Actual Price "25", Expected Price "26")

 

Since this is a required check for the PR, merging of the PR is blocked:

 

Success Scenario

As we can see below if the PR validation is successful, the changes can be approved and merged:

 

Limitations of this Approach and Other Considerations

There are a few limitations of this simplistic API testing during PR validation approach:

 

Thanks for reading this post. I hope you liked it. Please feel free to write your comments and views about the same over here or at @manisbindra

 

Exit mobile version