API Testing Tools: The Ultimate Guide (20+ Eye-Catching Reviews)
It's not a secret that choosing the right API Testing Tools can be overwhelming. And you are desperately looking for SoapUI Alternatives, I've been there too.
GUI Testing can be horribly slow, a nightmare to maintain and requires a permanent QA Engineer. It's no wonder more and more Devops oriented companies (who aim to unify software development and testing) tend to favor API Testing.
API (which means Application Programming Interface) Testing has proven to be faster and more reliable. Plus, developers can write tests too!
Looking for an ultimate guide to make your choice? Here we have made visual reviews of more than 20 Rest / Soap Testing software in one single place. No more web crawling to get an insight on all the tools!
Ready for some action? Let's go!
API Testing¶
What Is An API¶
An API is:
In computer programming, an application programming interface (API) is a set of subroutine definitions, protocols, and tools for building application software.
When it comes to APIs exposed on the web through HTTP servers, it's basically endpoints with different paths (URLs) each exposing its own logic.
What Is Needed to Test an API¶
We're talking here about Test Automation on HTTP Rest APIs.
Basically, being able to automate testing an API requires the following features:
- Support sending GET, POST, PUT, PATCH, DELETE (all Http methods) requests,
- Support injecting variables into requests coming from various sources (plain text, csv files etc.),
- Be able to extract data from servers responses and put them into variables for later use,
- Run a complex sequence of requests and responses with some logic (for/while loops, if/else conditions etc.)
- Mimic as closely as possible to interactions a web browser performs with the API.
Typically, those needs are almost the same as the one required by load testing. Except here we deal with functional testing instead.
Devops API Testing Tools¶
Devops tools are mostly targeted at developers. These tools require coding knowledge in various languages like Java or Javascript.
Citrus Framework¶
Citrus Framework is an Automated integration tests for message protocols and data formats including: HTTP REST, JMS, TCP/IP, SOAP, FTP, SSH, XML, JSON and more.
In a typical test scenario the system under test is deployed on some application server and interacts with Citrus over various message transports. During the test run Citrus is able to act on both sides as client and/or server simulating request/response messages.
With each test step you can validate the exchanged messages with expected control data. The test is fully automated and repeatable, so you can easily add the integration tests to your continuous build.
Key Features:
- Interface mocks and simulators: client and server side,
- Test automation: repeatable integration tests within your continuous build,
- Message transport connectivity: HTTP, JMS, TCP/IP, REST, SOAP, XML, JSON, and more,
- Validation: message header and body assertions.
Karate DSL¶
Introduction¶
Key features:
- Is based on Cucumber,
- Runs using Java code (needs a Java IDE),
- Run requests on Rest endpoints and check the expected response,
- Supports XML and SOAP Endpoints,
- Has API mocking / test-double capabilities.
see Karate DSL Features to see them all. There are 25+ unique features!
What Users like:
- Test reports are rich and provides insightful information.
What Users don't like:
- Requires some coding knowledge (which can be difficult for a QA, but okay for Devops).
Karate by Intuit enables you to script a sequence of calls to any kind of web-service and assert that the responses are as expected. It makes it really easy to build complex request payloads, traverse data within the responses, and chain data from responses into the next request. Karate's payload validation engine can perform a 'smart compare' of two JSON or XML documents without being affected by white-space or the order in which data-elements actually appear, and you can opt to ignore fields that you choose.
Since Karate is built on top of Cucumber-JVM, you can run tests and generate reports like any standard Java project. But instead of Java - you write tests in a language designed to make dealing with HTTP, JSON or XML - simple.
As mentioned above, most CI tools would be able to process the JUnit XML output of the parallel runner and determine the status of the build as well as generate reports.
The Karate Demo has a working example of the recommended parallel-runner set up. It also details how a third-party library can be easily used to generate some very nice-looking reports, from the JSON output of the parallel runner.
For example, here below is an actual report generated by the cucumber-reporting open-source library.
KarateDSL vs SoapUI¶
Our initial KarateDSL review clearly lacked some details about this tool, as suggested Peter Thomas, the Karate DSL creator:
@KarateDSL Karate tops the list for me.
— Testing Excellence (@TestingExcel) 26 mars 2018
Clearly, we would like to mention we have no affiliate link with SoapUI. The fact we indicate SoapUI as the potential winner on API Testing Tools only reflects our opinion.
Sadly, we don't have the expertise in our team to write a complete review of your tool, so we've just improved it with the details mentioned by Peter.
Mocha and Chai¶
We would like to thanks BigstickCarpet for his excellent guide SuperPower API Testing for most of the information provided below.
Mocha¶
Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple and fun.
To get started with Mocha:
$ npm install mocha
$ mkdir test
$ $EDITOR test/test.js # or open with your favorite editor
Here is an example test:
var assert = require('assert');
describe('Array', function() {
describe('#indexOf()', function() {
it('should return -1 when the value is not present', function() {
assert.equal([1,2,3].indexOf(4), -1);
});
});
});
Mocha: Tests from Command-Line¶
To Run mocha command-line (and executes all tests):
$ npm run mocha
Results are shown within the console. When results are green, everything is fine.
Results can be executed directly inside an IDE too.
Finally, results can be executed and viewed within the web browser. (Karma users should feel home here)
Chai¶
Chai is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any javascript testing framework.
To get started with Chai:
$ npm install chai
Here is an example test using should:
chai.should();
foo.should.be.a('string');
foo.should.equal('bar');
foo.should.have.lengthOf(3);
tea.should.have.property('flavors')
.with.lengthOf(3);
Combining Them¶
Mocha and Chai can be combined to run tests via a CLI interface or an IDE. The CLI works well for automated testing as well as CI/CD tools. There are also plugins for many popular IDEs and code editors, including VisualStudio, Atom, WebStorm and more.
Mobile and Browser-Testing¶
This is an important feature that only Mocha and Chai offers. There are several reasons why you would like to test this way:
- Testing API Same-Origin Policy and Content Security Policy,
- Check if CORS are implemented correctly,
- Check if the API is using Forbidden Header Names,
- The API cannot be proxied to record interactions.
Browsers and mobile devices have many limitations and restrictions in place for security, privacy, and latency reasons, which your API may need to account for.
Running real-world tests is always one of the best options since it's the closest to production.
Karma Runner or Nightwatch can be used to automate cross-browser testing.
Rest Assured¶
Testing and validating REST services in Java is harder than in dynamic languages such as Ruby and Groovy. REST Assured brings the simplicity of using these languages into the Java domain.
Rest Assured is a Java testing library to perform rest http calls to a Rest API and perform checks on server responses. The project sources are available on GitHub.
Here's an example of how to make a GET request and validate the JSON or XML response:
get("/lotto").then().assertThat().body("lotto.lottoId", equalTo(5));
Get and verify all winner ids:
get("/lotto").then().assertThat().body("lotto.winners.winnerId", hasItems(23, 54));
Key Features:
- XML and JSON response validation: parse XML and Json responses easily,
- Programming using Java Language,
- Supports various authentication methods.
What Users like:
- Java is a programming language which is widely used in a variety of tools,
- Good Documentation,
- Supports Behavior Driven Development.
- Developer friendly.
What Users don't like:
- Requires programming knowledge,
- Not QA Engineer friendly (at least those who don't program),
- Probably not the best Java lib for Rest API Testing (see alternatives below).
RestSharp¶
If you only have a small number of one-off requests to make to an API, you can use RestSharp like so:
using RestSharp;
using RestSharp.Authenticators;
var client = new RestClient();
client.BaseUrl = new Uri("http://twitter.com");
client.Authenticator = new HttpBasicAuthenticator("username", "password");
var request = new RestRequest();
request.Resource = "statuses/friends_timeline.xml";
IRestResponse response = client.Execute(request);
Key Features:
- Supports various Microsoft protocols including .NET 3.5+, Silverlight 5, Windows Phone 8, Mono, MonoTouch, Mono for Android,
- Easy setup using NuGet,
- All standard HTTP Methods supported (GET, POST, PUT, PATCH etc.).
Retrofit¶
Retrofit turns your HTTP API into a Java interface. It's a Java library which allows to write unit tests to automate testing your Rest API.
public interface GitHubService {
@GET("users/{user}/repos")
Call<List<Repo>> listRepos(@Path("user") String user);
}
The Retrofit class generates an implementation of the GitHubService interface.
Retrofit retrofit = new Retrofit.Builder()
.baseUrl("https://api.github.com/")
.build();
GitHubService service = retrofit.create(GitHubService.class);
Each Call from the created GitHubService can make a synchronous or asynchronous HTTP request to the remote webserver.
Call<List<Repo>> repos = service.listRepos("OctoPerf");
It supports all standard HTTP methods including multipart post requests.
Key Features:
- Programming using Java Language,
- Supports various authentication methods.
What Users like:
- Java is a programming language which is widely used in a variety of tools,
- Good Documentation,
- Developer friendly,
- We use it at OctoPerf to test our API!
What Users don't like:
- Requires programming knowledge,
- Not QA Engineer friendly (at least those who don't program).
- Lacks response assertions. (but Json/XML objects can be mapped to Java Objects)
GUI API Testing Tools¶
To be honest, after reviewing all the API Testing Tools below, it's quite clear SoapUI is significantly better than any other tool available on the market (and for free). If you can bear the dated UI which is not especially user-friendly, you're good to go!
Why providing a list of alternative tools then? Because you may:
- Be dissatisfied with SoapUI (not to mention again its UI),
- You have very simple needs that simpler tool can fulfill,
- Or Obiwan told you it's bad to rely on a single tool.
It's not a surprise to hear that QA and developers hate testing their APIs.
Whatever reason, we feel like it's always worth to mention alternative api testing tools doing best effort to provide alternatives with slick and modern interfaces.
Fiddler¶
Fiddler is a web debugging tool which allows to capture and replay https requests very easily.
Fiddler is designed to be extensible. APITest is definitely an extension worth to try: it enhances Fiddler to validate the behavior of web APIs.
At OctoPerf, our clients use Fiddler to capture the requests and responses during a recording.
Key Features:
- Capture Http Requests through a web proxy,
- Export captures as HAR,
What users like:
- Repeat and execute captured http requests at ease,
- Search function is great to find requests matching certain criterias,
- Give great insight on the http requests pipeline,
- Free to use.
What users don't like:
- No very user friendly for non-tech folks. Requires a bit of knowledge about how the HTTP protocol works.
- closed-source.
HttpMaster Express¶
HttpMaster is a tool to automate API endpoint testing. It allows to perform functional testing on HTTP APIs including Rest endpoints. SOAP has limited support.
Key Features:
- Advanced Support for API Applications and Web Services: Take advantage of the exceptional support for testing web APIs and RESTful web services,
- Universal Tool for Testing Any Type of Web Application: Apply product flexibility and broad feature set to test websites and other http-based applications,
- Practical and Easy to Use: Enjoy elegant and responsive user interface, easy test management, codeless tests, and more.
What Users like:
- SOAP Support, (only on Http transport)
- Clean user interface.
What Users don't like:
- No WSDL parsing support,
- Runs only on Windows,
- Newer projects generally cannot be opened with older HttpMaster versions.
Insomnia¶
Insomnia is a powerful HTTP toolbelt, in one intuitive app. Free and open source on Mac, Windows, and Linux. Insomnia is a cross-platform REST client, built on top of Electron.
The project source code is available on GitHub.
Wow, just wow! Just one word: beautiful!
Key Features:
- Create HTTP requests: Specify URL, payload, headers, and authorization all in one place. Then just hit send,
- Get all the details on every response: View status code, body, headers, cookies, and more,
- Create workspaces or folders, drag-and-drop requests, and easily import and export your data.
What users like:
- Slick, modern interface with minimal fuss,
- Efficient: does the job well,
- Easy to install (Chrome Extension).
What users don't like:
- Nothing found yet...
You can read Insomnia user reviews on the Chrome store.
This REST client has really a clean interface. The emphasis on a clean and efficient UI is evident.
Postman¶
Introduction¶
Key features:
- Works great for automated testing,
- OS Independent: runs on any machine (including Windows, Linux and MAC),
- Supports importing various formats like Swagger or plain text urls,
- Has a wonderful UI to design tests using point-and-click.
What Users like:
- Very nice graphical interface,
- Can setup multiple environments (dev, prod etc.),
- Supports a wide variety of Rest Calls.
What Users don't like:
- Interface can be improved (slow),
- Some features like sharing and collaboration are confusing,
- Collections ordering is somehow not working as expected,
- WYSIWYG editor could be improved.
Postman is a software that helps developers and companies create API workflow, it is efficient on Chrome to test, develop and document APIs and create complex requests, go back in time and view results in a beautiful way.
Postman Makes API Development Simple. Developers use Postman to build modern software for the API-first world.
Postman is used by 5 million developers and more than 100,000 companies to access 130 million APIs every month.
Some key information they provide:
- The only complete API development environment,
- Integrated tools for every stage of the API lifecycle,
- Plans for everyone from individuals to enterprises,
Postman has the following features:
- Postman Collections: An executable API description format. Run requests, test & debug, create automated tests, and mock, document & monitor an API.
- Postman Workspaces: Powerful collaboration spaces for teams of any size. Share collections, set permissions, and manage participation in multiple workspaces.
- Built-in Tools: Everything a developer needs to work with APIs. Design & mock, debug, test automation, documentation, monitor, and publish.
Postman tests are written in JavaScript using a fluent development API. This makes the tool Devops friendly, but makes the learning curve a little bit steep.
Isolated Testing¶
You can open each API request one-by-one in Postman's request builder. It's the preferred way to view and run single isolated tests.
If you click the Send button in the request builder, it will send the request and run the script. Results are displayed at the bottom of the screen.
Successful tests are green, and failed ones are red.
Collection Runner¶
Postman Collections can be run within the Postman app using the collection runner, from the command line using Postman’s Newman tool, or on scheduled intervals using Postman Monitors.
Running a collection is useful when you want to automate API testing.
Command-Line Runner¶
Newman is a command line collection runner for Postman.
Running tests via CLI is preferred when you want to automate tests using a CI/CD pipeline (like Jenkins)
Results have high contrast which make them very easy to read.
RESTLet¶
Introduction¶
Restlet is a free Chrome extension which is pretty easy to use to test Rest APIs.
According to their website:
Restlet Framework helps Java developers build better web APIs that follow the REST architecture style. Adopted and supported by a large community of Java developers, Restlet Framework benefits from numerous resources available all over the Internet. Fully open source, it is freely downloadable and can be used under the terms of the Apache Software License.
Graphical Interface¶
RESTLet features a GUI directly embedded into Google Chrome. It's as easy as other tools like Postman to design Http Requests and assert server responses.
Expression Builder¶
RestLet Expressions are structured strings that are parsed and evaluated by Restlet Client.
You can use them in all those places:
- Request URL,
- Headers (name and value),
- Query parameters (name and value),
- Form parameters (name and value),
- Request body (of type text),
- Assertions values.
Those are extremely helpful to extract content from a server response for later reuse in subsequent requests (like a session id).
This GUI Expression Builder makes it extremely easy to extract those variables. We feel like it's worth to mention because other tools are so sophisticated on this point.
GUI Runner¶
As with Postman, there is a nice UI which lets you run all the tests by clicking on the Play button. Again, green ticks indicate tests that ran successfully.
Command-Line Runner¶
Running Restlet tests from the command line requires Maven and the Java JDK.
Then, you'll need to create a pom.xml file and get license key from Restlet.
To run the test:
$ cd /path/to/tests
$ mvn test
As Maven perfectly well integrates with most CI/CD platforms, it's easy to automate tests using RestLet.
SoapUI¶
SoapUI is an API testing software edited by SmartBear.
SoapUI is a tool for testing Web Services; these can be the SOAP Web Services as well RESTful Web Services or HTTP based services. SoapUI is an Open Source and completely free tool.
Key features:
- API Functional Testing: Functionality doesn't stop at the GUI level. Learn about how to make sure your API functions as intended, every time,
- API Performance Testing: Performance is the bottom line of your API driven business. Load Test Virtual Users for your API and see how well it performs,
- API Security: Worried about third party APIs, public APIs, or vulnerable internal APIs? Learn how to safely secure them all,
- API Mocking: A properly designed, built and tested API can take valuable resources from your business. Save time and money with virtualization,
- Data-Driven Testing: Test faster and smarter with data-driven testing, increasing your API testing coverage overnight,
- Test Reporting: Need metrics, statistics and other testing data? Get detailed, comprehensive API test reporting to improve testing and measure success.
What Users like:
- Works great with both SOAP Web Services and Json Rest Services,
- Creates sample tests by providing a SOAP WSDL,
- Works great on both Windows and MacOS (Linux is also supported as it's a Java Application),
- Has portable version which can be kept on a USB Key,
- Easy web service mocking,
- Free version available!
What Users don't like:
- UI is dated and sometimes buggy,
- HiDPI displays seem not well supported (like on the latest MacBook Pro),
- Windows App UI works better than the MacOS version,
- Documentation can be improved,
- Requires a lot of memory to run smoothly (several GBs on big projects).
- Generally speaking, users find the UI awful to use.
StopLight¶
Introduction¶
StopLight is a commercial tool. StopLight provides a suite of products that cover the entire pre-production API lifecycle. Here is an overview of the platform:
As you can see, StopLight is not just an API Testing Tool, but a complete suite to Test, Document and ship Rest APIs.
The API Designer is in my opinion a great piece of software: it makes testing an API a breeze with its clean and slick UI.
Targeting Quality Engineers, the API Testing interface is designed to quickly stop issues in a bunch of api tests. You can run all of them by clicking the Run Scenario button.
As other tools, the CLI interface allows an easy integration with CI/CD pipelines.
OpenAPI Support¶
Stoplight is built on top of OpenAPI, so it can leverage the JSON Schemas in your API spec to generate verifications. Tests verifying your Json Schemas are automatically generated by the tool. That's cool!
Test Coverage¶
StopLight uses the information provided about your API definitions to know if the endpoint has been tested or not. That's how they generate coverage reports. Thanks to this information, you know which API Endpoints need to be tested. Lack of testing on newly added endpoints can be pinpointed too.
It should not be confused with code coverage (lines of code being tested server-side): it's API Coverage, equivalent to percent of exposed API Endpoints being tested.
UI Runner¶
Tests can be run using the CLI (Command-Line Interface) and integrated into external tools (mostly for test automation).
CLI Runner¶
Simply click on Run Collection button, and it will run all the tests within the collection.
Our Opinion¶
StopLight puts the emphasis on connecting QA, Devops and Writer teams together. The tool covers pretty much everything from designing an API to putting it into production.
We really like the polished UI and ease of use.
Storm¶
STORM is a free and open source tool for testing web services.
It is written mostly in F#. (I love this language!)
STORM allows you to:
- Test web services written using any technology (.NET , Java, etc.)
- Dynamically invoke web service methods even those that have input parameters of complex data types,
- Save development time and money. Creating throw-away test client apps just to test the web service is just too wasteful,
- Test multiple web services from within one UI,
- Edit/Manipulate the raw soap requests.
This tool is pretty old (latest release was in 2008), but still used by some people because it does the job well.
WebInject¶
Please don't focus on the poorly designed website, WebInject does the job too!
As explained on WebInject website:
WebInject is a free tool for automated testing of web applications and web services. It can be used to test individual system components that have HTTP interfaces (JSP, ASP, CGI, PHP, AJAX, Servlets, HTML Forms, XML/SOAP Web Services, REST, etc), and can be used as a test harness to create a suite of [HTTP level] automated functional, acceptance, and regression tests. A test harness allows you to run many test cases and collect/report your results. WebInject offers real-time results display and may also be used for monitoring system response times.
Wizdler¶
Parses the WSDL files and generates SOAP messages for you.
Recognizes WSDL information on the page to show you the available services and operations. Click the operation to generate the SOAP requests and view the response. By clicking the service, you can download WSDL and external XSD files in single ZIP file.
It’s really lightweight and makes it easy to quickly test a SOAP web service.
Key Features:
- SOAP WSDL Parsing,
- Capable of building SOAP XML Requests.
What Users like:
- UI is simple and clean,
- Perform tests quickly with the WSDL parser.
What Users don't like:
- Supports only SOAP Web Services,
- Very limited functionalities.
As Google plans to shut down Chrome Apps, it's possible this app will disappear in a near future.
Other Tools¶
The following tools have not been thoroughly reviewed but we would like to mention them. Just in case you're still looking for an alternative Api Testing Tool.
AirBorne¶
AirBorne: RSpec driven API testing framework.
ApiGee¶
ApiGee is a cross-cloud API testing tool.
Apiary¶
Apiary: Apiary allows monitoring the API during the design phase by capturing both request and response.
APIScience¶
APIScience: API science allows monitoring the health, availability, and performance of web APIs.
HippieSwagger¶
Hippie Swagger: hippie-swagger is a tool for testing RESTful APIs. In addition to validating api behavior, it will fail tests when swagger documentation is missing or inaccurate,
Ping API¶
Ping API is an Automated API Testing and Monitoring tool.
PyrestTest¶
PyrestTest: A REST testing and API micro-benchmarking tool.
REST Console¶
Rest Console: an HTTP Request Visualizer and Constructor tool, helps developers build, debug and test RESTful APIs
SOAPSonar¶
SOAPSonar: API Testing for SOAP, XML, REST, and JSON APIs.