A well-designed API is likely to be consumed by more developers, leading to a high adoption rate. Features of a well-designed API:
- Ease of use: Developers should be able to memorize its services and the resources it provides quickly.
- Complete and concise: An API should allow developers to build fully fledged applications using the data you expose. Completeness, in reality, is a process that takes time and is rarely achieved in the first version of an API. The developers who use your API will have feedback for improving it, and addressing this feedback will make integrating with it easier.
- Future-proof: It should be easy to extend your API in the future.
APIs generally fall into two categories: OpenAPIs – for use by the public, and private APIs – for internal use by the organization that developed them.
APIs can also be further classified according to the architecture and tech stack used to build them: RESTful, GraphQL, RPC, gRPC, or SOAP.
Regardless of the architecture of an API, there are conventional best practices to follow when designing them. This reference guide aims to explore these in terms of design patterns, architecture types, security measures, and testing techniques. We’ll finish off by taking a look at the OpenAPI Specification.
Good API Design Properties
It’s important to take the time to think about the design of your API before diving into building it, taking into consideration the principles of performance, ease of integration, and ease of use. A poorly designed API won’t meet its users’ needs, leading to a low adoption rate, which will waste the effort you’ve put into building it in the first place.
Let’s take a look at some of the common design patterns that are guaranteed to increase your chances of developing an API with a high adoption rate.
Ensure JSON Request and Responses
APIs should accept payload requests and send responses in JSON (JavaScript Object Notation) format. JSON is the standard for transferring data, as it is easily readable and many present-day technologies can decode it without the need for extra libraries or packages. To ensure that an API responds with JSON and that the client interprets it as such, set the Content-Type header in the response to application/json.
Encrypt Data
It is common for APIs to send and receive sensitive information like passwords, dates of birth, and other user profile data. Communication between a client and the server should therefore be private to protect this sensitive information. Using SSL/TLS to send data over secure channels when communicating with your API is a MUST.
Use Plural Nouns not Verbs in URLs
The endpoints or resource URIs should be based on a plural noun – usually the name of the resource being exposed – and not a verb. A common pitfall for using verbs is trying to depict which operation is being performed on the resource, for example http://localhost/create-order instead of http://localhost/orders. The first URI can only be used to create orders, while the second can be used to create, read, update, and delete orders when used with the appropriate HTTP method.
Table depicting good use of HTTP methods and plural nouns in URI design.
Table depicting bad use of HTTP methods and verbs in URI design.
Using verbs in URIs leads to the need to create multiple endpoints, which quickly complicates your API for no good reason. Rather, aim to use as many HTTP methods as you can on a single URI to cater for the different operations you might want to perform on the resource. This solution not only leads to cleaner code, but is also easier to maintain as you extend your API. Developers building clients to consume your API will also find the process of integrating with it more intuitive.
Include Filtering, Sorting, and Paging
In most queries, only a subset of a collection is required. But performing a simple GET request on a resource URI retrieves the whole collection. This is highly inefficient, as it wastes network bandwidth and processing power on the server hosting the API. It also leaves the client with the task of filtering the whole list retrieved from the data store.
Consider appending query parameters to the request, for example /orders?userid=25 to get only the orders belonging to the user with id=25.
In cases where the retrieved data is still large despite filtering it, the next good design principle would be to paginate it. When sending responses with paginated data, the response should include metadata which indicates the total number of resources available in the collection.
If the retrieved data also needs to be sorted, it’s better to do it on the server-side rather than the client. To this end, you can also add another query parameter to tell the API how you want to sort the data. An example of a request with filtering and sorting query parameters would be as follows: /orders?userid=25&&sort=ProductID.
Use HATEOAS
HATEOAS (Hypermedia as the Engine of Application State) is a REST principle that stipulates that responses from the server should contain all the necessary information to find all the resources related to the requested object. Clients should be able to navigate the entire set of resources without requiring prior knowledge of the URI scheme. To meet this constraint, responses should contain hyperlinks to the related resources and the operations that can be performed on each of them.
By keeping the actions that can be performed on the resources dynamic from the client’s perspective, HATEOAS allows the server to make URI changes as the API evolves without breaking client connections. Here is an example of a response that contains hyperlinks to the requested object’s related resources:
{“orderId”: 45, ”quantity”: 2, “links”: [{“rel”: customer, “href”: “https://dummy-link.com”, “action”: “GET”}, {“rel”: customer, “href”: “https://dummy-link.com/2”, “action”: “PUT”}]}
Include Bulk Endpoints
Most times, a client sends a request with the intent to only add a single record, but there are exceptions when a group of entries have to be created in the database in one go. In such cases, it would be inefficient to make an API call for each entry, as it would introduce a significant time delay depending on the number of entries to be made. While it is possible to scale the processing that happens server-side, it’s difficult to scale the networking that’s responsible for connecting the client and API. This is because each call needs to negotiate a protocol like TCP before data can be sent through securely, which introduces a time overhead.
To avoid this time delay, you can send all the data (entries) in a list or array in one API call. This implies that you have to design an endpoint specifically for handling bulk requests, since normal POST endpoints only expect one entry. Do this by looping through the list sent and performing the desired action on each one.
Manage Changes to Your API With Versioning
As the API evolves, future versions of the API should be able to add functionality independently of client applications. No version update should break applications that were previously consuming the API with no hiccups. In cases where new changes would otherwise break connections with existing clients, there are three options available to you to ensure already connected clients keep on working as expected.
These are URI versioning, query string versioning and header versioning. In URI versioning, you append the API version number to the endpoints with the new functionality. This way, old clients would keep on getting data at http://base-uri/customers while new clients would use this endpoint, http://base-uri/v2/customers. Query string versioning adds a version query parameter to the request, for example http://base-uri/customers?version=2 and header versioning adds the version number to the request header. Among the three, URI versioning is the least recommended as it leads to multiple URIs for the same resource, overcomplicating the API in the process.
Allow Partial Responses for Large Binary Files
Certain responses to GET requests may contain large binary fields for files and images. Such responses are prone to problems caused by unreliable and intermittent connections, since the payload will be huge. To overcome these problems, your API should allow for resources to be retrieved in chunks. Add this functionality by enabling your API to support the Accept-Ranges header for GET requests for large resources. If this header is present on a request, the responses can be partial, sending subsets of the requested resource, specified as a range of bytes.
Nest Resources
Group resources and endpoints that contain related information together. For example, if you want to get ratings for a particular product in an e-commerce store, you append the /ratings path to the end of the /products path. Using this example, you would get a product’s ratings by performing a GET request on this URI: /products/:productID/ratings. This nesting makes logical sense since the ratings are child objects of any given product.
Include Descriptions in Error Codes
When errors occur, they shouldn’t crash your API. Rather, the API should handle them and return appropriate error codes. The error codes should be accompanied by a brief description in the response body when a request fails. This eliminates confusion by giving the developer more information about what went wrong and accelerates the debugging process. Below is a table with some common HTTP error codes.
| Error Code | Meaning |
|---|---|
| 400 | Bad Request |
| 401 | Unauthorized |
| 403 | Forbidden |
| 404 | Not Found |
| 405 | Method Not Allowed |
| 429 | Too Many Requests |
| 500 | Internal Server Error |
| 501 | Not Implemented |
| 502 | Bad Gateway |
| 503 | Service Unavailable |
| 504 | Gateway Timeout |
In general, 4xx error codes signify a client error, while 5xx error codes signify a server error. Visit this link to see the full list of HTTP status codes, including informational, redirection, and success codes.
Cache Data
Consider caching data in the local memory cache to improve performance. This eliminates the need to query the database every time users request common data. There are many caching solutions available, like Redis, in-memory caching, and more.
Structure Response Data
Avoid meaningless nesting in responses, for example, nesting your “user” data in a data element. Aim to keep things as simple as you can.
{“data”: {“user”: {“userId”: bs32, “name”: “John”, “surname”: “Doe”}}} // Bad
{“user”: {“userId”: bs32, “name”: “John”, “surname”: “Doe”}} // Good
Use a Consistent Naming Convention
For clear and easy readability, you should pick a single naming convention and stick to it throughout the whole project. No particular naming convention is better than the other, but the goal here is to be consistent with your choice.
API Architecture Types
We’ll now look at the different API types and the use case each one excels in. Decisions in choosing an API architecture are generally guided by the type of data you want to send and receive and the performance requirements of the project.
REST APIs and GraphQL APIs
REST APIs and GraphQL APIs are the most popular API types at the time of writing this article. REST (Representational State Transfer) APIs are designed around resources that can be any object, data, or service that is accessible by the client. GraphQL APIs are based on the GraphQL query language. The specific nature of queries sent by GraphQL makes it desirable for use cases where performance is a top priority. Clients for both APIs communicate by exchanging representations of resources in JSON format.
The table below looks at some of the differences between the two APIs.
| REST APIs | GraphQL APIs |
|---|---|
| Server-driven | Client-driven |
| Resources are exposed via several endpoints | Resources are exposed via a single endpoint |
| Data transferred in JSON | Data transferred in JSON (with GZIP) |
| Sometimes need to make multiple calls, oftentimes retrieving unnecessary data in the process, thereby making it less efficient | GraphQL queries make it possible to retrieve the exact information a client requires in a single API call |
| Supports multiple languages (including Python, Java, Node.js) | It can only be implemented using the GraphQL query language |
| Large community that endorses the REST architecture, which greatly improves the chances of troubleshooting any problem | Growing community |
RPC and gRPC APIs
RPC (Remote Procedure Call) APIs entail the client sending a block of code, usually in binary format, that is to be executed by the server, and the server executing this code and sending back the result in the response. The most common RPC use case is when there is a need to run an action or procedure that doesn’t necessarily fit in the CRUD (Create, Read, Update, Delete) umbrella.
A modern open-source framework for RPC, gRPC (Google Remote Procedure Call) uses protocol buffers as its Interface Definition Language and its underlying message interchange format. It can be implemented in most of the popular programming languages of today and introduces a performance boost to normal RPC architecture. The connections established with gRPC are made efficient through its pluggable support for load balancing and tracing. The gRPC framework can also be applied in the last mile of distributed computing to connect devices, mobile applications, and browsers to backend services.
SOAP APIs
SOAP (Simple Object Access Protocol) APIs use an XML-based web services access protocol that was originally developed by Microsoft. They expose data via service interfaces like @WebService and have built-in error handling, which is particularly useful during the setup phase. Current trends in web development suggest that SOAP APIs are being phased out, as most developers find REST easier to implement. In addition, the XML used by SOAP makes for large payloads compared to the JSON-based payloads used in the REST architectural style. This difference in the sizes of the payloads leads to longer processing times in SOAP APIs, making REST the more efficient choice between the two.
Choosing an API Language and Framework
Choosing the language and framework to develop your API in will depend on your project’s requirements and your team’s strengths. Tight deadlines might mean you just want the quickest way to an MVP, and you’ll worry about optimizations in a later version, or performance may be your top priority from the get go.Let’s take a look at some of the options available to you to help you make that choice.
Python vs Golang vs JavaScript vs Java for API Development
Different languages have different pros and cons, and it is up to the developer to decide which one best suits the project. When choosing a language, you should think about:
- Functionality: Is the language capable of doing what you want without you having to look for workarounds?
- Performance: Does the language meet the performance needs for the project?
- Security: Does the language provide support to meet conventional API security measures?
- Scaling: Will the language scale well if your API becomes popular?
Python
Python is a high-level programming language with an easy-to-understand syntax, making it the language of choice if you want to get started quickly. Since Python has been around for some time, its frameworks, for example Django, are more mature, and Python documentation is comprehensive. There is also a considerable Python community, meaning developers are likely to find help with any problems they encounter in a timely fashion. On the downside, Python lacks type safety, which makes testing harder even with Python’s type annotations. It also has performance problems at scale, mainly because of GIL, which restricts multi-threading.
Golang
Golang is relatively new to the programming world but is quickly gaining popularity because of its high performance. It was developed by Google, and it has the best parallelism among today’s programming languages. Golang (Go) also supports cross-platform builds, which work very well on all operating systems. Golang is more modern, as it comes with built-in linting and style. These pros make it the best choice for huge teams that want to build products that run in parallel and scale well. Enforced linting and style ensure there’s only one correct way of writing Golang programs, which improves readability for the developers who work with it. Consider using Go if you’re starting a new project from scratch that has high performance and scaling needs.
As Golang is still relatively new on the scene, it’s harder to find experts who have experience with it and the available libraries aren’t as mature as Python libraries. You might have to build your own plugins, since the community is still small.
JavaScript / TypeScript
JavaScript is the most popular programming language in the world, so it’s easier to find developers who are experienced with it. JavaScript can be used in both the frontend and backend of applications, so collaboration and integration between frontend and backend teams is easier. Although JavaScript isn’t strongly typed, TypeScript fixes this by introducing variable types to the normal JavaScript syntax. Frontend developers working with JavaScript find it easier to transition into full stack by first using JavaScript in the backend, since it’d be a familiar syntax.
Consider using JavaScript if you’re a new web developer with little to no experience. Refrain from using JavaScript if you’re building a complex project with high performance needs, especially if you’ll need to scale it in the future. You will do well choosing Golang for such use cases.
Java
Java is a strongly typed language mainly used by enterprises that developed their applications over a decade ago. The JVM (Java Virtual Machine) works on nearly any hardware, meaning it’ll likely be around in the foreseeable future, although it has a low adoption rate among today’s startups since it isn’t modern. It still has good support from Oracle but is mostly found in legacy systems. Only consider using Java if you’re extending a project already built with Java. If you’re starting something new, try the more modern languages.
Django vs DRF vs Spring vs Gin vs Express vs Laravel vs TastyPie vs FastAPI vs Flask
Frameworks are tools that work with different libraries to make it possible to quickly build and deploy APIs by providing commonly used functionality as simple method calls. The popular frameworks for API development at the time of writing this article include Django (Python), Flask (Python), Spring (Java), and Laravel (PHP). Depending on the needs, priorities and language of your specific use case, some frameworks will be more suited than others for developing your API.
Django
The developers of Django dub it as “a framework for perfectionists with deadlines”. This is because of the rich set of built-in functions it comes with, making it possible to build full-fledged APIs quickly without installing additional libraries. Django is also secure and scales flexibly, making it a good choice for iterative projects that might get complex in the future.
DRF
Django Rest Framework (DRF) is a powerful toolkit for building web APIs with Django. To use DRF, you need to have installed Django first, then add it as a dependency to your Django project. DRF provides great built-in packages for OAuth and has serialization for both ORM and non-ORM data sources. This accelerates the development process as it allows developers to focus on code specific to the API’s functionality. DRF is highly recommended if you choose Django as your framework.
Gin
Gin is a Golang framework that offers a massive performance boost, up to 40 times faster than other web API frameworks. The performance increase is mainly due to the custom version of HttpRouter it uses and its good packet routing. We highly recommend using Gin when performance is a top priority for your API.
Express
Express is the standard server framework for Node.js. Once a team chooses Node.js as their programming language, it goes without saying that Express will be their preferred framework. Developers find Express easy to work with, and it provides a robust set of features common to API development while maintaining a lightweight, flexible configuration.
Flask
Flask is a lightweight Python minimalist framework for building APIs. It is easy to set up, making it a good choice if you want to quickly deploy something not too complex. Due to its lightweight configuration, it requires less server space in production compared to other heavy frameworks like Django.
Spring
The Spring framework is an all-in-one solution for Java applications of all types and sizes. It can be used to build microservices, and cloud and web applications, which REST APIs fall under. By removing much of the boilerplate code associated with web applications, Spring allows developers to streamline the development process of their APIs. Although Spring is a good framework, its language, Java, isn’t modern. You should therefore only consider using Spring for extending or maintaining a legacy codebase.
Laravel
Laravel is the most popular PHP web framework for developing APIs. It provides most of the basic functionality that every API needs out of the box, including unit testing, clean routing, and asynchronous queues for background jobs or long-running tasks. PHP isn’t a modern language, so we don’t recommend picking Laravel for your next project should you be starting from scratch. If, however, you’re working on a legacy system that was built on PHP, then Laravel is your best option compared to other PHP frameworks.
TastyPie
TastyPie is an API framework for Django. TastyPie is similar to Django Rest Framework in that it adds more basic functionalities to the ones provided by Django. Its code is more readable, and it’s a good choice for your project if your models are non-ORM. On the downside, TastyPie has no OAuth and is known for being harder to customize as a project gets bigger. We only recommend TastyPie for projects that have non-ORM models and aren’t too big.
FastAPI
FastAPI is a new, modern, high-performance API for Python that was released in 2018. It is faster than the older and more popular Flask framework as it is built on ASGI (Asynchronous Server Gateway Interface), which allows it to support asynchronous code. FastAPI is mainly suited for small- to medium-sized projects, as it doesn’t have plenty of features that support complex applications, like ORM. You will do well sticking to Django for larger projects.
API Security
Cyberattacks have been steadily increasing in the past decade and are likely to continue as more information is digitized. This trend means you cannot afford to release an online application with little to no security measures in place.
You should ensure that your API has a decent authentication system, like JWT or OAuth, to check whether requests are coming from your app users or the outside world. Avoid using basic authentication, as it sends information in an unencrypted format. If you opt to use JWT, make the token expiration as short as possible and use a complicated key for the token to reduce the chances of guessing the token through brute force. For OAuth, you should validate redirect_uri on the server side to allow only whitelisted URLs.
To avoid brute-force attacks on your API, you can implement throttling to limit the number of requests that can pass through in a given time interval. If the API you’re developing is for internal use, then you should only allow connections from a set of whitelisted IPs or hosts. For public APIs, where it is not possible to whitelist acceptable IPs or hosts, you can use HSTS (HTTP Strict Transport Security) together with SSL to avoid SSL stripping attacks. In SSL stripping attacks, hackers downgrade a client-server connection from the secure HTTPS version to the less-secure HTTP connection, which sets the stage for a classic man-in-the-middle attack. If this happens, attackers will be able to intercept the communication between the client and server, allowing them to steal sensitive information like payment details.
When designing endpoints, avoid mirroring your database structure in the URLs, as this could provide attackers information. Except for login and signup, each endpoint should check for an authentication token in the header to validate whether the request is coming from a genuine user. Avoid auto-incrementing user IDs when they sign up, and use UUID instead. User IDs shouldn’t be visible in the URL. For example, avoid URLs like users/5463/orders and use users/me/orders instead.
If the API you’re designing processes huge amounts of data, consider using workers and queues to process most of the data in the background. This enables you to send responses fast and avoid HTTP blocking. Examples of open-source software you can use for this functionality include RabbitMQ and Celery.
Another option for securing your API is Google’s Apigee, which you can use as a proxy layer. By adding this layer, you can filter requests before they reach your API and block all suspicious network traffic that won’t be coming from your whitelist of acceptable IP addresses. Google also has a free e-book you can download here that looks at good API design principles.
API Testing
API testing is a software testing technique that involves testing APIs directly to determine if they meet expectations for functionality, reliability, performance, and security. The procedure for all tests follows a general two-step process, during which requests are sent to the API and the responses validated against appropriate acceptance criteria. API tests can be classified into various groups, namely unit testing, integration testing, load testing, and security testing. We’ll explore each testing type to see its benefits and the assurance it brings if the test is passed.
Integration testing deals with testing the client-server connection and the server-database connection. It is usually the first test carried out, as all other tests depend on successful connections between the presentation layer, business/application layer, and the database layer.
Unit testing involves testing a single endpoint with a single request, expecting a single response or set of responses if the request allows for partial responses. Common tools for carrying out unit tests include curl, Postman, resty, and Insomnia, among others.
Load testing measures the performance of the API under strenuous conditions. Under this test, the API is subjected to many requests at once to see if it has the capacity to simultaneously handle multiple client connections when in production. For the test to pass, response times must remain reasonable, and the server must not crash. There are a number of online performance testing tools, like ReadyAPI and Katalon, that you can try out on a free trial, since it’s difficult to manually carry out a performance test.
Security tests entail checking that restricted resources aren’t available to the public or unauthorized personnel. To carry out security tests, perform different requests while authenticated as different types of user and validate the responses against the expected results. Unauthenticated users shouldn’t be able to interact with the API except at the authentication endpoints, admin users should have god-level access to all resources exposed by the API, while normal users shouldn’t be able to access another user’s data.
OpenAPI Specification
The OpenAPI Specification (OAS) is a standard for REST APIs that aims to simplify the capabilities of the service to ensure that consumers understand and interact with it with minimal implementation logic. OpenAPI promotes a contract-first approach as opposed to an implementation-first approach, meaning you have to design the API document first then write the corresponding code. If properly defined, an OpenAPI definition can be used by different tools like Swagger to generate servers, clients, and documentation to display the API, among other use cases.
- Versions: The OpenAPI Specification is versioned using Semantic Versioning and follows the semver specification. An OpenAPI document should have an openapi property that depicts which semantic version the OAS uses. When upgrading to a new version, the new version should be written in a way that ensures backward compatibility.
- Format: An OpenAPI document is itself a JSON object that may be represented in either JSON or YAML. The document schema exposes two types of fields, namely fixed fields, which have a declared name, and patterned fields, which declare a regex pattern for the field name. Patterned fields must also have a unique name within the containing object.
- Document Structure: An OpenAPI document can be made up of a single document or multiple documents that are joined. In the case of multiple documents, ref fields must be used in the specification to reference the documents.
- Data Types: OAS uses several known formats to define the primitive data type being used in granularity. The primitive data types defined by OAS are integer, number, string, and boolean, and are based on the JSON Schema Specification Wright Draft. Null is however not supported as a data type.
- Rich Text Formatting: In cases where OpenAPI tooling renders rich text, it must support, at a minimum, Markdown syntax as described by CommonMark. Some CommonMark features may be ignored by tooling to address security concerns.
- Relative References in URLs: The default for all properties that are URLs is that they be relative references. Relative references are resolved using the URLs defined in the Server Object as a base URI. Where references are used in $ref, they are processed as per JSON Reference using the URL of the current document as the base URI.
- Schema: The schema of the OAS defines the various objects that comprise the OpenAPI document. Among them is the OpenAPI object, which is the root document object of the OpenAPI document. The Info object provides metadata about the API, while the Contact object provides contact information for it. There are other object types, such as Server, Components, and Parameter to mention but a few. Each object documents a unique property of the API, which is vital for auto-generating tools to work properly.
- Specification Extensions: Additional data can be added to extend the OAS at certain points. Extension properties are implemented as patterned fields that are prefixed by “x-”. The value for such fields can have any valid JSON-format value.
- Security Filtering: Some OAS objects, for example the
Paths ObjectandPath Item Object, can be declared and remain empty even though they are core parts of the API documentation. This functionality allows for an additional layer of access control so that a viewer can see that a path exists but won’t see any parameters or operations without the correct authorization.
APIs allow information to be exchanged between different applications, and the OpenAPI Specification standardizes the way APIs are built. Ensuring that APIs are built in a uniform way that everyone can understand eliminates confusion when integrating with them. It is with this understanding that one can see that the OpenAPI Specification is an integral part of developing public APIs with a high adoption rate.
The other good API design patterns discussed in this guide are equally important, as they improve the API’s security and performance, as well as making it easier to extend it in the future as it evolves.