REST API Error Handling in Go: Behavioural Type Assertion

Error handling is, without doubt, one of the most fundamental topics that every developer should know by the heart. But its importance is usually underestimated. I have seen many projects in which exceptions are ignored at some point in the call stack without even logging them. Robust Go applications should deal with errors gracefully.

The following are helpful tips and conventions that are meant to make error handling better in your REST API projects. But remember, you don’t have to follow all of them like they are unchangeable rules. If you think something else makes more sense for your case, try it!

1. Always check errors. Never ignore them unless you have a very, very good reason.
2. Always log error details somewhere. Errors are the most valuable information we have, to fix bugs, potential failures and performance issues.
3. Add breadcrumbs to error logs, such as Client IP, request headers/body, user information/events. The more data you have, the easier it is to debug the problem.
4. Handle errors only once. Wrap them with additional context and return to caller function if necessary. It is much more maintainable to handle (taking an action on error such as logging) them in a specific exit point/middleware.
5. Differentiate client and server errors. Don’t overthink and come up with hundreds of different error types. Keep it simple. It is almost always enough to have just two generic types: Client Errors (4xx), which means something was probably wrong with the request data and can be corrected by the client. Server Errors (5xx), are unexpected errors and they usually point out that there are some bugs needs to be fixed in the code.
6. Use HTTP response status codes to indicate something went wrong (400 Bad Request, 404 Not Found…). Although there are different status codes you can use for different situations, they are usually not enough to describe the error on their own. It is best to add more context to the response body.
7. Define a good error response structure from the beginning. Whether it is a simple JSON response with only 'detail' and 'status' or a complex one with 'domain' , 'type' , 'track_id' , 'title' , 'helpUrl' ; stick with it. Try not to have different structures on different endpoints. Be consistent as much as possible. You can follow RFC7807.
8. Make your Client Error messages human readable. Use 'code' to identify errors, messages are for users. Don’t scare the user off with complex error messages, they are best to be descriptive and easy to grasp.
9. Don’t share the details/stack trace of Server Errors with the client. They might contain your code logic and secrets. Log them on a secure platform with a unique id and return this id to the client. So that you can find relevant records/metrics easily when a customer contacts you with this trace id.
10. Document your errors both for developers and users. Try to be as descriptive as possible with error details. It is better to suggest possible reasons/solutions for users in the documentation.

Some background — Errors in Go##

If you are coming from another language like Java, Python or Javascript, you might find the errors in Go a little bit strange and ugly. There are no exceptions, no try/catch blocks and you are checking errors using 'if' statements. By convention, errors are the last return value from functions, that implements the built-in interface error :

type error interface {
    Error() string
}

You can create a custom error just by implementing error interface:

type CustomError string
func (err CustomError) Error() string {
    return string(err)
}

Go encourage you to explicitly check errors where they occur:

f, err := os.Open(filename)
if err != nil {
    // Handle the error ...
}

I admit that checking for every single error using if statements can be frustrating at first. It becomes even worse if you have many custom error types. Your Go code becomes too verbose with all this conditional checks and type assertions.

Fortunately, there are some techniques you can use to reduce repetitive error handling and I want to talk about behavioural type assertion .

Handling errors in a sample Go REST API project##

Since we talked about what is an error in Go and the best practices for handling errors, we can implement some of them in a sample REST API project.

We are going to use net/http module. Let's start with a simple endpoint:

POST /login

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
)

type loginSchema struct {
	Username string `json:"username"`
	Password string `json:"password"`
}

func loginUser(username string, password string) (bool, error) {...}

func loginHandler(w http.ResponseWriter, r *http.Request) {
	if r.Method != http.MethodPost {
		w.WriteHeader(405) // Return 405 Method Not Allowed.
		return
	}
	// Read request body.
	body, err := ioutil.ReadAll(r.Body)
	if err != nil {
		log.Printf("Body read error, %v", err)
		w.WriteHeader(500) // Return 500 Internal Server Error.
		return
	}

	// Parse body as json.
	var schema loginSchema
	if err = json.Unmarshal(body, &schema); err != nil {
		log.Printf("Body parse error, %v", err)
		w.WriteHeader(400) // Return 400 Bad Request.
		return
	}

	ok, err := loginUser(schema.Username, schema.Password)
	if err != nil {
		log.Printf("Login user DB error, %v", err)
		w.WriteHeader(500) // Return 500 Internal Server Error.
		return
	}

	if !ok {
		log.Printf("Unauthorized access for user: %v", schema.Username)
		w.WriteHeader(401) // Wrong password or username, Return 401.
		return
	}
	w.WriteHeader(200) // Successfully logged in.
}

func main() {
	http.HandleFunc("/login/", loginHandler)
	log.Fatal(http.ListenAndServe(":8080", nil))
}

It is pretty self-explanatory. Client makes a POST request with password and username in a JSON body. If the credentials are correct, we respond with 200 OK . Otherwise, we respond with 401 Unauthorized .

For the sake of simplicity, do not think about the validation of the request body, implementation of loginUser function or error messages.

What can go wrong with the code above?

• ioutil.ReadAll(r.Body) may return an error in case of buffer overflow.

• json.Unmarshal(body, &schema) may fail and return error while parsing.

• loginUser(...) may fail for various database or connection reasons.

And we are handling all of these three cases, which is a good way to start. But there is something ugly about the code above. I am sure that you already noticed we are repeating some steps to handle errors:

• Log the error details.
• Present a simple error response to the client with HTTP status code.

Imagine that we have hundreds of endpoints. You will end up having copies of the same error handling code everywhere. And what if you want to add an extra step to error handling logic (maybe alerting developers)? You must add it one by one to each part.

Do you remember what I mentioned earlier:

Handle errors only once. Wrap them with additional information and return to caller function if necessary. It is more maintainable and scalable to handle (taking an action on error such as logging) them in a specific exit point/middleware.

How about moving error handling logic from handler to somewhere else? We can return errors directly from the handler function and process them in a single function outside.

So how does our new handler look like? func loginHandler(w http.ResponseWriter, r *http.Request) error {...}

It seems like a reasonable idea, except that http.HandleFunc doesn’t understand the signature of this function (since we are returning error). Instead of using http.HandleFunc , we can use http.Handle which accepts any type that implements http.Handler interface. In other words, as long as you pass a type that has ServeHTTP method, the http.Handle function will be happy.

All of the handler functions in our application will share the same error handling code, so we can define a new function type (rootHandler ) and write error handling part in ServeHTTP method.

Now we can move error handling logic from loginHandler to rootHandler.ServeHTTP function, which is:

• Call actual handler function.
• If there is no error returned from handler, just return from the function.
• If there is an error: log the error and return an HTTP response to client.

Okay, but what message or HTTP status code should we use in the response? 500 Internal Server Error, 400 Bad Request, 405 Method Not Allowed? How can we know what to do just by looking at the error returned by handler?

Remember that we want to have two different main error types: Client Error for 4xx errors and Server Error (or Internal Error) for 5xx. We can declare interfaces based on the behavior we expect from these two types and use type assertion on rootHandler to make some decisions about the error.

Let's start by declaring an interface for Client Errors. Each ClientError must have a response body and HTTP response status code:

// ClientError is an error whose details to be shared with client.
type ClientError interface {
	Error() string
	// ResponseBody returns response body.
	ResponseBody() ([]byte, error)
	// ResponseHeaders returns http status code and headers.
	ResponseHeaders() (int, map[string]string)
}

• ResponseBody() ([]byte, error) : Returns JSON response body of the error (title, message, error code…) in bytes. (*Getting response body as bytes from one method is not the best solution, see Further Improvements section.)
• ResponseHeaders() (int, map[string]string) : Returns HTTP status code (4xx, 5xx) and headers (content type, no-cache…) of response.
• Error() string , this is necessary to make every ClientError and error at the same time.

Now we can declare a struct, HTTPError that implements ClientError :

// HTTPError implements ClientError interface.
type HTTPError struct {
	Cause  error  `json:"-"`
	Detail string `json:"detail"`
	Status int    `json:"-"`
}

func (e *HTTPError) Error() string {
	if e.Cause == nil {
		return e.Detail
	}
	return e.Detail + " : " + e.Cause.Error()
}

// ResponseBody returns JSON response body.
func (e *HTTPError) ResponseBody() ([]byte, error) {
	body, err := json.Marshal(e)
	if err != nil {
		return nil, fmt.Errorf("Error while parsing response body: %v", err)
	}
	return body, nil
}

// ResponseHeaders returns http status code and headers.
func (e *HTTPError) ResponseHeaders() (int, map[string]string) {
	return e.Status, map[string]string{
		"Content-Type": "application/json; charset=utf-8",
	}
}

func NewHTTPError(err error, status int, detail string) error {
	return &HTTPError{
		Cause:  err,
		Detail: detail,
		Status: status,
	}
}

HTTPError has all the information we need to log the error and return a proper HTTP response to the client:

• Cause : Original error (unmarshall errors, network errors…) which caused this HTTP error, set it to nil if there isn’t any.
• Detail : message to return in JSON response. Ex: { "detail": "Wrong password" } .
• Status : HTTP response status code. Ex: 400, 401, 405…

Why did we introduce ClientError interface, rather than just having HTTPError struct and using it for type assertion?

Because if we assert for types, our error handler must know every custom error in every package that can be returned, to assert them:

switch e := err.(type) {
case ErrorType1:
    body := e.Message
    status := e.Status
    ...
case package1.ErrorType2:
    body := e.Detail
    status := e.HTTPStatus
case package2.ErrorType3:
    ...
}

This may not seem like a problem at first but as the application becomes bigger and complex, you might want to have different error types with different structures on different packages (for instance you can define domain-specific error types). Which makes the handler function tightly coupled with each error type.

As long as you have a strong definition of errors in your API for both users and developers, I find it more easy, elegant to define an Interface based on the behaviour we expect from the error and assert for this Interface in the main handler.

We have a good definition, every Client Error must have an HTTP status code, response body(in predefined JSON format) and original error for internal logging:

clientError, ok := err.(ClientError) // type assertion for behavior.
if ok {
    body := clientError.ResponseBody()
    status, headers := clientError.ResponseHeaders()
    w.WriteHeader(status)
    w.Write(body)
}

Now, let's rewrite the final version of our handler function:

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
)

type loginSchema struct {
	Username string `json:"username"`
	Password string `json:"password"`
}

// Return `true`, nil if given user and password exists in database.
func loginUser(username string, password string) (bool, error) {...}

// Use as a wrapper around the handler functions.
type rootHandler func(http.ResponseWriter, *http.Request) error

func loginHandler(w http.ResponseWriter, r *http.Request) error {
	if r.Method != http.MethodPost {
		return NewHTTPError(nil, 405, "Method not allowed.")
	}

	body, err := ioutil.ReadAll(r.Body) // Read request body.
	if err != nil {
		return fmt.Errorf("Request body read error : %v", err)
	}

	// Parse body as json.
	var schema loginSchema
	if err = json.Unmarshal(body, &schema); err != nil {
		return NewHTTPError(err, 400, "Bad request : invalid JSON.")
	}

	ok, err := loginUser(schema.Username, schema.Password)
	if err != nil {
		return fmt.Errorf("loginUser DB error : %v", err)
	}

	if !ok { // Authentication failed.
		return NewHTTPError(nil, 401, "Wrong password or username.")
	}
	w.WriteHeader(200) // Successfully authenticated. Return access token?
	return nil
}

// rootHandler implements http.Handler interface.
func (fn rootHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	err := fn(w, r) // Call handler function
	if err == nil {
		return
	}
	// This is where our error handling logic starts.
	log.Printf("An error accured: %v", err) // Log the error.

	clientError, ok := err.(ClientError) // Check if it is a ClientError.
	if !ok {
		// If the error is not ClientError, assume that it is ServerError.
		w.WriteHeader(500) // return 500 Internal Server Error.
		return
	}

	body, err := clientError.ResponseBody() // Try to get response body of ClientError.
	if err != nil {
		log.Printf("An error accured: %v", err)
		w.WriteHeader(500)
		return
	}
	status, headers := clientError.ResponseHeaders() // Get http status code and headers.
	for k, v := range headers {
		w.Header().Set(k, v)
	}
	w.WriteHeader(status)
	w.Write(body)
}

func main() {
	// Notice rootHandler.
	http.Handle("/login/", rootHandler(loginHandler))
	log.Fatal(http.ListenAndServe(":8080", nil))
}

What happens when we make POST request to /login/ endpoint with invalid username and password?

• http.Handle calls rootHandler.ServeHTTP with initialized Request and ResponseWriter (line 83)
• rootHandler.ServeHTTP calls loginHandler function (line 52)
• loginHandler checks the database, authorisation fails and returns an HTTPError (line 44)
• rootHandler.ServeHTTP asserts that returned error implements ClientError interface (line 59)
• rootHandler.ServeHTTP gets body, headers and returns a response to the client (line 57, 73, 77, 78)

Further improvements:

• Use https://github.com/pkg/errors to add stack trace and for wrapping errors.
• Define an Interface for Internal/Server errors
• Instead of having one ResponseBody() method that returns the whole JSON body, implement smaller methods like ResponseTitle() , ResponseMessage() , ResponseStatusCode() and assemble response on rootHandler.
• Make error messages more descriptive and keep them as constants in a file.
• Use an error tracking platform such as Sentry for logging.

Conclusion

Dealing with errors in a single shared place is a better choice especially for web projects. Bubbling them up to the main error handler and adding context at each step will be beneficial to keep track of what is happening at each stage. We might need to take different actions for different error types. But every error type should eventually fall under one of these two main categories: ClientError or ServerError . We can expect specific behaviours (method signatures) from errors based on their category. Using interface and type assertion for the interface behaviour at this point is particularly helpful for decoupling our handler from the rest of the project.