Golang http error handling

When developing your APIs always remember that an error scenario will always arise. The user may send an incorrect request or there might be some errors within your logic that might have been overlooked. In any condition, your ultimate aim is to never let the system stop execution.

Alec develops a set of APIs and tests out their basic functionalities. They worked just fine. Great! He releases his APIs for others to use, calls it a day and starts daydreaming about a nice vacation by the beach…..

Meanwhile,  on the other side of the world, someone decides to use Alec’s APIs. They don’t really know much about his system and they hit one of the services with some random input…. and….the service breaks, the system shuts down…..mayday!!

Seems that Alec only tested out the ideal conditions and never really saw this coming….

Take my advice, never be like Alec. Never! When developing your APIs always remember that an error scenario will always arise. The user may send an incorrect request or there might be some errors within your logic that might have been overlooked. In any condition, your ultimate aim is to never let the system stop execution .

You need to think of various situations where your API can break and take mitigative measures. Even if an error occurs, you need to handle it gracefully and inform the user about the error.

In this post, we are going to be doing exactly that. And like our previous series, we will continue with the petstore application example ….

In other programming languages, errors are mostly handled using «try»,»catch»,»except» blocks. Its pretty straightforward in Go though! Go has a built in type called «error» which basically looks like this.

type error interface {
    Error() string
}

In any case where an error may occur, the usual way to handle it is to check if the error returned is nil or not. If its nil, then that means the logic worked fine, else an error occurred.

func myFunc() error{
   //do something that may return an error
}

err := myFunc()
if err!=nil{
   //do something
}

The description of the error can be obtained by using the Error() function and printing it out

func myFunc() error{
   return errors.New("This is an error")
}

err := myFunc()
if err!=nil{
   fmt.Println(err.Error())
}

The panic() function is mostly used if you don’t know what to do with the error. Panic will cause the execution to exit with a non zero status code and will print out the error and its stack trace

if err!=nil{
	panic(err)
}

Error Codes in HTTP

When using HTTP, there are specific codes which have been defined for errors. Know that your API may throw an error in two conditions:

• Client Errors: When the Client side requests for invalid, missing or incorrect data
• Server Errors: When the Server fails to furnish the client request because of some error that has occurred internally

All error codes of format 4XX are Client side errors and 5XX format belong to server errors.

Let’s look at a few most commonly used error codes an analogy:

• 400 — Bad Request
  This ones like asking for a pizza at an ice cream shop. That’s a «bad request» !
  That means that the client has sent some request which cannot be understood by the server has to be modified.
• 404 — Not Found
  You ask for a mint chocolate ice cream but the shop doesn’t have mint chocolate. (See you asked for an ice cream in an ice cream shop so its not a «bad request», but the flavour you wanted was «not found»)
  This means that the client has asked for some data which the server couldn’t find.
• 500 — Internal Server Error
  You asked for an ice cream and the ice cream shop does have the flavour. But the vendor tells you can’t get the ice cream because there is some problem in the kitchen. Its an «internal error» you see!
  This means there was some error in the server’s function because of which you couldn’t get the response back

Error Handling in Our API

At the end of our previous series, we had a sweet set of working APIs. We’d tested them out by sending the appropriate input and got the correct output accordingly. But what if we sent a wrong input?

Remember we have an endpoint GET «/category/{id}» where we pass the category identifier to get the details of the category. Now think of situations where this API can break. We are listing down a few

• The identifier is supposed to be an integer. What if we pass a string value?
• We pass an identifier but there’s no such record not present in the database.

In our current situation let’s try out one of these situations.

Try calling the API with id=»hello». Another way is to set id=10 which doesn’t exist in our data. When calling from Postman, this is what you’ll see:

The API returns no response and any external user will have no clue as to why this happend ! But since you are the developer, you can easily see the logs:

See? It clearly says there are no records for this. We should definitely tell that to the user, shouldn’t we?

Let’s do that now. Take a look at the current handler function for the endpoint which is the GetCategory() function within internal/petstore/rest/category.go file:

//GetCategory get a single category
func GetCategory(c echo.Context) error {
	// Get id from the path param for category. eg /category/1
	id, err := strconv.Atoi(c.Param("id"))
	// Call category service to get category
	response, err := service.GetCategory(id)
	if err != nil {
		panic(err)
	}
	// send JSON response
	return c.JSON(http.StatusOK, response)

}

We will slightly modify this to handle our error.

Returning errors using Echo

Echo framework provides error handling HTTPError type which can be returned in the response. The default error response returned is:

{
"message":"<Your error string>"
}

The error can be returned by using echo.NewHTTPError().

return echo.NewHTTPError(<statuscode>,<error string>)

Using this, let’s handle each of the situations we mentioned above.

When an incorrect input value is sent

We are passing the value «hello» to the «id» path parameter in the GET /category/{id} endpoint. In this case, id should be an integer. Let’s look at the first

In our code, we are converting the id path parameter to integer using «strconv.Atoi()». If the value passed cannot be converted, then an error is thrown. In this case, we’ll return an error response with the status as «BadRequest» with code 400 since the type of value passed is incorrect.

//GetCategory get a single category
func GetCategory(c echo.Context) error {
	// Get id from the path param for category. eg /category/1
	id, err := strconv.Atoi(c.Param("id"))
    
   	//-- ADD ERROR HANDLING HERE --
    if err != nil {
	return echo.NewHTTPError(http.StatusBadRequest, "id sent is incorrect.Please send a valid integer value")
	}
    
    .....
    }

Lets see how the API response looks now:

When a record corresponding to the input value doesn’t exist

We are passing the value 10 to the «id» path parameter in the GET /category/{id} endpoint but a record for category with id=10 doesn’t exist.  To be honest, its not an error exactly. The user doesn’t really know what records you database has. And if the record isn’t there  we are bound to get an empty response.

In such a case, we have to pass a response with the status as  «NotFound» and code 404.  The error will be sent if the response object returned in your code  is empty or «nil».

//GetCategory get a single category
func GetCategory(c echo.Context) error {
	// Get id from the path param for category. eg /category/1
	id, err := strconv.Atoi(c.Param("id"))
	if err != nil {
		return echo.NewHTTPError(http.StatusBadRequest, "id sent is incorrect. Please send a valid integer value")
	}
	// Call category service to get category
	response, err := service.GetCategory(id)
	if response == nil {
		return echo.NewHTTPError(http.StatusNotFound, "No records found for given category id ")
	}
	....

}

Lets look at how this scenario looks like :

Returning internal errors

While returning any other error, we can return a response with status as «InternalServerError» with code 500.

if err != nil {
return echo.NewHTTPError(http.StatusInternalServerError, "Failure in fetching category for the given id")
	}

Overall, your entire function for the get category for an id will look like:

//GetCategory get a single category
func GetCategory(c echo.Context) error {
	// Get id from the path param for category. eg /category/1
	id, err := strconv.Atoi(c.Param("id"))
	if err != nil {
		return echo.NewHTTPError(http.StatusBadRequest, "id sent is incorrect. Please send a valid integer value")
	}
	// Call category service to get category
	response, err := service.GetCategory(id)
	if response == nil {
		return echo.NewHTTPError(http.StatusNotFound, "No records found for given category id ")
	}
	if err != nil {
		return echo.NewHTTPError(http.StatusInternalServerError, "Failure in fetching category for the given id")
	}
	// send JSON response
	return c.JSON(http.StatusOK, response)

}

Perfect! One of your APIs is now error proof! In a similar manner, you can handle all the errors in all the API handlers present under the /internal/petstore/rest folder.

In Summary

Although it seems trivial, error handling is always a crucial aspect while developing any software since it makes the system robust.

Your system is like a black box for the external users and APIs are the only way with which they can interact with the system. If your API response doesn’t return appropriate information in case of errors, the user will remain clueless and your accountability will decrease. So never forget to find and catch those errors!

References:

In this article, we’ll take a look at how to handle errors using build-in Golang functionality, how you can extract information from the errors you are receiving and the best practices to do so.

Error handling in Golang is unconventional when compared to other mainstream languages like Javascript, Java and Python. This can make it very difficult for new programmers to grasp Golangs approach of tackling error handling.

In this article, we’ll take a look at how to handle errors using build-in Golang functionality, how you can extract information from the errors you are receiving and the best practices to do so. A basic understanding of Golang is therefore required to follow this article. If you are unsure about any concepts, you can look them up here.

Errors in Golang

Errors indicate an unwanted condition occurring in your application. Let’s say you want to create a temporary directory where you can store some files for your application, but the directory’s creation fails. This is an unwanted condition and is therefore represented using an error.

package main

import (  
    "fmt"
    "ioutil"
)

func main() {  
    dir, err := ioutil.TempDir("", "temp")
		if err != nil {
			return fmt.Errorf("failed to create temp dir: %v", err)
		}
}

Golang represents errors using the built-in error type, which we will look at closer in the next section. The error is often returned as a second argument of the function, as shown in the example above. Here the TempDir function returns the name of the directory as well as an error variable.

Creating custom errors

As already mentioned errors are represented using the built-in error interface type, which has the following definition:

type error interface {  
    Error() string
}

The interface contains a single method Error() that returns an error message as a string. Every type that implements the error interface can be used as an error. When printing the error using methods like fmt.Println the Error() method is automatically called by Golang.

There are multiple ways of creating custom error messages in Golang, each with its own advantages and disadvantages.

String-based errors

String-based errors can be created using two out-of-the-box options in Golang and are used for simple errors that just need to return an error message.

err := errors.New("math: divided by zero")

The errors.New() method can be used to create new errors and takes the error message as its only parameter.

err2 := fmt.Errorf("math: %g cannot be divided by zero", x)

fmt.Errorf on the other hand also provides the ability to add formatting to your error message. Above you can see that a parameter can be passed which will be included in the error message.

Custom error with data

You can create your own error type by implementing the Error() function defined in the error interface on your struct. Here is an example:

type PathError struct {
    Path string
}

func (e *PathError) Error() string {
	return fmt.Sprintf("error in path: %v", e.Path)
}

The PathError implements the Error() function and therefore satisfies the error interface. The implementation of the Error() function now returns a string with the path of the PathError struct. You can now use PathError whenever you want to throw an error.

Here is an elementary example:

package main

import(
	"fmt"
)

type PathError struct {
    Path string
}

func (e *PathError) Error() string {
	return fmt.Sprintf("error in path: %v", e.Path)
}

func throwError() error {
	return &PathError{Path: "/test"}
}

func main() {
	err := throwError()

	if err != nil {
		fmt.Println(err)
	}
}

You can also check if the error has a specific type using either an if or switch statement:

if err != nil {
    switch e := err.(type) {
    case *PathError :
        // Do something with the path
    default:
        log.Println(e)
    }
}

This will allow you to extract more information from your errors because you can then call all functions that are implemented on the specific error type. For example, if the PathError had a second method called GetInfo you could call it like this.

e.GetInfo()

Error handling in functions

Now that you know how to create your own custom errors and extract as much information as possible from errors let’s take a look at how you can handle errors in functions.

Most of the time errors are not directly handled in functions but are returned as a return value instead. Here we can take advantage of the fact that Golang supports multiple return values for a function. Thus you can return your error alongside the normal result — errors are always returned as the last argument — of the function as follows:

func divide(a, b float64) (float64, error) {
	if b == 0 {
		return 0.0, errors.New("cannot divide through zero")
	}

	return a/b, nil
}

The function call will then look similar to this:

func main() {
	num, err := divide(100, 0)

	if err != nil {
		fmt.Printf("error: %s", err.Error())
	} else {
		fmt.Println("Number: ", num)
	}
}

If the returned error is not nil it usually means that there is a problem and you need to handle the error appropriately. This can mean that you use some kind of log message to warn the user, retry the function until it works or close the application entirely depending on the situation. The only drawback is that Golang does not enforce handling the retuned errors, which means that you could just ignore handling errors completely.

Take the following code for example:

package main

import (
	"errors"
	"fmt"
)

func main() {
	num2, _ := divide(100, 0)
	
	fmt.Println("Number: ", num2)
}

The so-called blank identifier is used as an anonymous placeholder and therefore provides a way to ignore values in an assignment and avoid compiler errors in the process. But remember that using the blank identifier instead of probably handling errors is dangerous and should not be done if it can be avoided.

Defer, panic and recover

Go does not have exceptions like many other programming languages, including Java and Javascript but has a comparable mechanism know as ,,Defer, panic and recover». Still the use-cases of panic and recover are very different from exceptions in other programming languages as they should only be used in unexpected and unrecoverable situations.

Defer

A defer statement is a mechanism used to defer a function by putting it into an executed stack once the function that contains the defer statement has finished, either normally by executing a return statement or abnormally panicking. Deferred functions will then be executed in reverse order in which they were deferred.

Take the following function for example:

func processHTML(url string) error {
  resp, err := http.Get(url)

  if err != nil {
    	return err
	}

	ct := resp.Header.Get("Content-Type")
	if ct != "text/html" && !strings.HasPrefix(ct, "text/html;") {
		resp.Body.Close()
		return fmt.Errorf("%s has content type %s which does not match text/html", url, ct)
	}

	doc, err := html.Parse(resp.Body)
	resp.Body.Close()

	// ... Process HTML ...
	
	return nil
}

Here you can notice the duplicated resp.Body.Close call, which ensures that the response is properly closed. Once functions grow more complex and have more errors that need to be handled such duplications get more and more problematic to maintain.

Since deferred calls get called once the function has ended, no matter if it succeeded or not it can be used to simplify such calls.

func processHTMLDefer(url string) error {
  resp, err := http.Get(url)

  if err != nil {
    	return err
	}
  defer resp.Body.Close()

	ct := resp.Header.Get("Content-Type")
	if ct != "text/html" && !strings.HasPrefix(ct, "text/html;") {
		return fmt.Errorf("%s has content type %s which does not match text/html", url, ct)
	}

	doc, err := html.Parse(resp.Body)

	// ... Process HTML ...

	return nil
}

All deferred functions are executed in reverse order in which they were deferred when the function finishes.

package main

import (
        "fmt"
)

func main() {
	first()
}

func first() {
	defer fmt.Println("first")
	second()
}

func second() {
	defer fmt.Println("second")
	third()
}

func third() {
	defer fmt.Println("third")
}

Here is the result of running the above program:

third
second
first

Panic

A panic statement signals Golang that your code cannot solve the current problem and it therefore stops the normal execution flow of your code. Once a panic is called, all deferred functions are executed and the program crashes with a log message that includes the panic values (usually an error message) and a stack trace.

As an example Golang will panic when a number is divided by zero.

package main

import "fmt"

func main() {
	divide(5)
}

func divide(x int) {
	fmt.Printf("divide(%d) n", x+0/x)
	divide(x-1)
}

Once the divide function is called using zero, the program will panic, resulting in the following output.

panic: runtime error: integer divide by zero

goroutine 1 [running]:
main.divide(0x0)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:16 +0xe6
main.divide(0x1)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:17 +0xd6
main.divide(0x2)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:17 +0xd6
main.divide(0x3)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:17 +0xd6
main.divide(0x4)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:17 +0xd6
main.divide(0x5)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:17 +0xd6
main.main()
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:11 +0x31
exit status 2

You can also use the built-in panic function to panic in your own programms. A panic should mostly only be used when something happens that the program didn’t expect and cannot handle.

func getArguments() {
	if len(os.Args) == 1 {
		panic("Not enough arguments!")
	}
}

As already mentioned, deferred functions will be executed before terminating the application, as shown in the following example.

package main

import (
	"fmt"
)

func main() {
	accessSlice([]int{1,2,5,6,7,8}, 0)
}

func accessSlice(slice []int, index int) {
	fmt.Printf("item %d, value %d n", index, slice[index])
	defer fmt.Printf("defer %d n", index)
	accessSlice(slice, index+1)
}

Here is the output of the programm:

item 0, value 1 
item 1, value 2 
item 2, value 5
item 3, value 6
item 4, value 7
item 5, value 8
defer 5
defer 4
defer 3
defer 2
defer 1
defer 0
panic: runtime error: index out of range [6] with length 6

goroutine 1 [running]:
main.accessSlice(0xc00011df48, 0x6, 0x6, 0x6)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:29 +0x250
main.accessSlice(0xc00011df48, 0x6, 0x6, 0x5)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:31 +0x1eb
main.accessSlice(0xc00011df48, 0x6, 0x6, 0x4)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:31 +0x1eb
main.accessSlice(0xc00011df48, 0x6, 0x6, 0x3)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:31 +0x1eb
main.accessSlice(0xc00011df48, 0x6, 0x6, 0x2)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:31 +0x1eb
main.accessSlice(0xc00011df48, 0x6, 0x6, 0x1)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:31 +0x1eb
main.accessSlice(0xc00011df48, 0x6, 0x6, 0x0)
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:31 +0x1eb
main.main()
        C:/Users/gabriel/articles/Golang Error handling/Code/panic/main.go:9 +0x99
exit status 2

Recover

In some rare cases panics should not terminate the application but be recovered instead. For example, a socket server that encounters an unexpected problem could report the error to the clients and then close all connections rather than leaving the clients wondering what just happened.

Panics can therefore be recovered by calling the built-in recover function within a deferred function in the function that is panicking. Recover will then end the current state of panic and return the panic error value.

package main

import "fmt"

func main(){
	accessSlice([]int{1,2,5,6,7,8}, 0)
}

func accessSlice(slice []int, index int) {
	defer func() {
		if p := recover(); p != nil {
			fmt.Printf("internal error: %v", p)
		}
	}()

	fmt.Printf("item %d, value %d n", index, slice[index])
	defer fmt.Printf("defer %d n", index)
	accessSlice(slice, index+1)
}

As you can see after adding a recover function to the function we coded above the program doesn’t exit anymore when the index is out of bounds by recovers instead.

Output:

item 0, value 1 
item 1, value 2
item 2, value 5
item 3, value 6
item 4, value 7
item 5, value 8
internal error: runtime error: index out of range [6] with length 6defer 5 
defer 4
defer 3
defer 2
defer 1
defer 0

Recovering from panics can be useful in some cases, but as a general rule you should try to avoid recovering from panics.

Error wrapping

Golang also allows errors to wrap other errors which provides the functionality to provide additional context to your error messages. This is often used to provide specific information like where the error originated in your program.

You can create wrapped errors by using the %w flag with the fmt.Errorf function as shown in the following example.

package main

import (
	"errors"
	"fmt"
	"os"
)

func main() {
	err := openFile("non-existing")

	if err != nil {
		fmt.Printf("error running program: %s n", err.Error())
	}
}

func openFile(filename string) error {
	if _, err := os.Open(filename); err != nil {
		return fmt.Errorf("error opening %s: %w", filename, err)
	}

	return nil
}

The output of the application would now look like the following:

error running program: error opening non-existing: open non-existing: no such file or directory

As you can see the application prints both the new error created using fmt.Errorf as well as the old error message that was passed to the %w flag. Golang also provides the functionality to get the old error message back by unwrapping the error using errors.Unwrap.

package main

import (
	"errors"
	"fmt"
	"os"
)

func main() {
	err := openFile("non-existing")

	if err != nil {
		fmt.Printf("error running program: %s n", err.Error())

		// Unwrap error
		unwrappedErr := errors.Unwrap(err)
		fmt.Printf("unwrapped error: %v n", unwrappedErr)
	}
}

func openFile(filename string) error {
	if _, err := os.Open(filename); err != nil {
		return fmt.Errorf("error opening %s: %w", filename, err)
	}

	return nil
}

As you can see the output now also displays the original error.

error running program: error opening non-existing: open non-existing: no such file or directory 
unwrapped error: open non-existing: no such file or directory

Errors can be wrapped and unwrapped multiple times, but in most cases wrapping them more than a few times does not make sense.

Casting Errors

Sometimes you will need a way to cast between different error types to for example, access unique information that only that type has. The errors.As function provides an easy and safe way to do so by looking for the first error in the error chain that fits the requirements of the error type. If no match is found the function returns false.

Let’s look at the official errors.As docs example to better understand what is happening.

package main

import (
	"errors"
	"fmt"
	"io/fs"
	"os"
)

func main(){
	// Casting error
	if _, err := os.Open("non-existing"); err != nil {
		var pathError *os.PathError
		if errors.As(err, &pathError) {
			fmt.Println("Failed at path:", pathError.Path)
		} else {
			fmt.Println(err)
		}
	}
}

Here we try to cast our generic error type to os.PathError so we can access the Path variable that that specific error contains.

Another useful functionality is checking if an error has a specific type. Golang provides the errors.Is function to do exactly that. Here you provide your error as well as the particular error type you want to check. If the error matches the specific type the function will return true, if not it will return false.

package main

import (
	"errors"
	"fmt"
	"io/fs"
	"os"
)

func main(){
	// Check if error is a specific type
	if _, err := os.Open("non-existing"); err != nil {
		if errors.Is(err, fs.ErrNotExist) {
			fmt.Println("file does not exist")
		} else {
			fmt.Println(err)
		}
	}
}

After checking, you can adapt your error message accordingly.

Sources

• Golang Blog — Working with Errors in Go 1.13
• The Go Programming language book
• Golang Blog — Defer, Panic, and Recover
• LogRocket — Error handling in Golang
• GolangByExample — Wrapping and Un-wrapping of error in Go
• Golang Documentation — Package errors

Conclusion

You made it all the way until the end! I hope this article helped you understand the basics of Go error handling and why it is an essential topic in application/software development.

If you have found this helpful, please consider recommending and sharing it with other fellow developers and subscribing to my newsletter. If you have any questions or feedback, let me know using my contact form or contact me on Twitter.