Golang http error response

Part I: When Errors cause API failures

When developing your APIs always remember that an error scenario will always arise. The user may send an incorrect request or there might be some errors within your logic that might have been overlooked. In any condition, your ultimate aim is to never let the system stop execution.

Alec develops a set of APIs and tests out their basic functionalities. They worked just fine. Great! He releases his APIs for others to use, calls it a day and starts daydreaming about a nice vacation by the beach.

Meanwhile, on the other side of the world, someone decides to use Alec’s APIs. They don’t really know much about his system and they hit one of the services with some random input. and. the service breaks, the system shuts down. mayday!!

Seems that Alec only tested out the ideal conditions and never really saw this coming.

Take my advice, never be like Alec. Never! When developing your APIs always remember that an error scenario will always arise. The user may send an incorrect request or there might be some errors within your logic that might have been overlooked. In any condition, your ultimate aim is to never let the system stop execution .

You need to think of various situations where your API can break and take mitigative measures. Even if an error occurs, you need to handle it gracefully and inform the user about the error.

In this post, we are going to be doing exactly that. And like our previous series, we will continue with the petstore application example .

Error Handling in Go

In other programming languages, errors are mostly handled using «try»,»catch»,»except» blocks. Its pretty straightforward in Go though! Go has a built in type called «error» which basically looks like this.

In any case where an error may occur, the usual way to handle it is to check if the error returned is nil or not. If its nil, then that means the logic worked fine, else an error occurred.

The description of the error can be obtained by using the Error() function and printing it out

The panic() function is mostly used if you don’t know what to do with the error. Panic will cause the execution to exit with a non zero status code and will print out the error and its stack trace

Error Codes in HTTP

When using HTTP, there are specific codes which have been defined for errors. Know that your API may throw an error in two conditions:

• Client Errors: When the Client side requests for invalid, missing or incorrect data
• Server Errors: When the Server fails to furnish the client request because of some error that has occurred internally

All error codes of format 4XX are Client side errors and 5XX format belong to server errors.

Let’s look at a few most commonly used error codes an analogy:

• 400 — Bad Request
  This ones like asking for a pizza at an ice cream shop. That’s a «bad request» !
  That means that the client has sent some request which cannot be understood by the server has to be modified.
• 404 — Not Found
  You ask for a mint chocolate ice cream but the shop doesn’t have mint chocolate. (See you asked for an ice cream in an ice cream shop so its not a «bad request», but the flavour you wanted was «not found»)
  This means that the client has asked for some data which the server couldn’t find.
• 500 — Internal Server Error
  You asked for an ice cream and the ice cream shop does have the flavour. But the vendor tells you can’t get the ice cream because there is some problem in the kitchen. Its an «internal error» you see!
  This means there was some error in the server’s function because of which you couldn’t get the response back

Error Handling in Our API

At the end of our previous series, we had a sweet set of working APIs. We’d tested them out by sending the appropriate input and got the correct output accordingly. But what if we sent a wrong input?

Remember we have an endpoint GET «/category/» where we pass the category identifier to get the details of the category. Now think of situations where this API can break. We are listing down a few

• The identifier is supposed to be an integer. What if we pass a string value?
• We pass an identifier but there’s no such record not present in the database.

In our current situation let’s try out one of these situations.

Try calling the API with . Another way is to set which doesn’t exist in our data. When calling from Postman, this is what you’ll see:

The API returns no response and any external user will have no clue as to why this happend ! But since you are the developer, you can easily see the logs:

See? It clearly says there are no records for this. We should definitely tell that to the user, shouldn’t we?

Let’s do that now. Take a look at the current handler function for the endpoint which is the GetCategory() function within internal/petstore/rest/category.go file:

We will slightly modify this to handle our error.

Returning errors using Echo

Echo framework provides error handling HTTPError type which can be returned in the response. The default error response returned is:

The error can be returned by using echo.NewHTTPError().

Using this, let’s handle each of the situations we mentioned above.

When an incorrect input value is sent

We are passing the value «hello» to the «id» path parameter in the GET /category/ endpoint. In this case, id should be an integer. Let’s look at the first

In our code, we are converting the id path parameter to integer using «strconv.Atoi()». If the value passed cannot be converted, then an error is thrown. In this case, we’ll return an error response with the status as «BadRequest» with code 400 since the type of value passed is incorrect.

Lets see how the API response looks now:

When a record corresponding to the input value doesn’t exist

We are passing the value 10 to the «id» path parameter in the GET /category/ endpoint but a record for category with doesn’t exist. To be honest, its not an error exactly. The user doesn’t really know what records you database has. And if the record isn’t there we are bound to get an empty response.

In such a case, we have to pass a response with the status as «NotFound» and code 404. The error will be sent if the response object returned in your code is empty or «nil».

Lets look at how this scenario looks like :

Returning internal errors

While returning any other error, we can return a response with status as «InternalServerError» with code 500.

Overall, your entire function for the get category for an id will look like:

Perfect! One of your APIs is now error proof! In a similar manner, you can handle all the errors in all the API handlers present under the /internal/petstore/rest folder.

In Summary

Although it seems trivial, error handling is always a crucial aspect while developing any software since it makes the system robust.

Your system is like a black box for the external users and APIs are the only way with which they can interact with the system. If your API response doesn’t return appropriate information in case of errors, the user will remain clueless and your accountability will decrease. So never forget to find and catch those errors!

Источник

Package http

Overview ▸

Overview ▾

Package http provides HTTP client and server implementations.

Get, Head, Post, and PostForm make HTTP (or HTTPS) requests:

The client must close the response body when finished with it:

For control over HTTP client headers, redirect policy, and other settings, create a Client:

For control over proxies, TLS configuration, keep-alives, compression, and other settings, create a Transport:

Clients and Transports are safe for concurrent use by multiple goroutines and for efficiency should only be created once and re-used.

ListenAndServe starts an HTTP server with a given address and handler. The handler is usually nil, which means to use DefaultServeMux. Handle and HandleFunc add handlers to DefaultServeMux:

More control over the server’s behavior is available by creating a custom Server:

Starting with Go 1.6, the http package has transparent support for the HTTP/2 protocol when using HTTPS. Programs that must disable HTTP/2 can do so by setting Transport.TLSNextProto (for clients) or Server.TLSNextProto (for servers) to a non-nil, empty map. Alternatively, the following GODEBUG environment variables are currently supported:

The GODEBUG variables are not covered by Go’s API compatibility promise. Please report any issues before disabling HTTP/2 support: https://golang.org/s/http2bug

The http package’s Transport and Server both automatically enable HTTP/2 support for simple configurations. To enable HTTP/2 for more complex configurations, to use lower-level HTTP/2 features, or to use a newer version of Go’s http2 package, import «golang.org/x/net/http2» directly and use its ConfigureTransport and/or ConfigureServer functions. Manually configuring HTTP/2 via the golang.org/x/net/http2 package takes precedence over the net/http package’s built-in HTTP/2 support.

Index ▸

Index ▾

Examples (Expand All)

Package files

Constants

Common HTTP methods.

Unless otherwise noted, these are defined in RFC 7231 section 4.3.

DefaultMaxHeaderBytes is the maximum permitted size of the headers in an HTTP request. This can be overridden by setting Server.MaxHeaderBytes.

DefaultMaxIdleConnsPerHost is the default value of Transport’s MaxIdleConnsPerHost.

TimeFormat is the time format to use when generating times in HTTP headers. It is like time.RFC1123 but hard-codes GMT as the time zone. The time being formatted must be in UTC for Format to generate the correct format.

For parsing this time format, see ParseTime.

TrailerPrefix is a magic prefix for ResponseWriter.Header map keys that, if present, signals that the map entry is actually for the response trailers, and not the response headers. The prefix is stripped after the ServeHTTP call finishes and the values are sent in the trailers.

This mechanism is intended only for trailers that are not known prior to the headers being written. If the set of trailers is fixed or known before the header is written, the normal Go trailers mechanism is preferred:

Variables

Errors used by the HTTP server.

DefaultClient is the default Client and is used by Get, Head, and Post.

DefaultServeMux is the default ServeMux used by Serve.

ErrAbortHandler is a sentinel panic value to abort a handler. While any panic from ServeHTTP aborts the response to the client, panicking with ErrAbortHandler also suppresses logging of a stack trace to the server’s error log.

ErrBodyReadAfterClose is returned when reading a Request or Response Body after the body has been closed. This typically happens when the body is read after an HTTP Handler calls WriteHeader or Write on its ResponseWriter.

ErrHandlerTimeout is returned on ResponseWriter Write calls in handlers which have timed out.

ErrLineTooLong is returned when reading request or response bodies with malformed chunked encoding.

ErrMissingFile is returned by FormFile when the provided file field name is either not present in the request or not a file field.

ErrNoCookie is returned by Request’s Cookie method when a cookie is not found.

ErrNoLocation is returned by Response’s Location method when no Location header is present.

ErrServerClosed is returned by the Server’s Serve, ServeTLS, ListenAndServe, and ListenAndServeTLS methods after a call to Shutdown or Close.

ErrSkipAltProtocol is a sentinel error value defined by Transport.RegisterProtocol.

ErrUseLastResponse can be returned by Client.CheckRedirect hooks to control how redirects are processed. If returned, the next request is not sent and the most recent response is returned with its body unclosed.

NoBody is an io.ReadCloser with no bytes. Read always returns EOF and Close always returns nil. It can be used in an outgoing client request to explicitly signal that a request has zero bytes. An alternative, however, is to simply set Request.Body to nil.

CanonicalHeaderKey returns the canonical format of the header key s. The canonicalization converts the first letter and any letter following a hyphen to upper case; the rest are converted to lowercase. For example, the canonical key for «accept-encoding» is «Accept-Encoding». If s contains a space or invalid header field bytes, it is returned without modifications.

func DetectContentType ¶

DetectContentType implements the algorithm described at https://mimesniff.spec.whatwg.org/ to determine the Content-Type of the given data. It considers at most the first 512 bytes of data. DetectContentType always returns a valid MIME type: if it cannot determine a more specific one, it returns «application/octet-stream».

func Error ¶

Error replies to the request with the specified error message and HTTP code. It does not otherwise end the request; the caller should ensure no further writes are done to w. The error message should be plain text.

func Handle ¶

Handle registers the handler for the given pattern in the DefaultServeMux. The documentation for ServeMux explains how patterns are matched.

func HandleFunc ¶

HandleFunc registers the handler function for the given pattern in the DefaultServeMux. The documentation for ServeMux explains how patterns are matched.

func ListenAndServe ¶

ListenAndServe listens on the TCP network address addr and then calls Serve with handler to handle requests on incoming connections. Accepted connections are configured to enable TCP keep-alives.

The handler is typically nil, in which case the DefaultServeMux is used.

ListenAndServe always returns a non-nil error.

func ListenAndServeTLS ¶

ListenAndServeTLS acts identically to ListenAndServe, except that it expects HTTPS connections. Additionally, files containing a certificate and matching private key for the server must be provided. If the certificate is signed by a certificate authority, the certFile should be the concatenation of the server’s certificate, any intermediates, and the CA’s certificate.

func MaxBytesReader ¶

MaxBytesReader is similar to io.LimitReader but is intended for limiting the size of incoming request bodies. In contrast to io.LimitReader, MaxBytesReader’s result is a ReadCloser, returns a non-nil error of type *MaxBytesError for a Read beyond the limit, and closes the underlying reader when its Close method is called.

MaxBytesReader prevents clients from accidentally or maliciously sending a large request and wasting server resources. If possible, it tells the ResponseWriter to close the connection after the limit has been reached.

func NotFound ¶

NotFound replies to the request with an HTTP 404 not found error.

func ParseHTTPVersion ¶

ParseHTTPVersion parses an HTTP version string according to RFC 7230, section 2.6. «HTTP/1.0» returns (1, 0, true). Note that strings without a minor version, such as «HTTP/2», are not valid.

func ParseTime ¶ 1.1

ParseTime parses a time header (such as the Date: header), trying each of the three formats allowed by HTTP/1.1: TimeFormat, time.RFC850, and time.ANSIC.

func ProxyFromEnvironment ¶

ProxyFromEnvironment returns the URL of the proxy to use for a given request, as indicated by the environment variables HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions thereof). HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.

The environment values may be either a complete URL or a «host[:port]», in which case the «http» scheme is assumed. The schemes «http», «https», and «socks5» are supported. An error is returned if the value is a different form.

A nil URL and nil error are returned if no proxy is defined in the environment, or a proxy should not be used for the given request, as defined by NO_PROXY.

As a special case, if req.URL.Host is «localhost» (with or without a port number), then a nil URL and nil error will be returned.

func ProxyURL ¶

ProxyURL returns a proxy function (for use in a Transport) that always returns the same URL.

func Redirect ¶

Redirect replies to the request with a redirect to url, which may be a path relative to the request path.

The provided code should be in the 3xx range and is usually StatusMovedPermanently, StatusFound or StatusSeeOther.

If the Content-Type header has not been set, Redirect sets it to «text/html; charset=utf-8» and writes a small HTML body. Setting the Content-Type header to any value, including nil, disables that behavior.

func Serve ¶

Serve accepts incoming HTTP connections on the listener l, creating a new service goroutine for each. The service goroutines read requests and then call handler to reply to them.

The handler is typically nil, in which case the DefaultServeMux is used.

HTTP/2 support is only enabled if the Listener returns *tls.Conn connections and they were configured with «h2» in the TLS Config.NextProtos.

Serve always returns a non-nil error.

func ServeContent ¶

ServeContent replies to the request using the content in the provided ReadSeeker. The main benefit of ServeContent over io.Copy is that it handles Range requests properly, sets the MIME type, and handles If-Match, If-Unmodified-Since, If-None-Match, If-Modified-Since, and If-Range requests.

If the response’s Content-Type header is not set, ServeContent first tries to deduce the type from name’s file extension and, if that fails, falls back to reading the first block of the content and passing it to DetectContentType. The name is otherwise unused; in particular it can be empty and is never sent in the response.

If modtime is not the zero time or Unix epoch, ServeContent includes it in a Last-Modified header in the response. If the request includes an If-Modified-Since header, ServeContent uses modtime to decide whether the content needs to be sent at all.

The content’s Seek method must work: ServeContent uses a seek to the end of the content to determine its size.

If the caller has set w’s ETag header formatted per RFC 7232, section 2.3, ServeContent uses it to handle requests using If-Match, If-None-Match, or If-Range.

Note that *os.File implements the io.ReadSeeker interface.

func ServeFile ¶

ServeFile replies to the request with the contents of the named file or directory.

If the provided file or directory name is a relative path, it is interpreted relative to the current directory and may ascend to parent directories. If the provided name is constructed from user input, it should be sanitized before calling ServeFile.

As a precaution, ServeFile will reject requests where r.URL.Path contains a «..» path element; this protects against callers who might unsafely use filepath.Join on r.URL.Path without sanitizing it and then use that filepath.Join result as the name argument.

As another special case, ServeFile redirects any request where r.URL.Path ends in «/index.html» to the same path, without the final «index.html». To avoid such redirects either modify the path or use ServeContent.

Outside of those two special cases, ServeFile does not use r.URL.Path for selecting the file or directory to serve; only the file or directory provided in the name argument is used.

func ServeTLS ¶ 1.9

ServeTLS accepts incoming HTTPS connections on the listener l, creating a new service goroutine for each. The service goroutines read requests and then call handler to reply to them.

The handler is typically nil, in which case the DefaultServeMux is used.

Additionally, files containing a certificate and matching private key for the server must be provided. If the certificate is signed by a certificate authority, the certFile should be the concatenation of the server’s certificate, any intermediates, and the CA’s certificate.

ServeTLS always returns a non-nil error.

func SetCookie ¶

SetCookie adds a Set-Cookie header to the provided ResponseWriter’s headers. The provided cookie must have a valid Name. Invalid cookies may be silently dropped.

func StatusText ¶

StatusText returns a text for the HTTP status code. It returns the empty string if the code is unknown.

type Client ¶

A Client is an HTTP client. Its zero value (DefaultClient) is a usable client that uses DefaultTransport.

The Client’s Transport typically has internal state (cached TCP connections), so Clients should be reused instead of created as needed. Clients are safe for concurrent use by multiple goroutines.

A Client is higher-level than a RoundTripper (such as Transport) and additionally handles HTTP details such as cookies and redirects.

When following redirects, the Client will forward all headers set on the initial Request except:

• when forwarding sensitive headers like «Authorization», «WWW-Authenticate», and «Cookie» to untrusted targets. These headers will be ignored when following a redirect to a domain that is not a subdomain match or exact match of the initial domain. For example, a redirect from «foo.com» to either «foo.com» or «sub.foo.com» will forward the sensitive headers, but a redirect to «bar.com» will not.

• when forwarding the «Cookie» header with a non-nil cookie Jar. Since each redirect may mutate the state of the cookie jar, a redirect may possibly alter a cookie set in the initial request. When forwarding the «Cookie» header, any mutated cookies will be omitted, with the expectation that the Jar will insert those mutated cookies with the updated values (assuming the origin matches). If Jar is nil, the initial cookies are forwarded without change.

func (*Client) CloseIdleConnections ¶ 1.12

CloseIdleConnections closes any connections on its Transport which were previously connected from previous requests but are now sitting idle in a «keep-alive» state. It does not interrupt any connections currently in use.

If the Client’s Transport does not have a CloseIdleConnections method then this method does nothing.

func (*Client) Do ¶

Do sends an HTTP request and returns an HTTP response, following policy (such as redirects, cookies, auth) as configured on the client.

An error is returned if caused by client policy (such as CheckRedirect), or failure to speak HTTP (such as a network connectivity problem). A non-2xx status code doesn’t cause an error.

If the returned error is nil, the Response will contain a non-nil Body which the user is expected to close. If the Body is not both read to EOF and closed, the Client’s underlying RoundTripper (typically Transport) may not be able to re-use a persistent TCP connection to the server for a subsequent «keep-alive» request.

The request Body, if non-nil, will be closed by the underlying Transport, even on errors.

On error, any Response can be ignored. A non-nil Response with a non-nil error only occurs when CheckRedirect fails, and even then the returned Response.Body is already closed.

Generally Get, Post, or PostForm will be used instead of Do.

If the server replies with a redirect, the Client first uses the CheckRedirect function to determine whether the redirect should be followed. If permitted, a 301, 302, or 303 redirect causes subsequent requests to use HTTP method GET (or HEAD if the original request was HEAD), with no body. A 307 or 308 redirect preserves the original HTTP method and body, provided that the Request.GetBody function is defined. The NewRequest function automatically sets GetBody for common standard library body types.

Any returned error will be of type *url.Error. The url.Error value’s Timeout method will report true if the request timed out.

func (*Client) Get ¶

Get issues a GET to the specified URL. If the response is one of the following redirect codes, Get follows the redirect after calling the Client’s CheckRedirect function:

An error is returned if the Client’s CheckRedirect function fails or if there was an HTTP protocol error. A non-2xx response doesn’t cause an error. Any returned error will be of type *url.Error. The url.Error value’s Timeout method will report true if the request timed out.

When err is nil, resp always contains a non-nil resp.Body. Caller should close resp.Body when done reading from it.

To make a request with custom headers, use NewRequest and Client.Do.

To make a request with a specified context.Context, use NewRequestWithContext and Client.Do.

func (*Client) Head ¶

Head issues a HEAD to the specified URL. If the response is one of the following redirect codes, Head follows the redirect after calling the Client’s CheckRedirect function:

To make a request with a specified context.Context, use NewRequestWithContext and Client.Do.

func (*Client) Post ¶

Post issues a POST to the specified URL.

Caller should close resp.Body when done reading from it.

If the provided body is an io.Closer, it is closed after the request.

To set custom headers, use NewRequest and Client.Do.

To make a request with a specified context.Context, use NewRequestWithContext and Client.Do.

See the Client.Do method documentation for details on how redirects are handled.

func (*Client) PostForm ¶

PostForm issues a POST to the specified URL, with data’s keys and values URL-encoded as the request body.

The Content-Type header is set to application/x-www-form-urlencoded. To set other headers, use NewRequest and Client.Do.

When err is nil, resp always contains a non-nil resp.Body. Caller should close resp.Body when done reading from it.

See the Client.Do method documentation for details on how redirects are handled.

To make a request with a specified context.Context, use NewRequestWithContext and Client.Do.

type CloseNotifier ¶ 1.1

The CloseNotifier interface is implemented by ResponseWriters which allow detecting when the underlying connection has gone away.

This mechanism can be used to cancel long operations on the server if the client has disconnected before the response is ready.

Deprecated: the CloseNotifier interface predates Go’s context package. New code should use Request.Context instead.

type ConnState ¶ 1.3

A ConnState represents the state of a client connection to a server. It’s used by the optional Server.ConnState hook.

func (ConnState) String ¶ 1.3

type Cookie ¶

A Cookie represents an HTTP cookie as sent in the Set-Cookie header of an HTTP response or the Cookie header of an HTTP request.

func (*Cookie) String ¶

String returns the serialization of the cookie for use in a Cookie header (if only Name and Value are set) or a Set-Cookie response header (if other fields are set). If c is nil or c.Name is invalid, the empty string is returned.

func (*Cookie) Valid ¶ 1.18

Valid reports whether the cookie is valid.

type CookieJar ¶

A CookieJar manages storage and use of cookies in HTTP requests.

Implementations of CookieJar must be safe for concurrent use by multiple goroutines.

The net/http/cookiejar package provides a CookieJar implementation.

type Dir ¶

A Dir implements FileSystem using the native file system restricted to a specific directory tree.

While the FileSystem.Open method takes ‘/’-separated paths, a Dir’s string value is a filename on the native file system, not a URL, so it is separated by filepath.Separator, which isn’t necessarily ‘/’.

Note that Dir could expose sensitive files and directories. Dir will follow symlinks pointing out of the directory tree, which can be especially dangerous if serving from a directory in which users are able to create arbitrary symlinks. Dir will also allow access to files and directories starting with a period, which could expose sensitive directories like .git or sensitive files like .htpasswd. To exclude files with a leading period, remove the files/directories from the server or create a custom FileSystem implementation.

An empty Dir is treated as «.».

func (Dir) Open ¶

Open implements FileSystem using os.Open, opening files for reading rooted and relative to the directory d.

type File ¶

A File is returned by a FileSystem’s Open method and can be served by the FileServer implementation.

The methods should behave the same as those on an *os.File.

type FileSystem ¶

A FileSystem implements access to a collection of named files. The elements in a file path are separated by slash (‘/’, U+002F) characters, regardless of host operating system convention. See the FileServer function to convert a FileSystem to a Handler.

This interface predates the fs.FS interface, which can be used instead: the FS adapter function converts an fs.FS to a FileSystem.

func FS ¶ 1.16

FS converts fsys to a FileSystem implementation, for use with FileServer and NewFileTransport.

type Flusher ¶

The Flusher interface is implemented by ResponseWriters that allow an HTTP handler to flush buffered data to the client.

The default HTTP/1.x and HTTP/2 ResponseWriter implementations support Flusher, but ResponseWriter wrappers may not. Handlers should always test for this ability at runtime.

Note that even for ResponseWriters that support Flush, if the client is connected through an HTTP proxy, the buffered data may not reach the client until the response completes.

type Handler ¶

A Handler responds to an HTTP request.

ServeHTTP should write reply headers and data to the ResponseWriter and then return. Returning signals that the request is finished; it is not valid to use the ResponseWriter or read from the Request.Body after or concurrently with the completion of the ServeHTTP call.

Depending on the HTTP client software, HTTP protocol version, and any intermediaries between the client and the Go server, it may not be possible to read from the Request.Body after writing to the ResponseWriter. Cautious handlers should read the Request.Body first, and then reply.

Except for reading the body, handlers should not modify the provided Request.

If ServeHTTP panics, the server (the caller of ServeHTTP) assumes that the effect of the panic was isolated to the active request. It recovers the panic, logs a stack trace to the server error log, and either closes the network connection or sends an HTTP/2 RST_STREAM, depending on the HTTP protocol. To abort a handler so the client sees an interrupted response but the server doesn’t log an error, panic with the value ErrAbortHandler.

func AllowQuerySemicolons ¶ 1.17

AllowQuerySemicolons returns a handler that serves requests by converting any unescaped semicolons in the URL query to ampersands, and invoking the handler h.

This restores the pre-Go 1.17 behavior of splitting query parameters on both semicolons and ampersands. (See golang.org/issue/25192). Note that this behavior doesn’t match that of many proxies, and the mismatch can lead to security issues.

AllowQuerySemicolons should be invoked before Request.ParseForm is called.

func FileServer ¶

FileServer returns a handler that serves HTTP requests with the contents of the file system rooted at root.

As a special case, the returned file server redirects any request ending in «/index.html» to the same path, without the final «index.html».

To use the operating system’s file system implementation, use http.Dir:

To use an fs.FS implementation, use http.FS to convert it:

func MaxBytesHandler ¶ 1.18

MaxBytesHandler returns a Handler that runs h with its ResponseWriter and Request.Body wrapped by a MaxBytesReader.

func NotFoundHandler ¶

NotFoundHandler returns a simple request handler that replies to each request with a “404 page not found” reply.

func RedirectHandler ¶

RedirectHandler returns a request handler that redirects each request it receives to the given url using the given status code.

The provided code should be in the 3xx range and is usually StatusMovedPermanently, StatusFound or StatusSeeOther.

func StripPrefix ¶

StripPrefix returns a handler that serves HTTP requests by removing the given prefix from the request URL’s Path (and RawPath if set) and invoking the handler h. StripPrefix handles a request for a path that doesn’t begin with prefix by replying with an HTTP 404 not found error. The prefix must match exactly: if the prefix in the request contains escaped characters the reply is also an HTTP 404 not found error.

func TimeoutHandler ¶

TimeoutHandler returns a Handler that runs h with the given time limit.

The new Handler calls h.ServeHTTP to handle each request, but if a call runs for longer than its time limit, the handler responds with a 503 Service Unavailable error and the given message in its body. (If msg is empty, a suitable default message will be sent.) After such a timeout, writes by h to its ResponseWriter will return ErrHandlerTimeout.

TimeoutHandler supports the Pusher interface but does not support the Hijacker or Flusher interfaces.

type HandlerFunc ¶

The HandlerFunc type is an adapter to allow the use of ordinary functions as HTTP handlers. If f is a function with the appropriate signature, HandlerFunc(f) is a Handler that calls f.

func (HandlerFunc) ServeHTTP ¶

ServeHTTP calls f(w, r).

A Header represents the key-value pairs in an HTTP header.

The keys should be in canonical form, as returned by CanonicalHeaderKey.

Add adds the key, value pair to the header. It appends to any existing values associated with key. The key is case insensitive; it is canonicalized by CanonicalHeaderKey.

Clone returns a copy of h or nil if h is nil.

Del deletes the values associated with key. The key is case insensitive; it is canonicalized by CanonicalHeaderKey.

Get gets the first value associated with the given key. If there are no values associated with the key, Get returns «». It is case insensitive; textproto.CanonicalMIMEHeaderKey is used to canonicalize the provided key. Get assumes that all keys are stored in canonical form. To use non-canonical keys, access the map directly.

Set sets the header entries associated with key to the single element value. It replaces any existing values associated with key. The key is case insensitive; it is canonicalized by textproto.CanonicalMIMEHeaderKey. To use non-canonical keys, assign to the map directly.

Values returns all values associated with the given key. It is case insensitive; textproto.CanonicalMIMEHeaderKey is used to canonicalize the provided key. To use non-canonical keys, access the map directly. The returned slice is not a copy.

Write writes a header in wire format.

WriteSubset writes a header in wire format. If exclude is not nil, keys where exclude[key] == true are not written. Keys are not canonicalized before checking the exclude map.

type Hijacker ¶

The Hijacker interface is implemented by ResponseWriters that allow an HTTP handler to take over the connection.

The default ResponseWriter for HTTP/1.x connections supports Hijacker, but HTTP/2 connections intentionally do not. ResponseWriter wrappers may also not support Hijacker. Handlers should always test for this ability at runtime.

type MaxBytesError ¶ 1.19

MaxBytesError is returned by MaxBytesReader when its read limit is exceeded.

func (*MaxBytesError) Error ¶ 1.19

type ProtocolError ¶

ProtocolError represents an HTTP protocol error.

Deprecated: Not all errors in the http package related to protocol errors are of type ProtocolError.

func (*ProtocolError) Error ¶

type PushOptions ¶ 1.8

PushOptions describes options for Pusher.Push.

type Pusher ¶ 1.8

Pusher is the interface implemented by ResponseWriters that support HTTP/2 server push. For more background, see https://tools.ietf.org/html/rfc7540#section-8.2.

type Request ¶

A Request represents an HTTP request received by a server or to be sent by a client.

The field semantics differ slightly between client and server usage. In addition to the notes on the fields below, see the documentation for Request.Write and RoundTripper.

func NewRequest ¶

NewRequest wraps NewRequestWithContext using context.Background.

func NewRequestWithContext ¶ 1.13

NewRequestWithContext returns a new Request given a method, URL, and optional body.

If the provided body is also an io.Closer, the returned Request.Body is set to body and will be closed by the Client methods Do, Post, and PostForm, and Transport.RoundTrip.

NewRequestWithContext returns a Request suitable for use with Client.Do or Transport.RoundTrip. To create a request for use with testing a Server Handler, either use the NewRequest function in the net/http/httptest package, use ReadRequest, or manually update the Request fields. For an outgoing client request, the context controls the entire lifetime of a request and its response: obtaining a connection, sending the request, and reading the response headers and body. See the Request type’s documentation for the difference between inbound and outbound request fields.

If body is of type *bytes.Buffer, *bytes.Reader, or *strings.Reader, the returned request’s ContentLength is set to its exact value (instead of -1), GetBody is populated (so 307 and 308 redirects can replay the body), and Body is set to NoBody if the ContentLength is 0.

func ReadRequest ¶

ReadRequest reads and parses an incoming request from b.

ReadRequest is a low-level function and should only be used for specialized applications; most code should use the Server to read requests and handle them via the Handler interface. ReadRequest only supports HTTP/1.x requests. For HTTP/2, use golang.org/x/net/http2.

func (*Request) AddCookie ¶

AddCookie adds a cookie to the request. Per RFC 6265 section 5.4, AddCookie does not attach more than one Cookie header field. That means all cookies, if any, are written into the same line, separated by semicolon. AddCookie only sanitizes c’s name and value, and does not sanitize a Cookie header already present in the request.

func (*Request) BasicAuth ¶ 1.4

BasicAuth returns the username and password provided in the request’s Authorization header, if the request uses HTTP Basic Authentication. See RFC 2617, Section 2.

func (*Request) Clone ¶ 1.13

Clone returns a deep copy of r with its context changed to ctx. The provided ctx must be non-nil.

For an outgoing client request, the context controls the entire lifetime of a request and its response: obtaining a connection, sending the request, and reading the response headers and body.

func (*Request) Context ¶ 1.7

Context returns the request’s context. To change the context, use WithContext.

The returned context is always non-nil; it defaults to the background context.

For outgoing client requests, the context controls cancellation.

For incoming server requests, the context is canceled when the client’s connection closes, the request is canceled (with HTTP/2), or when the ServeHTTP method returns.

func (*Request) Cookie ¶

Cookie returns the named cookie provided in the request or ErrNoCookie if not found. If multiple cookies match the given name, only one cookie will be returned.

func (*Request) Cookies ¶

Cookies parses and returns the HTTP cookies sent with the request.

func (*Request) FormFile ¶

FormFile returns the first file for the provided form key. FormFile calls ParseMultipartForm and ParseForm if necessary.

func (*Request) FormValue ¶

FormValue returns the first value for the named component of the query. POST and PUT body parameters take precedence over URL query string values. FormValue calls ParseMultipartForm and ParseForm if necessary and ignores any errors returned by these functions. If key is not present, FormValue returns the empty string. To access multiple values of the same key, call ParseForm and then inspect Request.Form directly.

func (*Request) MultipartReader ¶

MultipartReader returns a MIME multipart reader if this is a multipart/form-data or a multipart/mixed POST request, else returns nil and an error. Use this function instead of ParseMultipartForm to process the request body as a stream.

func (*Request) ParseForm ¶

ParseForm populates r.Form and r.PostForm.

For all requests, ParseForm parses the raw query from the URL and updates r.Form.

For POST, PUT, and PATCH requests, it also reads the request body, parses it as a form and puts the results into both r.PostForm and r.Form. Request body parameters take precedence over URL query string values in r.Form.

If the request Body’s size has not already been limited by MaxBytesReader, the size is capped at 10MB.

For other HTTP methods, or when the Content-Type is not application/x-www-form-urlencoded, the request Body is not read, and r.PostForm is initialized to a non-nil, empty value.

ParseMultipartForm calls ParseForm automatically. ParseForm is idempotent.

func (*Request) ParseMultipartForm ¶

ParseMultipartForm parses a request body as multipart/form-data. The whole request body is parsed and up to a total of maxMemory bytes of its file parts are stored in memory, with the remainder stored on disk in temporary files. ParseMultipartForm calls ParseForm if necessary. If ParseForm returns an error, ParseMultipartForm returns it but also continues parsing the request body. After one call to ParseMultipartForm, subsequent calls have no effect.

func (*Request) PostFormValue ¶ 1.1

PostFormValue returns the first value for the named component of the POST, PATCH, or PUT request body. URL query parameters are ignored. PostFormValue calls ParseMultipartForm and ParseForm if necessary and ignores any errors returned by these functions. If key is not present, PostFormValue returns the empty string.

func (*Request) ProtoAtLeast ¶

ProtoAtLeast reports whether the HTTP protocol used in the request is at least major.minor.

func (*Request) Referer ¶

Referer returns the referring URL, if sent in the request.

Referer is misspelled as in the request itself, a mistake from the earliest days of HTTP. This value can also be fetched from the Header map as Header[«Referer»]; the benefit of making it available as a method is that the compiler can diagnose programs that use the alternate (correct English) spelling req.Referrer() but cannot diagnose programs that use Header[«Referrer»].

func (*Request) SetBasicAuth ¶

SetBasicAuth sets the request’s Authorization header to use HTTP Basic Authentication with the provided username and password.

With HTTP Basic Authentication the provided username and password are not encrypted. It should generally only be used in an HTTPS request.

The username may not contain a colon. Some protocols may impose additional requirements on pre-escaping the username and password. For instance, when used with OAuth2, both arguments must be URL encoded first with url.QueryEscape.

func (*Request) UserAgent ¶

UserAgent returns the client’s User-Agent, if sent in the request.

func (*Request) WithContext ¶ 1.7

WithContext returns a shallow copy of r with its context changed to ctx. The provided ctx must be non-nil.

For outgoing client request, the context controls the entire lifetime of a request and its response: obtaining a connection, sending the request, and reading the response headers and body.

To create a new request with a context, use NewRequestWithContext. To change the context of a request, such as an incoming request you want to modify before sending back out, use Request.Clone. Between those two uses, it’s rare to need WithContext.

func (*Request) Write ¶

Write writes an HTTP/1.1 request, which is the header and body, in wire format. This method consults the following fields of the request:

If Body is present, Content-Length is Example

func Head ¶

Head issues a HEAD to the specified URL. If the response is one of the following redirect codes, Head follows the redirect, up to a maximum of 10 redirects:

Head is a wrapper around DefaultClient.Head.

To make a request with a specified context.Context, use NewRequestWithContext and DefaultClient.Do.

func Post ¶

Post issues a POST to the specified URL.

Caller should close resp.Body when done reading from it.

If the provided body is an io.Closer, it is closed after the request.

Post is a wrapper around DefaultClient.Post.

To set custom headers, use NewRequest and DefaultClient.Do.

See the Client.Do method documentation for details on how redirects are handled.

To make a request with a specified context.Context, use NewRequestWithContext and DefaultClient.Do.

func PostForm ¶

PostForm issues a POST to the specified URL, with data’s keys and values URL-encoded as the request body.

The Content-Type header is set to application/x-www-form-urlencoded. To set other headers, use NewRequest and DefaultClient.Do.

When err is nil, resp always contains a non-nil resp.Body. Caller should close resp.Body when done reading from it.

PostForm is a wrapper around DefaultClient.PostForm.

See the Client.Do method documentation for details on how redirects are handled.

To make a request with a specified context.Context, use NewRequestWithContext and DefaultClient.Do.

func ReadResponse ¶

ReadResponse reads and returns an HTTP response from r. The req parameter optionally specifies the Request that corresponds to this Response. If nil, a GET request is assumed. Clients must call resp.Body.Close when finished reading resp.Body. After that call, clients can inspect resp.Trailer to find key/value pairs included in the response trailer.

func (*Response) Cookies ¶

Cookies parses and returns the cookies set in the Set-Cookie headers.

func (*Response) Location ¶

Location returns the URL of the response’s «Location» header, if present. Relative redirects are resolved relative to the Response’s Request. ErrNoLocation is returned if no Location header is present.

func (*Response) ProtoAtLeast ¶

ProtoAtLeast reports whether the HTTP protocol used in the response is at least major.minor.

func (*Response) Write ¶

Write writes r to w in the HTTP/1.x server response format, including the status line, headers, body, and optional trailer.

This method consults the following fields of the response r:

The Response Body is closed after it is sent.

type ResponseWriter ¶

A ResponseWriter interface is used by an HTTP handler to construct an HTTP response.

A ResponseWriter may not be used after the Handler.ServeHTTP method has returned.

HTTP Trailers are a set of key/value pairs like headers that come after the HTTP response, instead of before.

type RoundTripper ¶

RoundTripper is an interface representing the ability to execute a single HTTP transaction, obtaining the Response for a given Request.

A RoundTripper must be safe for concurrent use by multiple goroutines.

DefaultTransport is the default implementation of Transport and is used by DefaultClient. It establishes network connections as needed and caches them for reuse by subsequent calls. It uses HTTP proxies as directed by the $HTTP_PROXY and $NO_PROXY (or $http_proxy and $no_proxy) environment variables.

func NewFileTransport ¶

NewFileTransport returns a new RoundTripper, serving the provided FileSystem. The returned RoundTripper ignores the URL host in its incoming requests, as well as most other properties of the request.

The typical use case for NewFileTransport is to register the «file» protocol with a Transport, as in:

type SameSite ¶ 1.11

SameSite allows a server to define a cookie attribute making it impossible for the browser to send this cookie along with cross-site requests. The main goal is to mitigate the risk of cross-origin information leakage, and provide some protection against cross-site request forgery attacks.

type ServeMux ¶

ServeMux is an HTTP request multiplexer. It matches the URL of each incoming request against a list of registered patterns and calls the handler for the pattern that most closely matches the URL.

Patterns name fixed, rooted paths, like «/favicon.ico», or rooted subtrees, like «/images/» (note the trailing slash). Longer patterns take precedence over shorter ones, so that if there are handlers registered for both «/images/» and «/images/thumbnails/», the latter handler will be called for paths beginning «/images/thumbnails/» and the former will receive requests for any other paths in the «/images/» subtree.

Note that since a pattern ending in a slash names a rooted subtree, the pattern «/» matches all paths not matched by other registered patterns, not just the URL with Path == «/».

If a subtree has been registered and a request is received naming the subtree root without its trailing slash, ServeMux redirects that request to the subtree root (adding the trailing slash). This behavior can be overridden with a separate registration for the path without the trailing slash. For example, registering «/images/» causes ServeMux to redirect a request for «/images» to «/images/», unless «/images» has been registered separately.

Patterns may optionally begin with a host name, restricting matches to URLs on that host only. Host-specific patterns take precedence over general patterns, so that a handler might register for the two patterns «/codesearch» and «codesearch.google.com/» without also taking over requests for «http://www.google.com/».

ServeMux also takes care of sanitizing the URL request path and the Host header, stripping the port number and redirecting any request containing . or .. elements or repeated slashes to an equivalent, cleaner URL.

func NewServeMux ¶

NewServeMux allocates and returns a new ServeMux.

func (*ServeMux) Handle ¶

Handle registers the handler for the given pattern. If a handler already exists for pattern, Handle panics.

Источник

Handling errors is really important in Go. Errors are first class citizens and there are many different approaches for handling them. Initially I started off basing my error handling almost entirely on a blog post from Rob Pike and created a carve-out from his code to meet my needs. It served me well for a long time, but found over time I wanted a way to easily get a stacktrace of the error, which led me to Dave Cheney’s https://github.com/pkg/errors package. I now use a combination of the two. The implementation below is sourced from my go-api-basic repo, indeed, this post will be folded into its README as well.

Requirements

My requirements for REST API error handling are the following:

• Requests for users who are not properly authenticated should return a 401 Unauthorized error with a WWW-Authenticate response header and an empty response body.
• Requests for users who are authenticated, but do not have permission to access the resource, should return a 403 Forbidden error with an empty response body.
• All requests which are due to a client error (invalid data, malformed JSON, etc.) should return a 400 Bad Request and a response body which looks similar to the following:

{
    "error": {
        "kind": "input_validation_error",
        "param": "director",
        "message": "director is required"
    }
}

• All requests which incur errors as a result of an internal server or database error should return a 500 Internal Server Error and not leak any information about the database or internal systems to the client. These errors should return a response body which looks like the following:

{
    "error": {
        "kind": "internal_error",
        "message": "internal server error - please contact support"
    }
}

All errors should return a Request-Id response header with a unique request id that can be used for debugging to find the corresponding error in logs.

Implementation

All errors should be raised using custom errors from the domain/errs package. The three custom errors correspond directly to the requirements above.

Typical Errors

Typically, errors raised throughout go-api-basic are the custom errs.Error, which looks like:

type Error struct {
   // User is the username of the user attempting the operation.
   User UserName
   // Kind is the class of error, such as permission failure,
   // or "Other" if its class is unknown or irrelevant.
   Kind Kind
   // Param represents the parameter related to the error.
   Param Parameter
   // Code is a human-readable, short representation of the error
   Code Code
   // The underlying error that triggered this one, if any.
   Err error
}

These errors are raised using the E function from the domain/errs package. errs.E is taken from Rob Pike’s upspin errors package (but has been changed based on my requirements). The errs.E function call is variadic and can take several different types to form the custom errs.Error struct.

Here is a simple example of creating an error using errs.E:

err := errs.E("seems we have an error here")

When a string is sent, an error will be created using the errors.New function from github.com/pkg/errors and added to the Err element of the struct, which allows retrieval of the error stacktrace later on. In the above example, User, Kind, Param and Code would all remain unset.

You can set any of these custom errs.Error fields that you like, for example:

func (m *Movie) SetReleased(r string) (*Movie, error) {
    t, err := time.Parse(time.RFC3339, r)
    if err != nil {
        return nil, errs.E(errs.Validation,
            errs.Code("invalid_date_format"),
            errs.Parameter("release_date"),
            err)
    }
    m.Released = t
    return m, nil
}

Above, we used errs.Validation to set the errs.Kind as Validation. Valid error Kind are:

const (
    Other           Kind = iota // Unclassified error. This value is not printed in the error message.
    Invalid                     // Invalid operation for this type of item.
    IO                          // External I/O error such as network failure.
    Exist                       // Item already exists.
    NotExist                    // Item does not exist.
    Private                     // Information withheld.
    Internal                    // Internal error or inconsistency.
    BrokenLink                  // Link target does not exist.
    Database                    // Error from database.
    Validation                  // Input validation error.
    Unanticipated               // Unanticipated error.
    InvalidRequest              // Invalid Request
)

errs.Code represents a short code to respond to the client with for error handling based on codes (if you choose to do this) and is any string you want to pass.

errs.Parameter represents the parameter that is being validated or has problems, etc.

Note in the above example, instead of passing a string and creating a new error inside the errs.E function, I am directly passing the error returned by the time.Parse function to errs.E. The error is then added to the Err field using errors.WithStack from the github.com/pkg/errors package. This will enable stacktrace retrieval later as well.

There are a few helpers in the errs package as well, namely the errs.MissingField function which can be used when validating missing input on a field. This idea comes from this Mat Ryer post and is pretty handy.

Here is an example in practice:

// IsValid performs validation of the struct
func (m *Movie) IsValid() error {
    switch {
    case m.Title == "":
        return errs.E(errs.Validation, errs.Parameter("title"), errs.MissingField("title"))

The error message for the above would read title is required

There is also errs.InputUnwanted which is meant to be used when a field is populated with a value when it is not supposed to be.

Typical Error Flow

As errors created with errs.E move up the call stack, they can just be returned, like the following:

func inner() error {
    return errs.E("seems we have an error here")
}

func middle() error {
    err := inner()
    if err != nil {
        return err
    }
    return nil
}

func outer() error {
    err := middle()
    if err != nil {
        return err
    }
    return nil
}

In the above example, the error is created in the inner function — middle and outer return the error as is typical in Go.

You can add additional context fields (errs.Code, errs.Parameter, errs.Kind) as the error moves up the stack, however, I try to add as much context as possible at the point of error origin and only do this in rare cases.

Handler Flow

At the top of the program flow for each service is the app service handler (for example, Server.handleMovieCreate). In this handler, any error returned from any function or method is sent through the errs.HTTPErrorResponse function along with the http.ResponseWriter and a zerolog.Logger.

For example:

response, err := s.CreateMovieService.Create(r.Context(), rb, u)
if err != nil {
    errs.HTTPErrorResponse(w, logger, err)
    return
}

errs.HTTPErrorResponse takes the custom error (errs.Error, errs.Unauthenticated or errs.UnauthorizedError), writes the response to the given http.ResponseWriter and logs the error using the given zerolog.Logger.

return must be called immediately after errs.HTTPErrorResponse to return the error to the client.

Typical Error Response

For the errs.Error type, errs.HTTPErrorResponse writes the HTTP response body as JSON using the errs.ErrResponse struct.

// ErrResponse is used as the Response Body
type ErrResponse struct {
    Error ServiceError `json:"error"`
}

// ServiceError has fields for Service errors. All fields with no data will
// be omitted
type ServiceError struct {
    Kind    string `json:"kind,omitempty"`
    Code    string `json:"code,omitempty"`
    Param   string `json:"param,omitempty"`
    Message string `json:"message,omitempty"`
}

When the error is returned to the client, the response body JSON looks like the following:

{
    "error": {
        "kind": "input_validation_error",
        "code": "invalid_date_format",
        "param": "release_date",
        "message": "parsing time "1984a-03-02T00:00:00Z" as "2006-01-02T15:04:05Z07:00": cannot parse "a-03-02T00:00:00Z" as "-""
    }
}

In addition, the error is logged. If zerolog.ErrorStackMarshaler is set to log error stacks (more about this in a later post), the logger will log the full error stack, which can be super helpful when trying to identify issues.

The error log will look like the following (I cut off parts of the stack for brevity):

{
    "level": "error",
    "ip": "127.0.0.1",
    "user_agent": "PostmanRuntime/7.26.8",
    "request_id": "bvol0mtnf4q269hl3ra0",
    "stack": [{
        "func": "E",
        "line": "172",
        "source": "errs.go"
    }, {
        "func": "(*Movie).SetReleased",
        "line": "76",
        "source": "movie.go"
    }, {
        "func": "(*MovieController).CreateMovie",
        "line": "139",
        "source": "create.go"
    }, {
    ...
    }],
    "error": "parsing time "1984a-03-02T00:00:00Z" as "2006-01-02T15:04:05Z07:00": cannot parse "a-03-02T00:00:00Z" as "-"",
    "HTTPStatusCode": 400,
    "Kind": "input_validation_error",
    "Parameter": "release_date",
    "Code": "invalid_date_format",
    "time": 1609650267,
    "severity": "ERROR",
    "message": "Response Error Sent"
}

Note: E will usually be at the top of the stack as it is where the errors.New or errors.WithStack functions are being called.

Internal or Database Error Response

There is logic within errs.HTTPErrorResponse to return a different response body if the errs.Kind is Internal or Database. As per the requirements, we should not leak the error message or any internal stack, etc. when an internal or database error occurs. If an error comes through and is an errs.Error with either of these error Kind or is unknown error type in any way, the response will look like the following:

{
    "error": {
        "kind": "internal_error",
        "message": "internal server error - please contact support"
    }
}

Unauthenticated Errors

type UnauthenticatedError struct {
    // WWWAuthenticateRealm is a description of the protected area.
    // If no realm is specified, "DefaultRealm" will be used as realm
    WWWAuthenticateRealm string

    // The underlying error that triggered this one, if any.
    Err error
}

The spec for 401 Unauthorized calls for a WWW-Authenticate response header along with a realm. The realm should be set when creating an Unauthenticated error. The errs.NewUnauthenticatedError function initializes an UnauthenticatedError.

I generally like to follow the Go idiom for brevity in all things as much as possible, but for Unauthenticated vs. Unauthorized errors, it’s confusing enough as it is already, I don’t take any shortcuts.

func NewUnauthenticatedError(realm string, err error) *UnauthenticatedError {
    return &UnauthenticatedError{WWWAuthenticateRealm: realm, Err: err}
}

Unauthenticated Error Flow

The errs.Unauthenticated error should only be raised at points of authentication as part of a middleware handler. I will get into application flow in detail later, but authentication for go-api-basic happens in middleware handlers prior to calling the app handler for the given route.

• The WWW-Authenticate realm is set to the request context using the defaultRealmHandler middleware in the app package prior to attempting authentication.
• Next, the Oauth2 access token is retrieved from the Authorization http header using the accessTokenHandler middleware. There are several access token validations in this middleware, if any are not successful, the errs.Unauthenticated error is returned using the realm set to the request context.
• Finally, if the access token is successfully retrieved, it is then converted to a User via the GoogleAccessTokenConverter.Convert method in the gateway/authgateway package. This method sends an outbound request to Google using their API; if any errors are returned, an errs.Unauthenticated error is returned.

In general, I do not like to use context.Context, however, it is used in go-api-basic to pass values between middlewares. The WWW-Authenticate realm, the Oauth2 access token and the calling user after authentication, all of which are request-scoped values, are all set to the request context.Context.

Unauthenticated Error Response

Per requirements, go-api-basic does not return a response body when returning an Unauthenticated error. The error response from cURL looks like the following:

HTTP/1.1 401 Unauthorized
Request-Id: c30hkvua0brkj8qhk3e0
Www-Authenticate: Bearer realm="go-api-basic"
Date: Wed, 09 Jun 2021 19:46:07 GMT
Content-Length: 0

Unauthorized Errors

type UnauthorizedError struct {
    // The underlying error that triggered this one, if any.
    Err error
}

The errs.NewUnauthorizedError function initializes an UnauthorizedError.

Unauthorized Error Flow

The errs.Unauthorized error is raised when there is a permission issue for a user when attempting to access a resource. Currently, go-api-basic’s placeholder authorization implementation Authorizer.Authorize in the domain/auth package performs rudimentary checks that a user has access to a resource. If the user does not have access, the errs.Unauthorized error is returned.

Per requirements, go-api-basic does not return a response body when returning an Unauthorized error. The error response from cURL looks like the following:

HTTP/1.1 403 Forbidden
Request-Id: c30hp2ma0brkj8qhk3f0
Date: Wed, 09 Jun 2021 19:54:50 GMT
Content-Length: 0

404 Error Illustration by Pixeltrue from Ouch!

Introduction

This guide explains error handling in Go and the best practices for handling API errors in Go. You should have a working knowledge of Go and understand how web APIs work to get the most benefit from this article.

Go has a unique approach to handling errors. Rather than utilizing a try..catch block like other programming languages such as C++, Go treats errors as first-class values with panic and recovery mechanisms.

Go has a built-in type called error, which exposes an interface that implements the Error() method:

type error interface {

  Error() string

}

Typically the usual way to handle errors in Go is to check if the returned error value is nil. If it’s equal to nil, then it means no errors occurred.

Go functions can also return multiple values. In cases where a function can fail, it’s a good idea to return the error status as a second return value. If the function doesn’t return anything, you should return the error status.

func myFunc() error {

  // do something

}



err := myFunc()



if err != nil {

  // do something

}

You can define errors using the errors.New() function, and have it printed out to the console using the Error() method:

func myFunc() error {

  myErr := errors.New(My Error”)

  return myErr

}



err := myFunc()



if err != nil {

  fmt.Println(err.Error())

}

This prints our defined error(My Error») to the screen.

The panic and recovery mechanisms work a bit differently. As panic halts program flow and causes the execution to exit with a non-zero status code, and then prints out the stack trace and error.

if err != nil {

  panic(err)

}

When a function calls panic(), it won’t be executed further, but all deferred functions will be called. Recover is typically used with this defer mechanism to rescue the program from the panic.

func panicAndRecover() {

  defer func() {

    if err := recover(); err != nil {

      fmt.Println(Successfully recovered from panic”)

    }

  }()



  panic(Panicking")

}

When the panicAndRecover function is called, it panics, but rather than returning a non-zero exit status and exiting, it runs the deferred anonymous function first. This recovers from the panic using recover(), and prints out to the screen. Normal program execution occurs without exiting immediately as we successfully recover from panic.

API Error Handling

When building web APIs, appropriately handling errors is an integral part of the development process. Examples of such errors include JSON parsing errors, wrong endpoint requests, etc.

Let’s dive into some of the best practices for handling API errors in Go:

Using Appropriate HTTP Status Codes

HTTP status codes communicate the status of an HTTP request; you must carefully return status codes representing the state of a request.

HTTP status codes are split into five categories:

• 1xx — Information

• 2xx — Success

• 3xx — Redirection

• 4xx — Client error

• 5xx — Server error

Many developers using your API might rely solely on the status code to see if the request was successful. Sending a 200 (success) code followed by an error is bad practice. Instead, it is proper to return a more appropriate code, such as 400 (bad request).

For instance, consider the following reqChecker function, which takes a request body and returns nil or some error value, depending on the condition of the request body. An http.StatusOK (200 status code) is misleading if the request body doesn’t conform to our standard.

func checker(w http.ResponseWriter, r *http.Request) {

  w.WriteHeader(http.StatusOK)

  if err := reqChecker(r.Body); err != nil {

    fmt.Fprintf(w, Invalid request body error: %s”, err.Error())

  }

}

We can refactor the handler to return a more appropriate error code:

func checker(w http.ResponseWriter, r *http.Request) {

  if err := reqChecker(r.Body); err != nil {

    w.WriteHeader(http.StatusBadRequest)     // 400 http status code

    fmt.Fprintf(w, Invalid request body error:%s”, err.Error())

  } else {

    w.writeHeader(http.StatusOK)

  }

}

There are quite a large number of HTTP status codes available, but just a subset of them are actually used in practice, with the most common ones being:

• 200 — Ok

• 201 — Created

• 304 — Not modified

• 400 — Bad Request

• 401 — Unauthorized

• 403 — Forbidden

• 404 — Not Found

• 500 — Internal Server Error

• 503 — Service unavailable

It’s not ideal to use too many status codes in your application. Keeping it to a minimum is recommended practice.

Descriptive Error Messages

While sending proper HTTP status error codes is a very important step in handling API errors, returning descriptive messages provides the client with additional information.

Multiple errors could return the same error code. For instance, posting a wrong JSON request format and malformed requests could spawn the same http.StatusBadRequest error (status code 400). The status code indicates that the request failed due to the client’s error but didn’t provide much about the nature of the error.

Returning a JSON response to the client alongside the error code like the following is more descriptive:

{

    "error": "Error parsing JSON request",

    "message": "Invalid JSON format",

    "detail": "Post the 'author' and 'id' fields in the request body."

}

Note: Descriptive error messages should be framed carefully not to expose the inner workings of the API to attackers.

The error field within the response should be unique across your application, and the detail field should provide more information about the error or how to fix it.

Avoid generic error messages, as it’s much better to know why specifically the request failed rather than a generic error code.

Exhaustive Documentation

Documentation is an essential part of API development. While API errors can be handled with ease by formulating proper responses when errors occur, comprehensive documentation helps the clients know all the relevant information concerning your API, such as what endpoints are provided by your API, the response and request formats, parameter options, and more.

The more exhaustive your documentation, the less likely clients will spend time battling API errors.

Conclusion

In this guide, we have learned about error handling in Go and some of the best practices for handling API errors.

•••

I wrote an article a while back on implementing custom handler types to avoid a few common problems with the existing http.HandlerFunc—the func MyHandler(w http.ResponseWriter, r *http.Request) signature you often see. It’s a useful “general purpose” handler type that covers the basics, but—as with anything generic—there are a few shortcomings:

• Having to remember to explicitly call a naked return when you want to stop processing in the handler. This is a common case when you want to raise a re-direct (301/302), not found (404) or internal server error (500) status. Failing to do so can be the cause of subtle bugs (the function will continue) and because the function signature doesn’t require a return value, the compiler won’t alert you.
• You can’t easily pass in additional arguments (i.e. database pools, configuration values). You end up having to either use a bunch of globals (not terrible, but tracking them can scale poorly) or stash those things into a request context and then type assert each of them out. Can be clunky.
• You end up repeating yourself. Want to log the error returned by your DB package? You can either call log.Printf in your database package (in each query func), or in every handler when an error is returned. It’d be great if your handlers could just return that to a function that centrally logs errors and raise a HTTP 500 on the ones that call for it.

My previous approach used the func(http.ResponseWriter, *http.Request) (int, error) signature. This has proven to be pretty neat, but a quirk is that returning “non error” status codes like 200, 302, 303 was often superfluous—you’re either setting it elsewhere or it’s effectively unused — e.g.

func SomeHandler(w http.ResponseWriter, r *http.Request) (int, error) {
    db, err := someDBcall()
    if err != nil {
        // This makes sense.
        return 500, err
    }

    if user.LoggedIn {
        http.Redirect(w, r, "/dashboard", 302)
        // Superfluous! Our http.Redirect function handles the 302, not 
        // our return value (which is effectively ignored).
        return 302, nil
    }

}

It’s not terrible, but we can do better.

A Little Different

So how can we improve on this? Let’s lay out some code:

package handler

// Error represents a handler error. It provides methods for a HTTP status 
// code and embeds the built-in error interface.
type Error interface {
	error
	Status() int
}

// StatusError represents an error with an associated HTTP status code.
type StatusError struct {
	Code int
	Err  error
}

// Allows StatusError to satisfy the error interface.
func (se StatusError) Error() string {
	return se.Err.Error()
}

// Returns our HTTP status code.
func (se StatusError) Status() int {
	return se.Code
}

// A (simple) example of our application-wide configuration.
type Env struct {
	DB   *sql.DB
	Port string
	Host string
}

// The Handler struct that takes a configured Env and a function matching
// our useful signature.
type Handler struct {
	*Env
	H func(e *Env, w http.ResponseWriter, r *http.Request) error
}

// ServeHTTP allows our Handler type to satisfy http.Handler.
func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	err := h.H(h.Env, w, r)
	if err != nil {
		switch e := err.(type) {
		case Error:
			// We can retrieve the status here and write out a specific
			// HTTP status code.
			log.Printf("HTTP %d - %s", e.Status(), e)
			http.Error(w, e.Error(), e.Status())
		default:
			// Any error types we don't specifically look out for default
			// to serving a HTTP 500
			http.Error(w, http.StatusText(http.StatusInternalServerError),
				http.StatusInternalServerError)
		}
	}
}

The code above should be self-explanatory, but to clarify any outstanding points:

• We create a custom Error type (an interface) that embeds Go’s built-in error interface and also has a Status() int method.
• We provide a simple StatusError type (a struct) that satisfies our handler.Error type. Our StatusError type accepts a HTTP status code (an int) and an error that allows us to wrap the root cause for logging/inspection.
• Our ServeHTTP method contains a type switch—which is the e := err.(type) part that tests for the errors we care about and allows us to handle those specific cases. In our example that’s just the handler.Error type. Other error types—be they from other packages (e.g. net.Error) or additional error types we have defined—can also be inspected (if we care about their details).

If we don’t want to inspect them, our default case catches them. Remember that the ServeHTTP method allows our Handler type to satisfy the http.Handler interface and be used anywhere http.Handler is accepted: Go’s net/http package and all good third party frameworks. This is what makes custom handler types so useful: they’re flexible about where they can be used.

Note that the net package does something very similar. It has a net.Error interface that embeds the built-in error interface and then a handful of concrete types that implement it. Functions return the concrete type that suits the type of error they’re returning (a DNS error, a parsing error, etc). A good example would be defining a DBError type with a Query() string method in a ‘datastore’ package that we can use to log failed queries.

Full Example

What does the end result look like? And how would we split it up into packages (sensibly)?

package handler

import (
    "net/http"
)

// Error represents a handler error. It provides methods for a HTTP status 
// code and embeds the built-in error interface.
type Error interface {
	error
	Status() int
}

// StatusError represents an error with an associated HTTP status code.
type StatusError struct {
	Code int
	Err  error
}

// Allows StatusError to satisfy the error interface.
func (se StatusError) Error() string {
	return se.Err.Error()
}

// Returns our HTTP status code.
func (se StatusError) Status() int {
	return se.Code
}

// A (simple) example of our application-wide configuration.
type Env struct {
	DB   *sql.DB
	Port string
	Host string
}

// The Handler struct that takes a configured Env and a function matching
// our useful signature.
type Handler struct {
	*Env
	H func(e *Env, w http.ResponseWriter, r *http.Request) error
}

// ServeHTTP allows our Handler type to satisfy http.Handler.
func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	err := h.H(h.Env, w, r)
	if err != nil {
		switch e := err.(type) {
		case Error:
			// We can retrieve the status here and write out a specific
			// HTTP status code.
			log.Printf("HTTP %d - %s", e.Status(), e)
			http.Error(w, e.Error(), e.Status())
		default:
			// Any error types we don't specifically look out for default
			// to serving a HTTP 500
			http.Error(w, http.StatusText(http.StatusInternalServerError),
				http.StatusInternalServerError)
		}
	}
}

func GetIndex(env *Env, w http.ResponseWriter, r *http.Request) error {
    users, err := env.DB.GetAllUsers()
    if err != nil {
        // We return a status error here, which conveniently wraps the error
        // returned from our DB queries. We can clearly define which errors 
        // are worth raising a HTTP 500 over vs. which might just be a HTTP 
        // 404, 403 or 401 (as appropriate). It's also clear where our 
        // handler should stop processing by returning early.
        return StatusError{500, err}
    }

    fmt.Fprintf(w, "%+v", users)
    return nil
}

… and in our main package:

package main

import (
    "net/http"
    "github.com/you/somepkg/handler"
)

func main() {
    db, err := sql.Open("connectionstringhere")
    if err != nil {
          log.Fatal(err)
    }

    // Initialise our app-wide environment with the services/info we need.
    env := &handler.Env{
        DB: db,
        Port: os.Getenv("PORT"),
        Host: os.Getenv("HOST"),
        // We might also have a custom log.Logger, our 
        // template instance, and a config struct as fields 
        // in our Env struct.
    }

    // Note that we're using http.Handle, not http.HandleFunc. The 
    // latter only accepts the http.HandlerFunc type, which is not 
    // what we have here.
    http.Handle("/", handler.Handler{env, handler.GetIndex})

    // Logs the error if ListenAndServe fails.
    log.Fatal(http.ListenAndServe(":8000", nil))
}

In the real world, you’re likely to define your Handler and Env types in a separate file (of the same package) from your handler functions, but I’ve keep it simple here for the sake of brevity. So what did we end up getting from this?

• A practical Handler type that satisfies http.Handler can be used with
  frameworks like net/http, gorilla/mux,
  Goji and any others that sensibly accept a http.Handler type.
• Clear, centralised error handling. We inspect the errors we want to handle
  specifically—our handler.Error type—and fall back to a default
  for generic errors. If you’re interested in better error handling practices in Go,
  read Dave Cheney’s blog post,
  which dives into defining package-level Error interfaces.
• A useful application-wide “environment” via our Env type. We don’t have to
  scatter a bunch of globals across our applications: instead we define them in
  one place and pass them explicitly to our handlers.

If you have questions about the post, drop me a line via @elithrar on Twitter,
or the Gopher community on Slack.

Posted on 02 July 2015

Nate Finch had a nice blog post on error flags
recently, and it caused me to think about error handling in my own
greenfield Go project at work.

Much of the Go software I write follows a common pattern: an HTTP JSON
API fronting some business logic, backed by a data store of some sort.
When an error occurs, I typically want to present a context-aware HTTP
status code and an a JSON payload containing an error message. I want
to avoid 400 Bad Request and 500 Internal Server Errors whenever
possible, and I also don’t want to expose internal implementation
details or inadvertently leak information to API consumers.

I’d like to share the pattern I’ve settled on for this type of application.

An API-safe error interface

First, I define a new interface that will be used throughout the
application for exposing “safe” errors through the API:

package app

type APIError interface {
    // APIError returns an HTTP status code and an API-safe error message.
    APIError() (int, string)
}

Common sentinel errors

In practice, most of the time there are a limited set of errors that I
want to return through the API. Things like a 401 Unauthorized for a
missing or invalid API token, or a 404 Not Found when referring to a
resource that doesn’t exist in the data store. For these I create a
create a private struct that implements APIError:

type sentinelAPIError struct {
    status int
    msg    string
}

func (e sentinelAPIError) Error() string {
    return e.msg
}

func (e sentinelAPIError) APIError() (int, string) {
    return e.status, e.msg
}

And then I publicly define common sentinel errors:

var (
    ErrAuth      = &sentinelAPIError{status: http.StatusUnauthorized, msg: "invalid token"}
    ErrNotFound  = &sentinelAPIError{status: http.StatusNotFound, msg: "not found"}
    ErrDuplicate = &sentinelAPIError{status: http.StatusBadRequest, msg: "duplicate"}
)

Wrapping sentinels

The sentinel errors provide a good foundation for reporting basic
information through the API, but how can I associate real errors with
them? ErrNoRows from the database/sql package is never going to
implement my APIError interface, but I can leverage the
error wrapping functionality introduced in Go 1.13.

One of the lesser-known features of error wrapping is the ability to
write a custom Is method on your own types. This is perhaps because
the implementation is privately hidden
within the errors package, and the package documentation
doesn’t give much information about why you’d want to use it. But it’s
a perfect fit for these sentinel errors.

First, I define a sentinel-wrapped error type:

type sentinelWrappedError struct {
    error
    sentinel *sentinelAPIError
}

func (e sentinelWrappedError) Is(err error) bool {
    return e.sentinel == err
}

func (e sentinelWrappedError) APIError() (int, string) {
    return e.sentinel.APIError()
}

This associates an error from elsewhere in the application with one of
my predefined sentinel errors. A key thing to note here is that
sentinelWrappedError embeds the original error, meaning its Error
method returns the original error’s message, while implementing
APIError with the sentinel’s API-safe message. The Is method allows
for comparisons of these wrapping errors with the sentinel errors using
errors.Is.

Then I need a public function to do the wrapping:

func WrapError(err error, sentinel *sentinelAPIError) error {
    return sentinelWrappedError{error: err, sentinel: sentinel}
}

(If you wanted to include additional context in the APIError, such as a resource name, this would be a good place to add it.)

When other parts of the application encounter an error, they wrap the
error with one of the sentinel errors. For example, the database layer
might have its own wrapError function that looks something like this:

package db

import "example.com/app"

func wrapError(err error) error {
    switch {
    case errors.Is(err, sql.ErrNoRows):
        return app.WrapError(err, app.ErrNotFound)
    case isMySQLError(err, codeDuplicate):
        return app.WrapError(err, app.ErrDuplicate)
    default:
        return err
    }
}

Because the wrapper implements Is against the sentinel, you can
compare errors to sentinels regardless of what the original error is:

err := db.DoAThing()
switch {
case errors.Is(err, ErrNotFound):
    // do something specific for Not Found errors
case errors.Is(err, ErrDuplicate):
    // do something specific for Duplicate errors
}

Handling errors in the API

The final task is to handle these errors and send them safely back
through the API. In my api package, I define a helper function that
takes an error and serializes it to JSON:

package api

import "example.com/app"

func JSONHandleError(w http.ResponseWriter, err error) {
    var apiErr app.APIError
    if errors.As(err, &apiErr) {
        status, msg := apiErr.APIError()
        JSONError(w, status, msg)
    } else {
        JSONError(w, http.StatusInternalServerError, "internal error")
    }
}

(The elided JSONError function is the one responsible for setting the
HTTP status code and serializing the JSON.)

Note that this function can take any error. If it’s not an
APIError, it falls back to returning a 500 Internal Server Error.
This makes it safe to pass unwrapped and unexpected errors without
additional care.

Because sentinelWrappedError embeds the original error, you can also
log any error you encounter and get the original error message. This
can aid debugging.

An example

Here’s an example HTTP handler function that generates an error, logs
it, and returns it to a caller.

package api

func exampleHandler(w http.ResponseWriter, r *http.Request) {
    // A contrived example that always throws an error.  Imagine this
    // is actually a function that calls into a data store.
    err := app.WrapError(fmt.Errorf("user ID %q not found", "archer"), app.ErrNotFound)
    if err != nil {
        log.Printf("exampleHandler: error fetching user: %v", err)
        JSONHandleError(w, err)
        return
    }

    // Happy path elided...
}

Hitting this endpoint will give you this HTTP response:

HTTP/1.1 404 Not Found
Content-Type: application/json

{"error": "not found"}

And send to your logs:

exampleHandler: error fetching user: user ID "archer" not found

If I had forgotten to call app.WrapError, the response instead would
have been:

HTTP/1.1 500 Internal Server Error
Content-Type: application/json

{"error": "internal error"}

But the message to the logs would have been the same.

Impact

Adopting this pattern for error handling has reduced the number of error
types and scaffolding in my code – the same problems that Nate
experienced before adopting his error flags scheme. It’s centralized
the errors I expose to the user, reduced the work to expose appropriate
and consistent error codes and messages to API consumers, and has an
always-on safe fallback for unexpected errors or programming mistakes.
I hope you can take inspiration to improve the error handling in your
own code.