My application is running on the host with IP 192.168.4.235 on the port 7080. When I curl the same IP and port, it seems connection has been set, but still I get 404 page not found error, as below:
$ curl --verbose http://192.168.4.235:7080
* Trying 192.168.4.235:7080...
* TCP_NODELAY set
* Connected to 192.168.4.235 (192.168.4.235) port 7080 (#0)
> GET / HTTP/1.1
> Host: 192.168.4.235:7080
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Wed, 28 Jul 2021 14:20:17 GMT
< Content-Length: 19
<
404 page not found
* Connection #0 to host 192.168.4.235 left intact
But when I run a postgres service on port 5432, and connect using curl on port 5432, it gets connected and returns an empty reply.
Can someone please guide me on what is wrong here? I think the curl should return an empty reply with the application running on port 7080, but returning 404 not found is something I can’t digest.
Curl was expected to return an empty reply.
curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1j zlib/1.2.11 libidn2/2.3.0 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL
5.0.0-23-generic #24~18.04.1-Ubuntu SMP Mon Jul 29 16:12:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Example site: http://web.de
in browser: working without any problems
in cURL : gives back error 404 not found.
curl options
$cookie_file_path = "/adm/cookie.txt";
$header[0] = "Accept: text/html,application/xhtml+xml,application/xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3";
$header[] = "Pragma: ";
$ch = curl_init();
curl_setopt ( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt ( $ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt ( $ch, CURLOPT_HEADER, 1);
curl_setopt ( $ch, CURLOPT_HTTPHEADER, $header);
curl_setopt ( $ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt ( $ch, CURLOPT_POST, 0);
curl_setopt ( $ch, CURLOPT_URL, $url );
curl_setopt ( $ch, CURLOPT_SSLVERSION,3);
curl_setopt ( $ch, CURLOPT_ENCODING, "");
curl_setopt ( $ch, CURLOPT_MAXREDIRS, 10);
curl_setopt ( $ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt ( $ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt ( $ch, CURLOPT_NOBODY, 0);
curl_exec($ch);
what could be the reason for this?
Содержание
- Как отловить ошибки cURL в PHP
- Ответ 1
- Ответ 2
- Ответ 3
- curl_error
- Description
- Parameters
- Return Values
- Changelog
- Examples
- See Also
- User Contributed Notes 4 notes
- curl_errno
- Описание
- Список параметров
- Возвращаемые значения
- Список изменений
- Примеры
- Смотрите также
- User Contributed Notes 10 notes
Как отловить ошибки cURL в PHP
Я использую функции PHP curl для отправки данных на веб-сервер с моей локальной машины. Мой код выглядит следующим образом:
curl_setopt($c, CURLOPT_URL, $url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
curl_setopt($c, CURLOPT_POST, true);
curl_setopt($c, CURLOPT_POSTFIELDS, $data);
if (curl_exec($c) === false) <
К сожалению, я не могу поймать ни одной ошибки типа 404, 500 или сетевого уровня. Как же мне узнать, что данные не были размещены или получены с удаленного сервера?
Ответ 1
Вы можете использовать функцию curl_error(), чтобы определить, произошла ли какая-то ошибка. Например:
curl_setopt($ch, CURLOPT_URL, $your_url);
curl_setopt($ch, CURLOPT_FAILONERROR, true); // Требуется для того, чтобы коды ошибок HTTP сообщались через наш вызов к curl_error($ch)
// TODO — Обработать ошибку cURL соответствующим образом
Ответ 2
Если CURLOPT_FAILONERROR равно false, ошибки http не будут вызывать ошибок curl.
header(‘HTTP/1.1 503 Service Temporarily Unavailable’);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
$http_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
echo «HTTP Status == 503
«;
echo «Curl Errno returned $curl_errno
«;
Ответ 3
Вы можете сгенерировать ошибку curl после его выполнения:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
echo ‘Request Error:’ . curl_error($ch);
И вот коды ошибок curl:
если кому-то нужна дополнительная информация об ошибках curl
Источник
curl_error
(PHP 4 >= 4.0.3, PHP 5, PHP 7, PHP
curl_error — Return a string containing the last error for the current session
Description
Returns a clear text error message for the last cURL operation.
Parameters
A cURL handle returned by curl_init() .
Return Values
Returns the error message or » (the empty string) if no error occurred.
Changelog
Version | Description |
---|---|
8.0.0 | handle expects a CurlHandle instance now; previously, a resource was expected. |
Examples
Example #1 curl_error() example
// Create a curl handle to a non-existing location
$ch = curl_init ( ‘http://404.php.net/’ );
curl_setopt ( $ch , CURLOPT_RETURNTRANSFER , true );
if( curl_exec ( $ch ) === false )
<
echo ‘Curl error: ‘ . curl_error ( $ch );
>
else
<
echo ‘Operation completed without any errors’ ;
>
// Close handle
curl_close ( $ch );
?>
See Also
User Contributed Notes 4 notes
For a 404 response to actually trigger an error as the example seems to be trying to demonstrate the following option should be set:
CURLE_HTTP_RETURNED_ERROR (22)
This is returned if CURLOPT_FAILONERROR is set TRUE and the HTTP server returns an error code that is >= 400. (This error code was formerly known as CURLE_HTTP_NOT_FOUND.)
If you’re using curl_multi and there’s an error, curl_error() will remain empty until you’ve called curl_multi_info_read(). That function «pumps» the information inside the curl libraries to the point where curl_error() will return a useful string.
This should really be added to the documentation, because it’s not at all obvious.
Источник
curl_errno
(PHP 4 >= 4.0.3, PHP 5, PHP 7, PHP
curl_errno — Возвращает код последней ошибки
Описание
Возвращает код ошибки последней операции cURL.
Список параметров
Дескриптор cURL, полученный из curl_init() .
Возвращаемые значения
Возвращает номер ошибки или 0 (ноль), если ошибки не произошло.
Список изменений
Версия | Описание |
---|---|
8.0.0 | handle теперь ожидает экземпляр CurlHandle ; раньше, ожидался ресурс ( resource ). |
Примеры
Пример #1 Пример использования curl_errno()
// Создаём дескриптор curl к несуществующему адресу
$ch = curl_init ( ‘http://404.php.net/’ );
// Запускаем
curl_setopt ( $ch , CURLOPT_RETURNTRANSFER , true );
curl_exec ( $ch );
// Проверяем наличие ошибки
if( curl_errno ( $ch ))
<
echo ‘Ошибка curl: ‘ . curl_error ( $ch );
>
// Закрываем дескриптор
curl_close ( $ch );
?>
Смотрите также
- curl_error() — Возвращает строку с описанием последней ошибки текущего сеанса
- » Коды ошибок cURL
User Contributed Notes 10 notes
if someone need more information about curl errors
=array(
[ 1 ] => ‘CURLE_UNSUPPORTED_PROTOCOL’ ,
[ 2 ] => ‘CURLE_FAILED_INIT’ ,
[ 3 ] => ‘CURLE_URL_MALFORMAT’ ,
[ 4 ] => ‘CURLE_URL_MALFORMAT_USER’ ,
[ 5 ] => ‘CURLE_COULDNT_RESOLVE_PROXY’ ,
[ 6 ] => ‘CURLE_COULDNT_RESOLVE_HOST’ ,
[ 7 ] => ‘CURLE_COULDNT_CONNECT’ ,
[ 8 ] => ‘CURLE_FTP_WEIRD_SERVER_REPLY’ ,
[ 9 ] => ‘CURLE_REMOTE_ACCESS_DENIED’ ,
[ 11 ] => ‘CURLE_FTP_WEIRD_PASS_REPLY’ ,
[ 13 ] => ‘CURLE_FTP_WEIRD_PASV_REPLY’ ,
[ 14 ]=> ‘CURLE_FTP_WEIRD_227_FORMAT’ ,
[ 15 ] => ‘CURLE_FTP_CANT_GET_HOST’ ,
[ 17 ] => ‘CURLE_FTP_COULDNT_SET_TYPE’ ,
[ 18 ] => ‘CURLE_PARTIAL_FILE’ ,
[ 19 ] => ‘CURLE_FTP_COULDNT_RETR_FILE’ ,
[ 21 ] => ‘CURLE_QUOTE_ERROR’ ,
[ 22 ] => ‘CURLE_HTTP_RETURNED_ERROR’ ,
[ 23 ] => ‘CURLE_WRITE_ERROR’ ,
[ 25 ] => ‘CURLE_UPLOAD_FAILED’ ,
[ 26 ] => ‘CURLE_READ_ERROR’ ,
[ 27 ] => ‘CURLE_OUT_OF_MEMORY’ ,
[ 28 ] => ‘CURLE_OPERATION_TIMEDOUT’ ,
[ 30 ] => ‘CURLE_FTP_PORT_FAILED’ ,
[ 31 ] => ‘CURLE_FTP_COULDNT_USE_REST’ ,
[ 33 ] => ‘CURLE_RANGE_ERROR’ ,
[ 34 ] => ‘CURLE_HTTP_POST_ERROR’ ,
[ 35 ] => ‘CURLE_SSL_CONNECT_ERROR’ ,
[ 36 ] => ‘CURLE_BAD_DOWNLOAD_RESUME’ ,
[ 37 ] => ‘CURLE_FILE_COULDNT_READ_FILE’ ,
[ 38 ] => ‘CURLE_LDAP_CANNOT_BIND’ ,
[ 39 ] => ‘CURLE_LDAP_SEARCH_FAILED’ ,
[ 41 ] => ‘CURLE_FUNCTION_NOT_FOUND’ ,
[ 42 ] => ‘CURLE_ABORTED_BY_CALLBACK’ ,
[ 43 ] => ‘CURLE_BAD_FUNCTION_ARGUMENT’ ,
[ 45 ] => ‘CURLE_INTERFACE_FAILED’ ,
[ 47 ] => ‘CURLE_TOO_MANY_REDIRECTS’ ,
[ 48 ] => ‘CURLE_UNKNOWN_TELNET_OPTION’ ,
[ 49 ] => ‘CURLE_TELNET_OPTION_SYNTAX’ ,
[ 51 ] => ‘CURLE_PEER_FAILED_VERIFICATION’ ,
[ 52 ] => ‘CURLE_GOT_NOTHING’ ,
[ 53 ] => ‘CURLE_SSL_ENGINE_NOTFOUND’ ,
[ 54 ] => ‘CURLE_SSL_ENGINE_SETFAILED’ ,
[ 55 ] => ‘CURLE_SEND_ERROR’ ,
[ 56 ] => ‘CURLE_RECV_ERROR’ ,
[ 58 ] => ‘CURLE_SSL_CERTPROBLEM’ ,
[ 59 ] => ‘CURLE_SSL_CIPHER’ ,
[ 60 ] => ‘CURLE_SSL_CACERT’ ,
[ 61 ] => ‘CURLE_BAD_CONTENT_ENCODING’ ,
[ 62 ] => ‘CURLE_LDAP_INVALID_URL’ ,
[ 63 ] => ‘CURLE_FILESIZE_EXCEEDED’ ,
[ 64 ] => ‘CURLE_USE_SSL_FAILED’ ,
[ 65 ] => ‘CURLE_SEND_FAIL_REWIND’ ,
[ 66 ] => ‘CURLE_SSL_ENGINE_INITFAILED’ ,
[ 67 ] => ‘CURLE_LOGIN_DENIED’ ,
[ 68 ] => ‘CURLE_TFTP_NOTFOUND’ ,
[ 69 ] => ‘CURLE_TFTP_PERM’ ,
[ 70 ] => ‘CURLE_REMOTE_DISK_FULL’ ,
[ 71 ] => ‘CURLE_TFTP_ILLEGAL’ ,
[ 72 ] => ‘CURLE_TFTP_UNKNOWNID’ ,
[ 73 ] => ‘CURLE_REMOTE_FILE_EXISTS’ ,
[ 74 ] => ‘CURLE_TFTP_NOSUCHUSER’ ,
[ 75 ] => ‘CURLE_CONV_FAILED’ ,
[ 76 ] => ‘CURLE_CONV_REQD’ ,
[ 77 ] => ‘CURLE_SSL_CACERT_BADFILE’ ,
[ 78 ] => ‘CURLE_REMOTE_FILE_NOT_FOUND’ ,
[ 79 ] => ‘CURLE_SSH’ ,
[ 80 ] => ‘CURLE_SSL_SHUTDOWN_FAILED’ ,
[ 81 ] => ‘CURLE_AGAIN’ ,
[ 82 ] => ‘CURLE_SSL_CRL_BADFILE’ ,
[ 83 ] => ‘CURLE_SSL_ISSUER_ERROR’ ,
[ 84 ] => ‘CURLE_FTP_PRET_FAILED’ ,
[ 84 ] => ‘CURLE_FTP_PRET_FAILED’ ,
[ 85 ] => ‘CURLE_RTSP_CSEQ_ERROR’ ,
[ 86 ] => ‘CURLE_RTSP_SESSION_ERROR’ ,
[ 87 ] => ‘CURLE_FTP_BAD_FILE_LIST’ ,
[ 88 ] => ‘CURLE_CHUNK_FAILED’ );
Источник
Chapter 6
The curl
requests we have used so far have all been expertly crafted to work. We haven’t tried to game the API or make it fail, because we are nice people.
Not everyone is like this, unfortunately! And it’s not only people with bad intentions we need to look out for, but also simple mistakes. Anyone can make a mistake — it happens. That does not mean we should laugh at them for trying and send back some indecipherable message.
Descriptive messages are a very important part of any web API. Web APIs are used by our fellow developers and, by building an API, we take the responsibility of providing a good service for them. Any service should be easy to use. Put yourself inside a user’s shoes and pinpoint the issues someone could have with your service.
The obvious problem in our current web API is that we don’t send back meaningful errors when something goes wrong.
Let’s take the following curl
request as an example. Notice anything wrong? We forgot to close the JSON
document. Before we can realize our mistake, we send the request…
curl -i -X POST http://localhost:4567/users -H "Content-Type: application/json" -d '{"first_name":"Samuel", "last_name":"Da Costa", "age":19'
and we end up with a scary 500 Internal Server Error
. This seems to mean that there was something wrong with the server, not with our request.
HTTP/1.1 500 Internal Server Error Content-Type: text/plain Content-Length: 4679 Connection: keep-alive Server: thin [JSON::ParserError]
Luckily, Sinatra sends back the whole stack trace, so we can easily debug it. Not every framework is that nice, though. That’s the server’s duty — to provide descriptive errors to the client. A 500
is as descriptive as looking at a black box that just says: “Sorry bro, it didn’t work”.
HTTP comes with a set of features to handle errors. Indeed, thanks to all its HTTP status codes, we can configure our API to send back descriptive error messages to a client in a format that it can understand.
6.1. The different classes of HTTP status codes
There are five classes of HTTP codes:
1XX
Informational: Status codes starting with1
are known as informational codes. Most of them are rarely used nowadays.2XX
Success: These codes indicate that the exchange between the server and the client was successful.3XX
Redirection:3XX
codes indicate that the client must take additional action before the request can be completed.4XX
Client Error: There was something wrong with the request the client sent and the server cannot process it.5XX
Server Error: The client sent a valid request but the server was not able to process it successfully.
We will be using status codes from some of these classes in this chapter.
6.2. Global
We will add error handling for each route, but let’s first see the errors we need to handle for every request.
6.2.1. 405 Method Not Allowed
This HTTP status code can be used when the client tries to access a resource using an unsupported HTTP method. For example, in our API, what would happen if a client tried to use the PUT
method with the /users
URI?
Let’s give it a try.
curl -i -X PUT http://localhost:4567/users
Output
HTTP/1.1 404 Not Found Content-Type: text/html;charset=utf-8 X-Cascade: pass Content-Length: 467 X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN Connection: keep-alive Server: thin <!DOCTYPE html> <html> <head> <MORE HTML>
That’s not very helpful… Plus, it returns 404
. While we do have a resource living at that URI, it’s not the right HTTP method… we can do better. All we need to do is catch any request accessing /users
with an unsupported method: put
, patch
and delete
. With a bit of metaprogramming, it’s not hard to do.
In the code below, we loop through each of the HTTP methods we don’t support and define a Sinatra route that returns 405 Method Not Allowed
for each one of them.
# webapi.rb # /users # head /users # get /users # post /users [:put, :patch, :delete].each do |method| send(method, '/users') do halt 405 end end # options /users/:first_name
Let’s give it another try after restarting the server, shall we?
curl -i -X PUT http://localhost:4567/users
Output
HTTP/1.1 405 Method Not Allowed Content-Type: text/html;charset=utf-8 Content-Length: 0 [MORE HEADERS]
Alright! It’s working correctly, let’s proceed.
6.2.2. 406 Not Acceptable
We already implemented one HTTP status code in the previous chapters. If the client requests a media type that our API does not support, we will return 406 Not Acceptable
. To the client receiving this, it means that the server was not able to generate a representation for the resource according to the criteria the client had fixed. The response should also include what formats are actually available for the client to pick from.
# webapi.rb # users # ... helpers do # def json_or_default? # def xml? def accepted_media_type return 'json' unless request.accept.any? request.accept.each do |type| return 'json' if json_or_default?(type) return 'xml' if xml?(type) end content_type 'text/plain' halt 406, 'application/json, application/xml' end # def type # def send_data end
Let’s try with a fake media type that we do not support.
curl -i http://localhost:4567/users -H "Accept: moar/curl"
Output
HTTP/1.1 406 Not Acceptable Content-Type: text/plain;charset=utf-8 Content-Length: 33 ... More Headers ... application/json, application/xml
With this, the client can understand what is actually supported and act accordingly. Sadly, it’s not defined more clearly in a standard.
Alternatively, we could actually return a JSON representation instead of telling the client that we can’t give it anything. We would set the Content-Type
header to application/json
to let the client do its own checking before it parses the response. Both this option and the previous one are acceptable as defined in the HTTP RFC.
# webapi.rb # users # ... helpers do # def json_or_default? # def xml? def accepted_media_type return 'json' unless request.accept.any? request.accept.each do |type| return 'json' if json_or_default?(type) return 'xml' if xml?(type) end 'json' end # def type # def send_data end
Don’t forget to restart the server.
curl -i http://localhost:4567/users -H "Accept: moar/curl"
Output
HTTP/1.1 200 OK Content-Type: application/json Content-Length: 203 ... More Headers ... [ {"first_name":"Thibault", "last_name":"Denizet", "age":25, "id":"thibault"}, {"first_name":"Simon", "last_name":"Random", "age":26, "id":"simon"}, {"first_name":"John", "last_name":"Smith", "age":28, "id":"john"} ]
If your server cannot generate a representation the way the client requests, you can either return 406
(with or without a list of actually supported media types), or just return the default format and let the client handle it.
6.3. POST /users
For the post /users
route, we are currently not handling any errors. If you run the following curl
request, it will crash and send you back 500
and the whole stack trace.
curl -X POST -i http://localhost:4567/users -H "Content-Type: application/fake" -d 'Weirdly Formatted Data'
Output
HTTP/1.1 500 Internal Server Error Content-Type: text/plain Content-Length: 4647 Connection: keep-alive Server: thin JSON::ParserError: 757: unexpected token at 'Weirdly Formatted Data' [Stack Trace]
This does not help the client in understanding what happened. It does help us humans, because we can read the Ruby stack trace and see that there was a JSON::ParserError
. But to an automated client, this only means that something went wrong on the server. We need to fix this.
6.3.1. 415 Unsupported Media Type
To fix it we will use the 415
HTTP status code. 415 Unsupported Media Type
is exactly what its name implies — we can return it to the client when we don’t understand the format of the data sent by the client.
To prevent the execution of the code there, we will first check if the data was sent as a JSON document, the only format we support for requests. Our code must be guarded from unwanted media types.
We just need the following line to do it.
halt 415 unless request.env['CONTENT_TYPE'] == 'application/json'
And it comes in the post /users
route like this:
# webapi.rb # Stuff # ... post '/users' do halt 415 unless request.env['CONTENT_TYPE'] == 'application/json' users[user['first_name'].downcase.to_sym] = user url = "http://localhost:4567/users/#{user['first_name']}" response.headers['Location'] = url status 201 end
Don’t forget to restart the server.
Now, let’s see what happens when we send the curl
request seen in the previous section.
curl -X POST -i http://localhost:4567/users -H "Content-Type: application/fake" -d 'Weirdly Formatted Data'
Output
HTTP/1.1 415 Unsupported Media Type [MORE HEADERS]
That’s much more descriptive, right? The only problem is that this does not tell the client which media types are actually supported. We will see what we can do about this later. For now, our endpoint won’t crash anymore whatever the received format is.
6.3.2. 400 Bad Request
The 400 Bad Request
is used way too often in modern web applications. Its real purpose is to indicate that the server could not understand the request due to some syntax issue. For example, a malformed JSON document.
Let’s give it a try and break some stuff.
curl -X POST -i http://localhost:4567/users -H "Content-Type: application/json" -d '{"first_name":"Mark"'
Output
HTTP/1.1 500 Internal Server Error Content-Type: text/plain Content-Length: 4645 Connection: keep-alive Server: thin JSON::ParserError: 757: unexpected token at '{"first_name":"Mark"' [Stack Trace]
Boom! Broken. Since our JSON document, {"first_name":"Mark"
, is not valid, our server crashes when trying to parse it. Here is how we can fix this. We are going to catch the exception raised when parsing an invalid JSON document (JSON::ParserError
). In the rescue
, we will send back the error to the client, in the format of its choice (specified by the Accept
header). Note that we are still using the send_data
method we wrote earlier.
# webapi.rb # Stuff # ... post '/users' do halt 415 unless request.env['CONTENT_TYPE'] == 'application/json' begin user = JSON.parse(request.body.read) rescue JSON::ParserError => e halt 400, send_data(json: -> { { message: e.to_s } }, xml: -> { { message: e.to_s } }) end users[user['first_name'].downcase.to_sym] = user url = "http://localhost:4567/users/#{user['first_name']}" response.headers['Location'] = url status 201 end
Don’t forget to restart your server!
curl -X POST -i http://localhost:4567/users -H "Content-Type: application/json" -d '{"first_name":"Mark"'
Output
HTTP/1.1 400 Bad Request Content-Type: application/json Content-Length: 65 X-Content-Type-Options: nosniff Connection: keep-alive Server: thin {"message":"757: unexpected token at '{"first_name":"Mark"'"}
Great! We are getting a 400
back and the body contains the error message.
6.3.3. 409 Conflict
Currently, if we try to re-create a user that already exists, it’s just going to override the existing one. Not good. This should throw up some kind of conflict error saying that a user cannot be overridden. The problem here is how we save users in a Ruby hash. Since we use the first name as a key, we don’t want to allow the override of an existing user.
Luckily, that’s what the 409 Conflict
HTTP code is for. Note that this code is only allowed in situations where the client can actually fix the conflict. In our case, the client can either change the first name or use another endpoint to update the user.
Our use of the first name as a key is there only for simplicity and any production application should not use such a mechanism. Unique generated IDs are way better!
To prevent any override, we will use this code. If we find a user with the first name sent by the client, we’ll immediately halt and send back a response to the client with a 409
status code and an explanation message.
if users[user['first_name'].downcase.to_sym] message = { message: "User #{user['first_name']} already in DB." } halt 409, send_data(json: -> { message }, xml: -> { message }) end
Here is our full POST
endpoint:
# webapi.rb # Stuff # ... post '/users' do halt 415 unless request.env['CONTENT_TYPE'] == 'application/json' begin user = JSON.parse(request.body.read) rescue JSON::ParserError => e halt 400, send_data(json: -> { { message: e.to_s } }, xml: -> { { message: e.to_s } }) end if users[user['first_name'].downcase.to_sym] message = { message: "User #{user['first_name']} already in DB." } halt 409, send_data(json: -> { message }, xml: -> { message }) end users[user['first_name'].downcase.to_sym] = user url = "http://localhost:4567/users/#{user['first_name']}" response.headers['Location'] = url status 201 end
Let’s try to recreate an existing user (for example, Thibault), after restarting the server.
curl -X POST -i http://localhost:4567/users -H "Content-Type: application/json" -d '{"first_name":"Thibault", "last_name":"Denizet", "age":25}'
Output
HTTP/1.1 409 Conflict Content-Type: application/json Content-Length: 42 X-Content-Type-Options: nosniff Connection: keep-alive Server: thin {"message":"User Thibault already in DB."}
Great, we’re telling the client that its request triggered a conflict that needs to be solved. We can now detect when updates conflict with the users’ data stored in the users
hash.
6.4. GET /users/:first_name
Let’s now turn our attention to the get /users/:first_name
route. We know our endpoint works fine when the user exists. But what happens when it doesn’t?
curl -i http://localhost:4567/users/frank
HTTP/1.1 200 OK Content-Type: application/json Content-Length: 4 ... More Headers ... null
What the heck is this null
doing here? Plus, we are telling the client it’s a JSON document with the Content-Type
. We really need to fix this!
6.4.1. 404 Not Found
The 404 Not Found
status code can help us. This code is meant to tell the client that nothing was found at the URI specified by the client; in other words, the requested resource does not exist.
This fits what we need for the get /users/:first_name
route pretty well. Let’s add a little halt that will return 404
if the user is not found in our hash.
# webapi.rb # Stuff # ... get '/users/:first_name' do |first_name| halt 404 unless users[first_name.to_sym] send_data(json: -> { users[first_name.to_sym].merge(id: first_name) }, xml: -> { { first_name => users[first_name.to_sym] } }) end
Restart the server and send the following request.
curl -i http://localhost:4567/users/frank
Output
HTTP/1.1 404 Not Found
... More Headers ...
Looks good!
6.4.2. 410 Gone
Sadly 404 Not Found
can seem quite generic to a client. We can use other codes to give a more specific response. One of them is 410 Gone
which indicates that a resource used to live at the given URI, but doesn’t anymore. It has probably been deleted since the server does not have the new location.
Currently, if we delete a user and then try to access it, we will just get 404
, as if it never existed.
It’s not always possible for a server to indicate a deleted resource and there is nothing forcing us to use this HTTP status. It is helpful for a client though, since it will let the client know that this URI does not point to anything anymore. It shouldn’t be considered as a bug since there used to be something there. The client should take note that it’s now gone and move on with its life.
Let’s quickly add a mechanism to keep track of what has been deleted. In a production application with a real database, it is possible to achieve this kind of mechanism by using a boolean in order to see if something has been deleted or not. There are other ways to do it, but we won’t be covering them.
The first thing we need is a hash, named deleted_users
, that will contain all the deleted users (duh!).
# webapi.rb # require users = { thibault: { first_name: 'Thibault', last_name: 'Denizet', age: 25 }, simon: { first_name: 'Simon', last_name: 'Random', age: 26 }, john: { first_name: 'John', last_name: 'Smith', age: 28 } } deleted_users = {} # Helpers # Routes
Then we need to update the get /users/:first_name
route to halt with 410
if the deleted_users
hash contains the specified user.
# webapi.rb # stuff get '/users/:first_name' do |first_name| halt 410 if deleted_users[first_name.to_sym] halt 404 unless users[first_name.to_sym] send_data(json: -> { users[first_name.to_sym].merge(id: first_name) }, xml: -> { { first_name => users[first_name.to_sym] } }) end
Finally, we need to fill the deleted_users
hash every time a user gets deleted in the delete /users/:first_name
route.
# webapi.rb # stuff delete '/users/:first_name' do |first_name| first_name = first_name.to_sym deleted_users[first_name] = users[first_name] if users[first_name] users.delete(first_name) status 204 end
Everything is in place now and we can start sending some test requests. First, restart the server. Then delete the simon
user with this command:
curl -i -X DELETE http://localhost:4567/users/simon
Output
HTTP/1.1 204 No Content [ ... ]
Now, what happens when we try to access the Simon resource?
curl -i http://localhost:4567/users/simon
HTTP/1.1 410 Gone Content-Type: text/html;charset=utf-8 Content-Length: 0 [More Headers]
Our little API is getting more expressive by the minute!
6.5. PUT /users/:first_name
For this route, we can handle 415 Unsupported Media Type
and 400 Bad Request
. However, I’m not going to show you how. Instead, I want you to do it.
Testing 415
curl -X PUT -i http://localhost:4567/users/thibault -H "Content-Type: application/fake" -d 'Weirdly Formatted Data'
Testing 400
curl -X PUT -i http://localhost:4567/users/thibault -H "Content-Type: application/json" -d '{"first_name":"Tibo"'
6.6. PATCH /users/:first_name
For this route, we can handle 415 Unsupported Media Type
, 404 Not Found
, 410 Gone
and 400 Bad Request
. You can get inspired by what we did for POST /users/:first_name
and GET /users/:first_name
. Below, you will find curl
commands to test if you have correctly implemented each status code.
Testing 415
curl -X PATCH -i http://localhost:4567/users/thibault -H "Content-Type: application/fake" -d 'Weirdly Formatted Data'
Testing 400
curl -X PATCH -i http://localhost:4567/users/thibault -H "Content-Type: application/json" -d '{"first_name":"Tibo"'
Testing 404
curl -X PATCH -i http://localhost:4567/users/mark -H "Content-Type: application/json" -d '{"first_name":"Marc"}'
Testing 410
curl -i -X DELETE http://localhost:4567/users/simon
curl -X PATCH -i http://localhost:4567/users/simon -H "Content-Type: application/json" -d '{"first_name":"Super Simon"}'
6.7. Wrap Up
In this chapter, we’ve seen how to implement errors to let our clients understand exactly what went wrong when something didn’t go as planned.
In the next chapter, we will tackle a sensitive subject: versioning!
Львиная доля всех моих работ связана с работами по сео-оптимизации сайтов по требованиям сео-специалистов. Доработать, исправить, добавить, устранить… Очень часто нужно посмотреть ответы сервера по тому или иному url-адресу.
Проверку можно делать разными способами. Самым простым будет воспользоваться онлайн-тестами. Но это не наш вариант:)
В данном посте рассмотрим, какие возможности для данной задачи нам предоставляет curl.
Коды ответа сервера
Для начала рассмотрим самые распространенные коды ответа сервера:
1xx — информационные:
- 100 — сервер принял первую часть запроса, можно подрожать передачу;
- 101 — нужно изменить протокол работы на более подходящий;
- 102 — на обработку запроса уйдет много времени, используется чтобы браузер не разрывал соединение раньше времени;
2хх — операция успешна:
- 200 — запрос выполнен успешно, отправляется для большинства запрашиваемых страниц;
- 201 — после выполнения запроса был создан ресурс;
- 202 — запрос принят, но еще не обработан;
- 203 — запрос выполнен успешно, но информация для ответа взята из прокси;
- 204 — запрос обработан, но контента для отображения нет;
- 205 — попросить пользователя ввести необходимые данные;
- 206 — запрос обработан, но передана только часть контента;
3xx — перенаправления:
- 300 — есть несколько страниц для этого запроса, например, на нескольких языках;
- 301 — страница навсегда перемещена по новому адресу;
- 302 — документ был временно перемещен;
- 303 — документ необходимо загрузить по указанному адресу с помощью протокола GET;
- 304 — документ не изменился с последнего запроса;
- 305 — нужно использовать прокси;
- 307 — ресурс временно перемещен на новый адрес.
4хх — ошибка в запросе:
- 400 — неверный запрос;
- 401 — необходимо аутентифицироваться;
- 403 — запрос принят, но у вас нет доступа;
- 404 — страница не найдена на сервере;
- 405 — используемый метод нельзя применять на сервере;
- 408 — время ожидания передачи запроса истекло;
- 410 — ресурс полностью удален;
- 411 — нужно указать длину запроса;
- 413 — запрос слишком длинный;
- 414 — URI запроса слишком длинная.
5хх — ошибка сервера:
- 500 — внутренняя ошибка сервера;
- 501 — нужная функция не поддерживается;
- 502 — прокси не может соединиться со шлюзом;
- 503 — сервер не может обрабатывать запросы по техническим причинам;
- 504 — прокси не дождался ответа от сервера;
- 505 — версия протокола HTTP не поддерживается.
Основные заголовки, отправляемые сервером
- Server — имя и версия веб-сервера;
- Date — дата осуществления запроса;
- Content-Type — MIME тип передаваемых данных, например, text/html, тут же задается кодировка;
- Connection — тип соединения, может быть closed — уже закрыто, или keep-alive — открыто для передачи данных;
- Vary — указывает при каких заголовках веб-сервер будет возвращать разные старины для одного URI;
- Set-Cookie — сохранить Cookie информацию для страницы;
- Expires — можно хранить страницу или ресурс в кэше до определенной даты;
- Cache-Control — настройка времени кэширования страницы браузером, а также разрешения на кэширования;
- ETag — содержит контрольную сумму для страницы, применимо для проверки кэша;
- Last-Modified — дата, когда страница последний раз была изменена;
Проверка кода ответа сервера с помощью curl
Чтобы увидеть только код ответа страницы достаточно выполнить такую команду:
curl -I https://pai-bx.com 2>/dev/null | head -n 1 | cut -d$' ' -f2
Как видим, сервер вернул 200 статус, что означает, что все ок. Страница доступна для чтения. Если проверить страницу, для которой должны быть настроены редиректы, получим 301-й статус:
Проверка http-заголовков с помощью curl
Чтобы вывести заголовки страницы необходимо запустить curl с опцией -I:
curl -I https://pai-bx.com
Проверка IF-MODIFIED-SINCE
Чтобы увидеть работает ли данный заголовок, для начала выполняем обычный запрос заголовков сервера. Далее, повторяем отправку запроса, но уже с заголовком If-Modified-Since и датой:
Видим, что второй запрос возвращает 304-й статус ответа, что означает, что с указанного времени страница не менялась и может быть взята из кеша браузера.
Таким образом, с помощью curl можно увидеть код ответа сервера и отдаваемые сервером заголовки.
I am trying to check whether a file is available at a particular location or not, if file doesn’t exists then it must send an email to an email list. The problem is that, the file needs to be downloaded from a URL (sharepoint) before checking if it exists or not.
If there is no file at the URL, the cURL command is downloading the file with «404 NOT FOUND» content in it. If there is no file at the source, I don’t want the cURL command to download any file. How can i achieve this?
cd $file_path
curl -k --ntlm -u $USER_GROUP:$PASS -O http://sharepoint.com/sites/dummy/file_list.txt
This is the command to download that i am using. Any help would be appreciated. Thanks.
asked Sep 30, 2020 at 8:30
Adding the -f
switch should achieve what you want.
From curl
manual:
-f/—fail
(HTTP) Fail silently (no output at all) on server errors. This is mostly done to better enable scripts etc to better deal with failed attempts. In normal cases when a HTTP server fails to deliver a document, it returns an HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and return error 22.This method is not fail-safe and there are occasions where non-successful response codes will slip through, especially when authentication is involved (response codes 401 and 407).
answered Sep 30, 2020 at 8:54
Dave WhiteDave White
2201 gold badge2 silver badges8 bronze badges
Improve Article
Save Article
Improve Article
Save Article
Checking if a Webpage URL exists or not is relatively easy in PHP. If the required URL does not exist, then it will return 404 error. The checking can be done with and without using cURL library.
cURL: The cURL stands for ‘Client for URLs’, originally with URL spelled in uppercase to make it obvious that it deals with URLs. It is pronounced as ‘see URL’. The cURL project has two products libcurl and curl.
- libcurl: A free and easy-to-use client-side URL transfer library, supporting FTP, TPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE, and LDAP. libcurl supports TTPS certificates, HTTP POST, HTTP PUT, FTP uploading, kerberos, HTTP based upload, proxies, cookies, user & password authentication, file transfer resume, HTTP proxy tunneling and many more. libcurl is free, thread-safe, IPv6 compatible, feature rich, well supported and fast.
- curl: A command line tool for getting or sending files using URL syntax. Since curl uses libcurl, it supports a range of common internal protocols, currently including HTTP, HTTPS, FTP, FTPS, GOPHER, TELNET, DICT, and FILE.
Example 1: This example test a URL for 404 error without using cURL approach.
<?php
$array
= @get_headers(
$url
);
$string
=
$array
[0];
if
(
strpos
(
$string
,
"200"
)) {
echo
'Specified URL Exists'
;
}
else
{
echo
'Specified URL does not exist'
;
}
?>
Output:
Specified URL exists
Example 2: This example test a URL for 404 error using cURL approach.
<?php
curl_setopt(
$ch
, CURLOPT_NOBODY, true);
curl_exec(
$ch
);
$retcode
= curl_getinfo(
$ch
, CURLINFO_HTTP_CODE);
if
(
$retcode
!= 200) {
echo
"Specified URL does not exist"
;
}
else
{
echo
"Specified URL exists"
;
}
curl_close(
$ch
);
?>
Output:
Specified URL exists