Curl 404 error code

I did this My application is running on the host with IP 192.168.4.235 on the port 7080. When I curl the same IP and port, it seems connection has been set, but still I get 404 page not found error...

My application is running on the host with IP 192.168.4.235 on the port 7080. When I curl the same IP and port, it seems connection has been set, but still I get 404 page not found error, as below:

$ curl --verbose http://192.168.4.235:7080

*   Trying 192.168.4.235:7080...
* TCP_NODELAY set
* Connected to 192.168.4.235 (192.168.4.235) port 7080 (#0)
> GET / HTTP/1.1
> Host: 192.168.4.235:7080
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Wed, 28 Jul 2021 14:20:17 GMT
< Content-Length: 19
<
404 page not found
* Connection #0 to host 192.168.4.235 left intact

But when I run a postgres service on port 5432, and connect using curl on port 5432, it gets connected and returns an empty reply.

Can someone please guide me on what is wrong here? I think the curl should return an empty reply with the application running on port 7080, but returning 404 not found is something I can’t digest.

Curl was expected to return an empty reply.

curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1j zlib/1.2.11 libidn2/2.3.0 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

5.0.0-23-generic #24~18.04.1-Ubuntu SMP Mon Jul 29 16:12:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Example site: http://web.de

in browser: working without any problems

in cURL : gives back error 404 not found.

curl options

$cookie_file_path   = "/adm/cookie.txt";


$header[0]          = "Accept: text/html,application/xhtml+xml,application/xml,";
$header[0]          .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[]           = "Cache-Control: max-age=0";
$header[]           = "Connection: keep-alive";
$header[]           = "Keep-Alive: 300";
$header[]           = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[]           = "Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3";
$header[]           = "Pragma: ";

$ch                 = curl_init();
curl_setopt ( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt ( $ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt ( $ch, CURLOPT_HEADER, 1);
curl_setopt ( $ch, CURLOPT_HTTPHEADER, $header);
curl_setopt ( $ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt ( $ch, CURLOPT_POST, 0);
curl_setopt ( $ch, CURLOPT_URL, $url );
curl_setopt ( $ch, CURLOPT_SSLVERSION,3);
curl_setopt ( $ch, CURLOPT_ENCODING, "");
curl_setopt ( $ch, CURLOPT_MAXREDIRS, 10);
curl_setopt ( $ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt ( $ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt ( $ch, CURLOPT_NOBODY, 0);


curl_exec($ch);

what could be the reason for this?

Содержание

  1. Как отловить ошибки cURL в PHP
  2. Ответ 1
  3. Ответ 2
  4. Ответ 3
  5. curl_error
  6. Description
  7. Parameters
  8. Return Values
  9. Changelog
  10. Examples
  11. See Also
  12. User Contributed Notes 4 notes
  13. curl_errno
  14. Описание
  15. Список параметров
  16. Возвращаемые значения
  17. Список изменений
  18. Примеры
  19. Смотрите также
  20. User Contributed Notes 10 notes

Как отловить ошибки cURL в PHP

Я использую функции PHP curl для отправки данных на веб-сервер с моей локальной машины. Мой код выглядит следующим образом:

curl_setopt($c, CURLOPT_URL, $url);

curl_setopt($c, CURLOPT_RETURNTRANSFER, true);

curl_setopt($c, CURLOPT_POST, true);

curl_setopt($c, CURLOPT_POSTFIELDS, $data);

if (curl_exec($c) === false) <

К сожалению, я не могу поймать ни одной ошибки типа 404, 500 или сетевого уровня. Как же мне узнать, что данные не были размещены или получены с удаленного сервера?

Ответ 1

Вы можете использовать функцию curl_error(), чтобы определить, произошла ли какая-то ошибка. Например:

curl_setopt($ch, CURLOPT_URL, $your_url);

curl_setopt($ch, CURLOPT_FAILONERROR, true); // Требуется для того, чтобы коды ошибок HTTP сообщались через наш вызов к curl_error($ch)

// TODO — Обработать ошибку cURL соответствующим образом

Ответ 2

Если CURLOPT_FAILONERROR равно false, ошибки http не будут вызывать ошибок curl.

header(‘HTTP/1.1 503 Service Temporarily Unavailable’);

curl_setopt($ch, CURLOPT_FAILONERROR, true);

$http_status = curl_getinfo($ch, CURLINFO_HTTP_CODE);

echo «HTTP Status == 503
«;

echo «Curl Errno returned $curl_errno
«;

Ответ 3

Вы можете сгенерировать ошибку curl после его выполнения:

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

echo ‘Request Error:’ . curl_error($ch);

И вот коды ошибок curl:

если кому-то нужна дополнительная информация об ошибках curl

Источник

curl_error

(PHP 4 >= 4.0.3, PHP 5, PHP 7, PHP 8)

curl_error — Return a string containing the last error for the current session

Description

Returns a clear text error message for the last cURL operation.

Parameters

A cURL handle returned by curl_init() .

Return Values

Returns the error message or » (the empty string) if no error occurred.

Changelog

Version Description
8.0.0 handle expects a CurlHandle instance now; previously, a resource was expected.

Examples

Example #1 curl_error() example

// Create a curl handle to a non-existing location
$ch = curl_init ( ‘http://404.php.net/’ );
curl_setopt ( $ch , CURLOPT_RETURNTRANSFER , true );

if( curl_exec ( $ch ) === false )
<
echo ‘Curl error: ‘ . curl_error ( $ch );
>
else
<
echo ‘Operation completed without any errors’ ;
>

// Close handle
curl_close ( $ch );
?>

See Also

User Contributed Notes 4 notes

For a 404 response to actually trigger an error as the example seems to be trying to demonstrate the following option should be set:

CURLE_HTTP_RETURNED_ERROR (22)
This is returned if CURLOPT_FAILONERROR is set TRUE and the HTTP server returns an error code that is >= 400. (This error code was formerly known as CURLE_HTTP_NOT_FOUND.)

If you’re using curl_multi and there’s an error, curl_error() will remain empty until you’ve called curl_multi_info_read(). That function «pumps» the information inside the curl libraries to the point where curl_error() will return a useful string.

This should really be added to the documentation, because it’s not at all obvious.

Источник

curl_errno

(PHP 4 >= 4.0.3, PHP 5, PHP 7, PHP 8)

curl_errno — Возвращает код последней ошибки

Описание

Возвращает код ошибки последней операции cURL.

Список параметров

Дескриптор cURL, полученный из curl_init() .

Возвращаемые значения

Возвращает номер ошибки или 0 (ноль), если ошибки не произошло.

Список изменений

Версия Описание
8.0.0 handle теперь ожидает экземпляр CurlHandle ; раньше, ожидался ресурс ( resource ).

Примеры

Пример #1 Пример использования curl_errno()

// Создаём дескриптор curl к несуществующему адресу
$ch = curl_init ( ‘http://404.php.net/’ );

// Запускаем
curl_setopt ( $ch , CURLOPT_RETURNTRANSFER , true );
curl_exec ( $ch );

// Проверяем наличие ошибки
if( curl_errno ( $ch ))
<
echo ‘Ошибка curl: ‘ . curl_error ( $ch );
>

// Закрываем дескриптор
curl_close ( $ch );
?>

Смотрите также

  • curl_error() — Возвращает строку с описанием последней ошибки текущего сеанса
  • » Коды ошибок cURL

User Contributed Notes 10 notes

if someone need more information about curl errors
=array(
[ 1 ] => ‘CURLE_UNSUPPORTED_PROTOCOL’ ,
[ 2 ] => ‘CURLE_FAILED_INIT’ ,
[ 3 ] => ‘CURLE_URL_MALFORMAT’ ,
[ 4 ] => ‘CURLE_URL_MALFORMAT_USER’ ,
[ 5 ] => ‘CURLE_COULDNT_RESOLVE_PROXY’ ,
[ 6 ] => ‘CURLE_COULDNT_RESOLVE_HOST’ ,
[ 7 ] => ‘CURLE_COULDNT_CONNECT’ ,
[ 8 ] => ‘CURLE_FTP_WEIRD_SERVER_REPLY’ ,
[ 9 ] => ‘CURLE_REMOTE_ACCESS_DENIED’ ,
[ 11 ] => ‘CURLE_FTP_WEIRD_PASS_REPLY’ ,
[ 13 ] => ‘CURLE_FTP_WEIRD_PASV_REPLY’ ,
[ 14 ]=> ‘CURLE_FTP_WEIRD_227_FORMAT’ ,
[ 15 ] => ‘CURLE_FTP_CANT_GET_HOST’ ,
[ 17 ] => ‘CURLE_FTP_COULDNT_SET_TYPE’ ,
[ 18 ] => ‘CURLE_PARTIAL_FILE’ ,
[ 19 ] => ‘CURLE_FTP_COULDNT_RETR_FILE’ ,
[ 21 ] => ‘CURLE_QUOTE_ERROR’ ,
[ 22 ] => ‘CURLE_HTTP_RETURNED_ERROR’ ,
[ 23 ] => ‘CURLE_WRITE_ERROR’ ,
[ 25 ] => ‘CURLE_UPLOAD_FAILED’ ,
[ 26 ] => ‘CURLE_READ_ERROR’ ,
[ 27 ] => ‘CURLE_OUT_OF_MEMORY’ ,
[ 28 ] => ‘CURLE_OPERATION_TIMEDOUT’ ,
[ 30 ] => ‘CURLE_FTP_PORT_FAILED’ ,
[ 31 ] => ‘CURLE_FTP_COULDNT_USE_REST’ ,
[ 33 ] => ‘CURLE_RANGE_ERROR’ ,
[ 34 ] => ‘CURLE_HTTP_POST_ERROR’ ,
[ 35 ] => ‘CURLE_SSL_CONNECT_ERROR’ ,
[ 36 ] => ‘CURLE_BAD_DOWNLOAD_RESUME’ ,
[ 37 ] => ‘CURLE_FILE_COULDNT_READ_FILE’ ,
[ 38 ] => ‘CURLE_LDAP_CANNOT_BIND’ ,
[ 39 ] => ‘CURLE_LDAP_SEARCH_FAILED’ ,
[ 41 ] => ‘CURLE_FUNCTION_NOT_FOUND’ ,
[ 42 ] => ‘CURLE_ABORTED_BY_CALLBACK’ ,
[ 43 ] => ‘CURLE_BAD_FUNCTION_ARGUMENT’ ,
[ 45 ] => ‘CURLE_INTERFACE_FAILED’ ,
[ 47 ] => ‘CURLE_TOO_MANY_REDIRECTS’ ,
[ 48 ] => ‘CURLE_UNKNOWN_TELNET_OPTION’ ,
[ 49 ] => ‘CURLE_TELNET_OPTION_SYNTAX’ ,
[ 51 ] => ‘CURLE_PEER_FAILED_VERIFICATION’ ,
[ 52 ] => ‘CURLE_GOT_NOTHING’ ,
[ 53 ] => ‘CURLE_SSL_ENGINE_NOTFOUND’ ,
[ 54 ] => ‘CURLE_SSL_ENGINE_SETFAILED’ ,
[ 55 ] => ‘CURLE_SEND_ERROR’ ,
[ 56 ] => ‘CURLE_RECV_ERROR’ ,
[ 58 ] => ‘CURLE_SSL_CERTPROBLEM’ ,
[ 59 ] => ‘CURLE_SSL_CIPHER’ ,
[ 60 ] => ‘CURLE_SSL_CACERT’ ,
[ 61 ] => ‘CURLE_BAD_CONTENT_ENCODING’ ,
[ 62 ] => ‘CURLE_LDAP_INVALID_URL’ ,
[ 63 ] => ‘CURLE_FILESIZE_EXCEEDED’ ,
[ 64 ] => ‘CURLE_USE_SSL_FAILED’ ,
[ 65 ] => ‘CURLE_SEND_FAIL_REWIND’ ,
[ 66 ] => ‘CURLE_SSL_ENGINE_INITFAILED’ ,
[ 67 ] => ‘CURLE_LOGIN_DENIED’ ,
[ 68 ] => ‘CURLE_TFTP_NOTFOUND’ ,
[ 69 ] => ‘CURLE_TFTP_PERM’ ,
[ 70 ] => ‘CURLE_REMOTE_DISK_FULL’ ,
[ 71 ] => ‘CURLE_TFTP_ILLEGAL’ ,
[ 72 ] => ‘CURLE_TFTP_UNKNOWNID’ ,
[ 73 ] => ‘CURLE_REMOTE_FILE_EXISTS’ ,
[ 74 ] => ‘CURLE_TFTP_NOSUCHUSER’ ,
[ 75 ] => ‘CURLE_CONV_FAILED’ ,
[ 76 ] => ‘CURLE_CONV_REQD’ ,
[ 77 ] => ‘CURLE_SSL_CACERT_BADFILE’ ,
[ 78 ] => ‘CURLE_REMOTE_FILE_NOT_FOUND’ ,
[ 79 ] => ‘CURLE_SSH’ ,
[ 80 ] => ‘CURLE_SSL_SHUTDOWN_FAILED’ ,
[ 81 ] => ‘CURLE_AGAIN’ ,
[ 82 ] => ‘CURLE_SSL_CRL_BADFILE’ ,
[ 83 ] => ‘CURLE_SSL_ISSUER_ERROR’ ,
[ 84 ] => ‘CURLE_FTP_PRET_FAILED’ ,
[ 84 ] => ‘CURLE_FTP_PRET_FAILED’ ,
[ 85 ] => ‘CURLE_RTSP_CSEQ_ERROR’ ,
[ 86 ] => ‘CURLE_RTSP_SESSION_ERROR’ ,
[ 87 ] => ‘CURLE_FTP_BAD_FILE_LIST’ ,
[ 88 ] => ‘CURLE_CHUNK_FAILED’ );

Источник

Chapter 6

The curl requests we have used so far have all been expertly crafted to work. We haven’t tried to game the API or make it fail, because we are nice people.

Not everyone is like this, unfortunately! And it’s not only people with bad intentions we need to look out for, but also simple mistakes. Anyone can make a mistake — it happens. That does not mean we should laugh at them for trying and send back some indecipherable message.

Descriptive messages are a very important part of any web API. Web APIs are used by our fellow developers and, by building an API, we take the responsibility of providing a good service for them. Any service should be easy to use. Put yourself inside a user’s shoes and pinpoint the issues someone could have with your service.

The obvious problem in our current web API is that we don’t send back meaningful errors when something goes wrong.

Let’s take the following curl request as an example. Notice anything wrong? We forgot to close the JSON document. Before we can realize our mistake, we send the request…

curl -i -X POST http://localhost:4567/users 
     -H "Content-Type: application/json" 
     -d '{"first_name":"Samuel", "last_name":"Da Costa", "age":19'

and we end up with a scary 500 Internal Server Error. This seems to mean that there was something wrong with the server, not with our request.

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain
Content-Length: 4679
Connection: keep-alive
Server: thin

[JSON::ParserError]

Luckily, Sinatra sends back the whole stack trace, so we can easily debug it. Not every framework is that nice, though. That’s the server’s duty — to provide descriptive errors to the client. A 500 is as descriptive as looking at a black box that just says: “Sorry bro, it didn’t work”.

HTTP comes with a set of features to handle errors. Indeed, thanks to all its HTTP status codes, we can configure our API to send back descriptive error messages to a client in a format that it can understand.

6.1. The different classes of HTTP status codes

There are five classes of HTTP codes:

  • 1XX Informational: Status codes starting with 1 are known as informational codes. Most of them are rarely used nowadays.
  • 2XX Success: These codes indicate that the exchange between the server and the client was successful.
  • 3XX Redirection: 3XX codes indicate that the client must take additional action before the request can be completed.
  • 4XX Client Error: There was something wrong with the request the client sent and the server cannot process it.
  • 5XX Server Error: The client sent a valid request but the server was not able to process it successfully.

We will be using status codes from some of these classes in this chapter.

6.2. Global

We will add error handling for each route, but let’s first see the errors we need to handle for every request.

6.2.1. 405 Method Not Allowed

This HTTP status code can be used when the client tries to access a resource using an unsupported HTTP method. For example, in our API, what would happen if a client tried to use the PUT method with the /users URI?

Let’s give it a try.

curl -i -X PUT http://localhost:4567/users

Output

HTTP/1.1 404 Not Found
Content-Type: text/html;charset=utf-8
X-Cascade: pass
Content-Length: 467
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Connection: keep-alive
Server: thin

<!DOCTYPE html>
<html>
<head>
<MORE HTML>

That’s not very helpful… Plus, it returns 404. While we do have a resource living at that URI, it’s not the right HTTP method… we can do better. All we need to do is catch any request accessing /users with an unsupported method: put, patch and delete. With a bit of metaprogramming, it’s not hard to do.

In the code below, we loop through each of the HTTP methods we don’t support and define a Sinatra route that returns 405 Method Not Allowed for each one of them.

# webapi.rb
# /users
# head /users
# get /users
# post /users

[:put, :patch, :delete].each do |method|
  send(method, '/users') do
    halt 405
  end
end

# options /users/:first_name

Let’s give it another try after restarting the server, shall we?

curl -i -X PUT http://localhost:4567/users

Output

HTTP/1.1 405 Method Not Allowed
Content-Type: text/html;charset=utf-8
Content-Length: 0
[MORE HEADERS]

Alright! It’s working correctly, let’s proceed.

6.2.2. 406 Not Acceptable

We already implemented one HTTP status code in the previous chapters. If the client requests a media type that our API does not support, we will return 406 Not Acceptable. To the client receiving this, it means that the server was not able to generate a representation for the resource according to the criteria the client had fixed. The response should also include what formats are actually available for the client to pick from.

# webapi.rb
# users
# ...

helpers do

  # def json_or_default?
  # def xml?

  def accepted_media_type
    return 'json' unless request.accept.any?

    request.accept.each do |type|
      return 'json' if json_or_default?(type)
      return 'xml' if xml?(type)
    end

    content_type 'text/plain'
    halt 406, 'application/json, application/xml'
  end

  # def type
  # def send_data

end

Let’s try with a fake media type that we do not support.

curl -i http://localhost:4567/users -H "Accept: moar/curl"

Output

HTTP/1.1 406 Not Acceptable
Content-Type: text/plain;charset=utf-8
Content-Length: 33
... More Headers ...

application/json, application/xml

With this, the client can understand what is actually supported and act accordingly. Sadly, it’s not defined more clearly in a standard.

Alternatively, we could actually return a JSON representation instead of telling the client that we can’t give it anything. We would set the Content-Type header to application/json to let the client do its own checking before it parses the response. Both this option and the previous one are acceptable as defined in the HTTP RFC.

# webapi.rb
# users
# ...

helpers do

  # def json_or_default?
  # def xml?

  def accepted_media_type
    return 'json' unless request.accept.any?

    request.accept.each do |type|
      return 'json' if json_or_default?(type)
      return 'xml' if xml?(type)
    end

    'json'
  end

  # def type
  # def send_data

end

Don’t forget to restart the server.

curl -i http://localhost:4567/users -H "Accept: moar/curl"

Output

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 203
... More Headers ...

[
  {"first_name":"Thibault", "last_name":"Denizet", "age":25, "id":"thibault"},
  {"first_name":"Simon", "last_name":"Random", "age":26, "id":"simon"},
  {"first_name":"John", "last_name":"Smith", "age":28, "id":"john"}
]

If your server cannot generate a representation the way the client requests, you can either return 406 (with or without a list of actually supported media types), or just return the default format and let the client handle it.

6.3. POST /users

For the post /users route, we are currently not handling any errors. If you run the following curl request, it will crash and send you back 500 and the whole stack trace.

curl -X POST -i http://localhost:4567/users 
     -H "Content-Type: application/fake" 
     -d 'Weirdly Formatted Data'

Output

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain
Content-Length: 4647
Connection: keep-alive
Server: thin

JSON::ParserError: 757: unexpected token at 'Weirdly Formatted Data'
[Stack Trace]

This does not help the client in understanding what happened. It does help us humans, because we can read the Ruby stack trace and see that there was a JSON::ParserError. But to an automated client, this only means that something went wrong on the server. We need to fix this.

6.3.1. 415 Unsupported Media Type

To fix it we will use the 415 HTTP status code. 415 Unsupported Media Type is exactly what its name implies — we can return it to the client when we don’t understand the format of the data sent by the client.

To prevent the execution of the code there, we will first check if the data was sent as a JSON document, the only format we support for requests. Our code must be guarded from unwanted media types.

We just need the following line to do it.

halt 415 unless request.env['CONTENT_TYPE'] == 'application/json'

And it comes in the post /users route like this:

# webapi.rb
# Stuff
# ...
post '/users' do
  halt 415 unless request.env['CONTENT_TYPE'] == 'application/json'

  users[user['first_name'].downcase.to_sym] = user

  url = "http://localhost:4567/users/#{user['first_name']}"
  response.headers['Location'] = url
  status 201
end

Don’t forget to restart the server.

Now, let’s see what happens when we send the curl request seen in the previous section.

curl -X POST -i http://localhost:4567/users 
     -H "Content-Type: application/fake" 
     -d 'Weirdly Formatted Data'

Output

HTTP/1.1 415 Unsupported Media Type
[MORE HEADERS]

That’s much more descriptive, right? The only problem is that this does not tell the client which media types are actually supported. We will see what we can do about this later. For now, our endpoint won’t crash anymore whatever the received format is.

6.3.2. 400 Bad Request

The 400 Bad Request is used way too often in modern web applications. Its real purpose is to indicate that the server could not understand the request due to some syntax issue. For example, a malformed JSON document.

Let’s give it a try and break some stuff.

curl -X POST -i http://localhost:4567/users 
     -H "Content-Type: application/json" 
     -d '{"first_name":"Mark"'

Output

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain
Content-Length: 4645
Connection: keep-alive
Server: thin

JSON::ParserError: 757: unexpected token at '{"first_name":"Mark"'
[Stack Trace]

Boom! Broken. Since our JSON document, {"first_name":"Mark", is not valid, our server crashes when trying to parse it. Here is how we can fix this. We are going to catch the exception raised when parsing an invalid JSON document (JSON::ParserError). In the rescue, we will send back the error to the client, in the format of its choice (specified by the Accept header). Note that we are still using the send_data method we wrote earlier.

# webapi.rb
# Stuff
# ...
post '/users' do
  halt 415 unless request.env['CONTENT_TYPE'] == 'application/json'

  begin
    user = JSON.parse(request.body.read)
  rescue JSON::ParserError => e
    halt 400, send_data(json: -> { { message: e.to_s } },
                        xml:  -> { { message: e.to_s } })
  end

  users[user['first_name'].downcase.to_sym] = user
  url = "http://localhost:4567/users/#{user['first_name']}"
  response.headers['Location'] = url
  status 201
end

Don’t forget to restart your server!

curl -X POST -i http://localhost:4567/users 
     -H "Content-Type: application/json" 
     -d '{"first_name":"Mark"'

Output

HTTP/1.1 400 Bad Request
Content-Type: application/json
Content-Length: 65
X-Content-Type-Options: nosniff
Connection: keep-alive
Server: thin

{"message":"757: unexpected token at '{"first_name":"Mark"'"}

Great! We are getting a 400 back and the body contains the error message.

6.3.3. 409 Conflict

Currently, if we try to re-create a user that already exists, it’s just going to override the existing one. Not good. This should throw up some kind of conflict error saying that a user cannot be overridden. The problem here is how we save users in a Ruby hash. Since we use the first name as a key, we don’t want to allow the override of an existing user.

Luckily, that’s what the 409 Conflict HTTP code is for. Note that this code is only allowed in situations where the client can actually fix the conflict. In our case, the client can either change the first name or use another endpoint to update the user.

Our use of the first name as a key is there only for simplicity and any production application should not use such a mechanism. Unique generated IDs are way better!

To prevent any override, we will use this code. If we find a user with the first name sent by the client, we’ll immediately halt and send back a response to the client with a 409 status code and an explanation message.

if users[user['first_name'].downcase.to_sym]
  message = { message: "User #{user['first_name']} already in DB." }
  halt 409, send_data(json: -> { message },
                      xml:  -> { message })
end

Here is our full POST endpoint:

# webapi.rb
# Stuff
# ...
post '/users' do
  halt 415 unless request.env['CONTENT_TYPE'] == 'application/json'

  begin
    user = JSON.parse(request.body.read)
  rescue JSON::ParserError => e
    halt 400, send_data(json: -> { { message: e.to_s } },
                        xml:  -> { { message: e.to_s } })
  end

  if users[user['first_name'].downcase.to_sym]
    message = { message: "User #{user['first_name']} already in DB." }
    halt 409, send_data(json: -> { message },
                        xml:  -> { message })
  end

  users[user['first_name'].downcase.to_sym] = user
  url = "http://localhost:4567/users/#{user['first_name']}"
  response.headers['Location'] = url
  status 201
end

Let’s try to recreate an existing user (for example, Thibault), after restarting the server.

curl -X POST -i http://localhost:4567/users 
     -H "Content-Type: application/json" 
     -d '{"first_name":"Thibault", "last_name":"Denizet", "age":25}'

Output

HTTP/1.1 409 Conflict
Content-Type: application/json
Content-Length: 42
X-Content-Type-Options: nosniff
Connection: keep-alive
Server: thin

{"message":"User Thibault already in DB."}

Great, we’re telling the client that its request triggered a conflict that needs to be solved. We can now detect when updates conflict with the users’ data stored in the users hash.

6.4. GET /users/:first_name

Let’s now turn our attention to the get /users/:first_name route. We know our endpoint works fine when the user exists. But what happens when it doesn’t?

curl -i http://localhost:4567/users/frank
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 4
... More Headers ...

null

What the heck is this null doing here? Plus, we are telling the client it’s a JSON document with the Content-Type. We really need to fix this!

6.4.1. 404 Not Found

The 404 Not Found status code can help us. This code is meant to tell the client that nothing was found at the URI specified by the client; in other words, the requested resource does not exist.

This fits what we need for the get /users/:first_name route pretty well. Let’s add a little halt that will return 404 if the user is not found in our hash.

# webapi.rb
# Stuff
# ...
get '/users/:first_name' do |first_name|
  halt 404 unless users[first_name.to_sym]

  send_data(json: -> { users[first_name.to_sym].merge(id: first_name) },
            xml:  -> { { first_name => users[first_name.to_sym] } })
end

Restart the server and send the following request.

curl -i http://localhost:4567/users/frank

Output

HTTP/1.1 404 Not Found
... More Headers ...

Looks good!

6.4.2. 410 Gone

Sadly 404 Not Found can seem quite generic to a client. We can use other codes to give a more specific response. One of them is 410 Gone which indicates that a resource used to live at the given URI, but doesn’t anymore. It has probably been deleted since the server does not have the new location.

Currently, if we delete a user and then try to access it, we will just get 404, as if it never existed.

It’s not always possible for a server to indicate a deleted resource and there is nothing forcing us to use this HTTP status. It is helpful for a client though, since it will let the client know that this URI does not point to anything anymore. It shouldn’t be considered as a bug since there used to be something there. The client should take note that it’s now gone and move on with its life.

Let’s quickly add a mechanism to keep track of what has been deleted. In a production application with a real database, it is possible to achieve this kind of mechanism by using a boolean in order to see if something has been deleted or not. There are other ways to do it, but we won’t be covering them.

The first thing we need is a hash, named deleted_users, that will contain all the deleted users (duh!).

# webapi.rb
# require
users = {
  thibault: { first_name: 'Thibault', last_name: 'Denizet', age: 25 },
  simon:    { first_name: 'Simon', last_name: 'Random', age: 26 },
  john:     { first_name: 'John', last_name: 'Smith', age: 28 }
}

deleted_users = {}

# Helpers
# Routes

Then we need to update the get /users/:first_name route to halt with 410 if the deleted_users hash contains the specified user.

# webapi.rb
# stuff
get '/users/:first_name' do |first_name|
  halt 410 if deleted_users[first_name.to_sym]
  halt 404 unless users[first_name.to_sym]

  send_data(json: -> { users[first_name.to_sym].merge(id: first_name) },
            xml:  -> { { first_name => users[first_name.to_sym] } })
end

Finally, we need to fill the deleted_users hash every time a user gets deleted in the delete /users/:first_name route.

# webapi.rb
# stuff
delete '/users/:first_name' do |first_name|
  first_name = first_name.to_sym
  deleted_users[first_name] = users[first_name] if users[first_name]
  users.delete(first_name)
  status 204
end

Everything is in place now and we can start sending some test requests. First, restart the server. Then delete the simon user with this command:

curl -i -X DELETE http://localhost:4567/users/simon

Output

HTTP/1.1 204 No Content
[ ... ]

Now, what happens when we try to access the Simon resource?

curl -i http://localhost:4567/users/simon
HTTP/1.1 410 Gone
Content-Type: text/html;charset=utf-8
Content-Length: 0
[More Headers]

Our little API is getting more expressive by the minute!

6.5. PUT /users/:first_name

For this route, we can handle 415 Unsupported Media Type and 400 Bad Request. However, I’m not going to show you how. Instead, I want you to do it.

Testing 415

curl -X PUT -i http://localhost:4567/users/thibault 
     -H "Content-Type: application/fake" 
     -d 'Weirdly Formatted Data'

Testing 400

curl -X PUT -i http://localhost:4567/users/thibault 
     -H "Content-Type: application/json" 
     -d '{"first_name":"Tibo"'

6.6. PATCH /users/:first_name

For this route, we can handle 415 Unsupported Media Type, 404 Not Found, 410 Gone and 400 Bad Request. You can get inspired by what we did for POST /users/:first_name and GET /users/:first_name. Below, you will find curl commands to test if you have correctly implemented each status code.

Testing 415

curl -X PATCH -i http://localhost:4567/users/thibault 
     -H "Content-Type: application/fake" 
     -d 'Weirdly Formatted Data'

Testing 400

curl -X PATCH -i http://localhost:4567/users/thibault 
     -H "Content-Type: application/json" 
     -d '{"first_name":"Tibo"'

Testing 404

curl -X PATCH -i http://localhost:4567/users/mark 
     -H "Content-Type: application/json" 
     -d '{"first_name":"Marc"}'

Testing 410

curl -i -X DELETE http://localhost:4567/users/simon
curl -X PATCH -i http://localhost:4567/users/simon 
     -H "Content-Type: application/json" 
     -d '{"first_name":"Super Simon"}'

6.7. Wrap Up

In this chapter, we’ve seen how to implement errors to let our clients understand exactly what went wrong when something didn’t go as planned.

In the next chapter, we will tackle a sensitive subject: versioning!

Львиная доля всех моих работ связана с работами по сео-оптимизации сайтов по требованиям сео-специалистов. Доработать, исправить, добавить, устранить… Очень часто нужно посмотреть ответы сервера по тому или иному url-адресу.

Проверку можно делать разными способами. Самым простым будет воспользоваться онлайн-тестами. Но это не наш вариант:)

В данном посте рассмотрим, какие возможности для данной задачи нам предоставляет curl.

Коды ответа сервера

Для начала рассмотрим самые распространенные коды ответа сервера:

1xx — информационные:

  • 100 — сервер принял первую часть запроса, можно подрожать передачу;
  • 101 — нужно изменить протокол работы на более подходящий;
  • 102 — на обработку запроса уйдет много времени, используется чтобы браузер не разрывал соединение раньше времени;

2хх — операция успешна:

  • 200 — запрос выполнен успешно, отправляется для большинства запрашиваемых страниц;
  • 201 — после выполнения запроса был создан ресурс;
  • 202 — запрос принят, но еще не обработан;
  • 203 — запрос выполнен успешно, но информация для ответа взята из прокси;
  • 204 — запрос обработан, но контента для отображения нет;
  • 205 — попросить пользователя ввести необходимые данные;
  • 206 — запрос обработан, но передана только часть контента;

3xx — перенаправления:

  • 300 — есть несколько страниц для этого запроса, например, на нескольких языках;
  • 301 — страница навсегда перемещена по новому адресу;
  • 302 — документ был временно перемещен;
  • 303 — документ необходимо загрузить по указанному адресу с помощью протокола GET;
  • 304 — документ не изменился с последнего запроса;
  • 305 — нужно использовать прокси;
  • 307 — ресурс временно перемещен на новый адрес.

4хх — ошибка в запросе:

  • 400 — неверный запрос;
  • 401 — необходимо аутентифицироваться;
  • 403 — запрос принят, но у вас нет доступа;
  • 404 — страница не найдена на сервере;
  • 405 — используемый метод нельзя применять на сервере;
  • 408 — время ожидания передачи запроса истекло;
  • 410 — ресурс полностью удален;
  • 411 — нужно указать длину запроса;
  • 413 — запрос слишком длинный;
  • 414 — URI запроса слишком длинная.

5хх — ошибка сервера:

  • 500 — внутренняя ошибка сервера;
  • 501 — нужная функция не поддерживается;
  • 502 — прокси не может соединиться со шлюзом;
  • 503 — сервер не может обрабатывать запросы по техническим причинам;
  • 504 — прокси не дождался ответа от сервера;
  • 505 — версия протокола HTTP не поддерживается.

Основные заголовки, отправляемые сервером

  • Server — имя и версия веб-сервера;
  • Date — дата осуществления запроса;
  • Content-Type — MIME тип передаваемых данных, например, text/html, тут же задается кодировка;
  • Connection — тип соединения, может быть closed — уже закрыто, или keep-alive — открыто для передачи данных;
  • Vary — указывает при каких заголовках веб-сервер будет возвращать разные старины для одного URI;
  • Set-Cookie — сохранить Cookie информацию для страницы;
  • Expires — можно хранить страницу или ресурс в кэше до определенной даты;
  • Cache-Control — настройка времени кэширования страницы браузером, а также разрешения на кэширования;
  • ETag — содержит контрольную сумму для страницы, применимо для проверки кэша;
  • Last-Modified — дата, когда страница последний раз была изменена;

Проверка кода ответа сервера с помощью curl

Чтобы увидеть только код ответа страницы достаточно выполнить такую команду:

curl -I https://pai-bx.com 2>/dev/null | head -n 1 | cut -d$' ' -f2

код ответа страницы

Как видим, сервер вернул 200 статус, что означает, что все ок. Страница доступна для чтения. Если проверить страницу, для которой должны быть настроены редиректы, получим 301-й статус:

Страница доступна для чтения.

Проверка http-заголовков с помощью curl

Чтобы вывести заголовки страницы необходимо запустить curl с опцией -I:

curl -I https://pai-bx.com

вывести заголовки страницы

Проверка IF-MODIFIED-SINCE

Чтобы увидеть работает ли данный заголовок, для начала выполняем обычный запрос заголовков сервера. Далее, повторяем отправку запроса, но уже с заголовком If-Modified-Since и датой:

IF-MODIFIED-SINCE

Видим, что второй запрос возвращает 304-й статус ответа, что означает, что с указанного времени страница не менялась и может быть взята из кеша браузера.

Таким образом, с помощью curl можно увидеть код ответа сервера и отдаваемые сервером заголовки.

I am trying to check whether a file is available at a particular location or not, if file doesn’t exists then it must send an email to an email list. The problem is that, the file needs to be downloaded from a URL (sharepoint) before checking if it exists or not.

If there is no file at the URL, the cURL command is downloading the file with «404 NOT FOUND» content in it. If there is no file at the source, I don’t want the cURL command to download any file. How can i achieve this?

cd $file_path    
curl -k --ntlm -u $USER_GROUP:$PASS -O http://sharepoint.com/sites/dummy/file_list.txt

This is the command to download that i am using. Any help would be appreciated. Thanks.

asked Sep 30, 2020 at 8:30

Mohammed Arif's user avatar

Adding the -f switch should achieve what you want.

From curl manual:

-f/—fail
(HTTP) Fail silently (no output at all) on server errors. This is mostly done to better enable scripts etc to better deal with failed attempts. In normal cases when a HTTP server fails to deliver a document, it returns an HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and return error 22.

This method is not fail-safe and there are occasions where non-successful response codes will slip through, especially when authentication is involved (response codes 401 and 407).

answered Sep 30, 2020 at 8:54

Dave White's user avatar

Dave WhiteDave White

2201 gold badge2 silver badges8 bronze badges

Improve Article

Save Article

  • Read
  • Discuss
  • Improve Article

    Save Article

    Checking if a Webpage URL exists or not is relatively easy in PHP. If the required URL does not exist, then it will return 404 error. The checking can be done with and without using cURL library.

    cURL: The cURL stands for ‘Client for URLs’, originally with URL spelled in uppercase to make it obvious that it deals with URLs. It is pronounced as ‘see URL’. The cURL project has two products libcurl and curl.

    • libcurl: A free and easy-to-use client-side URL transfer library, supporting FTP, TPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE, and LDAP. libcurl supports TTPS certificates, HTTP POST, HTTP PUT, FTP uploading, kerberos, HTTP based upload, proxies, cookies, user & password authentication, file transfer resume, HTTP proxy tunneling and many more. libcurl is free, thread-safe, IPv6 compatible, feature rich, well supported and fast.
    • curl: A command line tool for getting or sending files using URL syntax. Since curl uses libcurl, it supports a range of common internal protocols, currently including HTTP, HTTPS, FTP, FTPS, GOPHER, TELNET, DICT, and FILE.

    Example 1: This example test a URL for 404 error without using cURL approach.

    <?php

    $array = @get_headers($url);

    $string = $array[0];

    if(strpos($string, "200")) {

        echo 'Specified URL Exists';

    else {

        echo 'Specified URL does not exist';

    }

    ?>

    Output:

    Specified URL exists

    Example 2: This example test a URL for 404 error using cURL approach.

    <?php

    curl_setopt($ch, CURLOPT_NOBODY, true);

    curl_exec($ch);

    $retcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

    if($retcode != 200) {

        echo "Specified URL does not exist";

    }

    else {

        echo "Specified URL exists";

    }

    curl_close($ch);

    ?>

    Output:

    Specified URL exists

    Понравилась статья? Поделить с друзьями:
  • Csrf token mismatch ошибка что делать
  • Csrf error code
  • Curl 22 the requested url returned error 404 not found
  • Cura как изменить размер модели
  • Cups waiting for job completed ошибка linux