Curl http error before end of send stop sending

I am sending a multipart form POST via CURL from Linux shell script. The request works fine in Postman but from CURL it fails with:

Internal Server Error

then

HTTP error before end of send, stop sending

I even copy the Curl for Linux Shell code directly from Postman and paste into the shell script, so it should be exactly the same request that Postman is making.

Here is the command:

curl --request POST 
  --no-alpn 
  --url https://XXXXXXXXXXX/api/v1.0/XXXXX/XXXXXX/XXXXX 
  --header 'accept: text/plain' 
  --header 'cache-control: no-cache' 
  --header 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' 
  --header 'sessionid: $session_id' 
  --form filename=XXXXXX.zip 
  --form XXXXXX=XXXXXX 
  --form file=@$file_path 
  --trace-ascii /dev/stdout || exit $?
}

And here is the log from —trace-ascii:

https://XXXXXXXXXXXXXXXXX/api/v1.0/XXXXXX/XXXXX/XXXXXXXXX
Note: Unnecessary use of -X or --request, POST is already inferred.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0== Info:   Trying XXX.XXX.XXX.XXXX...
== Info: Connected to XXXXXXXXXXXXXXXX.com (XX.XX.XX.XX) port 443 (#0)
== Info: found 148 certificates in /etc/ssl/certs/ca-certificates.crt
== Info: found 592 certificates in /etc/ssl/certs
== Info: SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
== Info:     server certificate verification OK
== Info:     server certificate status verification SKIPPED
== Info:     common name: *.XXXXXX.com (matched)
== Info:     server certificate expiration date OK
== Info:     server certificate activation date OK
== Info:     certificate public key: RSA
== Info:     certificate version: #3
== Info:     subject: OU=Domain Control Validated,CN=*.XXXXXX.com
== Info:     start date: Mon, 15 Aug 2016 08:23:38 GMT
== Info:     expire date: Thu, 15 Aug 2019 08:23:38 GMT
== Info:     issuer: C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com, Inc.,OU=http://certs.godaddy.com/repository/,CN=Go Daddy Secure Certificate Authority - G2
== Info:     compression: NULL
=> Send header, 363 bytes (0x16b)
0000: POST /XXXXX/api/v1.0/XXXXXX/upload/XXXXXX HTTP/1.1
003b: Host: XXXXXX.XXXXXXX.com
0059: User-Agent: curl/7.47.0
0072: accept: text/plain
0086: cache-control: no-cache
009f: sessionid: $session_id
00b7: Content-Length: 1639
00cd: Expect: 100-continue
00e3: content-type: multipart/form-data; boundary=----WebKitFormBounda
0123: ry7MA4YWxkTrZu0gW; boundary=------------------------b059847fb557
0163: a899
0169: 
<= Recv header, 23 bytes (0x17)
0000: HTTP/1.1 100 Continue
=> Send data, 387 bytes (0x183)
0000: --------------------------b059847fb557a899
002c: Content-Disposition: form-data; name="filename"
005d: 
005f: xxxxxxxxxx.zip
006b: --------------------------b059847fb557a899
0097: Content-Disposition: form-data; name="XXXXXXXXXXX"
00cc: 
00ce: XXXXXXXXXXXXXXXXXXXX
00ea: --------------------------b059847fb557a899
0116: Content-Disposition: form-data; name="file"; filename="XXXXXX.zip
0156: "
0159: Content-Type: application/octet-stream
0181: 
=> Send data, 1204 bytes (0x4b4)
0000: PK........r~.K..D!....p.......output/XXXXXXX.XXXXX.....7Z..7Zux...
0040: ............{LSW....@.!`.9. F..Eh+.......JA..W.2.V...A.%>... #Q1
0080: T.....{Nb.]&..1.3M|.........w..z.]8..I.I>.....n?...hM/.h..?oy^.
00c0: ..... ..:.>J..Q...N...*A...l`...."..N...@.P'........d..._.....L
0100: .].......z....N6.B......Y5t...Zd.V...}..l...........EC..$..e...W
0140: .V`.lV...p..d._.....S...............d`.l..}.....f[...{....`....M
0180: .....kN..[.4.2w.9.bN....q.8.'.K.......'..~........sI.....K...s.
01c0: ...U.'..d,.......>......T.5....|.$,)o'bIy{...pN.....K.o..[..cWp.
0200: c.@..B.S........d.I..P./.F..0....=4.......d..#{K$..#.^=.......
0240: *....Bi...i....8j!T......|.Ld...x....>......A...|.I.}>.....Yt=..
0280: ..Tp.q...O&.. .....Ac..V....a......f.G...!x.f.i.gu}.2i.4....NK..
02c0: .G;..k~......=*....g..c#..c.M.oW........-...vW.~#u...#....cz.bu=
0300: .."Bs.js.z.1.....&|.MV..<a"4...IqRO.kKC.v.Gz.....].G..|...:om
0340: .C.v5G..X].kw......R/.........C.X].5<.B.'....z.O|@.v.P......
0380: ^...f~........9....YG~fum}....^,K.......F.vmIl....hI."h.FM.....f
03c0: ....Z...`um.}E...1;......_....yF.xV...BDh...U..z...*.o.`O..V.W.6
0400: ..kf.n...*.{..].].c~.w~K......4I.k.Y.....r.wV.................F
0440: .v..O..OPK..........r~.K..D!....p.....................output/xxxxxxx
.mldUT.....7Zux.............PK..........V...H.....
=> Send data, 48 bytes (0x30)
0000: 
0002: --------------------------b059847fb557a899--
<= Recv header, 36 bytes (0x24)
0000: HTTP/1.1 500 Internal Server Error
<= Recv header, 15 bytes (0xf)
0000: Server: nginx
<= Recv header, 37 bytes (0x25)
0000: Date: Mon, 18 Dec 2017 15:15:56 GMT
<= Recv header, 26 bytes (0x1a)
0000: Content-Type: text/plain
<= Recv header, 28 bytes (0x1c)
0000: Transfer-Encoding: chunked
<= Recv header, 24 bytes (0x18)
0000: Connection: keep-alive

100  1639    0     0  100  1639      0   2269 --:--:-- --:--:-- --:--:--  2266<= Recv header, 29 bytes (0x1d)
0000: X-FRAME-OPTIONS: SAMEORIGIN
<= Recv header, 83 bytes (0x53)
0000: Set-Cookie: JSESSIONID=XXXXXXXXXXXXXXXXXXXXXXXX; Path=/;
0040:  Secure; HttpOnly
== Info: HTTP error before end of send, stop sending
<= Recv header, 2 bytes (0x2)
0000: 
<= Recv data, 106 bytes (0x6a)
0000: 64
0004: <ErrorResponse><key/><localizedMessage/><httpError>Internal Serv
0044: er Error</httpError></ErrorResponse>
<= Recv data, 5 bytes (0x5)
0000: 0
0003: 

I should add that the CURL command is being run from a Docker container.

  HTTP/1.1 100 Continue
< HTTP/1.1 500 Internal Server Error
< Server: openresty/1.9.7.4
< Date: Mon, 23 Jan 2017 16:51:29 GMT
< Content-Type: application/json; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Powered-By: PHP/5.6.21
< X-Rate-Limit-Limit: 600
< X-Rate-Limit-Remaining: 599
< X-Rate-Limit-Reset: 0

HTTP error before end of send, stop sending
<
Closing connection 0`

$request_url = 'https://api.acrcloud.com/v1/audios';
$http_method = 'POST';
$http_uri = '/v1/audios';
$timestamp = time();
$signature_version = '1';
$account_access_key = '';
$account_access_secret = '';
$string_to_sign =
$http_method . "n" .
$http_uri . "n" .
$account_access_key . "n" .
$signature_version . "n" .
$timestamp;
$signature = base64_encode(hash_hmac("sha1", $string_to_sign, $account_access_secret, true));
$realpath = realpath('uploads/*****.mp3');

if(class_exists('CURLFile'))
$cfile = new CURLFile($realpath, "audio/mp3", basename($realpath));
else
$cfile = '@' . $realpath;
$fields = array(
'audio_id' => '30007',
'title' => 'aya number 19',
'audio_file' => $cfile,
'bucket_name' => 'whatever',
'data_type' => 'audio',  // if you upload fingerprint file please set 'data_type'=>'fingerprint'
'custom_key[0]' => 'track_id',
);
$headerArray = array();
$headers = array(
'access-key' => $account_access_key,
'signature' => $signature,
'signature-version' => '1',
'timestamp' => $timestamp,
);
foreach( $headers as $n => $v ) {
$headerArr[] = $n .':' . $v;
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $request_url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_VERBOSE, true);
$verbose = fopen('php://temp', 'w+');
curl_setopt($ch, CURLOPT_STDERR, $verbose);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headerArr);
$result = curl_exec($ch);
if ($result === FALSE) {
printf("cUrl error (#%d): %s<br>n", curl_errno($ch),
htmlspecialchars(curl_error($ch)));
}
rewind($verbose);
$verboseLog = stream_get_contents($verbose);

echo "Verbose information:n<pre>", htmlspecialchars($verboseLog), "</pre>n";
dd($result);
curl_close($ch);

Решение

Другие решения

Других решений пока нет …

Is that the full curl verbose output? If not, please post it all.

You should test this using the curl command line, or by putting the data in a file and using the default read function that reads from files, just to verify that curl can successfully talk to the server and PUT the data with the curl options you are setting (save READFUNCTION of course). Do that with verbose output so you can see all the header fields, options, etc used to get a successful transfer.

You should get familiar with the HTTP protocols. You can find the RFC for HTTP/1.0 here and HTTP/1.1 here. The status code sections 9 (1.0) and 10 (1.1) will be of use/interest.

Here’s a breakdown of what I think is going on (haven’t used curl in many years, and was never an HTTP protocol expert):

You sent the PUT request — well, you sent the header for the PUT request, but you didn’t send the whole thing yet. You send it using HTTP/1.1 protocol. That header contains lines 2-10 of your output. It states that the payload (Content-Length) will be 140 bytes. It also says that the client (your curl app) will be expecting a HTTP status code of 100 to continue the PUT — to actually put the 140 bytes of data.

Then the sever responds (lines 13-22 of your output). It responds in HTTP/1.0 protocol — this is not necessarily an issue, there is much overlap, and 1.1 is backwards compatible. It sends back a 302 code, saying the resource is temporarily moved. Note the difference, your request was the top one, the response with the new temporary location is the bottom:

Code:

 http://apidev-ds.awmdm.com/deviceservices/peripheralservice/v1/enrolldevice
https://apidev-ds.awmdm.com/deviceservices/peripheralservice/v1/enrolldevice

It’s asking you to use HTTPS (secure).

The Content-Length field on line 20 is the content of the server’s response (the 302), with 0 meaning there is no body/content in the response, just a header, which is perfectly fine.

If you read and compare the 1.0 and 1.1 specs carefully, you will notice that HTTP/1.0 doesn’t support 100-continue. Thus, if your client is talking HTTP/1.1 and says it expects a 100-continue from the sever, but the server only talks HTTP/1.0, then it wont be able to send a 100, and your client will run into trouble. I notice that the server you are using (response header field Server) is reported as BigIP. I’m not familiar with that and don’t know if it supports HTTP/1.1 — if you even have the ability to configure it.

Also: for logging/debugging purposes, you should try printing all the important values inside the callback: rarg->len, rarg->pos, len. It will help in any debugging once you get your custom read function to trigger.

Hi,

On 19.03.2013 16:31, Luka Perkov wrote:
> Hi all,
>
> I’m using nginx as a frontend for my SCGI application and I want to
> handle authentication in my SCGI code. I have to deal with POST
> requests. Is it ok that nginx sends «401 Unauthorized» after sending
> «100 Continue»?
>
> Are both requests bellow correct?
>

Both requests do look basically correct just like the nginx’s and
curl’s behavior. You didn’t supply any credentials for
authentication to continue.

You can use -u curl option to explicitly specify <user:password>
on the command line either -n option to read ’em from your netrc.
Curl will at least try to authenticate itself with the supplied
credentials then.

You could also find this document useful

http://curl.haxx.se/docs/httpscripting.html

> I’m asking because of this curl message: «HTTP error before end of send,
> stop sending».
>

Hope it helps.

Regards,
Vladimir

> Luka
>
> $ curl -v -X POST -d «@/path/to/some/blob» http://127.0.0.1:8000/
> * About to connect() to 127.0.0.1 port 8000 (#0)
> * Trying 127.0.0.1…
> * Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
>> POST / HTTP/1.1
>> User-Agent: curl/7.29.0
>> Host: 127.0.0.1:8000
>> Accept: */*
>> Content-Length: 3398
>> Content-Type: application/x-www-form-urlencoded
>> Expect: 100-continue
>>
> < HTTP/1.1 100 Continue
> < HTTP/1.1 401 Unauthorized
> < Server: nginx/1.3.14
> < Date: Tue, 19 Mar 2013 12:11:16 GMT
> < Transfer-Encoding: chunked
> < Connection: keep-alive
> < WWW-Authenticate: Basic realm=»bla-bla-bla»
> * HTTP error before end of send, stop sending
> <
> * Closing connection 0
>
>
> $ curl -v -X POST -d «@/dev/null» http://127.0.0.1:8000/
> * About to connect() to 127.0.0.1 port 8000 (#0)
> * Trying 127.0.0.1…
> * Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
>> POST / HTTP/1.1
>> User-Agent: curl/7.29.0
>> Host: 127.0.0.1:8000
>> Accept: */*
>> Content-Length: 0
>> Content-Type: application/x-www-form-urlencoded
>>
> < HTTP/1.1 401 Unauthorized
> < Server: nginx/1.3.14
> < Date: Tue, 19 Mar 2013 12:23:17 GMT
> < Transfer-Encoding: chunked
> < Connection: keep-alive
> < WWW-Authenticate: Basic realm=»bla-bla-bla»
> * HTTP error before end of send, stop sending
> <
> * Closing connection 0
>

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel