Context
-
Version of iperf3: 3.6
-
Hardware: raspberry pi 3 b
-
Operating system (and distribution, if any): Debian Linux (default pi OS)
Bug Report
I’m trying to run iperf3.6 over a 100kbps link to measuer the actual throughput. I’m running it on two pis.
On the server, I run «iperf3 -s —logfile /dev/tty
On the client, I run «iperf3 -c (IPv6 address of peer) -u -b 390k -l 1024 -t 12»
-
Expected Behavior I see 12 reports, one second apart, then the summary
-
Actual Behavior, on the client I see 11 reports, one second apart, then a pause, then iperf3: error — control socket has closed unexpectedly. On the server I see 16 reports one second apart then iperf3: error — select failed: Bad file descriptor
-
Steps to Reproduce: given above
Server side:
Server listening on 5201
Accepted connection from (IP addr), port 56688
[ 6] local (ip addr) port 5201 connected to (ip addr) port 54260
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 6] 0.00-1.00 sec 9.00 KBytes 73.7 Kbits/sec 37.226 ms 0/9 (0%)
[ 6] 1.00-2.00 sec 10.0 KBytes 81.9 Kbits/sec 55.150 ms 0/10 (0%)
[ 6] 2.00-3.00 sec 11.0 KBytes 90.1 Kbits/sec 65.500 ms 0/11 (0%)
[ 6] 3.00-4.00 sec 9.00 KBytes 73.7 Kbits/sec 72.033 ms 0/9 (0%)
[ 6] 4.00-5.00 sec 11.0 KBytes 90.1 Kbits/sec 73.836 ms 0/11 (0%)
[ 6] 5.00-6.00 sec 10.0 KBytes 81.9 Kbits/sec 74.872 ms 0/10 (0%)
[ 6] 6.00-7.00 sec 11.0 KBytes 90.1 Kbits/sec 74.872 ms 0/11 (0%)
[ 6] 7.00-8.00 sec 10.0 KBytes 81.9 Kbits/sec 75.292 ms 0/10 (0%)
[ 6] 8.00-9.00 sec 10.0 KBytes 81.9 Kbits/sec 75.182 ms 0/10 (0%)
[ 6] 9.00-10.00 sec 11.0 KBytes 90.1 Kbits/sec 75.403 ms 0/11 (0%)
[ 6] 10.00-11.00 sec 10.0 KBytes 81.9 Kbits/sec 89.071 ms 26/36 (72%)
[ 6] 11.00-12.00 sec 10.0 KBytes 81.9 Kbits/sec 53.561 ms 38/48 (79%)
[ 6] 12.00-13.00 sec 10.0 KBytes 81.9 Kbits/sec 36.800 ms 36/46 (78%)
[ 6] 13.00-14.00 sec 10.0 KBytes 81.9 Kbits/sec 25.029 ms 35/45 (78%)
[ 6] 14.00-15.00 sec 11.0 KBytes 90.1 Kbits/sec 17.210 ms 40/51 (78%)
[ 6] 15.00-16.00 sec 10.0 KBytes 81.9 Kbits/sec 13.892 ms 36/46 (78%)
iperf3: error — select failed: Bad file descriptor
Server listening on 5201
Client side:
Connecting to host (IP addr) port 5201
[ 5] local (IP addr) port 54260 connected to (ip addr) port 5201
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 1.00-2.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 2.00-3.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 3.00-4.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 4.00-5.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 5.00-6.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 6.00-7.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 7.00-8.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 8.00-9.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 9.00-10.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 10.00-11.00 sec 47.0 KBytes 385 Kbits/sec 47
iperf3: error — control socket has closed unexpectedly
This is on a private network that uses valid public network addresses.
Don’t know why part of the output above is in bold.
Problem only occurs with -l 256 and above. Finishes normally with -l 128
What kind of link is it?
I can run just fine using an ODROID-C2. I don’t have an RPI 3B. Amlogic S905 Quad Core Cortex-A53 (ODROID C2) vs Broadcom BCM2837B0 Quad Core Cortex-A53 (RPI). I am running Ubuntu 20.04.1 LTS.
# ./iperf36 --client 192.168.X.X --udp --bandwidth 390k -l 1024
Connecting to host 192.168.X.X, port 5201
[ 5] 0.00-1.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 1.00-2.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 2.00-3.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 3.00-4.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 4.00-5.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 5.00-6.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 6.00-7.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 7.00-8.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 8.00-9.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 9.00-10.00 sec 48.0 KBytes 393 Kbits/sec 48
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 477 KBytes 391 Kbits/sec 0.000 ms 0/477 (0%) sender
[ 5] 0.00-10.00 sec 477 KBytes 391 Kbits/sec 0.018 ms 0/477 (0%) receiver
iperf Done.
Same for IPv6 …
./iperf36 --client SERVER_IPv6_ADDRESS --udp --bandwidth 390k -l 1024
Connecting to host SERVER_IPv6_ADDRESS, port 5201
[ 5] local CLIENT_IPv6_ADDRESS port 52823 connected to SERVER_IPv6_ADDRESS port 5201
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 1.00-2.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 2.00-3.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 3.00-4.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 4.00-5.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 5.00-6.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 6.00-7.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 7.00-8.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 8.00-9.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 9.00-10.00 sec 48.0 KBytes 393 Kbits/sec 48
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 477 KBytes 391 Kbits/sec 0.000 ms 0/477 (0%) sender
[ 5] 0.00-10.00 sec 477 KBytes 391 Kbits/sec 0.019 ms 0/477 (0%) receiver
iperf Done.
Don’t know why part of the output above is in bold
A main issue is that while the client sends about 48KB/sec (390Kbits/sec), the server is receiving only 10KB/sec. It seems that the network throughput is only about 100Kbis/sec. The client probably have a send buffer of about 100KB since the first 100KB were properly received by the server.
After the client completed sending, sending of the 100KB buffered packets continue. This is why the server continue to receive data after the client completed the sending. If my calculations are correct it would take 10 seconds (after the first 12 secs) for the server to receive all buffered data.
Can you send the client and server output when -l 128 was used? In principle it should not change much.
Something happened that that caused the error, I am not sure what, but it may help to try a newer version of iperf3. Version 3.6 is quite old and there were some related fixes since.
Can you try with 3.9? I see this in the release notes for 3.7 …
«The delay for tearing down the control connection for the default timed tests has been increased, to more gracefully handle high-delay paths (#751/#859).»
May be related.
Client is an 1GbE ODROID-C2 running Ubuntu 20.04.1 LTS. Server is a 10GbE rack mount server running Ubuntu 20.04.2 LTS.
I think your link could be causing the issue. Can you provide more details on this 100Kbps link?
# ./iperf36 --client SERVER_IPv4_ADDRESS --udp --bandwidth 390k --time 12 --length 128
Connecting to host SERVER_IPv4_ADDRESS, port 5201
[ 5] local CLIENT_IPv4_ADDRESS port 58317 connected to SERVER_IPv4_ADDRESS port 5201
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 1.00-2.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 2.00-3.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 3.00-4.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 4.00-5.00 sec 47.5 KBytes 389 Kbits/sec 380
[ 5] 5.00-6.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 6.00-7.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 7.00-8.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 8.00-9.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 9.00-10.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 10.00-11.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 11.00-12.00 sec 47.5 KBytes 389 Kbits/sec 380
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-12.00 sec 571 KBytes 390 Kbits/sec 0.000 ms 0/4570 (0%) sender
[ 5] 0.00-12.03 sec 571 KBytes 389 Kbits/sec 0.042 ms 0/4570 (0%) receiver
# ./iperf36 --server
Server listening on 5201
-----------------------------------------------------------
Accepted connection from CLIENT_IPv4_ADDRESS, port 38364
[ 5] local SERVER_IPv4_ADDRESS port 5201 connected to CLIENT_IPv4_ADDRESS port 58317
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 46.0 KBytes 377 Kbits/sec 0.028 ms 0/368 (0%)
[ 5] 1.00-2.00 sec 47.6 KBytes 390 Kbits/sec 0.038 ms 0/381 (0%)
[ 5] 2.00-3.00 sec 47.6 KBytes 390 Kbits/sec 0.043 ms 0/381 (0%)
[ 5] 3.00-4.00 sec 47.6 KBytes 390 Kbits/sec 0.038 ms 0/381 (0%)
[ 5] 4.00-5.00 sec 47.6 KBytes 390 Kbits/sec 0.043 ms 0/381 (0%)
[ 5] 5.00-6.00 sec 47.6 KBytes 390 Kbits/sec 0.037 ms 0/381 (0%)
[ 5] 6.00-7.00 sec 47.6 KBytes 390 Kbits/sec 0.019 ms 0/381 (0%)
[ 5] 7.00-8.00 sec 47.5 KBytes 389 Kbits/sec 0.035 ms 0/380 (0%)
[ 5] 8.00-9.00 sec 47.6 KBytes 390 Kbits/sec 0.045 ms 0/381 (0%)
[ 5] 9.00-10.00 sec 47.6 KBytes 390 Kbits/sec 0.034 ms 0/381 (0%)
[ 5] 10.00-11.00 sec 47.6 KBytes 390 Kbits/sec 0.036 ms 0/381 (0%)
[ 5] 11.00-12.00 sec 47.6 KBytes 390 Kbits/sec 0.046 ms 0/381 (0%)
[ 5] 12.00-12.03 sec 1.50 KBytes 361 Kbits/sec 0.042 ms 0/12 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-12.03 sec 571 KBytes 389 Kbits/sec 0.042 ms 0/4570 (0%) receiver
# ./iperf36 --client SERVER_IPv6_ADDRESS --udp --bandwidth 390k --time 12 --length 128
Connecting to host SERVER_IPv6_ADDRESS, port 5201
[ 5] local CLIENT_IPv6_ADDRESS port 60491 connected to SERVER_IPv6_ADDRESS port 5201
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 1.00-2.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 2.00-3.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 3.00-4.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 4.00-5.00 sec 47.5 KBytes 389 Kbits/sec 380
[ 5] 5.00-6.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 6.00-7.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 7.00-8.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 8.00-9.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 9.00-10.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 10.00-11.00 sec 47.6 KBytes 390 Kbits/sec 381
[ 5] 11.00-12.00 sec 47.5 KBytes 389 Kbits/sec 380
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-12.00 sec 571 KBytes 390 Kbits/sec 0.000 ms 0/4570 (0%) sender
[ 5] 0.00-12.04 sec 571 KBytes 389 Kbits/sec 0.014 ms 0/4570 (0%) receiver
# ./iperf36 --server
Server listening on 5201
-----------------------------------------------------------
Accepted connection from CLIENT_IPv6_ADDRESS, port 55168
[ 5] local SERVER_IPv6_ADDRESS port 5201 connected to CLIENT_IPv6_ADDRESS port 60491
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 45.9 KBytes 376 Kbits/sec 0.030 ms 0/367 (0%)
[ 5] 1.00-2.00 sec 47.6 KBytes 390 Kbits/sec 0.015 ms 0/381 (0%)
[ 5] 2.00-3.00 sec 47.6 KBytes 390 Kbits/sec 0.022 ms 0/381 (0%)
[ 5] 3.00-4.00 sec 47.6 KBytes 390 Kbits/sec 0.031 ms 0/381 (0%)
[ 5] 4.00-5.00 sec 47.6 KBytes 390 Kbits/sec 0.024 ms 0/381 (0%)
[ 5] 5.00-6.00 sec 47.6 KBytes 390 Kbits/sec 0.036 ms 0/381 (0%)
[ 5] 6.00-7.00 sec 47.5 KBytes 389 Kbits/sec 0.030 ms 0/380 (0%)
[ 5] 7.00-8.00 sec 47.6 KBytes 390 Kbits/sec 0.025 ms 0/381 (0%)
[ 5] 8.00-9.00 sec 47.6 KBytes 390 Kbits/sec 0.034 ms 0/381 (0%)
[ 5] 9.00-10.00 sec 47.6 KBytes 390 Kbits/sec 0.014 ms 0/381 (0%)
[ 5] 10.00-11.00 sec 47.6 KBytes 390 Kbits/sec 0.030 ms 0/381 (0%)
[ 5] 11.00-12.00 sec 47.6 KBytes 390 Kbits/sec 0.030 ms 0/381 (0%)
[ 5] 12.00-12.04 sec 1.62 KBytes 366 Kbits/sec 0.014 ms 0/13 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-12.04 sec 571 KBytes 389 Kbits/sec 0.014 ms 0/4570 (0%) receiver
Содержание
- Regression in 3.1/3.2: iperf server fails after
- Regression in 3.1/3.2: iperf server fails after
- Comments
- Bug Report
- server dies on very long test #200
- Comments
- v3.5 error — control socket has closed unexpectedly #751
- Comments
- Bug Report
- SCTP test produces «iperf3: error — control socket has closed unexpectedly» #620
- Comments
- Context
- Bug Report
- Enhancement Request
- iperf3: error — control socket has closed unexpectedly v.3.6 #1149
- Comments
- Bug Report
- Server side:
- Server listening on 5201
- Accepted connection from (IP addr), port 56688 [ 6] local (ip addr) port 5201 connected to (ip addr) port 54260 [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 6] 0.00-1.00 sec 9.00 KBytes 73.7 Kbits/sec 37.226 ms 0/9 (0%) [ 6] 1.00-2.00 sec 10.0 KBytes 81.9 Kbits/sec 55.150 ms 0/10 (0%) [ 6] 2.00-3.00 sec 11.0 KBytes 90.1 Kbits/sec 65.500 ms 0/11 (0%) [ 6] 3.00-4.00 sec 9.00 KBytes 73.7 Kbits/sec 72.033 ms 0/9 (0%) [ 6] 4.00-5.00 sec 11.0 KBytes 90.1 Kbits/sec 73.836 ms 0/11 (0%) [ 6] 5.00-6.00 sec 10.0 KBytes 81.9 Kbits/sec 74.872 ms 0/10 (0%) [ 6] 6.00-7.00 sec 11.0 KBytes 90.1 Kbits/sec 74.872 ms 0/11 (0%) [ 6] 7.00-8.00 sec 10.0 KBytes 81.9 Kbits/sec 75.292 ms 0/10 (0%) [ 6] 8.00-9.00 sec 10.0 KBytes 81.9 Kbits/sec 75.182 ms 0/10 (0%) [ 6] 9.00-10.00 sec 11.0 KBytes 90.1 Kbits/sec 75.403 ms 0/11 (0%) [ 6] 10.00-11.00 sec 10.0 KBytes 81.9 Kbits/sec 89.071 ms 26/36 (72%) [ 6] 11.00-12.00 sec 10.0 KBytes 81.9 Kbits/sec 53.561 ms 38/48 (79%) [ 6] 12.00-13.00 sec 10.0 KBytes 81.9 Kbits/sec 36.800 ms 36/46 (78%) [ 6] 13.00-14.00 sec 10.0 KBytes 81.9 Kbits/sec 25.029 ms 35/45 (78%) [ 6] 14.00-15.00 sec 11.0 KBytes 90.1 Kbits/sec 17.210 ms 40/51 (78%) [ 6] 15.00-16.00 sec 10.0 KBytes 81.9 Kbits/sec 13.892 ms 36/46 (78%) iperf3: error — select failed: Bad file descriptor
- Server listening on 5201
Regression in 3.1/3.2: iperf server fails after
15 seconds with «Bad file descriptor» #645
Regression in 3.1/3.2: iperf server fails after
Version of iperf3: 3.2, 3.2rc1 and latest commit on master (cd5d89d).
Hardware: MBP (Arch Linux, 2015), MBP (MacOS, 2017), VPS (Arch Linux).
Operating system (and distribution, if any): Arch Linux, MacOS.
Bug Report
Expected behaviour: setting -t 0 , I expect iperf to run indefinitely.
Actual behaviour: the connection is dropped after about 15 seconds.
Steps to reproduce: run iperf 3.2 as a server with iperf3 -s and a client (any version) with iperf3 -c $HOST -t 0 (can be entirely local, with HOST=localhost ); after
15 seconds, the client will report
and the server will report
(this can also be reproduced in reverse mode).
The text was updated successfully, but these errors were encountered:
This is a weird problem on a couple levels. I see it on the tip of the master and 3.1-STABLE branches, but not the tip of 3.0-STABLE (tested variously on macOS Sierra and CentOS 7). The test shouldn’t stop like that, so there’s definitely a bug there.
I’m not sure what the correct behavior is, on the other hand. Your «run indefinitely» makes some sense, but on the other hand it’d also make sense to not allow this case at all, so that a client can’t conduct a DoS attack against a server (a counter-argument is that a server should be able to specify a maximum test length).
I’m going to dig a little more into this and try to understand what’s going on. So far I have the following interesting result:
Server (iperf-master), client (iperf-master): Failure
Server (iperf-3.0-stable), client (iperf-master): OK
Server (iperf-master), client (iperf-3.0-stable): Failure
Server (iperf-3.0-stable), client (iperf-3.0-stable): OK
This so far implies that there’s a problem on the server side with newer iperf.
BTW thanks for a well-formulated bug report!
Источник
server dies on very long test #200
iperf3 -c 10.10.0.2 -t3000
I get this error after about 10 minutes:
iperf3: error — control socket has closed unexpectedly
perhaps some counter is wrapping?
The text was updated successfully, but these errors were encountered:
Same issue as #191 perhaps?
It would be great to fix this before the perfSONAR 3.4 release (which will
be in about 3 weeks).
On Mon, Aug 25, 2014 at 7:21 AM, Bruce A. Mah notifications@github.com
wrote:
—
Reply to this email directly or view it on GitHub
#200 (comment).
Brian Tierney, http://www.es.net/tierney
Energy Sciences Network (ESnet), Berkeley National Lab
http://fasterdata.es.net
Can you give more info on the test environment? Any way to tell how much data was sent? Which side (client or server) gave the error message?
I’m trying to reproduce this in a controlled environment but I’m not sure what to expect. (i.e. is this on the ESnet 100G testbed?) Ideally I’d like to be able to reproduce this problem on a single host over a loopback connection.
I was using a switch buffer test environment that LBLnet put together. I
can add accounts for you there if you wish.
But I was running this command:
iperf3 -c sr-test-4 -u -b2G -t3000 -4 -l8972
and after about 10 min the client gave the error.
localhost should be able to reproduce the problem.
On Mon, Aug 25, 2014 at 9:36 AM, Bruce A. Mah notifications@github.com
wrote:
Can you give more info on the test environment? Any way to tell how much
data was sent? Which side (client or server) gave the error message?
I’m trying to reproduce this in a controlled environment but I’m not sure
what to expect. (i.e. is this on the ESnet 100G testbed?) Ideally I’d like
to be able to reproduce this problem on a single host over a loopback
connection.
—
Reply to this email directly or view it on GitHub
#200 (comment).
Brian Tierney, http://www.es.net/tierney
Energy Sciences Network (ESnet), Berkeley National Lab
http://fasterdata.es.net
I’ve been unable to reproduce this problem so far. After some thought and discussion it doesn’t seem likely that this is the same counter wraparound issue from #191, because it’s not possible (given the parameters) to send the 2 billion UDP packets needed to cause the overflow of a 32-bit signed value.
I created a bmah account on sr-test-1.lbl.gov and sr-test-2.lbl.gov, and
added your ssh key.
Those are the hosts where I saw the problem
On Tue, Aug 26, 2014 at 12:59 PM, Bruce A. Mah notifications@github.com
wrote:
I’ve been unable to reproduce this problem so far. After some thought and
discussion it doesn’t seem likely that this is the same counter wraparound
issue from #191 #191, because it’s
not possible (given the parameters) to send the 2 billion UDP packets
needed to cause the overflow of a 32-bit signed value.
—
Reply to this email directly or view it on GitHub
#200 (comment).
Источник
v3.5 error — control socket has closed unexpectedly #751
I use iperf3 as part of an automated test environment using ansible. We run various tests over different network conditions.
Version of iperf3: iperf 3.5 (cJSON 1.5.2)
Hardware: lxd containers
Operating system (and distribution, if any): Ubuntu 16.04.3
Other relevant information (for example, non-default compilers,
libraries, cross-compiling, etc.):
Bug Report
Randomly I get the following error in the json report:
«error»: «error — control socket has closed unexpectedly»
When it happens, it seems to always occur after the second last interval, e.g. using -t 60 the json report will contain the interval report for interval 58 — 59, and then ends with the error. So the 60th interval is never reported on.
The server is run with iperf3 -s -D
The client is run with iperf3 -c 1.1.1.1 -t 60 -P 8 -J
Unfortunately I do not have access to the output of the server.
Note — I never experienced this when I was still using iper3 version that ships with Ubuntu, think it was v3.1.x ?? This might only be coincidence or a regression.
Expected Behavior
Expects to complete the test without any errors.
Actual Behavior
Test ends with «error»: «error — control socket has closed unexpectedly»
Steps to Reproduce
It occurs randomly, so not easy to reproduce.
The text was updated successfully, but these errors were encountered:
Источник
SCTP test produces «iperf3: error — control socket has closed unexpectedly» #620
NOTE: The iperf3 issue tracker is for registering bugs, enhancement
requests, or submissions of code. It is not a means for asking
questions about building or using iperf3. Those are best directed
towards the iperf3 mailing list at iperf-dev@google-groups.com or
question sites such as Stack Overflow
(http://www.stackoverflow.com/). A list of frequently-asked questions
regarding iperf3 can be found at http://software.es.net/iperf/faq.html.
Context
Version of iperf3:
iperf 3.1.7
Hardware:
VM for linux 64bit kernel version 4.4.70
Operating system (and distribution, if any):
LFS
Please note: iperf3 is supported on Linux, FreeBSD, and macOS.
Support may be provided on a best-effort basis to other UNIX-like
platforms. We cannot provide support for building and/or running
iperf3 on Windows, iOS, or Android.
Other relevant information (for example, non-default compilers,
libraries, cross-compiling, etc.):
Please fill out one of the «Bug Report» or «Enhancement Request»
sections, as appropriate.
Bug Report
- Expected Behavior
iperf3 can send sctp traffic successfully. - Actual Behavior
- Steps to Reproduce
create sever in one VM with ./iperf3 -s 192.168.1.7
start send traffic on another VM with ./iperf3 -c 192.168.1.7 —sctp - Possible Solution
N/A
Please submit patches or code changes as a pull request.
Enhancement Request
If submitting a proposed implementation of an enhancement request,
please use the pull request mechanism.
EDIT: Do some MarkDown for legibility. —bmah
The text was updated successfully, but these errors were encountered:
I download iperf3.2 and try again, same issue still existed.
EDIT: MarkDown for legibility. —bmah
I agree it shouldn’t do that. Can you show server-side output as well?
No feedback after 2.5 months, and I can’t reproduce this with tip of master branch code running on CentOS 7. Closing for now, I can reopen this issue later if necessary.
hi. I am testing this on 3G with udp — packet length 1448B for a 30 second interval. using iperf 3.5 unable to go beyond 28 seconds.
pushing 3 Mbps upstream towards a server
Server output comes as » iperf3 : error — select failed : Bad file descripter.
I have figured out that iperf 3.5 fixes this. I actually had iperf 3.4 there. issue is resolved in iperf 3.5
How to install iperf 3.5 in linux?
I get this issue on debian buster x64 with the distribution’s version of iperf3.
problem persists with iperf3 3.7 built from source
Seeing this intermittently on:
I’m affected by this as well, please reopen this issue ticket if there isn’t a more suitable one. @bmah888
The issue happens immediately, it doesn’t seem like there is any data transferred at all.
I’m still not able to reproduce this issue, but I believe y’all. Reopening.
Is it possible that those of you with this problem are having some firewall-related issue? That’s a fairly common issue with iperf3 in general.
Thanks for your reply. This is happening inside my local network at home where I have no firewall restrictions at all in place. Is there any way I could give you more information like by adding a very verbose debugging mode? Also, is it possible to query the version of the distant iperf3 server over the wire? I’m experimenting with two Ubuntu servers running iperf3 and there the issue happens only rarely. However, when using my router as server (which has an iperf3 server builtin as well, according to the docs), I always get this message very early on or even before the transfer starts at all.
Thanks for your reply. This is happening inside my local network at home where I have no firewall restrictions at all in place. Is there any way I could give you more information like by adding a very verbose debugging mode? Also, is it possible to query the version of the distant iperf3 server over the wire? I’m experimenting with two Ubuntu servers running iperf3 and there the issue happens only rarely. However, when using my router as server (which has an iperf3 server builtin as well, according to the docs), I always get this message very early on or even before the transfer starts at all.
Well, «firewall» could also mean a host-based firewall like iptables or firewalld. But it looks like if you’re doing this between two Ubuntu servers and you get the issue some of the time but not always, that doesn’t really indicate a firewall/packet filtering issue.
There’s a —debug flag in iperf3. If you can run that on the iperf3 server side that might help us a bit. I wonder if having a packet trace (such as from tcpdump) might be helpful.
The server knows the version of the client but not vice versa.
This is a bit of a mystery to me because I have absolutely no idea what’s going wrong.
Er, wait a second @DL6ER . Are you trying this with SCTP (as per the original bug report) or is this with TCP (which is what your command lines indicate)?
Источник
iperf3: error — control socket has closed unexpectedly v.3.6 #1149
Version of iperf3: 3.6
Hardware: raspberry pi 3 b
Operating system (and distribution, if any): Debian Linux (default pi OS)
Bug Report
I’m trying to run iperf3.6 over a 100kbps link to measuer the actual throughput. I’m running it on two pis.
On the server, I run «iperf3 -s —logfile /dev/tty
On the client, I run «iperf3 -c (IPv6 address of peer) -u -b 390k -l 1024 -t 12»
Expected Behavior I see 12 reports, one second apart, then the summary
Actual Behavior, on the client I see 11 reports, one second apart, then a pause, then iperf3: error — control socket has closed unexpectedly. On the server I see 16 reports one second apart then iperf3: error — select failed: Bad file descriptor
Steps to Reproduce: given above
Server side:
Server listening on 5201
Accepted connection from (IP addr), port 56688
[ 6] local (ip addr) port 5201 connected to (ip addr) port 54260
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 6] 0.00-1.00 sec 9.00 KBytes 73.7 Kbits/sec 37.226 ms 0/9 (0%)
[ 6] 1.00-2.00 sec 10.0 KBytes 81.9 Kbits/sec 55.150 ms 0/10 (0%)
[ 6] 2.00-3.00 sec 11.0 KBytes 90.1 Kbits/sec 65.500 ms 0/11 (0%)
[ 6] 3.00-4.00 sec 9.00 KBytes 73.7 Kbits/sec 72.033 ms 0/9 (0%)
[ 6] 4.00-5.00 sec 11.0 KBytes 90.1 Kbits/sec 73.836 ms 0/11 (0%)
[ 6] 5.00-6.00 sec 10.0 KBytes 81.9 Kbits/sec 74.872 ms 0/10 (0%)
[ 6] 6.00-7.00 sec 11.0 KBytes 90.1 Kbits/sec 74.872 ms 0/11 (0%)
[ 6] 7.00-8.00 sec 10.0 KBytes 81.9 Kbits/sec 75.292 ms 0/10 (0%)
[ 6] 8.00-9.00 sec 10.0 KBytes 81.9 Kbits/sec 75.182 ms 0/10 (0%)
[ 6] 9.00-10.00 sec 11.0 KBytes 90.1 Kbits/sec 75.403 ms 0/11 (0%)
[ 6] 10.00-11.00 sec 10.0 KBytes 81.9 Kbits/sec 89.071 ms 26/36 (72%)
[ 6] 11.00-12.00 sec 10.0 KBytes 81.9 Kbits/sec 53.561 ms 38/48 (79%)
[ 6] 12.00-13.00 sec 10.0 KBytes 81.9 Kbits/sec 36.800 ms 36/46 (78%)
[ 6] 13.00-14.00 sec 10.0 KBytes 81.9 Kbits/sec 25.029 ms 35/45 (78%)
[ 6] 14.00-15.00 sec 11.0 KBytes 90.1 Kbits/sec 17.210 ms 40/51 (78%)
[ 6] 15.00-16.00 sec 10.0 KBytes 81.9 Kbits/sec 13.892 ms 36/46 (78%)
iperf3: error — select failed: Bad file descriptor
Server listening on 5201
Client side:
Connecting to host (IP addr) port 5201
[ 5] local (IP addr) port 54260 connected to (ip addr) port 5201
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 1.00-2.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 2.00-3.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 3.00-4.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 4.00-5.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 5.00-6.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 6.00-7.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 7.00-8.00 sec 47.0 KBytes 385 Kbits/sec 47
[ 5] 8.00-9.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 9.00-10.00 sec 48.0 KBytes 393 Kbits/sec 48
[ 5] 10.00-11.00 sec 47.0 KBytes 385 Kbits/sec 47
iperf3: error — control socket has closed unexpectedly
The text was updated successfully, but these errors were encountered:
Источник
Здравствуйте.
Вкратце хотим добиться:
Максимально возможное количество соединений на edge-сервере при раздаче потоков, лимит которых будет упираться в лимит CPU/и или памяти.
Edge имеют одно dns имя. Все edge за classic load balancer+auto scaling group. Соответственно потоки клиенты будут получать по dns имени балансировщика. Тесты проводились с одним edge.
Окружение:
клиент (тестирующий) — c5.4xlarge (CPU-16 RAM-32):
-Xmx16G
-Xms16G
edge — c5.4xlarge (CPU-16 RAM-32):
-Xmx16G
-Xms16G
origin — с5.2xlarge (CPU-8 RAM-16):
-Xmx8G
-Xms8G
Везде:
Версия:
5.2.838 (при обновлениях на более новые появляется проблема подключения по ws. По возможности хотел бы вернуться к этому позже. пока проверяю на той, что работает)
сборщик мусора — ZGC
ice_tcp_transport=true
net.ipv4.ip_local_port_range = 54001 65000
media_port_from=31008
media_port_to=47000
wcs_agent_port_from=49001
wcs_agent_port_to=53999
WCS_FD_LIMIT=100000
webrtc_cc2_twcc=false
Есть другие настройки, скорее всего ненужные.
iptables -S:
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A INPUT -s 127.0.0.0/8 -d 127.0.0.0/8 -i lo -j ACCEPT
-A INPUT -m state —state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -m state —state INVALID -j DROP
-A INPUT -p tcp -m tcp ! —tcp-flags FIN,SYN,RST,ACK SYN -m state —state NEW -j DROP
-A INPUT -p icmp -m icmp —icmp-type 0 -j ACCEPT
-A INPUT -p icmp -m icmp —icmp-type 3 -j ACCEPT
-A INPUT -p icmp -m icmp —icmp-type 11 -j ACCEPT
-A INPUT -p icmp -m icmp —icmp-type 8 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 22 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 443 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 4431 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 9100 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 8888 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 8443 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 1935 -j ACCEPT
-A INPUT -p udp -m udp —dport 1935 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 554 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 8080 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 8081 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 8084 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 8082 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 8445 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 8444 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 9091 -j ACCEPT
-A INPUT -p tcp -m tcp —dport 20000:53999 -j ACCEPT
-A INPUT -p udp -m udp —dport 20000:53999 -j ACCEPT
-A INPUT -j DROP
-A FORWARD -m state —state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -m state —state INVALID -j DROP
-A FORWARD -j DROP
-A OUTPUT -s 127.0.0.0/8 -d 127.0.0.0/8 -o lo -j ACCEPT
-A OUTPUT -o eth0 -j ACCEPT
-A OUTPUT -m state —state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp -m tcp ! —tcp-flags FIN,SYN,RST,ACK SYN -m state —state NEW -j DROP
(некоторые порты от прежних настроек, будем убирать)
/etc/sysctl.conf сожержит:
fs.file-max = 100000
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.ipv4.tcp_max_syn_backlog = 10240
net.ipv4.tcp_congestion_control=htcp
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_fin_timeout = 15
net.core.somaxconn = 1024
net.core.netdev_max_backlog = 65536
net.core.optmem_max = 25165824
net.ipv4.tcp_mem = 65536 131072 262144
net.ipv4.udp_mem = 65536 131072 262144
net.core.rmem_default = 25165824
net.core.rmem_max = 25165824
net.ipv4.tcp_rmem = 20480 12582912 25165824
net.ipv4.udp_rmem_min = 16384
net.core.wmem_default = 25165824
net.core.wmem_max = 25165824
net.ipv4.tcp_wmem = 20480 12582912 25165824
net.ipv4.udp_wmem_min = 16384
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_tw_reuse = 1
(по умолчанию не было, выставляли исходя из рекомендаций по настройке сети., ни на что не повлияло)
Проверка канала напрямую по внешнему ip (от тестирующего к edge):
iperf3 -c xxxxxx -p 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 579 MBytes 4.85 Gbits/sec 48 718 KBytes
[ 4] 1.00-2.00 sec 539 MBytes 4.52 Gbits/sec 103 870 KBytes
[ 4] 2.00-3.00 sec 566 MBytes 4.75 Gbits/sec 116 981 KBytes
[ 4] 3.00-4.00 sec 538 MBytes 4.51 Gbits/sec 153 397 KBytes
^C[ 4] 4.00-4.62 sec 349 MBytes 4.72 Gbits/sec 0 922 KBytes
— — — — — — — — — — — — — — — — — — — — — — — — —
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-4.62 sec 2.51 GBytes 4.67 Gbits/sec 420 sender
[ 4] 0.00-4.62 sec 0.00 Bytes 0.00 bits/sec receiver
iperf3 -c xxxxxx -p 5201 -R
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 572 MBytes 4.79 Gbits/sec
[ 4] 1.00-2.00 sec 571 MBytes 4.79 Gbits/sec
[ 4] 2.00-3.00 sec 571 MBytes 4.79 Gbits/sec
[ 4] 3.00-4.00 sec 571 MBytes 4.79 Gbits/sec
^C[ 4] 4.00-4.61 sec 346 MBytes 4.78 Gbits/sec
— — — — — — — — — — — — — — — — — — — — — — — — —
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-4.61 sec 0.00 Bytes 0.00 bits/sec sender
[ 4] 0.00-4.61 sec 2.57 GBytes 4.79 Gbits/sec receiver
Edge находятся в AWS Classic load balancer+ Autoscaling group (настраивал согласно статье https://flashphoner.com/cdn-s-balan…rovaniem-na-baze-amazon-web-services/?lang=ru)
Тестирующий, origin, edge находятся в одной placement group. тип — cluster (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/placement-groups.html)
Проверка канала по dns имени балансировщика edge (от тестирующего к edge):
iperf3 -c xxxxxx -p 5201:
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 371 MBytes 3.11 Gbits/sec 95 235 KBytes
[ 4] 1.00-2.00 sec 471 MBytes 3.95 Gbits/sec 115 505 KBytes
[ 4] 2.00-3.00 sec 516 MBytes 4.33 Gbits/sec 229 472 KBytes
[ 4] 3.00-4.00 sec 365 MBytes 3.06 Gbits/sec 274 404 KBytes
[ 4] 4.00-5.00 sec 455 MBytes 3.82 Gbits/sec 106 359 KBytes
[ 4] 5.00-6.00 sec 514 MBytes 4.31 Gbits/sec 148 626 KBytes
[ 4] 6.00-7.00 sec 469 MBytes 3.93 Gbits/sec 182 293 KBytes
[ 4] 7.00-8.00 sec 436 MBytes 3.66 Gbits/sec 266 221 KBytes
[ 4] 8.00-9.00 sec 402 MBytes 3.38 Gbits/sec 144 444 KBytes
[ 4] 9.00-10.00 sec 440 MBytes 3.69 Gbits/sec 157 614 KBytes
— — — — — — — — — — — — — — — — — — — — — — — — —
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 4.34 GBytes 3.72 Gbits/sec 1716 sender
[ 4] 0.00-10.00 sec 3.98 GBytes 3.42 Gbits/sec receiver
iperf3 -c xxxxxx -p 5201 -R:
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 100 KBytes 822 Kbits/sec
[ 4] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec
iperf3: error — control socket has closed unexpectedly
На тестирующей машине выставляется:
wcs_activity_timer_timeout=86400000
Тестировалось с помощью «pull streams» в количестве 1000 соединений. Указывался внешний IP непосредственно edge.
Исходный поток, опубликованный на origin имеет битрейт примерно от 1.5 до 2 мбит
При тестировании наблюдается следующее:
при достижении примерно 300 соединений (примерно 500 мбит/с) перестаёт расти скорость, так и остаётся — максимум 500 мбит/с. Количество соединений при этом увеличивается до 1000, как и было указано при запуске теста. При этом изображение начинает опаздывать, затем тормозить, если соединений больше, чем примерно 300.
Скорость замерялась утилитой nload.
Значительных нагрузок на CPU, память не наблюдается как на edge, так и на тестирующей машине.
Ставили много экспериментов, всё упирается в скорость раздачи в 500 мбит/с. Просьба помочь с настройками, дать рекомендации по настройке инфраструктуры, если допустили концептуальные ошибки.
Те же результаты наблюдались с типом машин edge, тестирующей- с5.2xlarge (CPU-8 RAM-16 -Xmx8G
-Xms8G).
Заранее спасибо.
A few days ago, I reviewed a USB 3.0 to 2.5 Gbps Ethernet adapter based on Realtek RTL8156B chip in Ubuntu 20.04, and let’s say the reliability and performance were underwhelming. I got some recommendations like changing cables, the MTU size, etc…
Playing around with cables did no help, but one comment mentioned the cdc_ncm driver could be the issue, followed by another saying that updating to Linux kernel 5.14 should install the correct r8152 driver… So I just did that:
|
sudo apt install linux—oem—20.04d |
This upgraded Linux 5.13 (shipped with Ubuntu 20.04 + HWE) to Linux 5.14, but still no luck as the system kept using the cdc_ncm driver with a half-duplex link:
|
jaufranc@cnx—laptop—4:~$ inxi —n Network: Device—1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8169 IF: enp2s0f1 state: down mac: 98:28:a6:0f:06:07 Device—2: Qualcomm Atheros QCA9377 802.11ac Wireless Network Adapter driver: ath10k_pci IF: wlp3s0 state: up mac: 70:c9:4e:b7:84:77 Device—3: Realtek USB 10/100/1G/2.5G LAN type: USB driver: cdc_ncm IF: enx1cbfced40321 state: up speed: 2500 Mbps duplex: half mac: 1c:bf:ce:d4:03:21 jaufranc@cnx—laptop—4:~$ uname —a Linux cnx—laptop—4 5.14.0—1022—oem #24-Ubuntu SMP Mon Jan 31 16:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
But then I thought I may have to use udev rules to prevent loading the cdc_ncm driver, and there’s indeed 50-usb-realtek-net.rules in r8152 driver to do just that. So I copied the file in /etc/udev/rules.d/ folder. Since I did not want to reboot, I unloaded the modules I did not need, and restart udev to try it out:
|
sudo rmmod cdc_mbim sudo rmmod cdc_ncm service udev restart |
Let’s see…
|
sudo inxi —n Network: Device—1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8169 IF: enp2s0f1 state: down mac: 98:28:a6:0f:06:07 Device—2: Qualcomm Atheros QCA9377 802.11ac Wireless Network Adapter driver: ath10k_pci IF: wlp3s0 state: up mac: 70:c9:4e:b7:84:77 Device—3: Realtek USB 10/100/1G/2.5G LAN type: USB driver: r8152 IF: enx1cbfced40321 state: up speed: 2500 Mbps duplex: full mac: 1c:bf:ce:d4:03:21 |
Great! It’s now using r8152 driver and we’ve got a full-duplex connection.
Let’s go through all our tests again to compare the results.
iperf2
upload
|
iperf —t 60 —c 192.168.31.12 —————————————————————————————— Client connecting to 192.168.31.12, TCP port 5001 TCP window size: 1.40 MByte (default) —————————————————————————————— [ 3] local 192.168.31.166 port 41266 connected with 192.168.31.12 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0—60.0 sec 16.4 GBytes 2.35 Gbits/sec |
download:
|
iperf —t 60 —c 192.168.31.166 —————————————————————————————— Client connecting to 192.168.31.166, TCP port 5001 TCP window size: 901 KByte (default) —————————————————————————————— [ 3] local 192.168.31.12 port 37188 connected with 192.168.31.166 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0—60.0 sec 10.8 GBytes 1.55 Gbits/sec |
There’s an improvement to the download speed (was 600 Mbps with cdc_ncm driver), but still not quite close to 2.3 Gbps.
Let’s try full-duplex for fun:
|
Client connecting to 192.168.31.166, TCP port 5001 TCP window size: 799 KByte (default) —————————————————————————————— [ 4] local 192.168.31.12 port 5001 connected with 192.168.31.166 port 41290 [ 6] local 192.168.31.12 port 37194 connected with 192.168.31.166 port 5001 [ 6] 0.0—60.1 sec 8.06 GBytes 1.15 Gbits/sec [ 4] 0.0—60.4 sec 16.3 GBytes 2.32 Gbits/sec |
That’s actually not too bad.
iperf3
upload:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
iperf3 —t 60 —c 192.168.31.12 —i 5 Connecting to host 192.168.31.12, port 5201 [ 5] local 192.168.31.166 port 32848 connected to 192.168.31.12 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00—5.00 sec 1.37 GBytes 2.36 Gbits/sec 0 847 KBytes [ 5] 5.00—10.00 sec 1.37 GBytes 2.35 Gbits/sec 0 889 KBytes [ 5] 10.00—15.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.14 MBytes [ 5] 15.00—20.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.14 MBytes [ 5] 20.00—25.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.14 MBytes [ 5] 25.00—30.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.14 MBytes [ 5] 30.00—35.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.73 MBytes [ 5] 35.00—40.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.73 MBytes [ 5] 40.00—45.00 sec 1.37 GBytes 2.35 Gbits/sec 0 3.92 MBytes [ 5] 45.00—50.00 sec 1.37 GBytes 2.35 Gbits/sec 0 3.92 MBytes [ 5] 50.00—55.00 sec 1.37 GBytes 2.35 Gbits/sec 0 3.92 MBytes [ 5] 55.00—60.00 sec 1.37 GBytes 2.35 Gbits/sec 0 3.92 MBytes — — — — — — — — — — — — — — — — — — — — — — — — — [ ID] Interval Transfer Bitrate Retr [ 5] 0.00—60.00 sec 16.4 GBytes 2.35 Gbits/sec 0 sender [ 5] 0.00—60.05 sec 16.4 GBytes 2.35 Gbits/sec receiver iperf Done. |
download:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
iperf3 —t 60 —c 192.168.31.166 —i 5 Connecting to host 192.168.31.166, port 5201 [ 5] local 192.168.31.12 port 53112 connected to 192.168.31.166 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00—5.00 sec 802 MBytes 1.35 Gbits/sec 641 222 KBytes [ 5] 5.00—10.00 sec 856 MBytes 1.44 Gbits/sec 618 83.4 KBytes [ 5] 10.00—15.00 sec 852 MBytes 1.43 Gbits/sec 583 87.7 KBytes [ 5] 15.00—20.00 sec 843 MBytes 1.41 Gbits/sec 592 168 KBytes [ 5] 20.00—25.00 sec 831 MBytes 1.39 Gbits/sec 642 91.9 KBytes [ 5] 25.00—30.00 sec 810 MBytes 1.36 Gbits/sec 666 97.6 KBytes [ 5] 30.00—35.00 sec 831 MBytes 1.39 Gbits/sec 590 123 KBytes [ 5] 35.00—40.00 sec 827 MBytes 1.39 Gbits/sec 652 298 KBytes [ 5] 40.00—45.00 sec 843 MBytes 1.41 Gbits/sec 605 93.3 KBytes [ 5] 45.00—50.00 sec 844 MBytes 1.42 Gbits/sec 635 96.2 KBytes [ 5] 50.00—55.00 sec 862 MBytes 1.45 Gbits/sec 565 84.8 KBytes [ 5] 55.00—60.00 sec 858 MBytes 1.44 Gbits/sec 583 82.0 KBytes — — — — — — — — — — — — — — — — — — — — — — — — — [ ID] Interval Transfer Bitrate Retr [ 5] 0.00—60.00 sec 9.82 GBytes 1.41 Gbits/sec 7372 sender [ 5] 0.00—60.00 sec 9.82 GBytes 1.41 Gbits/sec receiver iperf Done. |
About the same as iperf2.
iperf3 did not have support for full-duplex for many years, but version 3.7 reintroduced the feature. So let’s give it a go:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
iperf3 —t 60 —c 192.168.31.12 —bidir —i 5 Connecting to host 192.168.31.12, port 5201 [ 5] local 192.168.31.166 port 32858 connected to 192.168.31.12 port 5201 [ 7] local 192.168.31.166 port 32860 connected to 192.168.31.12 port 5201 [ ID][Role] Interval Transfer Bitrate Retr Cwnd [ 5][TX—C] 0.00—5.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.41 MBytes [ 7][RX—C] 0.00—5.00 sec 643 MBytes 1.08 Gbits/sec [ 5][TX—C] 5.00—10.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.48 MBytes [ 7][RX—C] 5.00—10.00 sec 673 MBytes 1.13 Gbits/sec [ 5][TX—C] 10.00—15.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.78 MBytes [ 7][RX—C] 10.00—15.00 sec 690 MBytes 1.16 Gbits/sec [ 5][TX—C] 15.00—20.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.78 MBytes [ 7][RX—C] 15.00—20.00 sec 695 MBytes 1.17 Gbits/sec [ 5][TX—C] 20.00—25.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.78 MBytes [ 7][RX—C] 20.00—25.00 sec 703 MBytes 1.18 Gbits/sec [ 5][TX—C] 25.00—30.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.78 MBytes [ 7][RX—C] 25.00—30.00 sec 704 MBytes 1.18 Gbits/sec [ 5][TX—C] 30.00—35.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.78 MBytes [ 7][RX—C] 30.00—35.00 sec 711 MBytes 1.19 Gbits/sec [ 5][TX—C] 35.00—40.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.78 MBytes [ 7][RX—C] 35.00—40.00 sec 697 MBytes 1.17 Gbits/sec [ 5][TX—C] 40.00—45.00 sec 28.8 MBytes 48.2 Mbits/sec 4 1.41 KBytes [ 7][RX—C] 40.00—45.00 sec 15.0 MBytes 25.2 Mbits/sec [ 5][TX—C] 45.00—50.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes [ 7][RX—C] 45.00—50.00 sec 0.00 Bytes 0.00 bits/sec [ 5][TX—C] 50.00—55.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes [ 7][RX—C] 50.00—55.00 sec 0.00 Bytes 0.00 bits/sec iperf3: error — control socket has closed unexpectedly |
Oops! What happened? There are some error messages in the kernel log as well.
|
[18424.279351] r8152 2—1:1.0 enx1cbfced40321: Tx status —71 [18424.287497] r8152 2—1:1.0 enx1cbfced40321: Tx status —71 [18424.295735] r8152 2—1:1.0 enx1cbfced40321: Tx status —71 [18424.303858] r8152 2—1:1.0 enx1cbfced40321: Tx status —71 [18430.885965] net_ratelimit: 107 callbacks suppressed [18430.885975] r8152 2—1:1.0 enx1cbfced40321: Tx status —71 [18431.251643] r8152 2—1:1.0 enx1cbfced40321: Tx status —71 [18431.909792] r8152 2—1:1.0 enx1cbfced40321: Tx status —71 [18437.797786] r8152 2—1:1.0 enx1cbfced40321: Tx status —71 |
I’m not the only one to have this problem, and that’s an open issue in r8156 driver’s Github repo. Here’s the answer from the developer for reference:
I suspect a problem on the ethernet adapter side, as there are many reports of it working with the DS918+. (Eg. design of power line, overheat, etc.)
https://github.com/bb-qq/r8152/wiki/CompatibilityCould you try another vendor? Also using a USB Hub with an external power supply might improve the situation.
At this point, the Ethernet did not work at all, and I had to unplug and reinsert the USB dongle. The second time “worked”.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
iperf3 —t 60 —c 192.168.31.12 —bidir —i 5 Connecting to host 192.168.31.12, port 5201 [ 5] local 192.168.31.166 port 32872 connected to 192.168.31.12 port 5201 [ 7] local 192.168.31.166 port 32874 connected to 192.168.31.12 port 5201 [ ID][Role] Interval Transfer Bitrate Retr Cwnd [ 5][TX—C] 0.00—5.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.59 MBytes [ 7][RX—C] 0.00—5.00 sec 515 MBytes 864 Mbits/sec [ 5][TX—C] 5.00—10.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.75 MBytes [ 7][RX—C] 5.00—10.00 sec 489 MBytes 820 Mbits/sec [ 5][TX—C] 10.00—15.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.75 MBytes [ 7][RX—C] 10.00—15.00 sec 530 MBytes 889 Mbits/sec [ 5][TX—C] 15.00—20.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.75 MBytes [ 7][RX—C] 15.00—20.00 sec 564 MBytes 947 Mbits/sec [ 5][TX—C] 20.00—25.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.75 MBytes [ 7][RX—C] 20.00—25.00 sec 560 MBytes 940 Mbits/sec [ 5][TX—C] 25.00—30.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.63 MBytes [ 7][RX—C] 25.00—30.00 sec 578 MBytes 970 Mbits/sec [ 5][TX—C] 30.00—35.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.63 MBytes [ 7][RX—C] 30.00—35.00 sec 561 MBytes 942 Mbits/sec [ 5][TX—C] 35.00—40.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.63 MBytes [ 7][RX—C] 35.00—40.00 sec 572 MBytes 960 Mbits/sec [ 5][TX—C] 40.00—45.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.63 MBytes [ 7][RX—C] 40.00—45.00 sec 570 MBytes 956 Mbits/sec [ 5][TX—C] 45.00—50.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.63 MBytes [ 7][RX—C] 45.00—50.00 sec 572 MBytes 960 Mbits/sec [ 5][TX—C] 50.00—55.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.63 MBytes [ 7][RX—C] 50.00—55.00 sec 570 MBytes 957 Mbits/sec [ 5][TX—C] 55.00—60.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.63 MBytes [ 7][RX—C] 55.00—60.00 sec 571 MBytes 958 Mbits/sec — — — — — — — — — — — — — — — — — — — — — — — — — [ ID][Role] Interval Transfer Bitrate Retr [ 5][TX—C] 0.00—60.00 sec 16.3 GBytes 2.34 Gbits/sec 0 sender [ 5][TX—C] 0.00—60.05 sec 16.3 GBytes 2.34 Gbits/sec receiver [ 7][RX—C] 0.00—60.00 sec 6.50 GBytes 931 Mbits/sec 58 sender [ 7][RX—C] 0.00—60.05 sec 6.50 GBytes 929 Mbits/sec receiver iperf Done. |
There are still retransmissions on the Rx side which could help explain the lower speed.
SAMBA
From my laptop with the RTL8156B dongle, and SATA SSD to UP Xtreme i11 mini PC with a 2.5GbE port, and a MINIX USB-C dock with a 480GB SSD.
Around 930 Mbps with r8152 driver against 750 Mbps with cdc_ncm driver.
Now from the mini PC to the laptop (aka download)…
It’s slower as expected at 837 Mbps, but better than the under 500 Mbps I got with the cdc_ncm driver.
scp
laptop to mini PC:
|
time scp Libero_SoC_v2021.2_lin.bin devkit@192.168.31.12:/home/devkit/NEO_Storage devkit@192.168.31.12‘s password: Libero_SoC_v2021.2_lin.bin 100% 10GB 115.0MB/s 01:32 real 1m34.981s user 0m55.750s sys 0m42.668s |
mini PC to laptop:
|
time scp devkit@192.168.31.12:/home/devkit/NEO_Storage/Libero_SoC_v2021.2_lin.bin . devkit@192.168.31.12‘s password: Libero_SoC_v2021.2_lin.bin 100% 10GB 111.7MB/s 01:35 real 1m36.896s user 0m56.926s sys 0m55.330s |
It’s almost the same speed between download and upload with scp which is odd. The bottleneck here looks to be my SATA SSD:
|
iozone —e —I —a —s 1000M —r 16384k —i 0 —i 1 Iozone: Performance Test of File I/O Version $Revision: 3.489 $ Compiled for 64 bit mode. Build: linux—AMD64 random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 1024000 16384 141794 139550 149591 145020 |
Read is limited by around 145 MB/s, while write is around 140 MB/s.
The SSD used in MINIX NEO Storage Plus USB-C dock is quite faster with, last time I tested, 379MB/s for sequential reads, and 240+MB/s for sequential writes.
So let’s see the speed we can get throwing away the data to /dev/null
download to laptop:
|
time scp —c aes128—ctr devkit@192.168.31.12:/home/devkit/NEO_Storage/Libero_SoC_v2021.2_lin.bin /dev/null devkit@192.168.31.12‘s password: Libero_SoC_v2021.2_lin.bin 100% 10GB 133.0MB/s 01:20 real 1m21.880s user 0m18.931s sys 0m33.838s |
upload from laptop:
|
time scp —c aes128—ctr Libero_SoC_v2021.2_lin.bin devkit@192.168.31.12:/dev/null devkit@192.168.31.12‘s password: Libero_SoC_v2021.2_lin.bin 100% 10GB 239.9MB/s 00:44 real 0m46.094s user 0m17.174s sys 0m36.793s |
So that’s more like it for the upload at least. I still have a problem with the download speed, but the performance was still greatly improved with the r8152 driver.
Testing with NanoPi R4S
While it’s much better, it’s not optimal. I have a NanoPi R4S router with two USB 3.0 ports, so I first tried it using the latest OpenWrt (FriendlyWrt) 21.02 image with Linux 5.15:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
root@FriendlyWrt:~# iperf3 -t 60 -c 192.168.2.207 —bidir -i 5 Connecting to host 192.168.2.207, port 5201 [ 5] local 192.168.2.161 port 21836 connected to 192.168.2.207 port 5201 [ 7] local 192.168.2.161 port 21838 connected to 192.168.2.207 port 5201 [ ID][Role] Interval Transfer Bitrate Retr Cwnd [ 5][TX—C] 0.00—5.00 sec 348 MBytes 583 Mbits/sec 4 413 KBytes [ 7][RX—C] 0.00—5.00 sec 220 MBytes 369 Mbits/sec [ 5][TX—C] 5.00—10.00 sec 480 MBytes 805 Mbits/sec 5 684 KBytes [ 7][RX—C] 5.00—10.00 sec 218 MBytes 366 Mbits/sec [ 5][TX—C] 10.00—15.00 sec 574 MBytes 963 Mbits/sec 47 557 KBytes [ 7][RX—C] 10.00—15.00 sec 183 MBytes 307 Mbits/sec [ 5][TX—C] 15.00—20.00 sec 481 MBytes 807 Mbits/sec 3 699 KBytes [ 7][RX—C] 15.00—20.00 sec 179 MBytes 301 Mbits/sec [ 5][TX—C] 20.00—25.00 sec 464 MBytes 779 Mbits/sec 18 701 KBytes [ 7][RX—C] 20.00—25.00 sec 214 MBytes 359 Mbits/sec [ 5][TX—C] 25.00—30.00 sec 549 MBytes 920 Mbits/sec 26 580 KBytes [ 7][RX—C] 25.00—30.00 sec 178 MBytes 298 Mbits/sec [ 5][TX—C] 30.00—35.00 sec 472 MBytes 792 Mbits/sec 3 526 KBytes [ 7][RX—C] 30.00—35.00 sec 207 MBytes 347 Mbits/sec [ 5][TX—C] 35.00—40.00 sec 465 MBytes 781 Mbits/sec 15 410 KBytes [ 7][RX—C] 35.00—40.00 sec 195 MBytes 326 Mbits/sec [ 5][TX—C] 40.00—45.00 sec 385 MBytes 645 Mbits/sec 0 376 KBytes [ 7][RX—C] 40.00—45.00 sec 217 MBytes 364 Mbits/sec [ 5][TX—C] 45.00—50.00 sec 497 MBytes 833 Mbits/sec 9 478 KBytes [ 7][RX—C] 45.00—50.00 sec 201 MBytes 338 Mbits/sec [ 5][TX—C] 50.00—55.00 sec 434 MBytes 728 Mbits/sec 0 543 KBytes [ 7][RX—C] 50.00—55.00 sec 208 MBytes 349 Mbits/sec [ 5][TX—C] 55.00—60.00 sec 451 MBytes 756 Mbits/sec 6 823 KBytes [ 7][RX—C] 55.00—60.00 sec 220 MBytes 370 Mbits/sec — — — — — — — — — — — — — — — — — — — — — — — — — [ ID][Role] Interval Transfer Bitrate Retr [ 5][TX—C] 0.00—60.00 sec 5.47 GBytes 783 Mbits/sec 136 sender [ 5][TX—C] 0.00—60.01 sec 5.47 GBytes 783 Mbits/sec receiver [ 7][RX—C] 0.00—60.00 sec 2.39 GBytes 342 Mbits/sec 94 sender [ 7][RX—C] 0.00—60.01 sec 2.38 GBytes 341 Mbits/sec receiver iperf Done. |
It’s really ugly with lots of retransmissions on both sides.
|
[ 276.268986] usb 8—1: New USB device found, idVendor=0bda, idProduct=8156, bcdDevice=31.00 [ 276.269798] usb 8—1: New USB device strings: Mfr=1, Product=2, SerialNumber=6 [ 276.270471] usb 8—1: Product: USB 10/100/1G/2.5G LAN [ 276.272112] usb 8—1: Manufacturer: Realtek [ 276.272519] usb 8—1: SerialNumber: 0013000000 [ 276.359625] cdc_ncm 8—1:2.0: MAC—Address: 1c:bf:ce:d4:03:21 [ 276.360178] cdc_ncm 8—1:2.0: setting rx_max = 16384 [ 276.360758] cdc_ncm 8—1:2.0: setting tx_max = 16384 [ 276.362948] cdc_ncm 8—1:2.0 eth2: register ‘cdc_ncm’ at usb—xhci—hcd.1.auto—1, CDC NCM, 1c:bf:ce:d4:03:21 [ 459.919597] IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready |
Looking at the kernel log, our RTL8156B USB dongle is using that CDC NCM driver again, just like in Ubuntu… There’s an r8152 driver too, but whatever module I remove in /etc/modules.d either the CDC NCM is loaded or the eth2 interface does not show up at all. So I’ve switched to the Ubuntu 20.04-based FriendlyCore OS, also featuring Linux 5.15, that will be closer to the setup on my laptop.
As one might have expected, the RTL8156B adapter uses the CDC NCM driver by default in Ubuntu:
|
[ 682.701529] usb 8—1: new SuperSpeed USB device number 3 using xhci—hcd [ 682.727125] usb 8—1: New USB device found, idVendor=0bda, idProduct=8156, bcdDevice=31.00 [ 682.727163] usb 8—1: New USB device strings: Mfr=1, Product=2, SerialNumber=6 [ 682.727179] usb 8—1: Product: USB 10/100/1G/2.5G LAN [ 682.727191] usb 8—1: Manufacturer: Realtek [ 682.727203] usb 8—1: SerialNumber: 0013000000 [ 682.806350] cdc_ncm 8—1:2.0: MAC—Address: 1c:bf:ce:d4:03:21 [ 682.806387] cdc_ncm 8—1:2.0: setting rx_max = 16384 [ 682.806561] cdc_ncm 8—1:2.0: setting tx_max = 16384 [ 682.807963] cdc_ncm 8—1:2.0 eth1: register ‘cdc_ncm’ at usb—xhci—hcd.1.auto—1, CDC NCM, 1c:bf:ce:d4:03:21 [ 682.834550] cdc_ncm 8—1:2.0 enx1cbfced40321: renamed from eth1 |
eth1 is not showing up with ifconfig, so I just went ahead and change udev rules to use the r8152 driver…
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
[ 882.081445] usb 7—1: new high—speed USB device number 3 using xhci—hcd [ 882.209358] usb 7—1: Device not responding to setup address. [ 882.417542] usb 7—1: Device not responding to setup address. [ 882.625432] usb 7—1: device not accepting address 3, error —71 [ 886.597570] usb 8—1: new SuperSpeed USB device number 4 using xhci—hcd [ 886.619484] usb 8—1: New USB device found, idVendor=0bda, idProduct=8156, bcdDevice=31.00 [ 886.619534] usb 8—1: New USB device strings: Mfr=1, Product=2, SerialNumber=6 [ 886.619556] usb 8—1: Product: USB 10/100/1G/2.5G LAN [ 886.619574] usb 8—1: Manufacturer: Realtek [ 886.619591] usb 8—1: SerialNumber: 0013000000 [ 886.911033] usb 8—1: reset SuperSpeed USB device number 4 using xhci—hcd [ 886.960598] r8152 8—1:1.0: Direct firmware load for rtl_nic/rtl8156b—2.fw failed with error —2 [ 886.960631] r8152 8—1:1.0: Falling back to sysfs fallback for: rtl_nic/rtl8156b—2.fw [ 947.165730] r8152 8—1:1.0: unable to load firmware patch rtl_nic/rtl8156b—2.fw (—110) [ 947.210940] r8152 8—1:1.0 (unnamed net_device) (uninitialized): netif_napi_add() called with weight 256 [ 947.237480] r8152 8—1:1.0 eth1: v1.12.11 [ 947.254755] usbcore: registered new interface driver cdc_ncm [ 947.257112] usbcore: registered new interface driver cdc_mbim [ 947.290865] r8152 8—1:1.0 enx1cbfced40321: renamed from eth1 |
It’s getting depressing. Let’s update the system first.
|
sudo apt update sudo apt install python3—pip sudo pip3 install apt—mirror—updater apt—mirror—updater —c http://ftp.tu-chemnitz.de/pub/linux/ubuntu-ports sudo apt dist—upgrade |
I followed all those steps since the FriendlyCore image relies on servers from China, which are very slow from where I have (just running apt update can take 15 minutes), so the update was possibly faster that way, although it still took a couple of hours! (See Changing Ubuntu Apt Mirror from the Command Line for details).
It did not help with the firmware issue. So instead I search for that rtl_nic/rtl8156b-2.fw file. It is in the firmware-realtek Debian package, or linux-firmware Ubuntu Impish package.
I downloaded the latter, extracted rtl8156b-2.fw and copied it to /etc/firmware/rtl_nic. It now works:
|
[ 2172.098871] usb 8—1: new SuperSpeed USB device number 4 using xhci—hcd [ 2172.120681] usb 8—1: New USB device found, idVendor=0bda, idProduct=8156, bcdDevice=31.00 [ 2172.120731] usb 8—1: New USB device strings: Mfr=1, Product=2, SerialNumber=6 [ 2172.120754] usb 8—1: Product: USB 10/100/1G/2.5G LAN [ 2172.120771] usb 8—1: Manufacturer: Realtek [ 2172.120788] usb 8—1: SerialNumber: 0013000000 [ 2172.183460] cdc_ncm 8—1:2.0: MAC—Address: 1c:bf:ce:d4:03:21 [ 2172.183494] cdc_ncm 8—1:2.0: setting rx_max = 16384 [ 2172.183620] cdc_ncm 8—1:2.0: setting tx_max = 16384 [ 2172.184904] cdc_ncm 8—1:2.0 eth1: register ‘cdc_ncm’ at usb—xhci—hcd.1.auto—1, CDC NCM, 1c:bf:ce:d4:03:21 [ 2172.189439] cdc_ncm 8—1:2.0 eth1: unregister ‘cdc_ncm’ usb—xhci—hcd.1.auto—1, CDC NCM [ 2172.451015] usb 8—1: reset SuperSpeed USB device number 4 using xhci—hcd [ 2172.535166] r8152 8—1:1.0: load rtl8156b—2 v1 04/15/21 successfully [ 2172.598459] r8152 8—1:1.0 eth1: v1.12.11 |
Somehow the interface is still not up, and adding it manually to /etc/network/interfaces.d, does not work.
Back to the Ubuntu laptop with rtl8156b-2.fw firmware
So instead, I went back to my laptop, and copied the firmware file to /lib/firmware/rtl_nic directory. The result was the same as in NanoPi R4S:
|
[23050.245495] usb 2—1: new SuperSpeed USB device number 3 using xhci_hcd [23050.266025] usb 2—1: New USB device found, idVendor=0bda, idProduct=8156, bcdDevice=31.00 [23050.266038] usb 2—1: New USB device strings: Mfr=1, Product=2, SerialNumber=6 [23050.266043] usb 2—1: Product: USB 10/100/1G/2.5G LAN [23050.266047] usb 2—1: Manufacturer: Realtek [23050.266050] usb 2—1: SerialNumber: 0013000000 [23050.325714] cdc_ncm 2—1:2.0: MAC—Address: 1c:bf:ce:d4:03:21 [23050.325720] cdc_ncm 2—1:2.0: setting rx_max = 16384 [23050.325762] cdc_ncm 2—1:2.0: setting tx_max = 16384 [23050.326199] cdc_ncm 2—1:2.0 eth0: register ‘cdc_ncm’ at usb—0000:04:00.3—1, CDC NCM, 1c:bf:ce:d4:03:21 [23050.334723] cdc_ncm 2—1:2.0 eth0: unregister ‘cdc_ncm’ usb—0000:04:00.3—1, CDC NCM [23050.559779] usb 2—1: reset SuperSpeed USB device number 3 using xhci_hcd [23050.617459] r8152 2—1:1.0: load rtl8156b—2 v1 04/15/21 successfully [23050.654344] r8152 2—1:1.0 eth0: v1.12.11 |
Let’s try iperf3 upload:
|
iperf3 —t 30 —c 192.168.31.12 —i 5 Connecting to host 192.168.31.12, port 5201 [ 5] local 192.168.31.166 port 50222 connected to 192.168.31.12 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00—5.00 sec 1.37 GBytes 2.36 Gbits/sec 0 609 KBytes [ 5] 5.00—10.00 sec 1.37 GBytes 2.35 Gbits/sec 0 691 KBytes [ 5] 10.00—15.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.35 MBytes [ 5] 15.00—20.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.35 MBytes [ 5] 20.00—25.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.35 MBytes [ 5] 25.00—30.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.35 MBytes — — — — — — — — — — — — — — — — — — — — — — — — — [ ID] Interval Transfer Bitrate Retr [ 5] 0.00—30.00 sec 8.22 GBytes 2.35 Gbits/sec 0 sender [ 5] 0.00—30.04 sec 8.22 GBytes 2.35 Gbits/sec receiver iperf Done. |
Same as before, so no regression. Now for the iperf3 download:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
iperf3 —t 30 —c 192.168.31.12 —i 5 —R Connecting to host 192.168.31.12, port 5201 Reverse mode, remote host 192.168.31.12 is sending [ 5] local 192.168.31.166 port 50226 connected to 192.168.31.12 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00—5.00 sec 969 MBytes 1.63 Gbits/sec [ 5] 5.00—10.00 sec 990 MBytes 1.66 Gbits/sec [ 5] 10.00—15.00 sec 982 MBytes 1.65 Gbits/sec [ 5] 15.00—20.00 sec 984 MBytes 1.65 Gbits/sec [ 5] 20.00—25.00 sec 1012 MBytes 1.70 Gbits/sec [ 5] 25.00—30.00 sec 1007 MBytes 1.69 Gbits/sec — — — — — — — — — — — — — — — — — — — — — — — — — [ ID] Interval Transfer Bitrate Retr [ 5] 0.00—30.02 sec 5.81 GBytes 1.66 Gbits/sec 3181 sender [ 5] 0.00—30.00 sec 5.80 GBytes 1.66 Gbits/sec receiver iperf Done. |
It’s getting better, although we are not there yet, and the retransmissions are through the roof. I’m really tired now… Let me know if you have other ideas.
Realtek RTL8156BG USB 3.0 to 2.5GbE dongle to the rescue.
[Update: Realtek sent me one of their USB 3.0 to 2.5GbE adapters and it does not have the same performance issue
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
$ iperf3 —t 60 —c 192.168.31.12 —bidir —i 5 Connecting to host 192.168.31.12, port 5201 [ 5] local 192.168.31.85 port 47164 connected to 192.168.31.12 port 5201 [ 7] local 192.168.31.85 port 47168 connected to 192.168.31.12 port 5201 [ ID][Role] Interval Transfer Bitrate Retr Cwnd [ 5][TX—C] 0.00—5.00 sec 1.35 GBytes 2.31 Gbits/sec 0 1.54 MBytes [ 7][RX—C] 0.00—5.00 sec 1.25 GBytes 2.14 Gbits/sec [ 5][TX—C] 5.00—10.00 sec 1.37 GBytes 2.35 Gbits/sec 0 1.54 MBytes [ 7][RX—C] 5.00—10.00 sec 1.36 GBytes 2.34 Gbits/sec [ 5][TX—C] 10.00—15.00 sec 1.36 GBytes 2.34 Gbits/sec 0 1.54 MBytes [ 7][RX—C] 10.00—15.00 sec 1.34 GBytes 2.31 Gbits/sec [ 5][TX—C] 15.00—20.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.34 MBytes [ 7][RX—C] 15.00—20.00 sec 1.32 GBytes 2.28 Gbits/sec [ 5][TX—C] 20.00—25.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.34 MBytes [ 7][RX—C] 20.00—25.00 sec 1.36 GBytes 2.33 Gbits/sec [ 5][TX—C] 25.00—30.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.34 MBytes [ 7][RX—C] 25.00—30.00 sec 1.34 GBytes 2.30 Gbits/sec [ 5][TX—C] 30.00—35.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.34 MBytes [ 7][RX—C] 30.00—35.00 sec 1.36 GBytes 2.33 Gbits/sec [ 5][TX—C] 35.00—40.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.34 MBytes [ 7][RX—C] 35.00—40.00 sec 1.33 GBytes 2.28 Gbits/sec [ 5][TX—C] 40.00—45.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.34 MBytes [ 7][RX—C] 40.00—45.00 sec 1.35 GBytes 2.32 Gbits/sec [ 5][TX—C] 45.00—50.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.34 MBytes [ 7][RX—C] 45.00—50.00 sec 1.35 GBytes 2.32 Gbits/sec [ 5][TX—C] 50.00—55.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.34 MBytes [ 7][RX—C] 50.00—55.00 sec 1.35 GBytes 2.31 Gbits/sec [ 5][TX—C] 55.00—60.00 sec 1.36 GBytes 2.34 Gbits/sec 0 2.34 MBytes [ 7][RX—C] 55.00—60.00 sec 1.32 GBytes 2.26 Gbits/sec — — — — — — — — — — — — — — — — — — — — — — — — — [ ID][Role] Interval Transfer Bitrate Retr [ 5][TX—C] 0.00—60.00 sec 16.3 GBytes 2.34 Gbits/sec 0 sender [ 5][TX—C] 0.00—60.05 sec 16.3 GBytes 2.34 Gbits/sec receiver [ 7][RX—C] 0.00—60.00 sec 16.0 GBytes 2.29 Gbits/sec 25 sender [ 7][RX—C] 0.00—60.05 sec 16.0 GBytes 2.29 Gbits/sec receiver iperf Done. |
]
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
I replaced my consumer wireless router with a linux box that has a quad-gigabit NIC PCIe card and a single gigabit NIC on the motherboard (for the WAN). After turning on IP forwarding, masquerading (via iptables), and setting up subnets on each of the four LAN interfaces I ran some speed tests.
$ ip route
default dev ppp0 scope link
10.0.0.0/16 dev enp3s0f0 proto kernel scope link src 10.0.0.1
10.64.0.0/16 dev enp3s0f1 proto kernel scope link src 10.64.0.1
10.192.0.0/16 dev enp4s0f1 proto kernel scope link src 10.192.0.1
aaa.bbb.ccc.ddd dev ppp0 proto kernel scope link src www.xxx.yyy.zzz
-
From a wireless device on one of the LAN subnets to a speedtest server on the WAN I get the full 40 Mbps / 5 Mbps I pay my ISP for.
-
From the router host to a wired LAN host using
iperf3I can consistently maintain 930+ Mbps for several minutes. -
From a wired device on one of the LAN subnets to a wired device on a different LAN subnet using
iperf3I initially get 80-95 Mbps for the first few seconds but it rapidly drops to zero. -
From a wired device on one of the LAN subnets to a wired device on a different LAN subnet using
iperf3with a target bitrate of 20 Mbps I see the similar results (see update at end), but can sustain about 10 Mpbs
.
Connecting to host 10.0.0.2, port 5201
[ 5] local 10.192.128.3 port 35620 connected to 10.0.0.2 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 10.2 MBytes 85.9 Mbits/sec 0 73.5 KBytes
[ 5] 1.00-2.00 sec 9.01 MBytes 75.6 Mbits/sec 0 82.0 KBytes
[ 5] 2.00-3.00 sec 8.26 MBytes 69.3 Mbits/sec 0 79.2 KBytes
[ 5] 3.00-4.00 sec 9.01 MBytes 75.6 Mbits/sec 0 73.5 KBytes
[ 5] 4.00-5.00 sec 5.28 MBytes 44.3 Mbits/sec 1 1.41 KBytes
[ 5] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
[ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
[ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
[ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
^C[ 5] 10.00-13.63 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-13.63 sec 41.8 MBytes 25.7 Mbits/sec 5 sender
[ 5] 0.00-13.63 sec 0.00 Bytes 0.00 bits/sec receiver
iperf3: interrupt - the client has terminated
This is suggesting to me that there’s some problem forwarding packets between the subnets. I first ensured that my iptables rules are as minimal as possible:
-t nat -A POSTROUTING -o ppp0 -j MASQUERADE
# WAN connection is PPPoE and VLAN tagged
-t filter -A FORWARD -o ppp0 -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
Dumping the iptables state I see low packet counts for both rules.
Next I checked for packet loss. There does seem to be a small but consistent amount of packet loss / retransmits.
$ sudo netstat -s | egrep -i 'retransmit|drop'
498 outgoing packets dropped
25848 fast retransmits
I then thought that maybe there was a buffer or queue that was filling and packets were getting dropped. I calculated the average bandwidth-delay product and compared that against the reserved memory.
$ sudo ping -f 10.0.0.2 -s $((1500-28))
PING 10.0.0.2 (10.0.0.2) 1472(1500) bytes of data.
.^C
--- 10.0.0.2 ping statistics ---
9036 packets transmitted, 9035 received, 0% packet loss, time 26512ms
rtt min/avg/max/mdev = 1.742/2.817/12.057/0.758 ms, pipe 2, ipg/ewma 2.934/3.091 ms
$ echo "1*(1024^3) * 0.003" | bc
3221225.472
$ cat /proc/sys/net/ipv4/tcp_mem
18396 24529 36792
$ getconf PAGESIZE
4096
That appears to be sufficient. So now I’m a bit stuck. I ran tcpdump on the iperf3 client and can see things moving along well for a bit. Then I see a long (almost 250ms) period of silence before lots of retransmits and duplicate acknowledgements.
Since I can pull sufficient download speeds from the WAN I don’t suspect that the onboard NIC is at fault. I’m looking for help to diagnose this quad-NIC (details below) and possibly a dumb layer-2 gigabit switch (Netgear GS-108) and any other kernel configuration that could be getting in the way. I doubt it’s the switch, as it’s never been a problem before and I can maintain speeds from the router’s loopback to that subnet. Only inter-subnet performance appears to be affected.
*-network:0
description: Ethernet interface
product: 82571EB Gigabit Ethernet Controller (Copper)
vendor: Intel Corporation
physical id: 0
bus info: pci@0000:03:00.0
logical name: enp3s0f0
version: 06
serial: 00:26:55:xx:xx:xx
size: 1Gbit/s
capacity: 1Gbit/s
width: 32 bits
clock: 33MHz
capabilities: pm msi pciexpress bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=3.2.6-k duplex=full firmware=5.12-2 ip=10.0.0.1 latency=0 link=yes multicast=yes port=twisted pair speed=1Gbit/s
resources: irq:24 memory:fe920000-fe93ffff memory:fe880000-fe8fffff ioport:d020(size=32)
UPDATE:
$ iperf3 -b 20m -c 10.0.0.2
Connecting to host 10.0.0.2, port 5201
[ 5] local 10.192.128.3 port 36554 connected to 10.0.0.2 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 2.49 MBytes 20.9 Mbits/sec 0 158 KBytes
[ 5] 1.00-2.00 sec 2.38 MBytes 19.9 Mbits/sec 0 150 KBytes
[ 5] 2.00-3.00 sec 2.38 MBytes 19.9 Mbits/sec 1 133 KBytes
[ 5] 3.00-4.00 sec 2.38 MBytes 19.9 Mbits/sec 0 73.5 KBytes
[ 5] 4.00-5.00 sec 2.38 MBytes 19.9 Mbits/sec 0 70.7 KBytes
[ 5] 5.00-6.00 sec 1.12 MBytes 9.44 Mbits/sec 2 1.41 KBytes
[ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 2 1.41 KBytes
[ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 1 1.41 KBytes
iperf3: error - control socket has closed unexpectedly
$ iperf3 -b 10m -c 10.0.0.2
Connecting to host 10.0.0.2, port 5201
[ 5] local 10.192.128.3 port 36564 connected to 10.0.0.2 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.24 MBytes 10.4 Mbits/sec 0 201 KBytes
[ 5] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec 0 118 KBytes
[ 5] 2.00-3.00 sec 1.12 MBytes 9.44 Mbits/sec 0 127 KBytes
[ 5] 3.00-4.00 sec 1.25 MBytes 10.5 Mbits/sec 0 107 KBytes
[ 5] 4.00-5.00 sec 1.12 MBytes 9.44 Mbits/sec 0 110 KBytes
[ 5] 5.00-6.00 sec 1.25 MBytes 10.5 Mbits/sec 0 90.0 KBytes
[ 5] 6.00-7.00 sec 1.12 MBytes 9.44 Mbits/sec 0 87.2 KBytes
[ 5] 7.00-8.00 sec 1.25 MBytes 10.5 Mbits/sec 0 81.6 KBytes
[ 5] 8.00-9.00 sec 1.12 MBytes 9.44 Mbits/sec 0 78.8 KBytes
[ 5] 9.00-10.00 sec 1.25 MBytes 10.5 Mbits/sec 0 112 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 12.0 MBytes 10.1 Mbits/sec 0 sender
[ 5] 0.00-10.04 sec 12.0 MBytes 10.0 Mbits/sec receiver
iperf Done.
Hi everyone,
I am at an impass. After several iperf3 tests and Blackmagic write/reads, I realized all connections work fine from all the computers at home, except uploads/writes from my Macintosh VM (under Proxmox server) to the NAS (freenas).
I posted on the Proxmox forums as well, but I figured the community here might shed light, and it could be a config I oversaw somewhere on the nas itself.
As you will see below, I’ve ran tests from all points (Proxmox node itself to and from several machines including over wifi and ethernet, machines to and from the NAS, etc) and the only thing that doesn’t work is from my Mac VM to the NAS.
It feels like the data stream gets saturated when I do so. It starts writing/sending, and everything related to internet begins to slow down: Freenas GUI unresponsive in my mac VM, write stops, etc. You can see through the iperf tests that a first packet is sent, then 0, 0, 0… etc until the end of the test.
I have not set jumbo frames anywhere, tried activating NAT acceleration on my router (ASUS Ac-88u, which is supposed to be a very capable router) and ethernet cable works fine as obviously showing with the proxmox server read/write tests.
Is vmxnet3 or any setup from my Proxmox machine the bottleneck? Could there be a setting I missed, or a conflict between Proxmox routing the data and the router receiving it? I am pretty sure I have a double NAT setup, but since there isn’t any issue with other machines except my VM, that seems unlikely.
Last thing is that I have set up Syncthing bterween the Mac VM and the NAS a few days ago. I am unsure I had those problems before setting that up (it’s a brand new build for both) but uninstalling the whole syncthing dependencies on my mac VM and turning off the jail (then ultimately deleting it) in freenas did nothing, so I have a feeling the issue is my proxmox setup (correct me if I’m wrong of course! I’m a noob on both and here to learn).
My machines for this test:
NAS = Freenas separate machine (LAN connection, 192.168.1.45) (hardware would seem irrelevant at this point, but it is by no means recommended hardware so here it is: AMD A8-7600 | MOBO: C.A68HM-K PLUS colorful plus V18 with the Realtek RTL8111/8168/8411)
Server = Proxmox machine (Proxmox 5.4, LAN connection, 192.168.1.191)
MacOSvm = My Mac VM (Mojave 10.14.4, VMXNet3 ethernet, 192.168.1.229)
iMacWifi = authentic iMac (Mojave 10.14.5, over wifi 5ghz, 192.168.1.6)
Router: brand new Asus AC88u acting as the router and switch (I have a huawei modem to delivery main WAN, supposedly bridged)
Feel free to let me know if I need to run other tests or output other settings to investigate! I haven’t setup any kind of interface, vlan or anything on Freenas so far.
Here is my interfaces config on proxmox running cat /etc/network/interfaces :
Code:
auto lo iface lo inet loopback iface eno2 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.1.191 netmask 255.255.255.0 gateway 192.168.1.1 bridge_ports eno2 bridge_stp off bridge_fd 0
Here are the iperf3 tests:
NAS >> Server
Code:
[ 5] local 192.168.1.45 port 62731 connected to 192.168.1.191 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 84.9 MBytes 712 Mbits/sec 909 598 KBytes [ 5] 1.00-2.00 sec 83.4 MBytes 700 Mbits/sec 0 713 KBytes [ 5] 2.00-3.00 sec 83.5 MBytes 700 Mbits/sec 0 813 KBytes [ 5] 3.00-4.00 sec 83.4 MBytes 700 Mbits/sec 4 483 KBytes [ 5] 4.00-5.00 sec 83.4 MBytes 700 Mbits/sec 0 622 KBytes [ 5] 5.00-6.00 sec 83.5 MBytes 701 Mbits/sec 0 734 KBytes [ 5] 6.00-7.00 sec 83.5 MBytes 700 Mbits/sec 0 827 KBytes [ 5] 7.00-8.00 sec 83.4 MBytes 700 Mbits/sec 0 843 KBytes [ 5] 8.00-9.00 sec 83.5 MBytes 700 Mbits/sec 0 858 KBytes [ 5] 9.00-10.00 sec 83.5 MBytes 701 Mbits/sec 0 872 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 836 MBytes 701 Mbits/sec 913 sender [ 5] 0.00-10.00 sec 835 MBytes 701 Mbits/sec receiver iperf Done.
Server >> NAS
Code:
Accepted connection from 192.168.1.191, port 39518 [ 5] local 192.168.1.45 port 5201 connected to 192.168.1.191 port 39520 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 111 MBytes 934 Mbits/sec [ 5] 1.00-2.00 sec 112 MBytes 937 Mbits/sec [ 5] 2.00-3.00 sec 112 MBytes 937 Mbits/sec [ 5] 3.00-4.00 sec 112 MBytes 936 Mbits/sec [ 5] 4.00-5.00 sec 112 MBytes 937 Mbits/sec [ 5] 5.00-6.00 sec 112 MBytes 937 Mbits/sec [ 5] 6.00-7.00 sec 112 MBytes 937 Mbits/sec [ 5] 7.00-8.00 sec 112 MBytes 937 Mbits/sec [ 5] 8.00-9.00 sec 112 MBytes 936 Mbits/sec [ 5] 9.00-10.00 sec 112 MBytes 937 Mbits/sec [ 5] 10.00-10.00 sec 250 KBytes 933 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 1.09 GBytes 936 Mbits/sec receiver ----------------------------------------------------------- Server listening on 5201 -----------------------------------------------------------
Server >> MacOSvm
Code:
Accepted connection from 192.168.1.191, port 36556 [ 5] local 192.168.1.229 port 5201 connected to 192.168.1.191 port 36558 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 158 MBytes 1.33 Gbits/sec [ 5] 1.00-2.00 sec 178 MBytes 1.49 Gbits/sec [ 5] 2.00-3.00 sec 177 MBytes 1.49 Gbits/sec [ 5] 3.00-4.00 sec 177 MBytes 1.49 Gbits/sec [ 5] 4.00-5.00 sec 178 MBytes 1.49 Gbits/sec [ 5] 5.00-6.00 sec 176 MBytes 1.48 Gbits/sec [ 5] 6.00-7.00 sec 178 MBytes 1.49 Gbits/sec [ 5] 7.00-8.00 sec 177 MBytes 1.48 Gbits/sec [ 5] 8.00-9.00 sec 178 MBytes 1.49 Gbits/sec [ 5] 9.00-10.00 sec 177 MBytes 1.48 Gbits/sec [ 5] 10.00-10.05 sec 8.33 MBytes 1.51 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.05 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.05 sec 1.72 GBytes 1.47 Gbits/sec
MacOSvm >> Server
Code:
Accepted connection from 192.168.1.229, port 64307 [ 5] local 192.168.1.191 port 5201 connected to 192.168.1.229 port 64308 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 195 MBytes 1.64 Gbits/sec [ 5] 1.00-2.00 sec 874 MBytes 7.34 Gbits/sec [ 5] 2.00-3.00 sec 455 MBytes 3.82 Gbits/sec [ 5] 3.00-4.00 sec 204 MBytes 1.71 Gbits/sec [ 5] 4.00-5.00 sec 977 MBytes 8.20 Gbits/sec [ 5] 5.00-6.00 sec 867 MBytes 7.27 Gbits/sec [ 5] 6.00-7.00 sec 1.16 GBytes 10.0 Gbits/sec [ 5] 7.00-8.00 sec 220 MBytes 1.84 Gbits/sec [ 5] 8.00-9.00 sec 818 MBytes 6.86 Gbits/sec [ 5] 9.00-10.00 sec 177 MBytes 1.48 Gbits/sec [ 5] 10.00-10.00 sec 0.00 Bytes 0.00 bits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.00 sec 5.84 GBytes 5.02 Gbits/sec receiver
MacOSvm >> NAS
Code:
Accepted connection from 192.168.1.229, port 64418 [ 5] local 192.168.1.45 port 5201 connected to 192.168.1.229 port 64419 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.01 sec 56.4 MBytes 469 Mbits/sec [ 5] 1.01-2.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 2.06-3.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 3.06-4.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 4.06-5.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 5.06-6.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 6.06-7.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 7.06-8.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 8.06-9.05 sec 0.00 Bytes 0.00 bits/sec [ 5] 9.05-10.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 10.06-11.01 sec 0.00 Bytes 0.00 bits/sec [ 5] 11.01-12.05 sec 0.00 Bytes 0.00 bits/sec [ 5] 12.05-13.06 sec 0.00 Bytes 0.00 bits/sec [ 5] 13.06-13.53 sec 0.00 Bytes 0.00 bits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-13.53 sec 56.4 MBytes 35.0 Mbits/sec receiver
that’s where it craps the bed…
NAS >> MacOSvm
Code:
Accepted connection from 192.168.1.45, port 24734 [ 5] local 192.168.1.229 port 5201 connected to 192.168.1.45 port 45358 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 55.8 MBytes 468 Mbits/sec [ 5] 1.00-2.00 sec 83.7 MBytes 702 Mbits/sec [ 5] 2.00-3.00 sec 83.6 MBytes 702 Mbits/sec [ 5] 3.00-4.00 sec 83.6 MBytes 701 Mbits/sec [ 5] 4.00-5.00 sec 83.6 MBytes 701 Mbits/sec [ 5] 5.00-6.00 sec 83.5 MBytes 701 Mbits/sec [ 5] 6.00-7.00 sec 83.7 MBytes 702 Mbits/sec [ 5] 7.00-8.00 sec 83.7 MBytes 702 Mbits/sec [ 5] 8.00-9.00 sec 83.5 MBytes 701 Mbits/sec [ 5] 9.00-10.00 sec 83.7 MBytes 702 Mbits/sec [ 5] 10.00-10.15 sec 12.9 MBytes 701 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.15 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.15 sec 821 MBytes 679 Mbits/sec receiver
all fine the other way !!!
iMacWifi >> MacOSvm
Code:
[ 4] local 192.168.1.229 port 65136 connected to 192.168.1.132 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 87.8 MBytes 736 Mbits/sec [ 4] 1.00-2.00 sec 101 MBytes 844 Mbits/sec [ 4] 2.00-3.00 sec 103 MBytes 865 Mbits/sec [ 4] 3.00-4.00 sec 105 MBytes 879 Mbits/sec [ 4] 4.00-5.00 sec 107 MBytes 898 Mbits/sec [ 4] 5.00-6.00 sec 82.8 MBytes 695 Mbits/sec [ 4] 6.00-7.00 sec 78.2 MBytes 656 Mbits/sec [ 4] 7.00-8.00 sec 84.4 MBytes 708 Mbits/sec [ 4] 8.00-9.00 sec 88.9 MBytes 746 Mbits/sec [ 4] 9.00-10.00 sec 94.5 MBytes 793 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 932 MBytes 782 Mbits/sec sender [ 4] 0.00-10.00 sec 931 MBytes 781 Mbits/sec receiver
iMacWifi >> NAS
Code:
Accepted connection from 192.168.1.132, port 54059 [ 5] local 192.168.1.45 port 5201 connected to 192.168.1.132 port 54060 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 51.8 MBytes 434 Mbits/sec [ 5] 1.00-2.00 sec 59.5 MBytes 500 Mbits/sec [ 5] 2.00-3.00 sec 60.4 MBytes 507 Mbits/sec [ 5] 3.00-4.00 sec 60.9 MBytes 511 Mbits/sec [ 5] 4.00-5.00 sec 61.3 MBytes 514 Mbits/sec [ 5] 5.00-6.00 sec 61.9 MBytes 519 Mbits/sec [ 5] 6.00-7.00 sec 62.2 MBytes 521 Mbits/sec [ 5] 7.00-8.00 sec 62.8 MBytes 527 Mbits/sec [ 5] 8.00-9.00 sec 62.1 MBytes 521 Mbits/sec [ 5] 9.00-10.00 sec 61.5 MBytes 516 Mbits/sec [ 5] 10.00-10.01 sec 441 KBytes 443 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.01 sec 605 MBytes 507 Mbits/sec receiver