Re: Network very very slow... was iSCSI and jumbo frames

[RVP <rvp%SDF.ORG@localhost>]

On Mon, 8 Feb 2021, Michael van Elst wrote:

> On Mon, Feb 08, 2021 at 11:40:43AM +0100, BERTRAND Joël wrote:
> > 	If I do that in write mode, I suppose I will kill filesystem on
> > /dev/rdk5, isn't it ?
>
> Definitely.
>
> You need a test volume for a write test.

A not quite off-topic data-point here: In one of JKB's logs, I
noticed this message:

[ 82537.321769] sd0: GPT GUID: a5d27c7c-8eda-40e8-a29b-e85a539a5bc7
[ 82537.321769] dk5 at sd0: "bacula", 23028563901 blocks at 34, type: ffs
[ 82537.331773] sd0: async, 8-bit transfers, tagged queueing

Here, dk5 on the disk sd0 exported by the NAS starts at sector
34--which is the standard starting sector for GPT partitioned disks,
but, this is not aligned to a 4k (physical sector) boundary. I did
a little test over the weekend pertaining to this, and I found that
there is a throughput difference when writing to aligned vs.
mis-aligned partitions.

I created a new 1MB aligned 50GB GPT partition, rdk14, in the free
space I had. This, I exported as the raw disk for istgt. Then,
after the kernel created the /dev/sd0, I created a new aligned
(2048-block) and then a mis-aligned (34-block) partition, rdk15,
for the test.  /dev/rdk15 was then formatted using `newfs -O 2
/dev/rdk15'. There is a ~20% decrease in write speed when the
partition is "mis-aligned".  Read speeds are the same. I don't
think JKB's NAS disk are "mis-aligned" like this, and it probably
doesn't matter to the NetBSD kernel that the exported partition
starts at sector 34--I think.

Anyway, here's the test (just single runs of each test-case--done
mainly because I was curious about this):

---

# rdk15 starts at block (sector) 2048 within 1M aligned sd0 (AKA. /dev/rdk14)
# Aligned write direct
$ sudo dd if=/dev/zero of=/mnt/out.dd bs=1m count=3000 iflag=direct oflag=creat,direct
3000+0 records in
3000+0 records out
3145728000 bytes transferred in 32.216 secs (97644896 bytes/sec)

# Aligned write buffered
$ sudo dd if=/dev/zero of=/mnt/out.dd bs=1m count=3000
3000+0 records in
3000+0 records out
3145728000 bytes transferred in 30.036 secs (104731921 bytes/sec)

# Aligned read direct
$ sudo dd if=/mnt/out.dd of=/dev/null bs=1m iflag=direct oflag=direct
3000+0 records in
3000+0 records out
3145728000 bytes transferred in 34.384 secs (91488134 bytes/sec)

# Aligned read buffered
$ sudo dd if=/mnt/out.dd of=/dev/null bs=1m 
3000+0 records in
3000+0 records out
3145728000 bytes transferred in 35.310 secs (89088870 bytes/sec)




# rdk15 starts at block (sector) 34 within 1M aligned sd0 (AKA. /dev/rdk14)
# Un-aligned write direct
$ sudo dd if=/dev/zero of=/mnt/out.dd bs=1m count=3000 iflag=direct oflag=creat,direct
3000+0 records in
3000+0 records out
3145728000 bytes transferred in 41.198 secs (76356327 bytes/sec)

# Un-aligned write buffered
$ sudo dd if=/dev/zero of=/mnt/out.dd bs=1m count=3000
3000+0 records in
3000+0 records out
3145728000 bytes transferred in 46.780 secs (67245147 bytes/sec)

# Un-aligned read direct
$ sudo dd if=/mnt/out.dd of=/dev/null bs=1m iflag=direct oflag=direct
3000+0 records in
3000+0 records out
3145728000 bytes transferred in 33.992 secs (92543186 bytes/sec)

# Un-aligned read buffered
$ sudo dd if=/mnt/out.dd of=/dev/null bs=1m
3000+0 records in
3000+0 records out
3145728000 bytes transferred in 35.331 secs (89035917 bytes/sec)

---

-RVP