Re: Network very very slow... was iSCSI and jumbo frames

[RVP <rvp%SDF.ORG@localhost>]

On Mon, 8 Feb 2021, BERTRAND Joël wrote:

> > Does it store its cache on /opt?
>
> 	No. Squid's cache is on /var (a local volume in Raid1).

OK.

> > Can you enable `noatime' and disable `log' when mounting /opt and
> > then run a dd read test on any 5GB+ file there (make sure the dd
> > cmd.  has `iflag=direct oflag=direct'). While the test is running,
> > do:
>
> 	I don't undestand how dd returns "no such file or directory" when I try :
>
> legendre# dd if=/dev/zero of=/opt/test iflag=direct oflag=direct
> count=5000 bs=1m
> dd: /opt/test: No such file or directory

This is mentioned in the dd man-page (see the oflag=flag operand).
If supply an `oflag=...' then the output file must exist; otherwise
you have to add the `creat' flag to `oflag=...'.

> 	I cannot remount /opt without atime and log until next thursday. Bacula
> full process is in progress.
>
> >    fstat -f /opt
>
> legendre# fstat -f /opt
> USER     CMD          PID   FD MOUNT       INUM MODE         SZ|DV R/W
> bacula-s bacula-sd   3861    7  /opt      114683956 -rw-r-----
> 50745719346 rw
> legendre#
>
> >    pstat -v | sed -n '/^\*\*\* MOUNT .*dk5/,/^\*\*\* MOUNT /p'
>
>
> > I want to see what other processes show an undue interest in
> > files being read/written in /opt. For the `pstat -v' command,
> > keep an eye on those with a USE count >1. One of them will be
> > the file being read/written.
>
> legendre# pstat -v | sed -n '/^\*\*\* MOUNT .*dk5/,/^\*\*\* MOUNT /p'
> *** MOUNT ffs /dev/dk5 on /opt (log,local)
> ADDR             TYP VFLAG  USE HOLD TAG NPAGE FILEID IFLAG RDEV|SZ
> ffffd45812906ac0 reg          1  319   1     0 114683952  - 53687079313
> ffffd459b0fab970 dir     R    1    1   1     0      2     -     512
> ffffd45628c05188 dir          1    1   1     0 114683904  -    6144
> ffffd45845df8ab8 reg          1  401   1     0 114683953  - 53687079313
> ffffd45737892970 reg          1  401   1     0 114683954  - 53687079692
> ffffd459328b32e0 reg     O    3  381   1 2258960 114683956 - 50745719346
> ffffd458711ed970 dir          1    0   1     0 697238784  -     512
> legendre#
>
> 	I understand, but please note that even if bacula is stopped, a simple
> dd doesn't exceed 10 MB/s on euclide.

OK. Looks like only bacula-sd is using /opt, and I don't think will
start injesting newly created files for no reason. You can forget the
dd test.

> > If bacula is stopped can you do the i3perf test between euclide
> > and legendre with the server port on euclide set to the iscsi-target
> > port? I want to make sure that the iscsi data isn't getting diverted
> > and filtered at some end, slowing down the throughput.
>
> 	I have done these tests without any transaction on iSCSI daemon. I
> don't remember if bacula was stopped, but I'm sure it was inactive.

I meant that if bacula was stopped, then you could also stop the
iscsi-target process on the NAS and then reuse port 3260 as the
server port for the i3perf throughput test.

> 	I suspect a latency issue somewhere in iscsid.

Maybe, but, then there isn't any latency when running over loopback
as all 3 of us (you, me & Michael) have seen. Plus, Michael mentioned
getting a speed of 680 Mbits using iscsid over his network (maybe a
9.99.XX version of iscsid & kernel?).

1. Chavdar Ivanov mentioned having a iSCSI-NAS storage. He could
chime in with the speeds he usually gets. (I'm of no use here
because at home I have just this one laptop which only has a
fast ethernet card; and at work we have all Linux & Windows boxes.)

2. You could test this yourself another way: Run istgt on one of
your FreeBSD/Linux boxes on the 192.168.10.0/24 network; then
connect to it using iscsid on legendre and do a dd test.

> I cannot test until next thirsday, but I think we should compare
> two tcpdump outputs. The first one between NetBSD initiator and
> euclide, the second one between Linux initiator and euclide.

We'll hold off on this until 2., above, is done.

-RVP