Re: Read/write slow on nvme drive - Supermicro X10DRL-CT

[mlelstv%serpens.de@localhost (Michael van Elst)]

derrick.lobo%givex.com@localhost (Derrick Lobo) writes:

>dd if=/dev/rdk0 of=/dev/null bs=1024k
>^C93836+0 records in
>93836+0 records out
>98394177536 bytes transferred in 258.920 secs (380017679 bytes/sec)

># dd if=/dev/rdk1 of=/dev/null bs=1024k
>^C57226+0 records in
>57226+0 records out
>60005810176 bytes transferred in 207.282 secs (289488764 bytes/sec)

Still doesn't look like nvme.


>So how do I increase read writes to my drive.. to I need to write with
>1024k block.. but the server is going to be used as database and not file
>sharing.  Any tweaks you can recommend.. I am using gpt for configure the
>drive..

Maybe you can write down all the details. The file /var/run/dmesg
should show the hardware configuration. E.g.:

nvme0 at pci1 dev 0 function 0: vendor 0x144d product 0xa802 (rev. 0x01)
nvme0: NVMe 1.1
nvme0: interrupting at ioapic0 pin 16
nvme0: Samsung SSD 950 PRO 512GB, firmware 1B0QBXX7, serial S2GMNXAH105150Z
ld0 at nvme0 nsid 1
ld0: 476 GB, 62260 cyl, 255 head, 63 sec, 512 bytes/sect x 1000215216 sectors
ld0: GPT GUID: 4f8e4d87-903c-422f-9520-2cabd2866689
dk0 at ld0: nvme-scratch

The gpt command shows the partitioning:

       start        size  index  contents
           0           1         PMBR
           1           1         Pri GPT header
           2          32         Pri GPT table
          34  1000215149      1  GPT part - NetBSD FFSv1/FFSv2
  1000215183          32         Sec GPT table
  1000215215           1         Sec GPT header

The mount command shows the filesystem:

/dev/dk0 on /mnt type ffs (local)

Reading from the raw devices yields:

>dd if=/dev/rdk0 of=/dev/null bs=1024k count=10000
10000+0 records in
10000+0 records out
10485760000 bytes transferred in 4.376 secs (2396197440 bytes/sec)


Your PCIe disk should show up like the above if it is a real nvme
device. You must run NetBSD-current, because the NVMe driver was
added recently and doesn't exist in NetBSD 7.

A SATA SSD shows up as a wd device instead. There are also PCIe SSDs that
aren't NVMe but instead a SATA controller and SATA SSD on a card. These
also show up as a wd device. Your numbers above suggest that this is what
you have.


There isn't much you can do for performance on SSD. You may see some
improvement if you align your partitions to a multiple of 512k or 1MB.
Avoiding WAPBL ("journaling") with FFS could also show some speedup
at the cost of additional fsck runs after a crash. SSDs also prefer
large blocks, formatting a filesystem with 64k blocks (the maximum)
can help at the cost of some wasted storage for small files.


I'm not sure what you mean with 'database' vs. 'file sharing'. If
you think about databases like postgres or mysql, these are using
files too.

-- 
-- 
                                Michael van Elst
Internet: mlelstv%serpens.de@localhost
                                "A potential Snark may lurk in every tree."