Re: Very slow transfers to/from micro SD card on a RPi B+

[Stephan <stephanwib%googlemail.com@localhost>]

While I was already sure that changing the I/O scheduler from disksort
to fcfs on the Pi increases sequential write speed from nearly 10MB/s
to 15 MB/s on average, it turned out that´s not true.

In fact, the writes speed varies for no recognizable reason. An
example of a "good" case:

rpi/root> dd if=/dev/zero of=myfile bs=4k count=10000
10000+0 records in
10000+0 records out
40960000 bytes transferred in 2.750 secs (14894545 bytes/sec)

      tty              ld0             CPU
 tin tout  KB/t  t/s  MB/s  us ni sy in id

   5   49 18.97   18 0.330   0  0  0  0 100
   4   49 0.000    0 0.000   0  0  0  1 99
   2   46 0.000    0 0.000   0  0  0  0 100
   1   48 55.83   12 0.648   0  0  2  1 97
   0   44 63.55  212 13.15   0  0  8  0 92
   0   44 64.00  194 12.13   0  0  6  1 93
   0  156 63.54  206 12.78   0  0  5  0 95
   0   44 0.000    0 0.000   0  0  0  0 100
   0   45 0.000    0 0.000   0  0  0  1 99

This is an example of a "bad" case:

rpi/root> dd if=/dev/zero of=myfile bs=4k count=10000
10000+0 records in
10000+0 records out
40960000 bytes transferred in 4.718 secs (8681644 bytes/sec)

  1   46 0.000    0 0.000   0  0  0  1 99
   6  266 0.000    0 0.000   0  0  1  0 99
   8  148 0.000    0 0.000   0  0  2  1 97
   1   47 63.54   69 4.301   0  0  5  0 95
   0   44 64.00  234 14.60   0  0  7  2 90
   0   44 62.60  142 8.656   0  0  3  0 97
   0   44 0.000    0 0.000   0  0  0  0 100
   0  156 58.02  185 10.49   0  0  2 11 86
   0   44 64.00   18 1.114   0  0  0  0 100
   0   45 0.000    0 0.000   0  0  0  1 99
   0   44 0.000    0 0.000   0  0  0  0 100


There is even one second in the process without a single xfer. Can
this be caused by the SD card itself, or may there be something
improvable in the kernel?



2015-08-25 15:05 GMT+02:00 Stephan <stephanwib%googlemail.com@localhost>:
> Nobody having an idea about DIRHASH? I´ve also seen that tmpfs has the
> same regression.
>
>
> I corrected 2 minor flaws in my tool, if it´s useful to somebody:
>
>
> ------8<---------------------------------
>
> #include <fcntl.h>
> #include <stdlib.h>
> #include <stdio.h>
> #include <sys/time.h>
>
> int main(int argc, char **argv) {
>   struct timeval tv_start, tv_finish;
>   unsigned long duration;
>   int i;
>   char filename[16];
>
>   if (argc < 2) { printf("Usage: %s <count>\n", argv[0]); exit(1); }
>   int howmany = atoi(argv[1]);
>
>   for (i = 0; i < howmany; i++)
>   {
>     snprintf(filename, sizeof(filename), "%d", i);
>     gettimeofday(&tv_start, NULL);
>     int fd = open(filename, O_CREAT);
>     gettimeofday(&tv_finish, NULL);
>     close(fd);
>     duration = (tv_finish.tv_sec * 1000000 + tv_finish.tv_usec) -
> (tv_start.tv_sec * 1000000 + tv_start.tv_usec);
>     printf("Duration: %lu\n", duration);
>   }
>
>   return 0;
> }
>
> 2015-08-20 18:16 GMT+02:00 Stephan <stephanwib%googlemail.com@localhost>:
>> 2015-08-19 15:51 GMT+02:00 Thor Lancelot Simon <tls%panix.com@localhost>:
>>> We should be shipping with DIRHASH turned on.
>>
>>
>> That´s it - top speed!
>>
>> /root/test/files> time seq 1 30000|xargs touch
>>     1.05s real     0.03s user     0.84s system
>>
>> Is DIRHASH still considered experimental? It fixes one of NetBSD´s
>> most annoying shortcomings.
>>
>> FreeBSD received dynamic memory allocation for DIRHASH which might be
>> even better.
>>
>>
>>
>> (re-sending this as my editor intermittently distributes HTML mails)
>>
>>
>>>
>>> On Tue, Aug 18, 2015 at 04:19:57PM +0000, Michael van Elst wrote:
>>>> stephanwib%googlemail.com@localhost (Stephan) writes:
>>>>
>>>> >This is from an amd64 server box with an aac raid controller.
>>>>
>>>> >/root/test/files> time seq 1 10000|xargs touch
>>>> >    3.10s real     0.01s user     3.07s system
>>>> >/root/test/files> rm *
>>>> >/root/test/files> time seq 1 20000|xargs touch
>>>> >    9.88s real     0.01s user     8.51s system
>>>> >/root/test/files> rm *
>>>> >/root/test/files> time seq 1 30000|xargs touch
>>>> >   23.45s real     0.04s user    20.41s system
>>>> >/root/test/files> time seq 1 40000|xargs touch
>>>> >   43.35s real     0.05s user    38.32s system
>>>>
>>>> >That is clearly not linear.
>>>>
>>>> Each lookup is linear (unless you use DIRHASH), so the total
>>>> time goes with n^2.
>>>>
>>>> --
>>>> --
>>>>                                 Michael van Elst
>>>> Internet: mlelstv%serpens.de@localhost
>>>>                                 "A potential Snark may lurk in every tree."
>>>
>>> --
>>>   Thor Lancelot Simon                                        tls%panix.com@localhost
>>>
>>>   "We cannot usually in social life pursue a single value or a single moral
>>>    aim, untroubled by the need to compromise with others."      - H.L.A. Hart