nfhd performance

nfhd performance

[Geert Uytterhoeven]

So far I never really used nfhd.

In a kernel that has both IDE and nfhd support, once everything is in the
buffer cache, I get:

atari:~# dd if=/dev/hda2 bs=1M of=/dev/null
134+1 records in
134+1 records out
141114880 bytes (141 MB) copied, 5.91 seconds, 23.9 MB/s
atari:~# dd if=/dev/nfhd8p2 bs=1M of=/dev/null
134+1 records in
134+1 records out
141114880 bytes (141 MB) copied, 17.09 seconds, 8.3 MB/s

So nfhd is slower than emulated IDE?

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: nfhd performance

[Michael Schmitz]

Geert,

> So far I never really used nfhd.

On that matter - I've used it once to good effect. My 160 GB disk was 
properly recognized by nfhd but not by IDE emulation (Falcon disk 
hooked up to PC via USB adapter). IDE appears to truncate the disk size 
(number of cylinders I presume) in my rather old ARAnyM version.

(had to use ARAnyM to revive the bootstrap on my second HD after the 
primary one died on me)

>
> In a kernel that has both IDE and nfhd support, once everything is in 
> the
> buffer cache, I get:
>
> atari:~# dd if=/dev/hda2 bs=1M of=/dev/null
> 134+1 records in
> 134+1 records out
> 141114880 bytes (141 MB) copied, 5.91 seconds, 23.9 MB/s
> atari:~# dd if=/dev/nfhd8p2 bs=1M of=/dev/null
> 134+1 records in
> 134+1 records out
> 141114880 bytes (141 MB) copied, 17.09 seconds, 8.3 MB/s
>
> So nfhd is slower than emulated IDE?

That seems odd indeed ... is the disk attached as IDE or via USB 
adapter? (Checking to see whether I should get similar results)

Cheers,

	Michael


>
> Gr{oetje,eeting}s,
>
>                         Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- 
> geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a 
> hacker. But
> when I'm talking to journalists I just say "programmer" or something 
> like that.
>                                 -- Linus Torvalds
> --
> To unsubscribe from this list: send the line "unsubscribe linux-m68k" 
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nfhd performance

[Geert Uytterhoeven]

On Sat, Aug 17, 2013 at 4:01 AM, Michael Schmitz
<schmitz@biophys.uni-duesseldorf.de> wrote:
>> In a kernel that has both IDE and nfhd support, once everything is in the
>> buffer cache, I get:
>>
>> atari:~# dd if=/dev/hda2 bs=1M of=/dev/null
>> 134+1 records in
>> 134+1 records out
>> 141114880 bytes (141 MB) copied, 5.91 seconds, 23.9 MB/s
>> atari:~# dd if=/dev/nfhd8p2 bs=1M of=/dev/null
>> 134+1 records in
>> 134+1 records out
>> 141114880 bytes (141 MB) copied, 17.09 seconds, 8.3 MB/s
>>
>> So nfhd is slower than emulated IDE?
>
>
> That seems odd indeed ... is the disk attached as IDE or via USB adapter?
> (Checking to see whether I should get similar results)

It's not a physical disk:

[IDE0]
Present = Yes
IsCDROM = No
ByteSwap = No
ReadOnly = No
Path = /scratch/geert/aranym/etch-m68k.img
Cylinders = 2102
Heads = 16
SectorsPerTrack = 63
ModelName = Sarge m68k

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: nfhd performance

[Petr Stehlik]

Geert Uytterhoeven píše v Pá 16. 08. 2013 v 23:57 +0200:
> 141114880 bytes (141 MB) copied, 17.09 seconds, 8.3 MB/s
> So nfhd is slower than emulated IDE?

nfhd is something like IDE with DMA (if everything is in buffer cache
then it's doing basically just a memcpy() from cache to the destination
place) - can't believe it would be that slow.

Don't have profiling numbers at hand but one of the main purposes of
nfhd was to be way way faster than the emulated IDE. Something must be
seriously wrong somewhere.

Petr

Re: nfhd performance

[Mikael Pettersson]

Geert Uytterhoeven writes:
 > On Sat, Aug 17, 2013 at 4:01 AM, Michael Schmitz
 > <schmitz@biophys.uni-duesseldorf.de> wrote:
 > >> In a kernel that has both IDE and nfhd support, once everything is in the
 > >> buffer cache, I get:
 > >>
 > >> atari:~# dd if=/dev/hda2 bs=1M of=/dev/null
 > >> 134+1 records in
 > >> 134+1 records out
 > >> 141114880 bytes (141 MB) copied, 5.91 seconds, 23.9 MB/s
 > >> atari:~# dd if=/dev/nfhd8p2 bs=1M of=/dev/null
 > >> 134+1 records in
 > >> 134+1 records out
 > >> 141114880 bytes (141 MB) copied, 17.09 seconds, 8.3 MB/s
 > >>
 > >> So nfhd is slower than emulated IDE?
 > >
 > >
 > > That seems odd indeed ... is the disk attached as IDE or via USB adapter?
 > > (Checking to see whether I should get similar results)
 > 
 > It's not a physical disk:
 > 
 > [IDE0]
 > Present = Yes
 > IsCDROM = No
 > ByteSwap = No
 > ReadOnly = No
 > Path = /scratch/geert/aranym/etch-m68k.img
 > Cylinders = 2102
 > Heads = 16
 > SectorsPerTrack = 63
 > ModelName = Sarge m68k

Part of the problem is that ByteSwap = No setting.  Contrary to what one might
think, "No" there actually means "yes, byteswap every sector read or written".

Grep for byteswap in aranym's src/natfeat/xhdi.cpp if you don't believe me...

As far as I can tell, this setting is only meaningful if you have an image which
must be shared with actual HW, and you don't want to byteswap it when migrating
it between these two roles (HW or aranym).  I don't have that requirement, so I
run my aranym VMs with ByteSwap = Yes, which eliminates that overhead.

There are also other things in xhdi.cpp I don't really like, such as going through
stdio when raw Unix file-descriptors would do, and the small I/O unit size.

Re: nfhd performance

[Petr Stehlik]

> Part of the problem is that ByteSwap = No setting.  Contrary to what one might
> think, "No" there actually means "yes, byteswap every sector read or written".
> 
> Grep for byteswap in aranym's src/natfeat/xhdi.cpp if you don't believe me...
> 
> As far as I can tell, this setting is only meaningful if you have an image which
> must be shared with actual HW, and you don't want to byteswap it when migrating
> it between these two roles (HW or aranym).  I don't have that requirement, so I
> run my aranym VMs with ByteSwap = Yes, which eliminates that overhead.

Very true.

> There are also other things in xhdi.cpp I don't really like, such as going through
> stdio when raw Unix file-descriptors would do, and the small I/O unit size.

Please suggest a better yet portable way. I've been thinking about
reading more than a single sector at once, that could help a bit. Should
write a benchmark for it first, it seems.

Petr

Re: nfhd performance

[Petr Stehlik]

Geert Uytterhoeven píše v Pá 16. 08. 2013 v 23:57 +0200:

> In a kernel that has both IDE and nfhd support, once everything is in the
> buffer cache, I get:
> 
> atari:~# dd if=/dev/hda2 bs=1M of=/dev/null
> 141114880 bytes (141 MB) copied, 5.91 seconds, 23.9 MB/s
> atari:~# dd if=/dev/nfhd8p2 bs=1M of=/dev/null
> 141114880 bytes (141 MB) copied, 17.09 seconds, 8.3 MB/s

FYI, I have just tried running XFERRATE.TTP (freeware by
AnodyneSoftware.com) under plain TOS with clock driven by RTC and disk
image without byteswap (it's still my original 1000 MB disk from Atari
Falcon). I got the following results with a small 30 MB partition that
most probably fully fit into host buffer cache:

Rwabs() transfer rate: 26660-26220 kb/sec
XHDI transfer rate: 26750 kb/sec

Please note that the Rwabs is a direct BIOS access. I think it's rather
close if not equivalent to your 'dd if=/dev/hda2'.

It basically means that since the XHDI path is slightly shorter code
path so the data transfer is very slightly faster. That's actually what
one could expect.

I will try more tests, especially with ByteSwap in ARAnyM enabled (to
see if there's a performance penalty). Will also try changing fread to
read and implement reading more sectors at once in the XHDI NatFeat
implementation. Will post my results here.

In summary, the slow nfhd above seems to be caused by the linux-m68k
implementation as I can't see such slow down under plain TOS.

Petr

Re: nfhd performance

[Petr Stehlik]

Mikael Pettersson píše v Po 19. 08. 2013 v 16:29 +0200:
> There are also other things in xhdi.cpp I don't really like, such as going through
> stdio when raw Unix file-descriptors would do

I think I recall that we did that for portability of LARGE FILES (64bit
offsets) at that time (think 10 years ago, various linux distros, MS
Windows, Mac OS, BeOS, various BSD Unices...). Perhaps the situation is
better (file descriptors with 64bit offsets are more portable) now?

Anyway, I have just converted the XHDI to using raw Unix file
descriptors and guess what - there's NO measurable difference (at least
using the previously mentioned XFERRATE.TTP).

Unless someone has a good reason for committing the Unix file
descriptors I'll throw it away and return to good old stdio.

Petr