Minimig HDF disk transfer speeds

User avatar
limi
Top Contributor
Posts: 619
Joined: Sun May 24, 2020 6:53 pm
Has thanked: 135 times
Been thanked: 418 times

Minimig HDF disk transfer speeds

Unread post by limi »

Saw the update today to make the Minimig core use “fast I/O for disk transfer”.

It’s great to see improvements in this area — if you’re curious, it went from ~1.7MB/s to ~2.1MB/s in my testing using SysInfo on the Amiga core.

Disk transfer speed is one of the few things I wish was better in the Amiga implementation — but definitely don’t take this as a complaint!

I’m just curious what the upper limit with the current architecture would be. Are we stuck with ~2MB/s speeds for HDFs, or are there ways to make disk access be faster if someone was interested in working on it? Or is it a limitation of how the system works?
mahen
Posts: 185
Joined: Sun May 24, 2020 8:25 pm
Has thanked: 19 times
Been thanked: 6 times

Re: Minimig HDF disk transfer speeds

Unread post by mahen »

Thanks for the bench ! Any feeling of speedup in "real conditions" applications like booting / loading whdload games ?
User avatar
limi
Top Contributor
Posts: 619
Joined: Sun May 24, 2020 6:53 pm
Has thanked: 135 times
Been thanked: 418 times

Re: Minimig HDF disk transfer speeds

Unread post by limi »

As the numbers suggest, marginal. For gaming, the performance is adequate. For managing a lot of files (e.g. unpacking a lot of WHDLoad games, working with media), that’s where it matters.
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Minimig HDF disk transfer speeds

Unread post by bbond007 »

limi wrote: Tue Jun 09, 2020 6:20 am It’s great to see improvements in this area — if you’re curious, it went from ~1.7MB/s to ~2.1MB/s in my testing using SysInfo on the Amiga core.
You might try IDEFix97 as well. --> https://aminet.net/package/driver/media/IDEfix97

The main purpose of it is to allow the use of more than two HDF (or ISO) files simultaneously, however it does increase overall throughput as well..

It might be good for an additional 200KB - 250KB / sec.

I was having some freezing issues while using it a while back but have not tested it on the Minimig core in a very long time.
User avatar
limi
Top Contributor
Posts: 619
Joined: Sun May 24, 2020 6:53 pm
Has thanked: 135 times
Been thanked: 418 times

Re: Minimig HDF disk transfer speeds

Unread post by limi »

This was also an interesting read:

Amiga A500 Accelerator Boots from SD Card
Performance is currently reduced significantly due to missing multiple block support in the IDE emulation, but around about 1MB/s is still achievable with most cards and the missing support is next on my list to add. The bottleneck is mainly due to the single-block access time of the card rather than any transfer rate issue.
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Minimig HDF disk transfer speeds

Unread post by bbond007 »

According to SysInfo, IDEFix takes the performance from 2.1MB/sec to 2.4MB/sec.

Unfortunately it still causes frequent friezes where the GUI is responsive (mouse pointer moves, windows move/resize) but the hard disk access totally stops working. I notice this when loading larger whdload games or copying large files.

Having the ability to select 4 HDFs is not really useful without this driver being stable...
User avatar
limi
Top Contributor
Posts: 619
Joined: Sun May 24, 2020 6:53 pm
Has thanked: 135 times
Been thanked: 418 times

Re: Minimig HDF disk transfer speeds

Unread post by limi »

I believe AmigaOS 3.1.4 fixes all the things that IDEFix was intended to improve: https://www.hyperion-entertainment.com/ ... migaos-314
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Minimig HDF disk transfer speeds

Unread post by bbond007 »

limi wrote: Thu Jun 11, 2020 11:21 pm I believe AmigaOS 3.1.4 fixes all the things that IDEFix was intended to improve: https://www.hyperion-entertainment.com/ ... migaos-314
Thanks, I decided to give it a try:

AmigaOS 3.1.4 does indeed improve IDE performance over AmigaOS 3.1 by 200KB/sec, however I have not yet figured out if it is possible to use 4 IDE devices. It only seems to recognize the first two by default.
3.1.4vs3.1.png
3.1.4vs3.1.png (629.86 KiB) Viewed 7891 times
Unrelated, ShapeShifter (MAC Emulator) seems to work with 3.1.4 without "PrepareEmul" or similar patches which is nice...
kolla
Posts: 188
Joined: Sat Jun 13, 2020 7:56 am
Has thanked: 17 times
Been thanked: 33 times

Re: Minimig HDF disk transfer speeds

Unread post by kolla »

You can just include this in your custom 3.1.4 kickstart, or load it with LoadModule, and you can use 4 devices sans IDEFix.

http://amiga.kolla.no/AtapiMagic

Btw - Cheers Limi! :D
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Minimig HDF disk transfer speeds

Unread post by bbond007 »

kolla wrote: Sat Jun 13, 2020 9:57 pm You can just include this in your custom 3.1.4 kickstart, or load it with LoadModule, and you can use 4 devices sans IDEFix.

http://amiga.kolla.no/AtapiMagik
Thanks, but the link does not seem to work, 404?

However, it was enough to get me on the right track and is working well so far...

I put LoadModule in SYS:C from --> http://aminet.net/package/util/boot/LoadModule

I put AtapiMagik into LIBS:modules -->
AtapiMagic.zip
(292 Bytes) Downloaded 237 times
I changed this in SYS:S/startup-sequence

Code: Select all

If Exists C:LoadModule
   C:Version exec.library version 46 >NIL:
   If Warn
      C:LoadModule ROMUPDATE
   EndIf
EndIf
to:

Code: Select all

If Exists C:LoadModule
   C:Version exec.library version 46 >NIL:
   If Warn
      C:LoadModule LIBS:modules/AtapiMagic ROMUPDATE
   Else
      C:LoadModule LIBS:modules/AtapiMagic
   EndIf
EndIf

Does that seem correct?

I guess the next step it to make a custom Kickstart ROM so I don't incur additional boot time due to LoadModule resetting.

I vaguely remember doing that a long time ago for my A1200 for use with BlizKick :)

Thanks for the info!
kolla
Posts: 188
Joined: Sat Jun 13, 2020 7:56 am
Has thanked: 17 times
Been thanked: 33 times

Re: Minimig HDF disk transfer speeds

Unread post by kolla »

Oops, I wrote k in stead of c - the name confusion of this module is that it really is called "ATAPIismajik" :)

I think your changes to S-S are correct (the atapimagic module works with any scsi.device since OS3.5), and yes - building a custom kickstart is often easier than messing around with LoadModule and on-disk files :)

As for access speed on MiSTer... in my experience it varies a lot, and most of all relies on the MiSTer firmware. If people really want higher speeds it can be achieved by building a dedicated "scsi.device clone" into the MiSTer "firmware", that talks with the host OS more directly and skip emulation of "gayle", just like UAE does with "uaehf.device".
User avatar
limi
Top Contributor
Posts: 619
Joined: Sun May 24, 2020 6:53 pm
Has thanked: 135 times
Been thanked: 418 times

Re: Minimig HDF disk transfer speeds

Unread post by limi »

I can confirm (that is, if you trust SysInfo) that larger HDFs have better transfer speeds than smaller ones.

On a 2GB image, I get ~2.1MB/s (as mentioned in the beginning of this thread, with a 6GB image, I get ~6.1GB, similar to what Solskogen posted about recently.
chaos wrote: Sun Aug 23, 2020 10:14 am Maybe if disk speed is of concern, a better option would be to implement direct SD card access in the FPGA. Actually, that is something I'll put on my TODO list ...
Would love to hear more about this, Chaos 😊
User avatar
Sorgelig
Site Admin
Posts: 877
Joined: Thu May 21, 2020 9:49 pm
Has thanked: 2 times
Been thanked: 211 times

Re: Minimig HDF disk transfer speeds

Unread post by Sorgelig »

Access speed can be improved by moving data path from traditional HPS-FPGA channel to DDR3 shared memory. But before that the IDE part in Minimig core needs to be greatly cleaned and moved out of chipset 7MHz constraint.

Honestly, i don't see a big problem in current 2.1MB transfer speed. It's pretty much fast for Amiga times.
ZigZag
Posts: 99
Joined: Sun Jun 28, 2020 7:05 am
Has thanked: 1 time
Been thanked: 5 times

Re: Minimig HDF disk transfer speeds

Unread post by ZigZag »

To give a bit context of computer speeds in days of yore, here's a little true story. Now boys & girls, settle down & listen...

In 1996 I was going to uni & realised I'd have limited space in my small room in halls of residence. I had an unexpanded A1200 with a 420mb PC HDD jammed in it. So I decided to take my 500+ floppy disk collection & make ADF images, then compress them into one huge (for the time, astronomical) LZH archive. That way I could take 20 floppies & just write out whatever images were required. This task took me weeks of copying using both floppy drives. Once I had all the images I started the compression. This was just days before leaving home. I needed to free as much space as possible so used maximum compression & it really became a race against time. Back in those days, creating such a large archive was ambitious lunacy - it took the Amiga days of uninterrupted crunching to complete the task. I was crapping myself that there might be a crash or a power outage. In the end, it worked brilliantly. I'd cleared the drive for working space, and just reinstalled the HDD software like Dpaint & Wordsworth after the archive was created. That archive is still on that HDD, though it's been in a box for 20 years or so. If it still works I'd love to get it & all my other data off the HDD. I'm aware that isn't exactly a story about HDD speed, as the archiving was limited mostly by the 2mb & CPU speed. However, the point is back then patience was a vital virtue. You can't expect blistering speed from old h/w.
apolkosnik
Posts: 24
Joined: Wed Jun 17, 2020 6:04 am

Re: Minimig HDF disk transfer speeds

Unread post by apolkosnik »

Weird, Sysinfo reports like 6MB/s for some partitions, and 2MB/s for another on the same hdf.
User avatar
limi
Top Contributor
Posts: 619
Joined: Sun May 24, 2020 6:53 pm
Has thanked: 135 times
Been thanked: 418 times

Re: Minimig HDF disk transfer speeds

Unread post by limi »

With the latest Minimig and Main core updates to add the FastIDE option, the transfer speeds have gone from ~2.1MB/s to ~4.8M/s. Nice!
zoopster
Posts: 78
Joined: Sun May 24, 2020 8:08 pm
Has thanked: 79 times
Been thanked: 15 times

Re: Minimig HDF disk transfer speeds

Unread post by zoopster »

@limi does any additional software need to be installed?
RetroP
Posts: 29
Joined: Sun May 24, 2020 8:55 pm
Has thanked: 1 time
Been thanked: 2 times

Re: Minimig HDF disk transfer speeds

Unread post by RetroP »

I have 3.1.4 and get 6.5MB/sec both before and after the update.
User avatar
limi
Top Contributor
Posts: 619
Joined: Sun May 24, 2020 6:53 pm
Has thanked: 135 times
Been thanked: 418 times

Re: Minimig HDF disk transfer speeds

Unread post by limi »

No special software, but check that Fast IDE is enabled under the Drives menu (it was by default for me). Directories load faster, so I’m pretty sure it’s not just placebo. :)

Larger drives seemed to have higher transfer speeds in the past, but could also be a bug in e.g. SysInfo.
User avatar
Sorgelig
Site Admin
Posts: 877
Joined: Thu May 21, 2020 9:49 pm
Has thanked: 2 times
Been thanked: 211 times

Re: Minimig HDF disk transfer speeds

Unread post by Sorgelig »

6.5MB/s instantly appear after pressing the button is bug of SysInfo (it also happen on real HW with fast IDE).
Real speed in Sysinfo now is ~4.9MB/s
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Minimig HDF disk transfer speeds

Unread post by bbond007 »

Sorgelig wrote: Tue Jul 06, 2021 4:23 pm 6.5MB/s instantly appear after pressing the button is bug of SysInfo (it also happen on real HW with fast IDE).
Real speed in Sysinfo now is ~4.9MB/s
The speed HD speed increase is welcome...

This makes HD on par with my A1200/060 with ElBox FastATA...

Benchmarking the dive speed under ShapeShifter shows similar increase to SysInfo (which often has unreliable results as you pointed out)

Thanks!
Post Reply