Why is rom loading so slow on MiSTer?

For topics which do not fit in other specific forums.
AtomicShroom
Posts: 170
Joined: Sun Mar 07, 2021 12:28 pm
Has thanked: 31 times
Been thanked: 48 times

Why is rom loading so slow on MiSTer?

Unread post by AtomicShroom »

I was wondering: Why is rom loading so slow on MiSTer?

Is there a bottleneck somewhere that is causing this? The SD Card I'm using is rated as rather fast (120MB/s), yet loading a 4MB SNES rom like Donkey Kong Country takes a good 3 seconds. (Which implies it's reading at 1.3MB/s). I mean, what the actual fuck? How can something in 2021 only move data at 1.3MB/s? So what is causing this slowness? Is the SD Card slot severely speed limited? Is it the OS itself? Is it the limited ARM resources? Is it something else? Just where is all that slowness coming from?

Thanks!
akeley
Top Contributor
Posts: 1303
Joined: Mon May 25, 2020 7:54 pm
Has thanked: 416 times
Been thanked: 399 times

Re: Why is rom loading so slow on MiSTer?

Unread post by akeley »

For me 3 seconds is blindingly fast :) I must be old or something....hailing from the tape/floppy era.

Seriously though, even if there is a bottleneck that could be eliminated and the speed improved, I wouldn't probably even notice.
AtomicShroom
Posts: 170
Joined: Sun Mar 07, 2021 12:28 pm
Has thanked: 31 times
Been thanked: 48 times

Re: Why is rom loading so slow on MiSTer?

Unread post by AtomicShroom »

I mean, a 1.3MB/s read speed would be pathetic even by 2005 standards. Even the first ever SD Cards had a read speed of 12.5MB/sec.
MostroW
Posts: 323
Joined: Tue Aug 18, 2020 3:32 pm
Has thanked: 140 times
Been thanked: 43 times

Re: Why is rom loading so slow on MiSTer?

Unread post by MostroW »

I think the limiting factor here are the GPIO pins used to transfer rom contents into RAM, not sure how much MB/s the MiSTer can pump through it, but i remember seeing Pi users testing it for a maximum of 1.9MB/s of data transfer speed.
Duffygag
Posts: 95
Joined: Tue Nov 03, 2020 10:57 am
Has thanked: 12 times
Been thanked: 25 times

Re: Why is rom loading so slow on MiSTer?

Unread post by Duffygag »

Who will give those time wasted back, whoo?!?! /s
User avatar
Aspie
Posts: 186
Joined: Sat Sep 26, 2020 1:07 pm
Location: Manchester
Has thanked: 20 times
Been thanked: 46 times
Contact:

Re: Why is rom loading so slow on MiSTer?

Unread post by Aspie »

Obviously you've never experienced an original ZX Spectrum or Commodore 64 and the joy of waiting 5 minutes for the tape to load, and then it crashes
User avatar
darksakul
Posts: 352
Joined: Mon May 25, 2020 4:34 pm
Has thanked: 397 times
Been thanked: 73 times

Re: Why is rom loading so slow on MiSTer?

Unread post by darksakul »

Also there the false equivalence of comparing the MiSTer to other emulation devices.
Yes a Raspberry Pi loads roms faster, but the entire inner workings of Emulation on the Raspberry Pi is different than FPGA "Emulation".
They are not was nor ever a 1 to 1 comparison of the inner workings.

I will mention the MiSTer boots up faster and we get the main menu faster on the MiSTer than we would with a Raspberry Pi running Retro Arch with a fancy front end. There also less controller input and video latency than with a Rasp Pi or PC emulation
Duffygag wrote: Fri Sep 03, 2021 2:52 pm Who will give those time wasted back, whoo?!?! /s
All the "wasted" time you spend playing video games?
akeley wrote: Fri Sep 03, 2021 2:24 pm For me 3 seconds is blindingly fast :) I must be old or something....hailing from the tape/floppy era.

Seriously though, even if there is a bottleneck that could be eliminated and the speed improved, I wouldn't probably even notice.
The old Microcomputer cassette tapes, you can go set a pot of coffee or tea to boil, prepare your cup sit down and enjoy your drink with those loading times.

I just now booted up my MiSTer and counted in my head how long from boot up till I got Donkey Kong Country running on the SNES Core.
I spent more time searching though the Smoke Monster pack looking for the game than it I spent loading the rom.
Counting in my head (by saying Mississippi) Donkey Kong Country loaded less than a seccond.

But I am also loading my roms off a 1TB Samsung SSD connected to a SATA to USB adapter going into the MiSTer USB Hub.
Maybe external/secondary storage is quicker?

My Mame roms exist on my SD card, and it took an almost 2 seconds to load X-Men children of the Atom on the JT CPS2 Core.
I take it that the X-Men CPS2 Rom is much bigger than a SNES Cart Rom.
Only rom that took me almost 3 seconds is the Neo Geo Strikers 1945 rom, which I think is at the upper reaches how big a Neo geo rom gets
It is my great regret that we live in an age that is proud of machines that think and suspicious of people who try to.
User avatar
darksakul
Posts: 352
Joined: Mon May 25, 2020 4:34 pm
Has thanked: 397 times
Been thanked: 73 times

Re: Why is rom loading so slow on MiSTer?

Unread post by darksakul »

I want to add certain Micro SD cards load faster than others

Generic Micro SD cards, the free pack in Micro SD card that comes with the DE 10 Nano, and a few others are slower than standard Micro SD card speeds. This latency in slower cards compound with the SD card read latency in the DE 10 Nano.

Certain brands, models of Cards, ect do load faster than others, regardless of the rated speed "written on paper".
It is my great regret that we live in an age that is proud of machines that think and suspicious of people who try to.
User avatar
Chris23235
Top Contributor
Posts: 841
Joined: Sun May 24, 2020 8:45 pm
Has thanked: 106 times
Been thanked: 167 times

Re: Why is rom loading so slow on MiSTer?

Unread post by Chris23235 »

The bottle neck is the SD Ram, 133MHz SD Ram has a transfer rate of about 1 MBps.
User avatar
lamarax
Top Contributor
Posts: 472
Joined: Wed Nov 11, 2020 6:28 pm
Has thanked: 33 times
Been thanked: 199 times

Re: Why is rom loading so slow on MiSTer?

Unread post by lamarax »

Aspie wrote: Fri Sep 03, 2021 3:32 pm Obviously you've never experienced an original ZX Spectrum or Commodore 64 and the joy of waiting 5 minutes for the tape to load, and then it crashes
Make that 8 minutes for Elite to load on a BBC B from cassette (not counting the save file) :mrgreen:

*in fact, several microcomputer cores have the 'feature' of virtually (or even actually -with the ADC board) loading from tape implemented, complete with screeching sound and waiting times. Of course no genius could ever be able to truly simulate crumpled tape, misaligned recorder heads and most of all, mismatch between volume and baud settings :lol:
Blitzwing
Posts: 103
Joined: Sat Sep 05, 2020 9:52 pm
Has thanked: 11 times
Been thanked: 24 times

Re: Why is rom loading so slow on MiSTer?

Unread post by Blitzwing »

darksakul wrote: Fri Sep 03, 2021 3:51 pm Also there the false equivalence of comparing the MiSTer to other emulation devices.
Yes a Raspberry Pi loads roms faster, but the entire inner workings of Emulation on the Raspberry Pi is different than FPGA "Emulation".
They are not was nor ever a 1 to 1 comparison of the inner workings.

I will mention the MiSTer boots up faster and we get the main menu faster on the MiSTer than we would with a Raspberry Pi running Retro Arch with a fancy front end. There also less controller input and video latency than with a Rasp Pi or PC emulation
Duffygag wrote: Fri Sep 03, 2021 2:52 pm Who will give those time wasted back, whoo?!?! /s
All the "wasted" time you spend playing video games?
akeley wrote: Fri Sep 03, 2021 2:24 pm For me 3 seconds is blindingly fast :) I must be old or something....hailing from the tape/floppy era.

Seriously though, even if there is a bottleneck that could be eliminated and the speed improved, I wouldn't probably even notice.
The old Microcomputer cassette tapes, you can go set a pot of coffee or tea to boil, prepare your cup sit down and enjoy your drink with those loading times.

I just now booted up my MiSTer and counted in my head how long from boot up till I got Donkey Kong Country running on the SNES Core.
I spent more time searching though the Smoke Monster pack looking for the game than it I spent loading the rom.
Counting in my head (by saying Mississippi) Donkey Kong Country loaded less than a seccond.

But I am also loading my roms off a 1TB Samsung SSD connected to a SATA to USB adapter going into the MiSTer USB Hub.
Maybe external/secondary storage is quicker?

My Mame roms exist on my SD card, and it took an almost 2 seconds to load X-Men children of the Atom on the JT CPS2 Core.
I take it that the X-Men CPS2 Rom is much bigger than a SNES Cart Rom.
Only rom that took me almost 3 seconds is the Neo Geo Strikers 1945 rom, which I think is at the upper reaches how big a Neo geo rom gets
As somebody whose first experience with a computer was my dad's Acorn Electron, where loading a game could sometimes take upwards of ten minutes I'm not sure what OP is expecting, kids today 😁
User avatar
lamarax
Top Contributor
Posts: 472
Joined: Wed Nov 11, 2020 6:28 pm
Has thanked: 33 times
Been thanked: 199 times

Re: Why is rom loading so slow on MiSTer?

Unread post by lamarax »

Fortunately, technology has advanced. Here's a solution to your slow and unreliable tape loading problems:
Sinclair_ZX_Microdrive_White_BG.jpg
Sinclair_ZX_Microdrive_White_BG.jpg (315.78 KiB) Viewed 4682 times
There was something akin to that for the Acorn micros called the "Hobbit" I think, but I've failed to come up with any Google result.
akeley
Top Contributor
Posts: 1303
Joined: Mon May 25, 2020 7:54 pm
Has thanked: 416 times
Been thanked: 399 times

Re: Why is rom loading so slow on MiSTer?

Unread post by akeley »

"Microdrive" sounded and looked great. When you discovered that it was actually a tiny tape, not so much :)

I used to think that tapes/decks are unreliable (in my ZX/C64 days), but now I know it was mostly because all the stuff I had was pirated and so recorded and recorded on crappy (or reused) tapes and very low quality decks. I got a little collection of originals together recently, plus a decent Sony deck, and must say that these ~40 year old tapes load 9/10 times no problem.
User avatar
darksakul
Posts: 352
Joined: Mon May 25, 2020 4:34 pm
Has thanked: 397 times
Been thanked: 73 times

Re: Why is rom loading so slow on MiSTer?

Unread post by darksakul »

But for real here your reasons your SD card could be going slow.

1. DE 10 Nano has a slow(ish) read/write speed

2. Your brand/model SD card (you never get the specs on paper anyways).
Also the following
A. Card Size will effect load times
B. Various specs Speed Class, UHS, Ect.
C. File Size

3. Way the FPGA loads data onto the ram, total ram speed pretty low.
To get the machine level timing down the MiSTer the MiSTer team went with SD Ram which it's total speed is 166 Mhz
If anyone can chime in and state whats the speed the MiSTer is running the SD ram at?

While my test methods is no way scientific, and is subject to a host of variables as well as it being anecdotal.
It is my great regret that we live in an age that is proud of machines that think and suspicious of people who try to.
User avatar
lamarax
Top Contributor
Posts: 472
Joined: Wed Nov 11, 2020 6:28 pm
Has thanked: 33 times
Been thanked: 199 times

Re: Why is rom loading so slow on MiSTer?

Unread post by lamarax »

akeley wrote: Fri Sep 03, 2021 7:02 pm [...]but now I know it was mostly because all the stuff I had was pirated and so recorded and recorded on crappy (or reused) tapes and very low quality decks.
How do you mean? Are you insinuating that those of us who happened to have double-deckers were into pirating swapping stuff? Outrageous! I used to place legit adds in magazines, in order to build genuine user groups!

*Fun fact: in my 3rd world country back then, even the official dealers used to sell badly copied software
User avatar
macro
Core Developer
Posts: 138
Joined: Sun May 24, 2020 4:12 pm
Been thanked: 171 times

Re: Why is rom loading so slow on MiSTer?

Unread post by macro »

darksakul wrote: Fri Sep 03, 2021 7:18 pm To get the machine level timing down the MiSTer the MiSTer team went with SD Ram which it's total speed is 166 Mhz
loading rom images and other data is usually handled by the core itself and runs at system clock speed, which varies (the main core passes the data to the FPGA core and that usually stores it somewhere - at least that is what happens in all of the arcade cores!)

see https://github.com/MiSTer-devel/Main_Mi ... -use-SDRAM for list of cores and what speed they clock SDRAM at (which is usually quicker than the system clock anyway)
Did I do something useful?

buy me a coffee
User avatar
jimmystones
Core Developer
Posts: 216
Joined: Sun Nov 22, 2020 1:26 pm
Location: Reading, UK
Has thanked: 32 times
Been thanked: 248 times
Contact:

Re: Why is rom loading so slow on MiSTer?

Unread post by jimmystones »

There are some cores which have a faster loading system which uses the shared DDRAM to speed up the process, but as it needs extra logic it's only really in the cores that have big ROMs that would take a really long time to load. GBA and CPS2 I think both use the technique - some CPS2 ROMs were getting on for 30 seconds to load before the fast loader was implemented!
User avatar
bazza_12
Top Contributor
Posts: 404
Joined: Sun May 24, 2020 7:49 pm
Location: Yorkshire, UK
Has thanked: 247 times
Been thanked: 112 times
Contact:

Re: Why is rom loading so slow on MiSTer?

Unread post by bazza_12 »

Aspie wrote: Fri Sep 03, 2021 3:32 pm Obviously you've never experienced an original ZX Spectrum or Commodore 64 and the joy of waiting 5 minutes for the tape to load, and then it crashes
then you'd get to the end of the loader and it would crash.. ahh good old days.. if I'd told my younger self games in the future would load in 3 seconds I wouldn't believe it
The music is reversible but time is not. Turn back. Turn back
User avatar
bazza_12
Top Contributor
Posts: 404
Joined: Sun May 24, 2020 7:49 pm
Location: Yorkshire, UK
Has thanked: 247 times
Been thanked: 112 times
Contact:

Re: Why is rom loading so slow on MiSTer?

Unread post by bazza_12 »

lamarax wrote: Fri Sep 03, 2021 7:28 pm
akeley wrote: Fri Sep 03, 2021 7:02 pm [...]but now I know it was mostly because all the stuff I had was pirated and so recorded and recorded on crappy (or reused) tapes and very low quality decks.
How do you mean? Are you insinuating that those of us who happened to have double-deckers were into pirating swapping stuff? Outrageous! I used to place legit adds in magazines, in order to build genuine user groups!

*Fun fact: in my 3rd world country back then, even the official dealers used to sell badly copied software
you don't know you're born til you've taped all your spectrum games on a c120 and marked down the number from the tape counter where each game begins :)
The music is reversible but time is not. Turn back. Turn back
User avatar
Chris23235
Top Contributor
Posts: 841
Joined: Sun May 24, 2020 8:45 pm
Has thanked: 106 times
Been thanked: 167 times

Re: Why is rom loading so slow on MiSTer?

Unread post by Chris23235 »

- deleted - didn't read the post I replied to close enough ;)
User avatar
darksakul
Posts: 352
Joined: Mon May 25, 2020 4:34 pm
Has thanked: 397 times
Been thanked: 73 times

Re: Why is rom loading so slow on MiSTer?

Unread post by darksakul »

macro wrote: Fri Sep 03, 2021 8:14 pm
darksakul wrote: Fri Sep 03, 2021 7:18 pm To get the machine level timing down the MiSTer the MiSTer team went with SD Ram which it's total speed is 166 Mhz
loading rom images and other data is usually handled by the core itself and runs at system clock speed, which varies (the main core passes the data to the FPGA core and that usually stores it somewhere - at least that is what happens in all of the arcade cores!)

see https://github.com/MiSTer-devel/Main_Mi ... -use-SDRAM for list of cores and what speed they clock SDRAM at (which is usually quicker than the system clock anyway)
Thanks for clearing that up.
It is my great regret that we live in an age that is proud of machines that think and suspicious of people who try to.
MostroW
Posts: 323
Joined: Tue Aug 18, 2020 3:32 pm
Has thanked: 140 times
Been thanked: 43 times

Re: Why is rom loading so slow on MiSTer?

Unread post by MostroW »

jimmystones wrote: Fri Sep 03, 2021 8:37 pm There are some cores which have a faster loading system which uses the shared DDRAM to speed up the process, but as it needs extra logic it's only really in the cores that have big ROMs that would take a really long time to load. GBA and CPS2 I think both use the technique - some CPS2 ROMs were getting on for 30 seconds to load before the fast loader was implemented!
Figures why it felt to me like these roms were loaded so blazingly fast :lol:
Post Reply