Page 1 of 2

Savegames randomly stop working.

Posted: Tue Jun 02, 2020 4:00 pm
by BamaMatt
I've only noticed this behavior with the SNES core. I haven't tried saving in another core. Here's a link to a reddit post where others are having the same problem. https://www.reddit.com/r/fpgagaming/com ... urce=share

My experience is with Chronotrigger, Super Mario World, and Super Mario World 2. Core set to autosave, play game and save in game, power off MiSTer and reload game, game loads just as expected. Until one time it loads up and ignores the save game. After it does this once, it never loads the save file regardless of repeated attempts.

Re: Savegames randomly stop working.

Posted: Tue Jun 02, 2020 5:39 pm
by bootsector
You're probably powering off before HPS Linux had a chance to flush save file buffer to disk/SD.

As a general rule, and this is what I always do aside of setting autosave on, is performing a cold reboot from core's menu. This ensures buffers are flushed before you turn your MiSTer off.

Re: Savegames randomly stop working.

Posted: Tue Jun 02, 2020 6:16 pm
by BamaMatt
Thanks I'll see how that goes. Would it be safe to just load a different game instead of a cold load?

Re: Savegames randomly stop working.

Posted: Tue Jun 02, 2020 10:53 pm
by retrorepair
I'm actually a bit confused by the sram save behaviour. When the sram has data (I imagine has changed) the green led lights up and goes out when the menu is accessed. I assume this is then saving to the sd card.

My question is, why wait to save until the menunis accessed? Is there a reason this is a manual process? I don't think there are many games which constantly write to sram are there? Maybe pokemon games or systems with an RTC but should this really be default behaviour for all systems?

Just asking.

Re: Savegames randomly stop working.

Posted: Tue Jun 02, 2020 11:05 pm
by Peaceism
I just had the same issue. I did "save backup ram" during SMW, no shutting down or anything, and now my FFVI save will not load. The save file is still on the SD card however.
I am 35 hours into the game so I really hope there is a way to fix this. :cry:

Re: Savegames randomly stop working.

Posted: Wed Jun 03, 2020 3:42 am
by Sorgelig
retrorepair wrote: Tue Jun 02, 2020 10:53 pm I'm actually a bit confused by the sram save behaviour. When the sram has data (I imagine has changed) the green led lights up and goes out when the menu is accessed. I assume this is then saving to the sd card.

My question is, why wait to save until the menunis accessed? Is there a reason this is a manual process? I don't think there are many games which constantly write to sram are there? Maybe pokemon games or systems with an RTC but should this really be default behaviour for all systems?

Just asking.
MiSTer is tuned to instant off. So random unattended writes are disabled by default. Enable auto-write (when you open OSD) and save the setting.

Re: Savegames randomly stop working.

Posted: Wed Jun 03, 2020 3:47 am
by Sorgelig
I need exact steps to reproduce the issue. Otherwise it won't be fixed as i don't know how to reproduce it.

Re: Savegames randomly stop working.

Posted: Wed Jun 03, 2020 4:27 am
by BamaMatt
Sorgelig wrote: Wed Jun 03, 2020 3:47 am I need exact steps to reproduce the issue. Otherwise it won't be fixed as i don't know how to reproduce it.
I haven't figured out a way to reliably reproduce the issue. I've been keeping an eye on a couple of game saves for the last couple of days, loading the game several times a day to make sure the save was still working and hoping to uncover something I was doing to cause it to corrupt. Everything seems to work fine until one time I load the game and it no longer works. I guess the only steps are start a game of Chronotrigger and save at the first opportunity. Open the core menu to make sure it saves, then turn the De 10 Nano off. Turn it back on and load Chronotrigger to check the save. If it appears to load the save game, turn it off and repeat. It eventually will corrupt the save file.

Re: Savegames randomly stop working.

Posted: Wed Jun 03, 2020 3:03 pm
by Hahnsoulo
You say the save file is still on the SD card, but Mister is ignoring it. One thing you might try is copying that save file from the SD card to your PC and try running the game with an emulator, just to see if an emulator will load that save file. If so, then that would indicate that Mister isn't corrupting the file, but rather it's just ignoring it for some reason. If an emulator also won't load the file, then that would indicate file corruption.

Re: Savegames randomly stop working.

Posted: Wed Jun 03, 2020 6:17 pm
by BamaMatt
The save didn't work in Snes9x eX+ on Android. When using a hex editor it appears blank.

Re: Savegames randomly stop working.

Posted: Thu Jun 04, 2020 11:20 am
by bootsector
BamaMatt wrote: Wed Jun 03, 2020 6:17 pm The save didn't work in Snes9x eX+ on Android. When using a hex editor it appears blank.
Are you able to test the same procedure you've described with another SD card?

Re: Savegames randomly stop working.

Posted: Thu Jun 04, 2020 12:58 pm
by BamaMatt
Yes I can set up the original 8 gig card and use it for a while. I haven't had a repeat for a day or so since I started cold booting out of gaming sessions. Should I keep up the cold boot procedure or.go back to just turning it off to see if the issue pops back up?

Re: Savegames randomly stop working.

Posted: Thu Jun 04, 2020 4:48 pm
by bootsector
BamaMatt wrote: Thu Jun 04, 2020 12:58 pm Yes I can set up the original 8 gig card and use it for a while. I haven't had a repeat for a day or so since I started cold booting out of gaming sessions. Should I keep up the cold boot procedure or.go back to just turning it off to see if the issue pops back up?
So cold booting is always a success in your tests? Yeah, I would suggest the original approach that led to save corruption with the new 8GB card and see how it goes.

Re: Savegames randomly stop working.

Posted: Thu Jun 04, 2020 5:14 pm
by BamaMatt
I don't know yet if cold booting actually prevents the issue, only that it hasn't reoccurred since beginning cold booting upon your recommendation. My experience so far has been that everything seems fine for a day or so until out of the blue save file disaster strikes.

Re: Savegames randomly stop working.

Posted: Wed Jun 10, 2020 2:20 pm
by BamaMatt
Just a quick update. I decided to test the cold boot method and after several days I thought it was working but today disaster struck once more. I think the solution is going to have to be to back up the save folder every playthrough.

Re: Savegames randomly stop working.

Posted: Wed Jun 10, 2020 2:46 pm
by dshadoff
When you save in the game, do you then go to the OSD menu (or is autosave on) ?
The in-game save places it in memory, but to move that to SDCard requires the OSD (or autosave).

Re: Savegames randomly stop working.

Posted: Wed Jun 10, 2020 4:00 pm
by BamaMatt
I've always had autosave on and opened OSD since the beginning. The problem isn't saving games and then coming back and loading them. I've been able to do that many times. The problem is eventually something will happen that causes the save to no longer load, in MiSTer or an emulator. It occasionally occurs even when cold booting after save. I am setting up a new sd card to ensure my SD card is not at fault.
.

Re: Savegames randomly stop working.

Posted: Sun Jul 05, 2020 3:24 pm
by paulbnl
I found an issue with autosave where it will write a blank save during rom loading. This only happens if there is already a game with saves running and it writes to save ram just before the new rom starts loading.

I have sent a pull request for the SNES core but it probably also affects other cores like Genesis and Turbografx16.

Re: Savegames randomly stop working.

Posted: Sun Jul 05, 2020 4:33 pm
by Lemonici
We don't deserve you

Re: Savegames randomly stop working.

Posted: Mon Jul 06, 2020 1:18 pm
by aberu
I had a similar issue on the Genesis and referred to your commit in my issue on the Genesis core and again for another person's issue that is similar, hopefully that draws attention to this being potentially a larger issue.

Re: Savegames randomly stop working.

Posted: Mon Jul 06, 2020 3:15 pm
by paulbnl
I checked the Genesis save code and it looks like it is not affected by this issue. The code is slightly different which prevents writing the save file during rom loading.

Re: Savegames randomly stop working.

Posted: Tue Jul 07, 2020 4:18 pm
by aberu
Ah okay, thanks for checking that!

It must have been either something I did or an actual glitch in the game itself. I closed the issue on github since it's highly likely this was not an actual issue with the core at all.

Re: Savegames randomly stop working.

Posted: Sat Jul 18, 2020 12:58 am
by Poeskas
I've had this happen a few times as well, thanks for looking into it

Re: Savegames randomly stop working.

Posted: Sun Aug 16, 2020 5:03 pm
by Johnny Love
paulbnl wrote: Sun Jul 05, 2020 3:24 pm I found an issue with autosave where it will write a blank save during rom loading. This only happens if there is already a game with saves running and it writes to save ram just before the new rom starts loading.

I have sent a pull request for the SNES core but it probably also affects other cores like Genesis and Turbografx16.
Unfortunately this issue is still here. Open OSD, pick the same ROM you are running, since it is saving as you do this, the save gets wiped out. It is incredibly easy to reproduce. Just open Super Mario World with some progress, open OSD, load Super Mario World again, all progress is gone.

Re: Savegames randomly stop working.

Posted: Mon Aug 17, 2020 3:59 pm
by aberu
Johnny Love wrote: Sun Aug 16, 2020 5:03 pm
paulbnl wrote: Sun Jul 05, 2020 3:24 pm I found an issue with autosave where it will write a blank save during rom loading. This only happens if there is already a game with saves running and it writes to save ram just before the new rom starts loading.

I have sent a pull request for the SNES core but it probably also affects other cores like Genesis and Turbografx16.
Unfortunately this issue is still here. Open OSD, pick the same ROM you are running, since it is saving as you do this, the save gets wiped out. It is incredibly easy to reproduce. Just open Super Mario World with some progress, open OSD, load Super Mario World again, all progress is gone.
https://github.com/MiSTer-devel/SNES_Mi ... its/master

This is because the newest release doesn't have paulbnl's latest commit (which, ostensibly, fixes the problem). you would have to download Quartus and compile the latest SNES build yourself and use that to fix the issue.

I compile the latest builds on my own typically when important features come out like this to test. You can take a look here, I've decided to start storing those since I'm bad at keeping track of things locally with no tracking. These are not to be considered official, and they could be highly unstable. Use at your own risk! I haven't tested the SNES core or any others up here, except for the Genesis core (recompiling that now). You have been warned. I'm going to test the cores other than Genesis later tonight, just been wanting to do this very thing a while.

https://github.com/birdybro/lilymistercores

Re: Savegames randomly stop working.

Posted: Tue Aug 18, 2020 10:25 am
by Harrumph
In the SNES thread on autosaving, there was a suggestion of Mister creating a backup of save game files automatically. I think that would be a good idea in general, but certainly as a stop-gap measure until this issue is tracked down and fixed.
E.g. every time SRAM is comitted to SD card for saving, the previous save file (if existing) gets renamed to gamename_old.suffix before the new save file is written.

Re: Savegames randomly stop working.

Posted: Tue Aug 18, 2020 3:39 pm
by BamaMatt
If the system displayed text notifying you when save is in progress and when it's complete I think everyone could have peace of mind that their save is intact.

Re: Savegames randomly stop working.

Posted: Tue Sep 15, 2020 3:08 am
by Johnny Love
aberu wrote: Mon Aug 17, 2020 3:59 pm
Johnny Love wrote: Sun Aug 16, 2020 5:03 pm
paulbnl wrote: Sun Jul 05, 2020 3:24 pm I found an issue with autosave where it will write a blank save during rom loading. This only happens if there is already a game with saves running and it writes to save ram just before the new rom starts loading.

I have sent a pull request for the SNES core but it probably also affects other cores like Genesis and Turbografx16.
Unfortunately this issue is still here. Open OSD, pick the same ROM you are running, since it is saving as you do this, the save gets wiped out. It is incredibly easy to reproduce. Just open Super Mario World with some progress, open OSD, load Super Mario World again, all progress is gone.
https://github.com/MiSTer-devel/SNES_Mi ... its/master

This is because the newest release doesn't have paulbnl's latest commit (which, ostensibly, fixes the problem). you would have to download Quartus and compile the latest SNES build yourself and use that to fix the issue.

I compile the latest builds on my own typically when important features come out like this to test. You can take a look here, I've decided to start storing those since I'm bad at keeping track of things locally with no tracking. These are not to be considered official, and they could be highly unstable. Use at your own risk! I haven't tested the SNES core or any others up here, except for the Genesis core (recompiling that now). You have been warned. I'm going to test the cores other than Genesis later tonight, just been wanting to do this very thing a while.

https://github.com/birdybro/lilymistercores
I didn't try your version, but the newest release contains paulbnl's latest commit. I tried the Super Mario World example I gave before and on my first try the save file was wiped out.

Re: Savegames randomly stop working.

Posted: Tue Sep 15, 2020 1:07 pm
by paulbnl
I have tried loading Super Mario World a few times in a row on the title screen and I can't reproduce it.

Re: Savegames randomly stop working.

Posted: Tue Sep 15, 2020 11:04 pm
by Johnny Love
paulbnl wrote: Tue Sep 15, 2020 1:07 pm I have tried loading Super Mario World a few times in a row on the title screen and I can't reproduce it.
Sorry. by "open Super Mario World with some progress" I meant start the game and resume any of the 3 slots that have the progress. You only have to be on the map. I didn't try it from the title screen, but i'm pretty sure the title screen doesn't trigger a save.

After closer inspection, it turns out that I never mentioned I was using the "Vitor Vilela SA-1 Enhanced v1.32" version of SMW, at least not here. I did mention it on reddit ( https://redd.it/hqy45b ) which might have more useful information other than that.

I tried reproducing it in the standard rom even right after saving at the chocolate ghost house 3 times in a row without triggering the bug. I'm not sure if that proves anything though, because nobody really had a foolproof way that I saw to trigger it even before the patch. The SA-1 enhanced version of SMW triggered it easily before and after the patch, and this isn't an obscure ROM hack by any means. It has been worked on for years, and has no history of losing saves that I know of in emulators or flash carts, so it may be the best way we have for now of reproducing this bug in MiSTer.