Lets actually try Hybrid Emulation

foft
Posts: 334
Joined: Thu Dec 03, 2020 11:05 am
Has thanked: 29 times
Been thanked: 120 times

Re: Lets actually try Hybrid Emulation

Unread post by foft »

Thanks Bas and Kolla.

Be great to have some better integration with MiSTer and to have some tests of correctness of the CPU.

I noticed too that qemu benchmarks faster, but feels slower. Not sure if its jit delays or something else is going on. I did notice a bunch of privilege violation exceptions and not sure if that is normal or not! Pretty sure I turned off all the debug stuff in the v5 of qemu.

qemu is currently set as 68020. When I set it to 68040 I got some MMU assertions and then qemu exited iirc. Probably 68020/030 + FPU (040!) might be a reasonable next try. Though given that Laurent said 68040 is the 'main' qemu version (built for Quadra emulation) that might be the best to use, if we can work out the MMU issue.
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Lets actually try Hybrid Emulation

Unread post by bbond007 »

foft wrote: Fri Apr 23, 2021 12:07 pm Yesterday I uploaded it all as a single archive. So just extract it to /media/fat. Then make a ./68000 (or was it 68000.sh) that calls ./qemu_system_test/go.

I wonder if the stop and restart works the same way as with the other one...
I actually extracted mine to my "Amiga" directory, so for "/media/fat" you'll just need to alter "CPU_DIR"
68000.zip
(583 Bytes) Downloaded 151 times
Musashi seems to run better for me. With the qemu v4 I notice the original Amiga "boing ball" demo runs like a slideshow. Something must be wrong for that...

To switch back to testing with Musashi:

Code: Select all

#CPU68000="ld-linux-armhf.so.3 qemu-system-m68kv5"
#CPU68000_START="qemu_system_test/go"
CPU68000="musashi_68040_mister"
EDIT: found an issue switching to Musashi - uploaded new script.
foft wrote: Fri Apr 23, 2021 12:09 pm @bbond007, come and help us out here:-)
Sorry it took so long. Had a reaction to my CV19 booster and was not feeling well the last few days :(
User avatar
ron
Posts: 159
Joined: Sun May 24, 2020 7:02 pm
Has thanked: 67 times
Been thanked: 58 times

Re: Lets actually try Hybrid Emulation

Unread post by ron »

Hi ! Thx for the effort and work.
I have tryed with musashi_020 and worked fine. Sysinfo shows 5.31 Mips

But, when launching 040...: Segmentation fault. Do I need to configure/copy/install anything else ?
I tryed both binaries.

Code: Select all

/media/fat# ./musashi_68040_mister 
Segmentation fault

/media/fat# file musashi_68040_mister
musashi_68040_mister: ERROR: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3 error reading (Invalid argument)

/media/fat# ./musashi_68040_mister 
Segmentation fault
User avatar
ron
Posts: 159
Joined: Sun May 24, 2020 7:02 pm
Has thanked: 67 times
Been thanked: 58 times

Re: Lets actually try Hybrid Emulation

Unread post by ron »

Hi there:

It Works I Fixed copying with scp. Don't know what happen to FTP transfer.

/media/fat# ./musashi_68040_mister
3:4
0xb5f90000:0xb4090000:0xb6fa0000:0x9c090000


I tryed half a dozen demos and worked fine. Sysinfo says 1.75 Mips xD :D
foft
Posts: 334
Joined: Thu Dec 03, 2020 11:05 am
Has thanked: 29 times
Been thanked: 120 times

Re: Lets actually try Hybrid Emulation

Unread post by foft »

Pleased you have it working.

Perhaps ftp was in ascii mode?

@bbond007: Uff, pleased you are feeling better
User avatar
ron
Posts: 159
Joined: Sun May 24, 2020 7:02 pm
Has thanked: 67 times
Been thanked: 58 times

Re: Lets actually try Hybrid Emulation

Unread post by ron »

foft wrote: Wed Apr 28, 2021 7:26 am Pleased you have it working.

Perhaps ftp was in ascii mode?

@bbond007: Uff, pleased you are feeling better
I've been talking to shanshe, who is the one who is implementing part of the musashi in the PiStorm. He has told me that the patches are in the repository. Let's try to test how it goes.

This is its repo: https://github.com/shanshe/pistorm/tree/retrowiki

We will simply check if there is any speed increase although the main interest is compatibility.
Thank you all very much, the truth is that it is a fascinating project.

I'll keep you informed. Regards
Bas
Top Contributor
Posts: 518
Joined: Fri Jan 22, 2021 4:36 pm
Has thanked: 60 times
Been thanked: 225 times

Re: Lets actually try Hybrid Emulation

Unread post by Bas »

The main MiSTer binary's code is sparsely documented and I've had less time than anticipated for this. What I'm looking to do, is to add a variable to the ini system to define a preload/postexit variable that will allow one to point to any kind of executable per core through the ini file. The main MiSTer binary should call the preload binary just before invoking the fpga_load_rbf function. This will ensure the system will have the CPU binary (or whatever Linux plumbing required) running before the FPGA core kicks in. That's the easy bit.

I have yet to find a clean exit-point where I can hook in to unload the CPU binary after the core is unloaded. For now it looks to me like cores do not actually get unloaded at all but simply overwritten with the next one the user chooses. So there's a bit of a thing there: the CPU binary will remain running and hogging ARM cycles long after we're done with our hybrid Minimig. That won't do at all. The way to fix this with the least amount of code touched, would be to add a flag somewhere so the main binary remembers that we still have the payload of a previous 'preload' (and which one) still running and then trigger the cleanup actions of the appropriate 'post-exit' script at that point in time.

Feels very very dirty to me, as I'd rather have a clean 'load core' and 'unload core' semantic (like a constructor/destructor) to plug things like this into. I don't think the next core's loading process should worry about any mess left behind by its predecessor. Coding that would seem to uproot a significant part of the current code base because I'd refactor things into a more OOP coding style. If I'm ever going to attempt that, then I'd prefer to have this discussed with and blessed by the project's maintainer beforehand. Investing the effort would be fine with me, but I'd rather not do it only to be shot down later.

For now I'll see what I can do by means of a proof-of-concept on the first idea, so we can at least see if the whole idea works at all and is workable in practice.
ByteMavericks
Posts: 53
Joined: Tue Oct 27, 2020 4:52 pm
Has thanked: 69 times
Been thanked: 11 times

Re: Lets actually try Hybrid Emulation

Unread post by ByteMavericks »

Just a thought bas, is there an input queue from the arm side to the core? If that fills and isn’t emptied it would suggest the core isn’t running…. Either way, checking with sorg or other maintainer that we’re not reinventing the wheel sounds wise

Good luck!
zakk4223
Posts: 270
Joined: Sun May 24, 2020 10:55 pm
Been thanked: 107 times

Re: Lets actually try Hybrid Emulation

Unread post by zakk4223 »

Bas wrote: Wed Apr 28, 2021 8:28 am The main MiSTer binary's code is sparsely documented and I've had less time than anticipated for this. What I'm looking to do, is to add a variable to the ini system to define a preload/postexit variable that will allow one to point to any kind of executable per core through the ini file. The main MiSTer binary should call the preload binary just before invoking the fpga_load_rbf function. This will ensure the system will have the CPU binary (or whatever Linux plumbing required) running before the FPGA core kicks in. That's the easy bit.

I have yet to find a clean exit-point where I can hook in to unload the CPU binary after the core is unloaded. For now it looks to me like cores do not actually get unloaded at all but simply overwritten with the next one the user chooses. So there's a bit of a thing there: the CPU binary will remain running and hogging ARM cycles long after we're done with our hybrid Minimig. That won't do at all. The way to fix this with the least amount of code touched, would be to add a flag somewhere so the main binary remembers that we still have the payload of a previous 'preload' (and which one) still running and then trigger the cleanup actions of the appropriate 'post-exit' script at that point in time.

Feels very very dirty to me, as I'd rather have a clean 'load core' and 'unload core' semantic (like a constructor/destructor) to plug things like this into. I don't think the next core's loading process should worry about any mess left behind by its predecessor. Coding that would seem to uproot a significant part of the current code base because I'd refactor things into a more OOP coding style. If I'm ever going to attempt that, then I'd prefer to have this discussed with and blessed by the project's maintainer beforehand. Investing the effort would be fine with me, but I'd rather not do it only to be shot down later.

For now I'll see what I can do by means of a proof-of-concept on the first idea, so we can at least see if the whole idea works at all and is workable in practice.
Just do it in app_restart() in fpga_io.cpp. Before the execl() call. Main_MiSTer re-execs itself on every core load. There's no real concept of 'unload a core', just programming the FPGA and 'restarting' MiSTer process. That's probably the best place to do it so the 'original' calling process is responsible for shutting it down.
Bas
Top Contributor
Posts: 518
Joined: Fri Jan 22, 2021 4:36 pm
Has thanked: 60 times
Been thanked: 225 times

Re: Lets actually try Hybrid Emulation

Unread post by Bas »

Restarting the binary would destroy state, unless we store that on the SD which I'd very much want to avoid.. this thing requires some more thought.

Edit: Writing a bit of state to /tmp (which is a tmpfs) should be viable. It'll be gone after a reboot, but so will any superfluous running binaries from a hybrid core so hey.. that'll do and I won't be needlessly wearing out the SD card this way.
foft
Posts: 334
Joined: Thu Dec 03, 2020 11:05 am
Has thanked: 29 times
Been thanked: 120 times

Re: Lets actually try Hybrid Emulation

Unread post by foft »

ron wrote: Wed Apr 28, 2021 8:26 am I've been talking to shanshe, who is the one who is implementing part of the musashi in the PiStorm. He has told me that the patches are in the repository. Let's try to test how it goes.

This is its repo: https://github.com/shanshe/pistorm/tree/retrowiki

We will simply check if there is any speed increase although the main interest is compatibility.
Thank you all very much, the truth is that it is a fascinating project.

I'll keep you informed. Regards
I didn't realise they had patches to Mushashi that aren't in the upstream repo. I think its well worth trying a build with.
zakk4223
Posts: 270
Joined: Sun May 24, 2020 10:55 pm
Been thanked: 107 times

Re: Lets actually try Hybrid Emulation

Unread post by zakk4223 »

Bas wrote: Wed Apr 28, 2021 12:20 pm Restarting the binary would destroy state, unless we store that on the SD which I'd very much want to avoid.. this thing requires some more thought.

Edit: Writing a bit of state to /tmp (which is a tmpfs) should be viable. It'll be gone after a reboot, but so will any superfluous running binaries from a hybrid core so hey.. that'll do and I won't be needlessly wearing out the SD card this way.
What state are you having to store? Clean up before the execl, you shouldn't have to do anything post exec(). MiSTer destroys all state every core load.
Bas
Top Contributor
Posts: 518
Joined: Fri Jan 22, 2021 4:36 pm
Has thanked: 60 times
Been thanked: 225 times

Re: Lets actually try Hybrid Emulation

Unread post by Bas »

If Minimig hybrid needs a CPU emulator to run on the ARM, then that running emulator needs to be stopped (preferably in a clean way) when Minimig hybrid stops or it will be orphaned and just keep running after the user moves on to other cores. The state that needs to be kept, is the fact that Minimig hybrid still has its CPU running and a pointer to how to clean this up. It's not very long-lived, but needs to survive a restart of the MiSTer binary so RAM is not an option.
zakk4223
Posts: 270
Joined: Sun May 24, 2020 10:55 pm
Been thanked: 107 times

Re: Lets actually try Hybrid Emulation

Unread post by zakk4223 »

Bas wrote: Wed Apr 28, 2021 3:44 pm If Minimig hybrid needs a CPU emulator to run on the ARM, then that running emulator needs to be stopped (preferably in a clean way) when Minimig hybrid stops or it will be orphaned and just keep running after the user moves on to other cores. The state that needs to be kept, is the fact that Minimig hybrid still has its CPU running and a pointer to how to clean this up. It's not very long-lived, but needs to survive a restart of the MiSTer binary so RAM is not an option.
What I'm saying is there is no situation in which a core 'stops' and the MiSTer main doesn't exec itself. That is done every time a new RBF is selected.

If your intent is to have a menu option in the core that allows it to toggle the 'helper' on the fly, that would involve the core having a config string (possibly a new "directive" if you want this to be generic) and handling the selection of that option in menu.cpp

If you just want to clean up when a user switches cores, app_restart() is probably where to do it. The MiSTer's sequence after a user selects a core is program fpga -> (effectively) execl("media/fat/MiSTer <rbfname>"). You should be cleaning up before MiSTer restarts, not trying to save state across an exec() just to kill a child process you explicitly started.
foft
Posts: 334
Joined: Thu Dec 03, 2020 11:05 am
Has thanked: 29 times
Been thanked: 120 times

Re: Lets actually try Hybrid Emulation

Unread post by foft »

foft wrote: Wed Apr 28, 2021 2:53 pm I didn't realise they had patches to Mushashi that aren't in the upstream repo. I think its well worth trying a build with.
Completely untested pistorm build. Does it work? Better?
Attachments
musashi_68040_mister.gz
(1.19 MiB) Downloaded 124 times
foft
Posts: 334
Joined: Thu Dec 03, 2020 11:05 am
Has thanked: 29 times
Been thanked: 120 times

Re: Lets actually try Hybrid Emulation

Unread post by foft »

If someone kills MiSTer manually it'd be good if it stops the hybrid emulation process. Or the hybrid emulator detects that it is an orphan (reparented) and exits itself.
User avatar
ron
Posts: 159
Joined: Sun May 24, 2020 7:02 pm
Has thanked: 67 times
Been thanked: 58 times

Re: Lets actually try Hybrid Emulation

Unread post by ron »

musashi040_0428.png
musashi040_0428.png (261.59 KiB) Viewed 6091 times
Seems to work, at least it started ok.
Now testing some demos and compatibility.

Thanks
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Lets actually try Hybrid Emulation

Unread post by bbond007 »

Bas wrote: Wed Apr 28, 2021 3:44 pm If Minimig hybrid needs a CPU emulator to run on the ARM, then that running emulator needs to be stopped (preferably in a clean way) when Minimig hybrid stops or it will be orphaned and just keep running after the user moves on to other cores. The state that needs to be kept, is the fact that Minimig hybrid still has its CPU running and a pointer to how to clean this up. It's not very long-lived, but needs to survive a restart of the MiSTer binary so RAM is not an option.
As an existing example:

The /sbin/uartmode script is not only called when the UART mode is changed in OSD, but also whenever a core is started or reset.

Currently, The uartmode script directly (and indirectly via MidiLink) starts a number of processes such as pppd, getty, FluidSynth, Munt...

These processes don't get cleaned up when the core exit which is why the first thing the script kills all of these daemons which could potentially be running before starting anything new:

Code: Select all

kill_all() {
	rm -f /tmp/uartmode*
	rm -f /tmp/ML_BAUD
	killall midilink
	killall pppd
	killall agetty
	killall login
	killall mt32d
	killall fluidsynth
	killall mpg123
}
I think a similar approach could be taken with the hybrid CPU.

I'm not sure if you would consider killall "clean" but its used extensively now - and I've been using it successfully in my 68000.sh
foft
Posts: 334
Joined: Thu Dec 03, 2020 11:05 am
Has thanked: 29 times
Been thanked: 120 times

Re: Lets actually try Hybrid Emulation

Unread post by foft »

Hmm, that pistorm one seems slower and crashier.
Here is an 020 of the 'official' Musashi with the 384MB fast.

Sure this was a fair bit faster on the early builds...
Attachments
musashi_68020_mister.gz
(1.34 MiB) Downloaded 127 times
User avatar
ron
Posts: 159
Joined: Sun May 24, 2020 7:02 pm
Has thanked: 67 times
Been thanked: 58 times

Re: Lets actually try Hybrid Emulation

Unread post by ron »

foft wrote: Wed Apr 28, 2021 5:53 pm Hmm, that pistorm one seems slower and crashier.
Here is an 020 of the 'official' Musashi with the 384MB fast.

Sure this was a fair bit faster on the early builds...
it works, making some more tests ....
foft
Posts: 334
Joined: Thu Dec 03, 2020 11:05 am
Has thanked: 29 times
Been thanked: 120 times

Re: Lets actually try Hybrid Emulation

Unread post by foft »

Here, have an FPU:
http://www.64kib.com/qemu_system_testv6.tar.xz

The patch was sent to me by Laurent (qemu maintainer) a week or so back but afraid I only just got around to applying it. It enables 68040 style stack frames on the fpu, even though we're in 68020 mode. Since they are not implemented for 68020 - which is the original problem I had with it booting.
caffeinekid
Posts: 75
Joined: Wed Nov 04, 2020 10:03 am
Has thanked: 21 times
Been thanked: 14 times

Re: Lets actually try Hybrid Emulation

Unread post by caffeinekid »

Would this hybrid work help with the possibility of a Atari Falcon core now that higher motorola chips and fpu are in the mix?
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Lets actually try Hybrid Emulation

Unread post by bbond007 »

foft wrote: Wed Apr 28, 2021 6:24 pm The patch was sent to me by Laurent (qemu maintainer) a week or so back but afraid I only just got around to applying it. It enables 68040 style stack frames on the fpu, even though we're in 68020 mode. Since they are not implemented for 68020 - which is the original problem I had with it booting.

Thanks!

Benchmarks aside, Musashi certainly seems "feel" faster than qemu...

Here is a update 68000.sh with an updated MiSTer built last night (with reset hack).
68000.zip
(582.94 KiB) Downloaded 126 times
User avatar
Caldor
Top Contributor
Posts: 930
Joined: Sat Jul 25, 2020 11:20 am
Has thanked: 112 times
Been thanked: 111 times

Re: Lets actually try Hybrid Emulation

Unread post by Caldor »

I have been trying this out, but so far no luck getting it to run. I cannot run the 68000.sh script without getting the "wrong core", I tried changing it so it would have the right core name and QEMU file name, but both just crash. I then tried running the script from Putty, which helped a bit I guess, but same problem as when I run the musashi_68020_mister or musashi_68040_mister directly, I see some hex code kind of thing in the prompt and the MiSTer itself freezes and then reboots after something like a minute.

I found Midnight Commander seems to be installed on the MiSTer by default, so that is nice, but has not helped much. I will be busy this weekend but I will probably try experimenting with this again Sunday or Monday. Seems really promising, and I feel like trying to experiment with it :)

Have I understood it correctly that the QEMU Test files are QEMU source code for building a musashi file by compiling that source code? Or do I need to do something with that as well, to be able to use the CPU file?
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Lets actually try Hybrid Emulation

Unread post by bbond007 »

kolla wrote: Mon Apr 26, 2021 6:09 am Some first impressions... :)
* turned off PPP, set UART to "none" (does it matter? I see no difference really)
I've not been able to to get PPP to connect with the hybrid core. Maybe its too much lag or something because serial does seem to work.

It probably makes very little difference if PPP, Console or Modem are selected, but MUNT and FluidSynth use not and insignificant portion of CPU just sitting idle.

I just added "uartmode 0" to my 68000.sh just to make sure CPU sucking softSynths are disabled for testing.
lordoftime79
Posts: 97
Joined: Sun Feb 14, 2021 6:29 pm
Has thanked: 1 time
Been thanked: 2 times

Re: Lets actually try Hybrid Emulation

Unread post by lordoftime79 »

I just srtuggle with this full stop - could somebody perhaps zip up a working setup that we can just un zip to the sd card and go?
Bas
Top Contributor
Posts: 518
Joined: Fri Jan 22, 2021 4:36 pm
Has thanked: 60 times
Been thanked: 225 times

Re: Lets actually try Hybrid Emulation

Unread post by Bas »

@lorsdoftime79 the whole thing is still very much in development flux with bits flying left and right. There is no working setup at the moment. Some things function but only until they break. There's still a lot of leg work left to be done before a just working setup can be packaged for any kind of stable general use.
kolla
Posts: 188
Joined: Sat Jun 13, 2020 7:56 am
Has thanked: 17 times
Been thanked: 33 times

Re: Lets actually try Hybrid Emulation

Unread post by kolla »

Noone really has a working setup yet, we are all experimenting. Btw you don’t need ssh, you can just press F9 in the menu core, log in and start the 68k emulation from there, after for example a 20 sec sleep, enough so you can F12 and start the hybrid core before CPU emulation kicks in.
kolla
Posts: 188
Joined: Sat Jun 13, 2020 7:56 am
Has thanked: 17 times
Been thanked: 33 times

Re: Lets actually try Hybrid Emulation

Unread post by kolla »

What I find really strange is how all benchmarks say everything is much faster than tg68, while in reality it is so much slower, heh... :)
ByteMavericks
Posts: 53
Joined: Tue Oct 27, 2020 4:52 pm
Has thanked: 69 times
Been thanked: 11 times

Re: Lets actually try Hybrid Emulation

Unread post by ByteMavericks »

kolla wrote: Thu Apr 29, 2021 5:44 am What I find really strange is how all benchmarks say everything is much faster than tg68, while in reality it is so much slower, heh... :)
Could that be because benchmarks can execute from the cache on the arm side, and the challenge in performance is in the fpga to arm bridge?
Post Reply