Lets actually try Hybrid Emulation

[foft]

https://linux-mips.linux-mips.narkive.c ... ry-so-slow
"mmap will create uncached mappings for anything above the highest RAM address."

So, how do I enable the cache, even though its not in /proc/iomem?

[foft]

Here are builds using malloc instead:
http://www.64kib.com/qemu_system_testv5.tar.xz

musashi_68040_mister.gz

(1.34 MiB) Downloaded 173 times

+ some pics of bustest and qemu benchmark results

[LamerDeluxe]

Wow, that is really impressively fast.

[robinsonb5]

That's more like it!

(Interesting that the Dhrystone score is so much lower than the other two...)

Nice work, anyway!

[foft]

Yeah, that dhrystone score is weird. I get much more with 68k user mode qemu. I'm going to set up a .vhd and install gcc and see how a local build looks.

[foft]

So, what do we need to do to get this 'releasable':
i) Figure out caching for mmap of 'spare' DDR.
ii) Plumb in reset
iii) Plumb in config params: fastram etc.
iv) Figure out common crash reasons - possible unimplemented instructions etc
v) Enable FPU on 68020 (maintainer sent me a patch to enable fsave 68040 style on 68020) or enable 68040
vi) Make it switchable: 68000, 68020 (tg68) or Hybrid qemu.
vii) Merge sys framework
viii) A generic method to start hybrid cpu from MiSTer binary

Who wants to help?

[foft]

This might be a solution for cache:
https://github.com/ikwzm/udmabuf

[Bas]

From that list I'd only qualify for VIII. I'll see if I can come up with something sensible.

[kolla]

I have some test code that was used to find bugs in tg68, which surely can be used again, and also software that use FPU... what kind of precision can we expect?

[kolla]

Some first impressions...
* turned off PPP, set UART to "none" (does it matter? I see no difference really)
* more or less untampered OS 3.1.4.1
* turned off Minimig RTG, use P96 with "native" driver and HD720 mode (32 pens)
* installed CPU libraries (+ mmu.library - http://aminet.net/package/util/sys/Mu680x0Libs)

* with musashi_68040_mister, all software thinks there is a 68030+68881
* with qemu (testv5), all software thinks there is just a 68020
* qemu ("testv5") seems very sluggish (debug output somewhere?), but sysinfo etc show that it is fast
* musashi is quite snappier, but sysinfo etc show that it is slow
* with musashi, I got a couple of "yellow guru" recoverable alerts on boot
* both passed the MULU.L test (which tg68 recently didn't), hehe

[foft]

Thanks Bas and Kolla.

Be great to have some better integration with MiSTer and to have some tests of correctness of the CPU.

I noticed too that qemu benchmarks faster, but feels slower. Not sure if its jit delays or something else is going on. I did notice a bunch of privilege violation exceptions and not sure if that is normal or not! Pretty sure I turned off all the debug stuff in the v5 of qemu.

qemu is currently set as 68020. When I set it to 68040 I got some MMU assertions and then qemu exited iirc. Probably 68020/030 + FPU (040!) might be a reasonable next try. Though given that Laurent said 68040 is the 'main' qemu version (built for Quadra emulation) that might be the best to use, if we can work out the MMU issue.

[bbond007]

Yesterday I uploaded it all as a single archive. So just extract it to /media/fat. Then make a ./68000 (or was it 68000.sh) that calls ./qemu_system_test/go.

I wonder if the stop and restart works the same way as with the other one...

I actually extracted mine to my "Amiga" directory, so for "/media/fat" you'll just need to alter "CPU_DIR"

68000.zip

(583 Bytes) Downloaded 153 times

Musashi seems to run better for me. With the qemu v4 I notice the original Amiga "boing ball" demo runs like a slideshow. Something must be wrong for that...

To switch back to testing with Musashi:

Code: Select all

#CPU68000="ld-linux-armhf.so.3 qemu-system-m68kv5"
#CPU68000_START="qemu_system_test/go"
CPU68000="musashi_68040_mister"

EDIT: found an issue switching to Musashi - uploaded new script.

@bbond007, come and help us out here:-)

Sorry it took so long. Had a reaction to my CV19 booster and was not feeling well the last few days

[ron]

Hi ! Thx for the effort and work.
I have tryed with musashi_020 and worked fine. Sysinfo shows 5.31 Mips

But, when launching 040...: Segmentation fault. Do I need to configure/copy/install anything else ?
I tryed both binaries.

Code: Select all

/media/fat# ./musashi_68040_mister 
Segmentation fault

/media/fat# file musashi_68040_mister
musashi_68040_mister: ERROR: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3 error reading (Invalid argument)

/media/fat# ./musashi_68040_mister 
Segmentation fault

[ron]

Hi there:

It Works I Fixed copying with scp. Don't know what happen to FTP transfer.

/media/fat# ./musashi_68040_mister
3:4
0xb5f90000:0xb4090000:0xb6fa0000:0x9c090000

I tryed half a dozen demos and worked fine. Sysinfo says 1.75 Mips xD

[foft]

Pleased you have it working.

Perhaps ftp was in ascii mode?

@bbond007: Uff, pleased you are feeling better

[ron]

Pleased you have it working.

Perhaps ftp was in ascii mode?

@bbond007: Uff, pleased you are feeling better

I've been talking to shanshe, who is the one who is implementing part of the musashi in the PiStorm. He has told me that the patches are in the repository. Let's try to test how it goes.

This is its repo: https://github.com/shanshe/pistorm/tree/retrowiki

We will simply check if there is any speed increase although the main interest is compatibility.
Thank you all very much, the truth is that it is a fascinating project.

I'll keep you informed. Regards

[Bas]

The main MiSTer binary's code is sparsely documented and I've had less time than anticipated for this. What I'm looking to do, is to add a variable to the ini system to define a preload/postexit variable that will allow one to point to any kind of executable per core through the ini file. The main MiSTer binary should call the preload binary just before invoking the fpga_load_rbf function. This will ensure the system will have the CPU binary (or whatever Linux plumbing required) running before the FPGA core kicks in. That's the easy bit.

I have yet to find a clean exit-point where I can hook in to unload the CPU binary after the core is unloaded. For now it looks to me like cores do not actually get unloaded at all but simply overwritten with the next one the user chooses. So there's a bit of a thing there: the CPU binary will remain running and hogging ARM cycles long after we're done with our hybrid Minimig. That won't do at all. The way to fix this with the least amount of code touched, would be to add a flag somewhere so the main binary remembers that we still have the payload of a previous 'preload' (and which one) still running and then trigger the cleanup actions of the appropriate 'post-exit' script at that point in time.

Feels very very dirty to me, as I'd rather have a clean 'load core' and 'unload core' semantic (like a constructor/destructor) to plug things like this into. I don't think the next core's loading process should worry about any mess left behind by its predecessor. Coding that would seem to uproot a significant part of the current code base because I'd refactor things into a more OOP coding style. If I'm ever going to attempt that, then I'd prefer to have this discussed with and blessed by the project's maintainer beforehand. Investing the effort would be fine with me, but I'd rather not do it only to be shot down later.

For now I'll see what I can do by means of a proof-of-concept on the first idea, so we can at least see if the whole idea works at all and is workable in practice.

[ByteMavericks]

Just a thought bas, is there an input queue from the arm side to the core? If that fills and isn’t emptied it would suggest the core isn’t running…. Either way, checking with sorg or other maintainer that we’re not reinventing the wheel sounds wise

Good luck!

[zakk4223]

The main MiSTer binary's code is sparsely documented and I've had less time than anticipated for this. What I'm looking to do, is to add a variable to the ini system to define a preload/postexit variable that will allow one to point to any kind of executable per core through the ini file. The main MiSTer binary should call the preload binary just before invoking the fpga_load_rbf function. This will ensure the system will have the CPU binary (or whatever Linux plumbing required) running before the FPGA core kicks in. That's the easy bit.

I have yet to find a clean exit-point where I can hook in to unload the CPU binary after the core is unloaded. For now it looks to me like cores do not actually get unloaded at all but simply overwritten with the next one the user chooses. So there's a bit of a thing there: the CPU binary will remain running and hogging ARM cycles long after we're done with our hybrid Minimig. That won't do at all. The way to fix this with the least amount of code touched, would be to add a flag somewhere so the main binary remembers that we still have the payload of a previous 'preload' (and which one) still running and then trigger the cleanup actions of the appropriate 'post-exit' script at that point in time.

Feels very very dirty to me, as I'd rather have a clean 'load core' and 'unload core' semantic (like a constructor/destructor) to plug things like this into. I don't think the next core's loading process should worry about any mess left behind by its predecessor. Coding that would seem to uproot a significant part of the current code base because I'd refactor things into a more OOP coding style. If I'm ever going to attempt that, then I'd prefer to have this discussed with and blessed by the project's maintainer beforehand. Investing the effort would be fine with me, but I'd rather not do it only to be shot down later.

For now I'll see what I can do by means of a proof-of-concept on the first idea, so we can at least see if the whole idea works at all and is workable in practice.

Just do it in app_restart() in fpga_io.cpp. Before the execl() call. Main_MiSTer re-execs itself on every core load. There's no real concept of 'unload a core', just programming the FPGA and 'restarting' MiSTer process. That's probably the best place to do it so the 'original' calling process is responsible for shutting it down.

[Bas]

Restarting the binary would destroy state, unless we store that on the SD which I'd very much want to avoid.. this thing requires some more thought.

Edit: Writing a bit of state to /tmp (which is a tmpfs) should be viable. It'll be gone after a reboot, but so will any superfluous running binaries from a hybrid core so hey.. that'll do and I won't be needlessly wearing out the SD card this way.

[foft]

I've been talking to shanshe, who is the one who is implementing part of the musashi in the PiStorm. He has told me that the patches are in the repository. Let's try to test how it goes.

This is its repo: https://github.com/shanshe/pistorm/tree/retrowiki

We will simply check if there is any speed increase although the main interest is compatibility.
Thank you all very much, the truth is that it is a fascinating project.

I'll keep you informed. Regards

I didn't realise they had patches to Mushashi that aren't in the upstream repo. I think its well worth trying a build with.

[zakk4223]

Restarting the binary would destroy state, unless we store that on the SD which I'd very much want to avoid.. this thing requires some more thought.

Edit: Writing a bit of state to /tmp (which is a tmpfs) should be viable. It'll be gone after a reboot, but so will any superfluous running binaries from a hybrid core so hey.. that'll do and I won't be needlessly wearing out the SD card this way.

What state are you having to store? Clean up before the execl, you shouldn't have to do anything post exec(). MiSTer destroys all state every core load.

[Bas]

If Minimig hybrid needs a CPU emulator to run on the ARM, then that running emulator needs to be stopped (preferably in a clean way) when Minimig hybrid stops or it will be orphaned and just keep running after the user moves on to other cores. The state that needs to be kept, is the fact that Minimig hybrid still has its CPU running and a pointer to how to clean this up. It's not very long-lived, but needs to survive a restart of the MiSTer binary so RAM is not an option.

[zakk4223]

If Minimig hybrid needs a CPU emulator to run on the ARM, then that running emulator needs to be stopped (preferably in a clean way) when Minimig hybrid stops or it will be orphaned and just keep running after the user moves on to other cores. The state that needs to be kept, is the fact that Minimig hybrid still has its CPU running and a pointer to how to clean this up. It's not very long-lived, but needs to survive a restart of the MiSTer binary so RAM is not an option.

What I'm saying is there is no situation in which a core 'stops' and the MiSTer main doesn't exec itself. That is done every time a new RBF is selected.

If your intent is to have a menu option in the core that allows it to toggle the 'helper' on the fly, that would involve the core having a config string (possibly a new "directive" if you want this to be generic) and handling the selection of that option in menu.cpp

If you just want to clean up when a user switches cores, app_restart() is probably where to do it. The MiSTer's sequence after a user selects a core is program fpga -> (effectively) execl("media/fat/MiSTer <rbfname>"). You should be cleaning up before MiSTer restarts, not trying to save state across an exec() just to kill a child process you explicitly started.

[foft]

I didn't realise they had patches to Mushashi that aren't in the upstream repo. I think its well worth trying a build with.

Completely untested pistorm build. Does it work? Better?

[foft]

If someone kills MiSTer manually it'd be good if it stops the hybrid emulation process. Or the hybrid emulator detects that it is an orphan (reparented) and exits itself.

[ron]

musashi040_0428.png (261.59 KiB) Viewed 6196 times

Seems to work, at least it started ok.
Now testing some demos and compatibility.

Thanks

[bbond007]

If Minimig hybrid needs a CPU emulator to run on the ARM, then that running emulator needs to be stopped (preferably in a clean way) when Minimig hybrid stops or it will be orphaned and just keep running after the user moves on to other cores. The state that needs to be kept, is the fact that Minimig hybrid still has its CPU running and a pointer to how to clean this up. It's not very long-lived, but needs to survive a restart of the MiSTer binary so RAM is not an option.

As an existing example:

The /sbin/uartmode script is not only called when the UART mode is changed in OSD, but also whenever a core is started or reset.

Currently, The uartmode script directly (and indirectly via MidiLink) starts a number of processes such as pppd, getty, FluidSynth, Munt...

These processes don't get cleaned up when the core exit which is why the first thing the script kills all of these daemons which could potentially be running before starting anything new:

Code: Select all

kill_all() {
	rm -f /tmp/uartmode*
	rm -f /tmp/ML_BAUD
	killall midilink
	killall pppd
	killall agetty
	killall login
	killall mt32d
	killall fluidsynth
	killall mpg123
}

I think a similar approach could be taken with the hybrid CPU.

I'm not sure if you would consider killall "clean" but its used extensively now - and I've been using it successfully in my 68000.sh

[foft]

Hmm, that pistorm one seems slower and crashier.
Here is an 020 of the 'official' Musashi with the 384MB fast.

Sure this was a fair bit faster on the early builds...

[ron]

Hmm, that pistorm one seems slower and crashier.
Here is an 020 of the 'official' Musashi with the 384MB fast.

Sure this was a fair bit faster on the early builds...

it works, making some more tests ....