ROMweasel 0.9.9

Kernel, Main, Utilities & Applications, Miscellaneous Devices.
User avatar
mrchrister
Scripting Wizard
Posts: 260
Joined: Tue Mar 30, 2021 6:23 pm
Location: Canada
Has thanked: 19 times
Been thanked: 102 times

Re: ROMweasel 0.9.9

Unread post by mrchrister »

SAM's button detection might give you a way to map additional joystick buttons:

https://github.com/mrchrisster/MiSTer_S ... SAM_joy.py

They wouldn't be assigned to MiSTer's input map file though so buttons will be randomly assigned. You could however have a button mapping script. SAM is doing this currently to map the start button of a controller to play the current game (any other button brings you back to MiSTer menu)

Test it with:

Code: Select all

/media/fat/Scripts/.MiSTer_SAM/MiSTer_SAM_joy.py /dev/input/js0 start
Push a button on your controller and it will give you the button number

You should connect with @wizzo who was also planning on mapping additional buttons for his scripts.
User avatar
wizzo
Scripting Wizard
Posts: 183
Joined: Sat Mar 12, 2022 11:32 am
Has thanked: 15 times
Been thanked: 240 times

Re: ROMweasel 0.9.9

Unread post by wizzo »

Koston wrote: Tue Nov 08, 2022 6:10 pm I investigated this a bit, before ending up implementing "Simple Mode". There are two ways how it could be done, if I'm not mistaken. The script could parse USB device node tree, fork event readers for each found controller event node and parse the event data. Safe to say, that's not happening.

Second way would be patching the main MiSTer binary to map unused buttons to keyboard events. No idea what chances such a patch would have to get merged upstream, so I'm a bit wary about spending a lot of time on it.
For the problem of the list select dialog not working with a controller, my solution was to implement my own list select using the menu dialog:
https://github.com/wizzomafizzo/MiSTer_ ... nu.sh#L294

For the page up/down stuff, I added 2 extra buttons on screen. But at this point I was writing a custom curses interface. I recall you're pretty limited on number of buttons available in dialog, and you're already wasting an extra one for the toggle button in the above solution.

I think your problem will be slightly different to SAM, because the mister binary actually releases the uinput devices when launching a script. The workarounds we were looking at are only required because SAM runs as a daemon, when the devices are locked out again. You already know how much work that will be just to get page up/down working

Having page up/down mapped in the mister binary is definitely the best solution, but I'm not sure how that idea will go down. Afaik l1 and r1 have already been poached for vibration testing, and l2/r2 are not part of the standard controller map. Who knows though, it could be fine. I might just check with sorg
FoxbatStargazer
Top Contributor
Posts: 995
Joined: Thu Dec 10, 2020 5:44 pm
Has thanked: 309 times
Been thanked: 228 times

Re: ROMweasel 0.9.9

Unread post by FoxbatStargazer »

Not sure I have the script working right. Tried to download three CD games for different cores (hook sega CD, puyo puyo tsu PCE, puyo puyo tsu ketteiban PSX), got to "download checksum verified!" after each, then the script freezes and my key presses turn to characters instead of doing anything. Seem to be stuck in the script so I have to reboot (isn't there an escape key combo?) The game files are there on my drive but if I try to load them in each core the ARM side of Mister seems to freeze entirely (CD does not change, menu will not come up, usb controls unresponsive).

I'm using a USB drive if it matters, NTFS formatted. Edited the settings.sh to point to the right place and it seems to be making files/folders where it should... just that the files themselves aren't in great condition.
Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

FoxbatStargazer wrote: Wed Nov 09, 2022 4:55 am Not sure I have the script working right. Tried to download three CD games for different cores (hook sega CD, puyo puyo tsu PCE, puyo puyo tsu ketteiban PSX), got to "download checksum verified!" after each, then the script freezes and my key presses turn to characters instead of doing anything. Seem to be stuck in the script so I have to reboot (isn't there an escape key combo?) The game files are there on my drive but if I try to load them in each core the ARM side of Mister seems to freeze entirely (CD does not change, menu will not come up, usb controls unresponsive).

I'm using a USB drive if it matters, NTFS formatted. Edited the settings.sh to point to the right place and it seems to be making files/folders where it should... just that the files themselves aren't in great condition.
Thank you for bug report. I filed a github issue about it: https://github.com/Koston-0xDEADBEEF/Mi ... l/issues/1

The script shouldn't be stuck, it's just moving the .chd file from cache to your game directory and because they're on different devices, it's taking a while. You can take a look in /media/fat/Scripts/.config/romweasel/cache if the .chd files are still there, and compare the sizes to those you have on USB drive.
Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

wizzo wrote: Wed Nov 09, 2022 2:59 am For the problem of the list select dialog not working with a controller, my solution was to implement my own list select using the menu dialog:
https://github.com/wizzomafizzo/MiSTer_ ... nu.sh#L294

For the page up/down stuff, I added 2 extra buttons on screen. But at this point I was writing a custom curses interface. I recall you're pretty limited on number of buttons available in dialog, and you're already wasting an extra one for the toggle button in the above solution.
This is quite impressive innovation to workaround the button limitations. I might go down this route; we'll see.

p.s. I love your on-screen keyboard! I definitely want to steal that, but I haven't yet managed to summon the courage to see just how complex the implementation is..
mrchrister wrote: Tue Nov 08, 2022 10:19 pm SAM's button detection might give you a way to map additional joystick buttons:

https://github.com/mrchrisster/MiSTer_S ... SAM_joy.py

They wouldn't be assigned to MiSTer's input map file though so buttons will be randomly assigned. You could however have a button mapping script. SAM is doing this currently to map the start button of a controller to play the current game (any other button brings you back to MiSTer menu)
I saw this script earlier when investigating options, but since it involves parsing binary data (not something a shellscript should ever be doing) I didn't study it in further detail. If I would use this approach, I'd likely first rip out the joystick reading and button mapping logic from SAM, turn it into a more formal re-usable input mapping API and have SAM consume that instead. But I'm currently far too ignorant on the details to determine if this makes any actual sense.
wizzo wrote: Having page up/down mapped in the mister binary is definitely the best solution, but I'm not sure how that idea will go down. Afaik l1 and r1 have already been poached for vibration testing, and l2/r2 are not part of the standard controller map. Who knows though, it could be fine. I might just check with sorg
This would definitely be ideal. I'll study the MiSTer code to get a better idea on how it works - and what kind of input device APIs Linux exposes in the first place.

There's just so much I'm clueless about..
grizzly
Posts: 376
Joined: Tue Jun 16, 2020 12:22 pm
Has thanked: 55 times
Been thanked: 76 times

Re: ROMweasel 0.9.9

Unread post by grizzly »

FoxbatStargazer wrote: Wed Nov 09, 2022 4:55 am I'm using a USB drive if it matters, NTFS formatted.
NTFS is not recommended to use since it can cause all sorts of problem and you can find MANY here on this forum.
Use EXFAT (works in windows and with big files), FAT32 (works in windows but can not handle bigger then 4gb files), EXT2/3/4 (do not work in windows without drivers) instead.
FoxbatStargazer
Top Contributor
Posts: 995
Joined: Thu Dec 10, 2020 5:44 pm
Has thanked: 309 times
Been thanked: 228 times

Re: ROMweasel 0.9.9

Unread post by FoxbatStargazer »

Koston wrote: Wed Nov 09, 2022 8:35 am
FoxbatStargazer wrote: Wed Nov 09, 2022 4:55 am Not sure I have the script working right. Tried to download three CD games for different cores (hook sega CD, puyo puyo tsu PCE, puyo puyo tsu ketteiban PSX), got to "download checksum verified!" after each, then the script freezes and my key presses turn to characters instead of doing anything. Seem to be stuck in the script so I have to reboot (isn't there an escape key combo?) The game files are there on my drive but if I try to load them in each core the ARM side of Mister seems to freeze entirely (CD does not change, menu will not come up, usb controls unresponsive).

I'm using a USB drive if it matters, NTFS formatted. Edited the settings.sh to point to the right place and it seems to be making files/folders where it should... just that the files themselves aren't in great condition.
Thank you for bug report. I filed a github issue about it: https://github.com/Koston-0xDEADBEEF/Mi ... l/issues/1

The script shouldn't be stuck, it's just moving the .chd file from cache to your game directory and because they're on different devices, it's taking a while. You can take a look in /media/fat/Scripts/.config/romweasel/cache if the .chd files are still there, and compare the sizes to those you have on USB drive.
Yep you were right, just needed some patience! In my defense, some kind of feedback to the user would probably be helpful, as copying ISOs over a USB 2 connection is always going to be slow.
Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

FoxbatStargazer wrote: Wed Nov 09, 2022 4:00 pm
Koston wrote: Wed Nov 09, 2022 8:35 am The script shouldn't be stuck, it's just moving the .chd file from cache to your game directory and because they're on different devices, it's taking a while. You can take a look in /media/fat/Scripts/.config/romweasel/cache if the .chd files are still there, and compare the sizes to those you have on USB drive.
Yep you were right, just needed some patience! In my defense, some kind of feedback to the user would probably be helpful, as copying ISOs over a USB 2 connection is always going to be slow.
True, user should always be informed when any process takes more than a few seconds. Need to implement a generic file copy function that displays a progress bar. Another thing is avoiding ever just copying files across different filesystems, but this might be a bit more tricky to handle cleanly since theoretically every core could have their ROMs in separate filesystems.
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: ROMweasel 0.9.9

Unread post by Malor »

Koston wrote: Sat Nov 05, 2022 1:35 am
Neocaron wrote: Sat Nov 05, 2022 12:47 am I'd be happy with a "select all" as well. It might come in handy.
This feature is currently absent on purpose.

I'm considering adding a separate code path for downloading full libraries, but only if the total library size is reasonable; I don't want to encourage wasting archive.org's bandwidth. They're struggling as it is, and their work preserving digital history is very important.
If you single-thread the download, it shouldn't hurt them too badly. I downloaded a huge PS1 library awhile back, but deliberately did it in single-threaded mode, and it ended up taking like a week or ten days. The steady rate was like 500k or 1MB/sec. I didn't feel particularly guilty, because I was using their official "ia" bulk download tool, and didn't use any of the tricks for parallel downloads.
User avatar
LamerDeluxe
Top Contributor
Posts: 1181
Joined: Sun May 24, 2020 10:25 pm
Has thanked: 826 times
Been thanked: 266 times

Re: ROMweasel 0.9.9

Unread post by LamerDeluxe »

I see page up and down buttons are being added to MiSTer.
Latest MiSTer unstable build: https://github.com/MiSTer-unstable-nigh ... 111_11e078
[Sorgelig 2022-11-11 e078f7f] input: add pageup/pagedown buttons to controller in linux console.
User avatar
PistolsAtDawn
Posts: 340
Joined: Fri Feb 18, 2022 7:29 pm
Has thanked: 248 times
Been thanked: 90 times

Re: ROMweasel 0.9.9

Unread post by PistolsAtDawn »

Thank you so much for doing this, Koston!

If it isn't too difficult, could we have the ability to filter out keywords while using the filter search (i.e. filter out beta versions with "-beta" or something like that)? Please don't put much time into this if it looks at all difficult. It's most definitely not necessary.
Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

PistolsAtDawn wrote: Fri Nov 11, 2022 8:40 pm If it isn't too difficult, could we have the ability to filter out keywords while using the filter search (i.e. filter out beta versions with "-beta" or something like that)? Please don't put much time into this if it looks at all difficult. It's most definitely not necessary.
Filed this as a feature request on github issues list.
Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

Just so you know - I'm busy catching up with a new job, it'll be a while until I have enough free brain cycles to dedicate to this.
User avatar
PistolsAtDawn
Posts: 340
Joined: Fri Feb 18, 2022 7:29 pm
Has thanked: 248 times
Been thanked: 90 times

Re: ROMweasel 0.9.9

Unread post by PistolsAtDawn »

Hey, we're thankful for whatever you want to share on your schedule. You don't owe us anything.
Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

https://github.com/Koston-0xDEADBEEF/MiSTer-ROMweasel

Code: Select all

 minor release bump to 0.9.666

To celebrate Saturn core receiving long-awaited CHD file support, here's a new
version with practically nothing new - except a cool version number. Also:

 * MiSTer can now operate normal mode with only a joystick (with enough
   buttons), so make normal mode default.

 * Place Saturn CHD images to region-segregated subdirectories, following
   MegaCD lead.

This script will probably not receive further updates beyond trivial
maintenance. The code is just too terrible to look at, it hurts my eye balls.

There you have it. Hope it makes someone happy; at least I have found it useful many times.

Disclaimer: Games are first downloaded to a cache directory on /media/fat/ before being moved to their final location. If these two are not on the same partition or even the same physical device, copying data over can take a while and there is currently no indicator showing the progress.

Defkyle
Posts: 4
Joined: Sat Sep 19, 2020 6:50 pm
Has thanked: 2 times
Been thanked: 1 time

Re: ROMweasel 0.9.9

Unread post by Defkyle »

So, for me the script just spits out a ton of text really fast and then does nothing. Any idea what could be causing this?
Thanks.

Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

Defkyle wrote: Wed Nov 08, 2023 3:01 pm

So, for me the script just spits out a ton of text really fast and then does nothing. Any idea what could be causing this?
Thanks.

Never seen that happen before. Can you run the script by logging in via command-line and capture the output:

Code: Select all

cd /media/fat/Scripts
./romweasel.sh >/tmp/debug.txt 2>&1
less /tmp/debug.txt

This will save the output to /tmp/debug.txt file; you can send it to me as-is, but as fallback the last command views the file for taking photos of it.

Defkyle
Posts: 4
Joined: Sat Sep 19, 2020 6:50 pm
Has thanked: 2 times
Been thanked: 1 time

Re: ROMweasel 0.9.9

Unread post by Defkyle »

Sure thing.

Code: Select all

/media/fat/Scripts# less /tmp/debug.txt
./romweasel.sh: command substitution: line 2: syntax error near unexpected token `('
./romweasel.sh: command substitution: line 2: `$XMLLINT ${CORE_FILES_XML} --xpath \"string(files/file[@name=\\\"\"$tag\"\\\"]/@name)\")","}","get_tag_filesize () {","    local tag=\"${1}\"","    local human_readable=${2:-false}","    local res=$($XMLLINT ${CORE_FILES_XML} --xpath \"string(files/file[@name=\\\"\"$tag\"\\\"]/size)\")","    $human_readable && print $(humanise $res) || print $res","}","get_tag_sha1sum () {","    local tag=\"${1}\"","    print $($XMLLINT ${CORE_FILES_XML} --xpath \"string(files/file[@name=\\\"\"$tag\"\\\"]/sha1)\")","}","","# Convert input bytes into more human-readable form","humanise () { print $(${NUMFMT} --to=iec-i --suffix=B --format=\"%9.2f\" ${1}) }","","# URL encode a string, including parenthesis but not a slash","urlencode () {","    local input=(${(s::)1})","    # Set by backreference glob (#b), and since they get set for each character to","    # encode, option WARN_NESTED_VAR will complain loudly if they're not declared local.","    local match mbegin mend","    print ${(j::)input/(#b)([^A-Za-z0-9_.!~*\\-\\/])/%${(l:2::0:)$(([##16]#match))}}","}","","cleanup () {","    [[ -f $DIALOG_TEMPFILE ]] && rm $DIALOG_TEMPFILE","    [[ $(ls -A $CACHE_DIR) ]] && print \"Warning: cache dir $CACHE_DIR not empty\"","    exit 0","}","","# Download XML files containing all ROM metadata","fetch_metadata () {","    curl_opts=(--connect-timeout 5 --retry 3 --retry-delay 5 -skLO)","","    # Loop through the list of ROM repositories","    (for (( i=1; i<${#SUPPORTED_CORES}; i+=2 )) ; do","        # Print some calming statistics via dialog gauge widget while downloading","        printf \"%s\\n\" \"XXX\"","        printf \"%i\\n\" $(( 100.0 / ${#SUPPORTED_CORES} * $i ))","        printf \"%s\\n\\n\" \"Downloading ROM repository metadata XML files\"","        printf \"%s\\n\" \"Currently downloading $(((${i}+1)/2)) of $((${#SUPPORTED_CORES}/2)):\"","        printf \"%s\\n\" \"${SUPPORTED_CORES[$(($i+1))]}\"","        printf \"%s\\n\" \"XXX\"","        select_core ${SUPPORTED_CORES[i]}","        [[ -f ${CORE_FILES_XML} ]] || $CURL $curl_opts ${CORE_URL}/${CORE_FILES_XML}","        [[ -f ${CORE_META_XML} ]] || $CURL $curl_opts ${CORE_URL}/${CORE_META_XML}","    done) |\\","        $DIALOG --title $TITLE --gauge \\","            \"Downloading ROM repository metadata XML files (total: $((${#SUPPORTED_CORES}/2)))\" \\","            16 $(($MAXWIDTH / 2)) 0","","    [[ $? -ne $DIALOG_OK ]] && cleanup","    unset curl_opts i","}","","# Display information for selected ROMs","get_rom_info () {","    local -a tags=($*)","    local rominfo=\"\" totalsize=0 romsize file_name tag dest","    for tag in $tags; do","        romsize=$(get_tag_filesize \"$tag\")","        # MiSTer Zsh is compiled with only 4-byte integers, so shell","        # arithmetic is unfit to keep count of the total size","        totalsize=$(print \"$totalsize + $romsize\" | ${BC})","        file_name=\"$(get_tag_filename \"$tag\")\"","        rominfo+=\"File name: ${file_name##*/}\\n\"","        rominfo+=\"File size: $(humanise $romsize)\\n\"","        dest=\"$(get_rom_gamedir \"$tag\")\"","        if [[ $? -ne 0 ]]; then","            rominfo+=\"\\\\\\Zb\\\\\\ZrSave path\\\\\\Zn: \\\\\\Z4${dest}\\\\\\Zn\\n\\n\"","        else rominfo+=\"Save path: ${dest}\\n\\n\"","        fi","","    done","    rominfo+=\"\\nTotal size: $(humanise $totalsize)\\n\"","    print $rominfo","}","","# Get destination directory path for a given tag","get_rom_gamedir () {","    local tag=$*","    local odir=\"${CORE_GAMEDIR}/\"","    local match mbegin mend # Set by backreference glob (#b)","","    # For compressed files, it's always just the core main ROM directory","    [[ -z ${tag##*.7z} ]] && { print \"$odir\" ; return }","","    # Strip prefix subdir and file extension","    tag=${${(Q)tag%.chd}##*/}","","    # MegaCD and Saturn have additional region specific subdirectories","    if [[ $CORE = \"MCD\" || $CORE = \"SS\" ]]; then","        : ${tag/(#b)\\((Europe|Japan|USA)\\)}","        # If we can't deduce region, well just skip it","        [[ -z $match ]] || odir+=\"${match}/\"","    fi","","    # If this isn't a multi-CD game, just use the game base name","    local base=\"${tag% \\(Disc [0-9AB]\\)*}\"","    (( $#base == $#tag )) && { print \"${odir}${base}/\" ; return }","","    # Search XML for games with same base name","    local filter=\"$base\"","    tmpdata=$($XMLLINT $CORE_FILES_XML --xpath \"files/file[sha1][contains(translate(\\","        @name, \\\"${(U)filter}\\\", \\\"${(L)filter}\\\"), \\\"${(L)filter}\\\")]/@name\")","","    local -a ntags=(${${${${${${(@f)tmpdata}#*\\\"}%\\\"*}##*/}:#^*.chd}//\\&amp\\;/&})","    unset tmpdata ; local nbase","    nbase=$(find_basename \"$tag\" $ntags)","    if [[ $? -eq 0 ]] && { print \"${odir}${nbase}/\" ; return }","","    # Failure","    print $odir ; return 1","}","","# Download selected ROMs","download_roms () {","    local -a tags=(${*})","    local tag url ofile","    local rominfo=\"$(get_rom_info $tags)\"","    rominfo+=\"\\nDownload selected game(s)?\\n\"","","    $DIALOG --title \"Information for selected ROM(s)\" --clear --cr-wrap --colors \\","        --yesno \"$rominfo\" $(( $MAXHEIGHT / 2 )) $MAXWIDTH 2>$DIALOG_TEMPFILE","    local retval=$?","    [[ $retval -eq $DIALOG_CANCEL ]] && return","    [[ $retval -ne $DIALOG_OK ]] && cleanup","","    # In case the file exists already, cURL will attempt to continue the download","    local curl_opts=(--connect-timeout 5 --retry 3 --retry-delay 5 -C - -kL)","","    # Make sure target directory exists or if user wants it to be created","    if [[ ! -d $CORE_GAMEDIR ]]; then","        $DIALOG --title \"Warning\" --clear --cr-wrap --yesno \\","            \"Directory \\\"$CORE_GAMEDIR\\\" doesn't exist.\\n\\nCreate it?\" \\","            10 82 2>$DIALOG_TEMPFILE","        retval=$?","        [[ $retval -eq $DIALOG_CANCEL ]] && return","        [[ $retval -ne $DIALOG_OK ]] && cleanup","        mkdir -p $CORE_GAMEDIR","    fi","","    for tag in $tags; do","        # Confirm final destination directory","        local dest=$(get_rom_gamedir $tag)","        [[ -n $dest ]] && { [[ -d $dest ]] || mkdir -p \"$dest\" }","","        # Encoded URL to fetch from","        url=\"${CORE_URL}/$(urlencode \"$(get_tag_filename \"$tag\")\")\"","        # Destination file with full path","        ofile=\"${CACHE_DIR}/${tag##*/}\"","        # Download the file","        $CURL $curl_opts \"$url\" -o \"$ofile\"","","        # Verify file checksum","        local filesum=\"${${(z):-$($SHA1SUM \"$ofile\")}[1]}\"","        local metasum=\"$(get_tag_sha1sum \"$tag\")\"","        if [[ $filesum = $metasum ]]; then","            print \"Downloaded file checksum verified successfully!\"","        else","            print \"ERROR: Checksum mismatch!\"","            print \"Downloaded file checksum:  $filesum\"","            print \"Metadata claimed checksum: $metasum\"","            cleanup","        fi","","        # If the file is compressed, extract it, otherwise just move to destination","        if [[ -z ${tag##*.7z} ]]; then","            $JOY_MODE && local clobber=\"-y\" || unset clobber","            $SZR e \"$ofile\" -o\"$dest\" $clobber","            rm \"$ofile\"","        else","            mv \"$ofile\" \"$dest\"","        fi","    done","","    $DIALOG --title $TITLE --cr-wrap --msgbox \"Download complete!\\n\\nPress OK to return.\" \\","        12 32 2>$DIALOG_TEMPFILE","    [[ $? -ne $DIALOG_OK ]] && cleanup","}","","# Organise ROM files in directory $* as we would when downloading","organise_chd_dir () {","    local gamedir=\"${*%/}\"","    local tag base nbase","    [[ -d $gamedir ]] || { print \"ERROR: $gamedir is not a directory?\" ; return 1 }","","    local -a tags=(${gamedir}/*.chd)","    for (( i=1; i <= $#tags; i++ )) ; do","        tag=\"${${(Q)tags[i]%.chd}##*/}\"","        base=\"${tag% \\(Disc [0-9AB]\\)*}\"","        if (( $#base == $#tag )); then # Not multi-CD game","            print \"\\e[33m${tags[i]##*/}\\e[0m -> \\e[34m${base}\\e[0m/\"","            [[ -d \"${gamedir}/${base}\" ]] || mkdir \"${gamedir}/${base}\"","            mv \"${tags[i]}\" \"${gamedir}/${base}\"","            continue","        fi","","        # Find other files with same basename and send off to neural network quantum AI","        local -a ntags=(${(M)${${(@f)tags%.chd}##*/}:#${base}*})","        nbase=$(find_basename \"$tag\" $ntags)","        if [[ $? -eq 0 ]]; then","            print \"\\e[33m${${tags[i]}##*/}\\e[0m -> \\e[36m${nbase}\\e[0m/\"","            [[ -d \"${gamedir}/${nbase}\" ]] || mkdir \"${gamedir}/${nbase}\"","            mv \"${tags[i]}\" \"${gamedir}/${nbase}\"","        else","            print \"\\e[35m${${tags[i]}##*/}\\e[0m -> \\e[31mFAILED TO COMPUTE SUITABLE NAME\\e[0m\"","        fi","    done","}","","# Find suitable game directory name when multiple base names are identical","find_basename () {","    local tag=${(Q)1##*/}","    local -a ntags=(${(Q)@[2,-1]##*/})","    local s subdir ntag match mbegin mend","    local base=${tag//(#b) \\(Disc [0-9AB]\\)(*)/}","    local suff=\"${match}\"","","    # All CD based system games should have their own subdirectories, for","    # detecting if CD change warrants a core reset (multi-CD games), and a","    # least for PSX core to automatically create a matching save file (mcd)","    #","    # Because file naming in the repositories isn't quite uniform, it's a bit","    # of a pain in the ass. Some multi-CD titles have multiple versions and","    # each disk additionally has a unique name.","    #","    # For deducing correct directory name for multi-CD games, filename is cut","    # into three parts:","    #","    #   `Example Multi-CD Game (Disc 1) (Ugly hack) (Proto)`","    #    +-------------------+ +------+ +-----------------+","    #            base            disc         suffix","","    typeset -A discset=() # discset[base]=\"disc:suffix\\x00disc:suffix\\x00\"","    for ntag in $ntags; do","        local nbase=${ntag//(#b)( \\(Disc [0-9AB]\\))(*)/}","        [[ ! $nbase = $base ]] && continue # This should never happen","        [[ -z ${match[2]} ]] && match[2]=\"0xDEADBEEF\" # Placeholder for no suffix","        discset[${base}]+=${:-${match[1]}\":\"${match[2]}\x00}","    done","","    # If there's only one file suffix, use it","    local -a nsuff=(${(u)${(0)discset[$base]}##*:})","    (( $#nsuff == 1 )) && { print \"${base}${suff}\" ; return }","","    # If there's multiple suffixes but only one set of discs, just use base name","    local -a discs=(${${(0)discset[$base]}%%:*})","    (( $#discs == ${#${(@u)discs}} )) && { print \"${base}\" ; return }","","    # If the number of disc sets matches the number of different suffixes,","    # *assume* there's a unique suffix per set","    local dsets=$(( ${#discs} / ${#${(@u)discs}} ))","    (( $dsets == $#nsuff )) && { print \"${base}${suff}\" ; return }","","    # This is as far as I'm willing to go with programmatical heuristics","    print ; return 1","}","","game_menu () {","    local -a all_tags selected_tags menu_tags menu_items subdirs submenu","    local -i itemwidth retval i","    local filter tmpdata st rominfo sub match mbegin mend n","","    # Full list of all games in current core XML","    tmpdata=$($XMLLINT $CORE_FILES_XML --xpath \"files/file[sha1]/@name\")","    all_tags=(${${${${${(@f)tmpdata}#*\\\"}%\\\"*}:#^*.(7z|chd)}//\\&amp\\;/&})","    unset tmpdata","","    # This *tarded sorting method crashes the whole MiSTer with bigger","    # repositories - reserves *far* too much memory, sigh. Looks cool tho.","    #all_tags=(${${(o)all_tags:t}/(#b)(*)/${(M)all_tags:#*${match}}})","","    if [[ -n ${(M)all_tags:#*/*} ]]; then","    # Sorting with an associative array instead.. lame lol","    local -A tt","    for n in $all_tags ; tt[${n:t}]=${n:h}","    all_tags=()","    for n in ${(ok)tt} ; all_tags+=(${tt[$n]}/$n)","    unset tt","    fi","    # First check if repository contains subdirectories and if","    # user wants to only look at a specific one or all of them","    subdirs=(${(u)all_tags//(#b)(*\\/)*/$match[1]})","    if (( $#subdirs != $#all_tags )) && (( $#subdirs > 1 )); then","        submenu=(\"ALL\" \"[[ All of them ]]\")","        for sub in $subdirs; submenu+=($sub $sub)","        $DIALOG --clear --title $TITLE --no-tags --menu \\","            \"This repository contains subdirectories, please select which one to browse.\" \\","            0 0 0 $submenu 2>$DIALOG_TEMPFILE","        (( $? != $DIALOG_OK )) && return","        sub=$(<$DIALOG_TEMPFILE)","        if [[ ! \"$sub\" = \"ALL\" ]] ; then","            all_tags=(${(M)all_tags:#${sub}*})","            sub=\"subdirectory: ${sub%\\/}, \"","        else unset sub","        fi","    fi","","    # Main loop","    while true; do","        if [[ -z $selected_tags ]]; then","            # Optional filter string for narrowing down the game list","            if [[ -n $filter ]]; then","                menu_tags=(${(M)all_tags:#(#i)*${filter}*})","            else","                menu_tags=($all_tags)","            fi","        fi","","        # Due to cdialog bug, checklist doesn't wrap correctly.","        # For display, remove any prefix subdirectories and file extension, then trim length if needed.","        # XXX: ${array[(r)${(l.${#${(O@)array//?/X}[1]}..?.)}]} <- only cut prefix if needed?","        itemwidth=$(( $MAXWIDTH - 14 ))","        menu_items=()","        for (( i=1 ; i<=${#menu_tags}; ++i )) ; do","            # Restore selected items, if any","            (( ${selected_tags[(Ie)${menu_tags[$i]}]} )) && st=\"On\" || st=\"0\"","            $JOY_MODE && unset st","            menu_items+=(${menu_tags[$i]} ${${${menu_tags[$i]##*/}%.(7z|chd)}:0:$itemwidth} $st)","        done","","        if [[ -z $menu_items ]]; then","            $DIALOG --msgbox \"No games found with filter: $filter\\n\" 5 42","            # If user does not press ok, bail out instead of reloading default set","            [[ $? -ne $DIALOG_OK ]] && break","            unset filter ; continue","        fi","","        ###############","        # Main ROM menu","        if $JOY_MODE; then","            $DIALOG --clear --title $TITLE --extra-button --extra-label \"ROM info\" \\","                --no-tags --cancel-label \"Back\" --ok-label \"Download\" --default-item \"$selected_tags\"\\","                --menu \"Choose game to download (core: ${CORE}, ${sub}games total: ${#menu_tags})\" \\","                $MAXHEIGHT $MAXWIDTH $#menu_tags $menu_items 2>$DIALOG_TEMPFILE","        else","            $DIALOG --clear --title $TITLE --separate-output --extra-button --extra-label \"ROM info\" \\","                --no-tags --cancel-label \"Back\" --help-button --help-tags --help-label \"Filter...\" \\","                --ok-label \"Download\" --default-item \"${selected_tags[1]}\" \\","                --checklist \"Choose game(s) to download (core: ${CORE}, ${sub}games total: $#menu_tags)\" \\","                $MAXHEIGHT $MAXWIDTH $#menu_tags $menu_items 2>$DIALOG_TEMPFILE","        fi","        retval=$?","        # List of user selected tags","        selected_tags=(${${(f)\"$(<$DIALOG_TEMPFILE)\"}//&amp\\;/&})","","        case $retval in","            # Download selected games","            $DIALOG_OK)","                if [[ -z $selected_tags ]]; then","                    $DIALOG --title $TITLE --msgbox \"No ROMs selected!\" 0 0","                    continue","                fi","                download_roms $selected_tags","                # In simple mode, return navigation to last downloaded item","                $JOY_MODE || unset selected_tags filter","                continue ;;","","            # Help button is for filtering the ROM list","            $DIALOG_HELP)","                $DIALOG --title \"Game list filter\" --clear --no-cancel \\","                    --inputbox \"Type search keyword (case-insensitive) or clear to reset list:\" \\","                    0 80 $filter 2>$DIALOG_TEMPFILE","                # ESC was pressed, or something else than Ok button","                [[ $? -ne $DIALOG_OK ]] && cleanup","                filter=\"$(<$DIALOG_TEMPFILE)\"","                unset selected_tags","                continue ;;","","            # Show some data for selected ROM(s)","            $DIALOG_EXTRA)","                if [[ -z $selected_tags ]]; then","                    $DIALOG --title $TITLE --msgbox \"No ROMs selected!\" 0 0","                    continue","                fi","                rominfo=\"$(get_rom_info $selected_tags)\"","                $DIALOG --title \"Information for selected ROM(s)\" --clear --cr-wrap --colors \\","                    --msgbox \"$rominfo\" $(( $MAXHEIGHT / 2 )) $MAXWIDTH 2>$DIALOG_TEMPFILE","                [[ $? -ne $DIALOG_OK ]] && cleanup","                continue ;;","","            $DIALOG_CANCEL) break ;;","            *) cleanup ;;","        esac","    done","}","","################################################################################################################","#","# MAIN SCREEN TURN ON","#","","main () {","    local -i retval","    local jm t d","","    init_static_globals","","    # Work directory contains:","    # - Downloaded ROM repository XML metadata files, indicated by $DLDONE file","    # - User configurable settings in $SETTINGS_SH","    # - Cache dir for temporarily storing downloaded ROMs","    [[ -d $WRK_DIR ]] || mkdir -p $WRK_DIR","    [[ -d $CACHE_DIR ]] || mkdir $CACHE_DIR","    pushd $WRK_DIR","","    # Cleanup in case of unclean exit","    trap 'cleanup' $SIG_HUP $SIG_INT $SIG_QUIT $SIG_TERM","","    # Fetch user-configurable configuration settings from ${SETTINGS_SH} or create it if it","    # doesn't yet exist, then set defaults for all which weren't explicitly set by the user.","    get_config","","    # Download ROM repository metadata XML files, if they haven't already been downloaded.","    fetch_metadata","","    # Secret feature, optional cmdline argument is a directory with .CHD files","    # to sort into their own subdirectories.","    [[ -n $* ]] && { organise_chd_dir $* ; return }","","    ###########","    # Main loop","    while true; do","        # Restore menu position, if any","        local default_item=${CORE:-0}","","        # Set special title for simple mode","        $JOY_MODE && jm=\" (Simple Mode)\" || unset jm","        typeset -g TITLE=\"${ROMWEASEL_VERSION}${jm}\"","","        # Show main ROM repository menu","        $JOY_MODE && jm=\"Normal Mode\" || jm=\"Simple Mode\"","        $DIALOG --title $TITLE --cancel-label \"Quit\" --help-button --help-tags --help-status \\","            --default-item \"$default_item\" --extra-button --extra-label \"Info\" --help-label $jm \\","            --menu \"Choose target system/repository:\" 0 80 0 $SUPPORTED_CORES 2>$DIALOG_TEMPFILE","        retval=$?","","        case $retval in","            # Open game list for selected ROM repository","            $DIALOG_OK)","                select_core $(<$DIALOG_TEMPFILE)","                game_menu ;;","","            # Repurposed for toggling simplified joystick mode on and off","            $DIALOG_HELP)","                select_core ${(@f)$(<$DIALOG_TEMPFILE)[2]}","                $JOY_MODE && { JOY_MODE=false ; jm='\\Z6Disabled!\\Zn' } || { JOY_MODE=true ; jm='\\Z5Enabled!\\Zn' }","                $DIALOG --title $TITLE --cr-wrap --colors --msgbox \"Simplified joystick mode:\\n\\n$jm\" \\","                    8 0 2>$DIALOG_TEMPFILE","                [[ $? -ne $DIALOG_OK ]] && cleanup","                ;;","","            # Show information for currently selected ROM repository","            $DIALOG_EXTRA)","                select_core $(<$DIALOG_TEMPFILE)","                t=$($XMLLINT $CORE_META_XML --xpath \"string(metadata/title)\")","                d=$($XMLLINT $CORE_META_XML --xpath \"string(metadata/addeddate)\")","                $DIALOG --title \"ROM repository info\" --msgbox \"\\","Core:  $CORE \\n\\","URL:   $CORE_URL \\n\\","Title: $t \\n\\","Added: $d\" 10 $MAXWIDTH","                unset t d","                ;;","","            *)","                break ;;","            esac","    done","","    # Clean up temporary files","    cleanup","}","","## For easy debug/test entry point","main $*"],"stylingDirectives":[[{"start":0,"end":10,"cssClass":"pl-c"},{"start":0,"end":2,"cssClass":"pl-c"}],[],[{"start":0,"end":65,"cssClass":"pl-c"},{"start":0,"end":1,"cssClass":"pl-c"}],[],[],[{"start":0,"end":42,"cssClass":"pl-c"},{"start":0,"end":1,"cssClass":"pl-c"}],[{"start":0,"end":19,"cssClass":"pl-en"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":34,"end":61,"cssClass":"pl-s"},{"start":34,"end":35,"cssClass":"pl-pds"},{"start":60,"end":61,"cssClass":"pl-pds"}],[],[{"start":4,"end":30,"cssClass":"pl-c"},{"start":4,"end":5,"cssClass":"pl-c"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":24,"end":40,"cssClass":"pl-s"},{"start":24,"end":26,"cssClass":"pl-pds"},{"start":39,"end":40,"cssClass":"pl-pds"},{"start":44,"end":46,"cssClass":"pl-k"},{"start":55,"end":83,"cssClass":"pl-s"},{"start":55,"end":56,"cssClass":"pl-pds"},{"start":82,"end":83,"cssClass":"pl-pds"},{"start":84,"end":85,"cssClass":"pl-k"},{"start":86,"end":92,"cssClass":"pl-k"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":21,"end":34,"cssClass":"pl-s"},{"start":21,"end":23,"cssClass":"pl-pds"},{"start":33,"end":34,"cssClass":"pl-pds"},{"start":44,"end":46,"cssClass":"pl-k"},{"start":55,"end":80,"cssClass":"pl-s"},{"start":55,"end":56,"cssClass":"pl-pds"},{"start":79,"end":80,"cssClass":"pl-pds"},{"start":81,"end":82,"cssClass":"pl-k"},{"start":83,"end":89,"cssClass":"pl-k"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":23,"end":38,"cssClass":"pl-s"},{"start":23,"end":25,"cssClass":"pl-pds"},{"start":37,"end":38,"cssClass":"pl-pds"},{"start":44,"end":46,"cssClass":"pl-k"},{"start":55,"end":82,"cssClass":"pl-s"},{"start":55,"end":56,"cssClass":"pl-pds"},{"start":81,"end":82,"cssClass":"pl-pds"},{"start":83,"end":84,"cssClass":"pl-k"},{"start":85,"end":91,"cssClass":"pl-k"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":24,"end":40,"cssClass":"pl-s"},{"start":24,"end":26,"cssClass":"pl-pds"},{"start":39,"end":40,"cssClass":"pl-pds"},{"start":44,"end":46,"cssClass":"pl-k"},{"start":55,"end":83,"cssClass":"pl-s"},{"start":55,"end":56,"cssClass":"pl-pds"},{"start":82,"end":83,"cssClass":"pl-pds"},{"start":84,"end":85,"cssClass":"pl-k"},{"start":86,"end":92,"cssClass":"pl-k"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":20,"end":32,"cssClass":"pl-s"},{"start":20,"end":22,"cssClass":"pl-pds"},{"start":31,"end":32,"cssClass":"pl-pds"},{"start":44,"end":46,"cssClass":"pl-k"},{"start":55,"end":79,"cssClass":"pl-s"},{"start":55,"end":56,"cssClass":"pl-pds"},{"start":78,"end":79,"cssClass":"pl-pds"},{"start":80,"end":81,"cssClass":"pl-k"},{"start":82,"end":88,"cssClass":"pl-k"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":23,"end":38,"cssClass":"pl-s"},{"start":23,"end":25,"cssClass":"pl-pds"},{"start":37,"end":38,"cssClass":"pl-pds"},{"start":44,"end":46,"cssClass":"pl-k"},{"start":55,"end":82,"cssClass":"pl-s"},{"start":55,"end":56,"cssClass":"pl-pds"},{"start":81,"end":82,"cssClass":"pl-pds"},{"start":83,"end":84,"cssClass":"pl-k"},{"start":85,"end":91,"cssClass":"pl-k"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":19,"end":30,"cssClass":"pl-s"},{"start":19,"end":21,"cssClass":"pl-pds"},{"start":29,"end":30,"cssClass":"pl-pds"},{"start":44,"end":46,"cssClass":"pl-k"},{"start":55,"end":78,"cssClass":"pl-s"},{"start":55,"end":56,"cssClass":"pl-pds"},{"start":77,"end":78,"cssClass":"pl-pds"},{"start":79,"end":80,"cssClass":"pl-k"},{"start":81,"end":87,"cssClass":"pl-k"}],[],[{"start":4,"end":29,"cssClass":"pl-c"},{"start":4,"end":5,"cssClass":"pl-c"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":24,"end":62,"cssClass":"pl-s"},{"start":24,"end":25,"cssClass":"pl-pds"},{"start":61,"end":62,"cssClass":"pl-pds"}],[{"start":4,"end":42,"cssClass":"pl-c"},{"start":4,"end":5,"cssClass":"pl-c"}],[{"start":4,"end":32,"cssClass":"pl-c"},{"start":4,"end":5,"cssClass":"pl-c"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":28,"end":52,"cssClass":"pl-s"},{"start":28,"end":29,"cssClass":"pl-pds"},{"start":29,"end":39,"cssClass":"pl-smi"},{"start":51,"end":52,"cssClass":"pl-pds"}],[{"start":4,"end":44,"cssClass":"pl-c"},{"start":4,"end":5,"cssClass":"pl-c"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":26,"end":44,"cssClass":"pl-s"},{"start":26,"end":27,"cssClass":"pl-pds"},{"start":27,"end":37,"cssClass":"pl-smi"},{"start":43,"end":44,"cssClass":"pl-pds"}],[{"start":4,"end":62,"cssClass":"pl-c"},{"start":4,"end":5,"cssClass":"pl-c"}],[{"start":4,"end":11,"cssClass":"pl-k"},{"start":23,"end":44,"cssClass":"pl-s"},{"start":23,"end":24,"cssClass":"pl-pds"},{"start":24,"end":34,"cssClass":"pl-smi"},{"start":43,"end":44,"cssClass":"pl-pds"}],[],[{"start":4,"end":32,"cssClass":"pl-c"},{"start":4,"end":5,"cssClass":"pl-c"}],[{"start":4,"end":11,"cssClass":"pl-k"}],[{"start":8,"end":13,"cssClass":"pl-s"},{"start":8,"end":9,"cssClass":"pl-pds"},{"start":12,"end":13,"cssClass":"pl-pds"},{"start":20,"end":51,"cssClass":"pl-s"},{"start":20,"end":21,"cssClass":"pl-pds"},{"start":50,"end":51,"cssClass":"pl-pds"}],[{"start":8,"end":14,"cssClass":"pl-s"},{"start":8,"end":9,"cssClass":"pl-pds"},{"start":13,"end":14,"cssClass":"pl-pds"},{"start":20,"end":36,"cssClass":"pl-s"},{"start":20,"end":21,"cssClass":"pl-pds"},{"start":35,"end":36,"cssClass":"pl-pds"}],[{"start":8,"end":12,"cssClass":"pl-s"},{"start":8,"end":9,"cssClass":"pl-pds"},{"start":11,"end":12,"cssClass":"pl-pds"},{"start":20,"end":38,"cssClass":"pl-s"},{"start":20,"end":21,"cssClass":"pl-pds"},{"start":37,"end":38,"cssClass":"pl-pds"}],[{"start":8,"end":13,"cssClass":"pl-s"},{"start":8,"end":9,"cssClass":"pl-pds"},{"start":12,"end":13,"cssClass":"pl-pds"},{"start":20,"end":44,"cssClass":"pl-s"},{"start":20,"end":21,"cssClass":"pl-pds"},{"start":43,"end":44,"cssClass":"pl-pds"}],[{"start":8,"end":13,"cssClass":"pl-s"},{"start":8,"end":9,"cssClass":"pl-pds"},{"start":12,"end":13,"cssClass":"pl-pds"},{"start":20,"end":37,"cssClass":"pl-s"},{"start":20,"end":21,"cssClass":"pl-pds"},{"start":36,"end":37,"cssClass":"pl-pds"}:
Defkyle
Posts: 4
Joined: Sat Sep 19, 2020 6:50 pm
Has thanked: 2 times
Been thanked: 1 time

Re: ROMweasel 0.9.9

Unread post by Defkyle »

Never mind, another script was conflicting. I have the BGM script running all the time and as soon as I stopped it, the script ran.
Thank you!

Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

Defkyle wrote: Wed Nov 08, 2023 3:25 pm

Never mind, another script was conflicting. I have the BGM script running all the time and as soon as I stopped it, the script ran.
Thank you!

Glad you got it working.

Now, if only someone could explain me why the two can't live together? :shock:

Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.7

Unread post by Koston »

New version 0.9.7, intended to add Nintendo 64 to the list, but instead ended up removing all cart based systems, since all the sources had been taken down. :(

Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

It seems the disappeared sources are back online, so I've re-added them.

New version 0.9.8 pushed, with Nintendo 64 as new supported platform!

Koston
Posts: 62
Joined: Sat Feb 13, 2021 12:25 pm
Has thanked: 47 times
Been thanked: 39 times

Re: ROMweasel 0.9.9

Unread post by Koston »

Another new release 0.9.9, with support for the excellent 0MHz DOS Collection!

Respect to the authors of this collection. The absurdly high quality of Amigavision is making its way to AO486.

User avatar
dickhardpill
Posts: 28
Joined: Tue Apr 09, 2024 9:28 pm
Has thanked: 41 times
Been thanked: 5 times

Re: ROMweasel 0.9.9

Unread post by dickhardpill »

I don't know if this helps anyone but in settings I use something similar to;

Code: Select all

NES_GAMEDIR=$(find /media | grep -m 1 games/NES)

Seems to work. I did this because I use 2 USB drives and they used to switch back and forth between USB0, USB1 and USB2. I have fixed that problem now by defining them in fstab but this may help someone else.

ETA-

if someone needed to find a bunch of cores it may be better to do something like

Code: Select all

find /media/*/games/ -maxdepth 2 -path /media/fat/games/mame -prune -o -print > /some/text/file

Code: Select all

NES_GAMEDIR=$(cat /some/text/file |grep -m 1 games/NES)

so it only runs find once instead of once per custom core location.

Code: Select all

cat /some/text/file |grep JP|grep -m 1 PSX

for sub-types?

Apologies if this is dumb.

Post Reply