Update All 2.0 Is Here

Kernel, Main, Utilities & Applications, Miscellaneous Devices.
Xbytez
Site Admin
Posts: 469
Joined: Wed May 20, 2020 3:36 pm
Has thanked: 214 times
Been thanked: 787 times

Update All 2.0 Is Here

Unread post by Xbytez »

theypsilon (@josembarroso) has after more than two years since his update_all scripts initial release, now released the second major version of Update All.

See the public patreon post for full details of the changes:

https://www.patreon.com/posts/73589304

UpdateAll2.jpg
UpdateAll2.jpg (56.36 KiB) Viewed 11027 times
akeley
Top Contributor
Posts: 1303
Joined: Mon May 25, 2020 7:54 pm
Has thanked: 416 times
Been thanked: 399 times

Re: Update All 2.0 Is Here

Unread post by akeley »

Good stuff, it really grew to be quite a powerful tool. It's great to see wizzo's scripts are now included too.
User avatar
clarkkent
Posts: 98
Joined: Sun Nov 21, 2021 1:59 pm
Has thanked: 1 time
Been thanked: 15 times

Re: Update All 2.0 Is Here

Unread post by clarkkent »

The documentation notes downloader has to be activated to update to 2.0. How can the downloader be activated?
Higgy
Posts: 83
Joined: Mon May 25, 2020 9:37 am
Has thanked: 4 times
Been thanked: 27 times

Re: Update All 2.0 Is Here

Unread post by Higgy »

Just doing it now.
Looks like you just run the Update_All Script as normal. It will say about the 2.0 update and you just need to run the Script again.
Seems to be running through.
theypsilon
Scripting Wizard
Posts: 105
Joined: Sun May 24, 2020 8:20 pm
Been thanked: 40 times

Re: Update All 2.0 Is Here

Unread post by theypsilon »

clarkkent wrote: Sat Oct 22, 2022 6:10 am The documentation notes downloader has to be activated to update to 2.0. How can the downloader be activated?
When you go to the Settings Screen by pressing up during the countdown, it will ask you to activate it. If it's not asking you it means you already activated it.

The upgrade to Update All 2.0 only happens within MiSTer, not with the updater-pc btw.
User avatar
lamarax
Top Contributor
Posts: 472
Joined: Wed Nov 11, 2020 6:28 pm
Has thanked: 33 times
Been thanked: 199 times

Re: Update All 2.0 Is Here

Unread post by lamarax »

theypsilon wrote: Sat Oct 22, 2022 9:46 am When you go to the Settings Screen by pressing up during the countdown, it will ask you to activate it. If it's not asking you it means you already activated it.

The upgrade to Update All 2.0 only happens within MiSTer, not with the updater-pc btw.
Good evening José,

Thank you for bringing us this huge milestone! Every module seems to have been "up-tuned", and the script runs faster than ever.

However, there's something strange going on with the bios_db sequence; upon first launch yesterday, it has gone ahead to delete the bios files from my ..fat/games/PSX folder specifically, but doesn't download them anew.

No big deal as I put them back on manually once I figured out what's happened, just something that you might want to investigate (sorry, can't file a github issue).

[Edit]I think the problem hides in here somewhere,

Code: Select all

"PSX": {
        "boot.rom": {
            "file": "SCPH7001.BIN",
            "hash": "1e68c231d0896b7eadcad1d7d8e76129",
            "size": 524288
        },
        "boot1.rom": {
            "file": "SCPH7000.BIN",
            "hash": "8e4c14f567745eff2f0408c8129f72a6",
            "size": 524288
        },
        "boot2.rom": {
            "file": "SCPH7002.BIN",
            "hash": "b9d9a0286c33dc6b7237bb13cd46fdee",
            "size": 524288
        },
        "sbi.zip": {
            "file": "sbi.zip",
            "hash": "706d60ab2dfdbfdd53be2069cb85a1fe",
            "size": 218953
        }
    },
as it doesn't convert SCPH700..bin into boot..rom according to build_db.py at line 44 :?
User avatar
CartoonDonkey
Posts: 121
Joined: Fri Aug 06, 2021 8:22 am
Has thanked: 17 times
Been thanked: 29 times

Re: Update All 2.0 Is Here

Unread post by CartoonDonkey »

Slightly off topic:
Any way to simply change the overall font for scripts shown in the console? Default font is huge on my 1084s crt. And the whole console screen gets shifted to the left. (I'm using this screenmode for the main menu, which is the only thing that really works for me.)

[Menu]
video_mode=640,6,56,66,224,14,8,18,12000
vga_scaler=1
User avatar
lamarax
Top Contributor
Posts: 472
Joined: Wed Nov 11, 2020 6:28 pm
Has thanked: 33 times
Been thanked: 199 times

Re: Update All 2.0 Is Here

Unread post by lamarax »

CartoonDonkey wrote: Sat Oct 22, 2022 6:26 pm Slightly off topic:
Any way to simply change the overall font for scripts shown in the console? Default font is huge on my 1084s crt. And the whole console screen gets shifted to the left. (I'm using this screenmode for the main menu, which is the only thing that really works for me.)

[Menu]
video_mode=640,6,56,66,224,14,8,18,12000
vga_scaler=1
That's an issue pertaining to the linux console properties, not any script in particular.
theypsilon
Scripting Wizard
Posts: 105
Joined: Sun May 24, 2020 8:20 pm
Been thanked: 40 times

Re: Update All 2.0 Is Here

Unread post by theypsilon »

lamarax wrote: Sat Oct 22, 2022 5:13 pm Good evening José,

Thank you for bringing us this huge milestone! Every module seems to have been "up-tuned", and the script runs faster than ever.

However, there's something strange going on with the bios_db sequence; upon first run yesterday, it has deleted the bios files from my ..fat/games/PSX folder specifically, but doesn't download them anew.

No big deal as I put them back on manually once I'd figured out what's happened, just something that you might want to investigate (sorry, can't file a github issue).

[Edit]I think the problem hides in here somewhere,

Code: Select all

"PSX": {
        "boot.rom": {
            "file": "SCPH7001.BIN",
            "hash": "1e68c231d0896b7eadcad1d7d8e76129",
            "size": 524288
        },
        "boot1.rom": {
            "file": "SCPH7000.BIN",
            "hash": "8e4c14f567745eff2f0408c8129f72a6",
            "size": 524288
        },
        "boot2.rom": {
            "file": "SCPH7002.BIN",
            "hash": "b9d9a0286c33dc6b7237bb13cd46fdee",
            "size": 524288
        },
        "sbi.zip": {
            "file": "sbi.zip",
            "hash": "706d60ab2dfdbfdd53be2069cb85a1fe",
            "size": 218953
        }
    },
as it doesn't convert SCPH700..bin into boot..rom according to build_db.py at line 44 :?
This is not related to this release, as you are describing Downloader/BiosDB behavior and these hasn't been touched since August/May respectively. But it might be that you plugged a USB drive with the games folder, and now it got installed there instead.
FoxbatStargazer
Top Contributor
Posts: 994
Joined: Thu Dec 10, 2020 5:44 pm
Has thanked: 309 times
Been thanked: 228 times

Re: Update All 2.0 Is Here

Unread post by FoxbatStargazer »

Yeah I recall having to edit some ini to point to my USB drive for arcade ROMs. But there's no setting here in 2.0 and yet it seems to have figured it out on its own?
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Update All 2.0 Is Here

Unread post by Malor »

CartoonDonkey wrote: Sat Oct 22, 2022 6:26 pm Slightly off topic:
Any way to simply change the overall font for scripts shown in the console? Default font is huge on my 1084s crt. And the whole console screen gets shifted to the left. (I'm using this screenmode for the main menu, which is the only thing that really works for me.)

[Menu]
video_mode=640,6,56,66,224,14,8,18,12000
vga_scaler=1
You might want to check that you have this setting:

Code: Select all

fb_size=1
That sets the size of the console font. 1 is smallest, 2 is twice as large, 4 is four times as large.

I'm not sure how to move your picture to the right. You should be able to move the menu screen right and left with the CRT dials, but video generated by cores may end up in a different place, meaning you'd have to adjust it again. That would get very old.
georgemog
Posts: 18
Joined: Wed Aug 19, 2020 5:03 pm
Been thanked: 15 times

Re: Update All 2.0 Is Here

Unread post by georgemog »

This is a good script and I am enjoying using it. It did have me confused when I tried using it via SSH and the terminal was set for vt100, since every time I tried to bring up the settings menu it would crash. This was easily solved by changing the terminal type to xterm-256 colours. It might be worth checking the term type at some point.

This is the error that I see:-

Launching Update All


-------- Update All 2.0 ---------
The All-in-One Updater for MiSTer
---------------------------------
- Powered by Downloader 1.5 -


╔═════════════════════════════════════════════════╗
║ Become a patron to unlock exclusive features! ║
www.patreon.com/theypsilon
╚═════════════════════════════════════════════════╝

Reading sections from /media/fat/downloader.ini

Sequence:
- Main Distribution: MiSTer-devel
- JTCORES for MiSTer
- Names TXT: CHAR18 Common EU
- BIOS Database
- Arcade ROMs Database
- Coin-Op Collection
- MiSTer Extensions (wizzo)
- Arcade Organizer


*Press <UP>, To enter the SETTINGS screen.
*Press <DOWN>, To continue now.

Traceback (most recent call last):
File "/tmp/dont_download.zip/update_all/main.py", line 32, in main
exit_code = execute_update_all(
File "/tmp/dont_download.zip/update_all/main.py", line 48, in execute_update_all
return factory.create(env).full_run()
File "/tmp/dont_download.zip/update_all/update_all_service.py", line 122, in full_run
self._countdown_for_settings_screen()
File "/tmp/dont_download.zip/update_all/update_all_service.py", line 170, in _countdown_for_settings_screen
self._settings_screen.load_main_menu()
File "/tmp/dont_download.zip/update_all/settings_screen.py", line 57, in load_main_menu
run_ui_engine('main_menu', settings_screen_model(), self)
File "/tmp/dont_download.zip/update_all/ui_engine.py", line 76, in run_ui_engine
curses.wrapper(loader)
File "/usr/lib/python3.10/curses/__init__.py", line 94, in wrapper
return func(stdscr, *args, **kwds)
File "/tmp/dont_download.zip/update_all/ui_engine.py", line 74, in loader
ui.execute()
File "/tmp/dont_download.zip/update_all/ui_engine.py", line 101, in execute
ui_section_factory = self._ui_application.initialize_ui(self, self._screen)
File "/tmp/dont_download.zip/update_all/settings_screen.py", line 102, in initialize_ui
drawer_factory, theme_manager = self._settings_screen_printer.initialize_screen(screen)
File "/tmp/dont_download.zip/update_all/settings_screen_standard_printer.py", line 80, in initialize_screen
curses.init_pair(COLOR_PAIR_RED_OVER_WHITE, color_red, color_white)
ValueError: Color number is greater than COLORS-1 (-1).

Update All failed!
Insert Disk Two
Posts: 192
Joined: Sun May 24, 2020 7:59 pm
Has thanked: 10 times
Been thanked: 23 times

Re: Update All 2.0 Is Here

Unread post by Insert Disk Two »

Hopefully one day it will update console games too :)
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Update All 2.0 Is Here

Unread post by Malor »

The normal solution for terminal problems like that is for the script to use the ncurses library, which has a giant list of most terminals ever made. It does automatic translation when you tell it, say, 'move to position 36,14 and print "HI THERE!"'. The Mister distro has a small subset of that full library; there's about 20 terminal defintions there.

It's awkward to look at the script because theypsilon forces it to be downloaded fresh each time, and I don't feel like reverse-engineering that whole system yet *again*, but if he's using ncurses, then the display should work if you declare your terminal to be any of the twenty-ish under /usr/share/terminfo.

If that's how things are working, then the instant fix is to do what you did, and just run in xterm-256color, or any other terminal in /usr/share/terminfo. If for some reason you have some alternate terminal that you want instead, you should be able to source the file from a bigger distro and copy it into the right place under terminfo. That would last until the next Linux update, probably. The permanent solution would be to get them to add it to the base Mister image.

Of course, if the script is using command codes directly, then setting to xterm-256color is probably your only option. Well, that, or talking theypsilon into using ncurses.
jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Update All 2.0 Is Here

Unread post by jca »

Thanks for this update.
A few observations:
1. The first run takes a while to complete as it has to build a store by scanning all the files.
2. Arcade ROMs DB downloaded 53 files which I already had on my MISTer. 11 associated with Jotego cores were due to hash problem and the others to size difference. It is somewhat puzzling as previously I was using Mame Getter but is not a big deal.
3. I update my MISTer frequently but there were a few core updates when I ran V2, the cores were downloaded but the old ones were not deleted. I suppose it is related to the store not yet created and this will not happen the next time cores are updated.
Phantoon
Posts: 2
Joined: Sun May 24, 2020 9:12 pm

Re: Update All 2.0 Is Here

Unread post by Phantoon »

I've run Update All 1.4 a few times with Downloader activated from the Miscellaneous menu (there has been no pop up so I've activate it manually). It's done the usual job but I'm still on 1.4. Any idea what I'm doing wrong?

theypsilon
Scripting Wizard
Posts: 105
Joined: Sun May 24, 2020 8:20 pm
Been thanked: 40 times

Re: Update All 2.0 Is Here

Unread post by theypsilon »

Phantoon wrote: Sat Dec 24, 2022 1:29 am

I've run Update All 1.4 a few times with Downloader activated from the Miscellaneous menu (there has been no pop up so I've activate it manually). It's done the usual job but I'm still on 1.4. Any idea what I'm doing wrong?

Could you try again? I updated the launcher recently to insert some ntpdate mitigation code and forgot to update the MD5 hash for it on the 1.4, which needs to be correct for the migration to Update All 2.0, but now it's fixed.

Phantoon
Posts: 2
Joined: Sun May 24, 2020 9:12 pm

Re: Update All 2.0 Is Here

Unread post by Phantoon »

Thanks, it worked! Thanks for all of the incredible work everyone does here.

jd213
Posts: 93
Joined: Sun Aug 15, 2021 2:44 pm
Has thanked: 15 times
Been thanked: 16 times

Re: Update All 2.0 Is Here

Unread post by jd213 »

Update All removed most of the cores in the YCConsole folder and added links to empty folders. As far as I can tell, the YC functionality isn't yet included in most of the standard console cores, at least I couldn't find the YC option in the menus...

User avatar
sid
Posts: 31
Joined: Mon May 25, 2020 5:00 pm
Has thanked: 10 times
Been thanked: 10 times

Re: Update All 2.0 Is Here

Unread post by sid »

Does anybody know the difference between updateall, update and downloader script? Thx

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Update All 2.0 Is Here

Unread post by jca »

The downloader concerns only what is in the official repository: cores, mras, Linux, MISTer Main ... It is used by the update all script for the official downloads and can also be configured to download everything else.
I am not sure for the update script but as the update all script is not official the update script can be used after a fresh install to update MISTer.

FoxbatStargazer
Top Contributor
Posts: 994
Joined: Thu Dec 10, 2020 5:44 pm
Has thanked: 309 times
Been thanked: 228 times

Re: Update All 2.0 Is Here

Unread post by FoxbatStargazer »

jd213 wrote: Tue Feb 28, 2023 2:42 pm

Update All removed most of the cores in the YCConsole folder and added links to empty folders. As far as I can tell, the YC functionality isn't yet included in most of the standard console cores, at least I couldn't find the YC option in the menus...

See these new lines in mister.ini.

Code: Select all

vga_mode=rgb           ; supported modes: rgb, ypbpr, svideo, cvbs. rgb is default.
ntsc_mode=0            ; Only for S-Video and CVBS vga_mode. 0 - normal NTSC, 1 - PAL-60, 2 - PAL-M.

Always check the online mister.ini for settings for new features, the updater will not overwrite or add them to your existing mister.ini!

jd213
Posts: 93
Joined: Sun Aug 15, 2021 2:44 pm
Has thanked: 15 times
Been thanked: 16 times

Re: Update All 2.0 Is Here

Unread post by jd213 »

OK, thanks. I saw discussion here in the forum about the new framework but I didn't see a mention of ini changes.

Post Reply