Individually Bootable VHDs for Each Game

Bas
Top Contributor
Posts: 489
Joined: Fri Jan 22, 2021 4:36 pm
Has thanked: 55 times
Been thanked: 210 times

Re: Individually Bootable VHDs for Each Game

Unread post by Bas »

Oh it's ugly as fudge right now. It uses yq but in very ad-hoc ways. I should really wrap that mess into functions that can deal with errors.

User avatar
NML32
Posts: 288
Joined: Sun May 24, 2020 6:57 pm
Has thanked: 268 times
Been thanked: 43 times

Re: Individually Bootable VHDs for Each Game

Unread post by NML32 »

I haven't had any luck getting Beneath a Steel Sky Floppy version to work.
I've tried with different memory settings.
Either I get an error when launching sky.exe. Error 16 bit code too large or just a frozen screen.
I know the install files are good, as I was able to load the game on my DOS 7.1 and DOS 8.0 VHDs.
My DOS 8 uses QEMM memory manager.

I also noticed a bug using the DOSContainer EMM386 default setting. A special character (a back arrow) is added to the command.

One note: Beneath a Steel Sky CD version loaded fine building from DOSContainer.

Maybe someone else could try getting the floppy version working on DOS 6.22.

Emubastard
Posts: 14
Joined: Wed May 27, 2020 10:06 pm

Re: Individually Bootable VHDs for Each Game

Unread post by Emubastard »

Found a booter VHD thing on archive.org for Star Trek 25th Anniversary edition. Maybe the author will most more like this.

Search "mister fpga startrek"

I like how this just boots and plays. I couldn't get this particular game working right in the Top 300 game collection. Also don't like having to deal with extra menus and memory manager selections.

User avatar
NML32
Posts: 288
Joined: Sun May 24, 2020 6:57 pm
Has thanked: 268 times
Been thanked: 43 times

Re: Individually Bootable VHDs for Each Game

Unread post by NML32 »

I really like this project. As of now, I've made a little over 100 DOS VHDs using the DOSContainer script. Now, if I could only get the VHDs to load correctly using a TapTo NFC.
Has anyone had any success loading an ao486 VHD using TapTo NFCs?

Bas
Top Contributor
Posts: 489
Joined: Fri Jan 22, 2021 4:36 pm
Has thanked: 55 times
Been thanked: 210 times

Re: Individually Bootable VHDs for Each Game

Unread post by Bas »

Time for an update. I've been silent for a while. That's a combination of my day job plus a rewrite of DOSContainer in Rust. I figured that the shell script is usable for now. It works, which is better than the vaporware DOSContainer was for a long time, but it's a maintenance nightmare. I'm currently writing the whole thing in Rust: no more leaning on external dependencies of any kind, just a single binary that you can drop on your MiSTer itself and things will always Just Work™ and be Bloody Fast™.

For now "things" are still extremely limited. As I mentioned before, DOSContainer must be 100% accurate to contemporary operating systems. That's the entire point of it. You should not be able to spot any difference between a hard drive formatted in a period appropriate PC versus an image created by DOSContainer. Now while Rust is a powerhouse of a language compared to Bash, I'm effectively rewriting large parts of GNU mtools for this. The other dependencies fortunately do have libraries available. Dealing with all the FAT versions and the fallout of Microsoft's shenanigans from MS-DOS 2.00 on up to Win95 is quite a task (there are 34 numbered versions that I know of, ignoring differences between the many OEM's of the time),

As with the Bash version, I will eventually open source the Rust code as well and provide builds for Windows, macOS (Intel/ARM) and the ARM-side of MiSTer itself. Since you'll have the source, I expect self-service for other platforms like Linux PC's or Raspberry Pi's to not be a problem.

Current progress:

  • Read+verify and create real, valid VHD format disk images.
  • Add a VHD footer to an existing raw disk image, promoting them to valid VHD images (having an out-of-band place to store physical disk geometry if vital!).
  • Read/write MBR to/from VHD images, read/write partition tables + entries.
  • Read/write FAT12 and FAT16(B) allocation tables.
  • Parse the existing YAML format.

Lots of todo's:

  • Finish the FAT implementation (FAT32, BPB, bootcode, read/write of files + attributes)
  • Downloading of assets
  • Read/write ZIP-files
  • Validate hashes of ZIP-files
  • Copy files from ZIP to the disk image's FAT filesystem.
  • Configuration file templating for the DOS system.
  • Implement all of the quirks from MS-DOS 2.00 on up to Win95

Stretch goals:

  • Support IBM PC-DOS
  • Support DR-DOS
  • Support FreeDOS

Plans for now: finish the Rust version in favor of the Bash version. If you spot any glaring bugs in the Bash version, let me know. Other than that, I'm stopping development on that in favor of the Rust version. Expected timeline: this is going to take months if not over a year.

AmintaMister
Posts: 274
Joined: Thu Sep 16, 2021 10:54 pm
Has thanked: 707 times
Been thanked: 43 times

Re: Individually Bootable VHDs for Each Game

Unread post by AmintaMister »

Bas wrote: Wed Feb 21, 2024 1:59 pm

Time for an update. I've been silent for a while. That's a combination of my day job plus a rewrite of DOSContainer in Rust. I figured that the shell script is usable for now. It works, which is better than the vaporware DOSContainer was for a long time, but it's a maintenance nightmare. I'm currently writing the whole thing in Rust: no more leaning on external dependencies of any kind, just a single binary that you can drop on your MiSTer itself and things will always Just Work™ and be Bloody Fast™.

For now "things" are still extremely limited. As I mentioned before, DOSContainer must be 100% accurate to contemporary operating systems. That's the entire point of it. You should not be able to spot any difference between a hard drive formatted in a period appropriate PC versus an image created by DOSContainer. Now while Rust is a powerhouse of a language compared to Bash, I'm effectively rewriting large parts of GNU mtools for this. The other dependencies fortunately do have libraries available. Dealing with all the FAT versions and the fallout of Microsoft's shenanigans from MS-DOS 2.00 on up to Win95 is quite a task (there are 34 numbered versions that I know of, ignoring differences between the many OEM's of the time),

As with the Bash version, I will eventually open source the Rust code as well and provide builds for Windows, macOS (Intel/ARM) and the ARM-side of MiSTer itself. Since you'll have the source, I expect self-service for other platforms like Linux PC's or Raspberry Pi's to not be a problem.

Current progress:

  • Read+verify and create real, valid VHD format disk images.
  • Add a VHD footer to an existing raw disk image, promoting them to valid VHD images (having an out-of-band place to store physical disk geometry if vital!).
  • Read/write MBR to/from VHD images, read/write partition tables + entries.
  • Read/write FAT12 and FAT16(B) allocation tables.

Lots of todo's:

  • Finish the FAT implementation (FAT32, BPB, bootcode, read/write of files + attributes)
  • Downloading of assets
  • Read/write ZIP-files
  • Validate hashes of ZIP-files
  • Copy files from ZIP to the disk image's FAT filesystem.
  • Parse the existing YAML format.
  • Configuration file templating for the DOS system.
  • Implement all of the quirks from MS-DOS 2.00 on up to Win95

Stretch goals:

  • Support IBM PC-DOS
  • Support DR-DOS
  • Support FreeDOS

Plans for now: finish the Rust version in favor of the Bash version. If you spot any glaring bugs in the Bash version, let me know. Other than that, I'm stopping development on that in favor of the Rust version. Expected timeline: this is going to take months if not over a year.

Only one word from a DOS games philological preservation lover:

THANKS!

Emubastard
Posts: 14
Joined: Wed May 27, 2020 10:06 pm

Re: Individually Bootable VHDs for Each Game

Unread post by Emubastard »

How does one go about releasing a dos collection? I assume putting link in here is not valid.

loki666
Posts: 6
Joined: Mon Oct 30, 2023 3:54 pm

Re: Individually Bootable VHDs for Each Game

Unread post by loki666 »

Hi @Bas

First thanks for your work, I was looking for something like this for my MiSTer.

I gave a run at your build.sh script, but the resulting .vhd doesn't seems to be bootable (or I'm missing something)
I do get some errors while executing the scipt, something like Error: no matches found (I guess it comes from the yaml parser)
I tried both flashback and lemmings.

Philippe

Bas
Top Contributor
Posts: 489
Joined: Fri Jan 22, 2021 4:36 pm
Has thanked: 55 times
Been thanked: 210 times

Re: Individually Bootable VHDs for Each Game

Unread post by Bas »

Errors usually mean that the output won't work. This is a bit shoddy and it's the main reason why I'm rewriting the whole thing in a language that gives me full control over the whole process. What platform are you running the script from? And which version of yq do you have installed? It turns out that yq is quite sensitive. You're welcome to PM the output.

loki666
Posts: 6
Joined: Mon Oct 30, 2023 3:54 pm

Re: Individually Bootable VHDs for Each Game

Unread post by loki666 »

Im' on Linux
yq (https://github.com/mikefarah/yq/) version v4.40.5

Bas
Top Contributor
Posts: 489
Joined: Fri Jan 22, 2021 4:36 pm
Has thanked: 55 times
Been thanked: 210 times

Re: Individually Bootable VHDs for Each Game

Unread post by Bas »

Which Linux is that exactly? I'd like to try and reproduce the bug.

loki666
Posts: 6
Joined: Mon Oct 30, 2023 3:54 pm

Re: Individually Bootable VHDs for Each Game

Unread post by loki666 »

Code: Select all

Linux bob 5.15.0-94-generic #104~20.04.1-Ubuntu SMP Tue Jan 16 13:34:09 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Post Reply