Right Way to Create a Blank HDD With PCEm?

Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by Malor »

Somewhere along the way, I'm pretty sure DOS added the "/mbr" argument to fdisk, which I believe loads that version's boot code. I think you have to partition the drive, make it active, MBR it, format it, SYS it from an existing DOS floppy, and then copy whatever parts of DOS you want.

I'm not sure how much of that the DOS installer does, but if you wipe the partition info and then launch the installer, I would imagine it would write a new, valid MBR.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

You are right:
"Secret fdisk switches
FDISK /MBR Command used to rewrite the master boot record, see: Do you have additional information on fdisk /mbr?

Do you have additional information on fdisk /mbr?
The fdisk /mbr command is an undocumented switch used with the fdisk command (MS-DOS 5.0 and higher) that recreates the master boot record on a hard drive."

I will try it on my unbootable VHD.

akeley
Top Contributor
Posts: 1309
Joined: Mon May 25, 2020 7:54 pm
Has thanked: 422 times
Been thanked: 409 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by akeley »

The hdds I've created in PCem or in MiSTer boot ok, but I'm stumped by the problem where I can only fill them with 500 GB of content, after this I get "run out of space error" and NC shows 0 bytes free (despite disks having between 1-1.3 GB capacity and their size being displahhhyed correctly) . This happened both when I was copying files to them in Windows and also via ao486.

@kitune-san: could you try this with your 1.2GB image when you have a spare minute? Just fill it up with anything and see if you can do better than 500GB? Unless somebody just knows the answer to this problem, which I suppose is something simple I'm missing.

Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by Malor »

Weren't the first BIOSes limited to 512MB? There was a limitation in Cylinder/Head/Sector addressing. Then I think there was a limit at 2.1TB that also needed to be overcome, but perhaps you're running into the original size limit?

If you partition, format, and use them strictly with DOS 7, I'm pretty sure that'll get around the 512MB limit. I think 7 still had the 2TB limit.

An earlier DOS may also work. I'm not sure when the first version of CHS addressing was fixed. DOS 6.22 might be just fine.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

So FDISK /MBR worked and my unbootable disk became bootable.
There is not really any information on this switch. In my case I booted from a floppy and had 2 VHDs mounted. The command does not ask which disk to fix, it just fixes C: and also does not destroy the partition table which is also in the MBR.
There is also another switch FDISK /CMBR <Disk Number> but it does not exist in MSDOS 6.22.

kitune-san
Top Contributor
Posts: 401
Joined: Wed May 18, 2022 11:20 am
Has thanked: 127 times
Been thanked: 412 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by kitune-san »

if the addressing mode for that drive is NORMAL in the xtide boot menu, it is restricted to 528MB.

Seems to automatically go into this mode when the cylinder count is small.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

A few hours ago I created a VHD of 1GB using the dd command. I used FDISK to create a partition of the max size, formatted it.
Now I am runnings ScanDisk on it and it has been scanning the disk for a few hours. It is now at 43%, let see what happens.

kitune-san
Top Contributor
Posts: 401
Joined: Wed May 18, 2022 11:20 am
Has thanked: 127 times
Been thanked: 412 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by kitune-san »

After copying a 500+ MB file using Windows, I copied a 10 MB file using PCXT.
I am not sure if this method is correct, but I was able to copy more than 500 MB.(not GB)

Your disk may be using all FAT allocation units or has reached the maximum number of files that can be created.

Attachments
2022-12-15 205609.png
2022-12-15 205609.png (56.53 KiB) Viewed 6074 times
kitune-san
Top Contributor
Posts: 401
Joined: Wed May 18, 2022 11:20 am
Has thanked: 127 times
Been thanked: 412 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by kitune-san »

akeley wrote: Wed Dec 14, 2022 6:54 pm

This happened both when I was copying files to them in Windows and also via ao486.

If same problem occurs on AO486 or Windows, it does not seem to be a bug in PCXT core.

Be careful when creating many files in the FAT16 root directory.
Only up to 512 files can be created in it.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

By design, FAT carries a few limitations:

Code: Select all

Maximum file size is limited to 4,294,967,295 bytes
Maximum files within folder is 65,536 (i.e. a directory must not be larger than 2,097,152 bytes).
A root directory on FAT12 and FAT16 can hold max 512 files/folders (there is no such limitation for FAT32)
FAT type depends on maximum and minimum number of clusters:
    FAT12 maximum clusters: 4,084
    FAT16 minimum clusters: 4,085
    FAT16 maximum clusters: 65,524
    FAT32 minimum clusters: 65,525
Maximum volume size depends on FAT type:
    FAT12: 256MB (for 64kB clusters)
    FAT16: 4GB (for 64kB clusters)
    FAT32: 2TB (4G of 512B sectors)

The big limitation is the root diectory.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

@akeley you must have reached the 512 entries possible in the root directory of your drive.
I had no problem filling up my 1GB VHD.

Attachments
20221215_130058-screen.png
20221215_130058-screen.png (184.47 KiB) Viewed 5991 times
akeley
Top Contributor
Posts: 1309
Joined: Mon May 25, 2020 7:54 pm
Has thanked: 422 times
Been thanked: 409 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by akeley »

I don't have many files in the root, just a handful of normal DOS ones. No more than 10 I suppose.

Overall the collection I need it for has a huge amount of files - approx 74K files in 4000+ directories. But all this is kept in separate directories, not root. Not sure if the amount of files can affect things in general, I was reading about space loss due to cluster sizes (so 100kb file becomes 128kb) in FAT 16 but still even with that considered not sure it could've accounted for my case (only 530 GB fitting into a 1GB+ hdd).

This 530MB limit seems more related to some of the other ATA/HDD limits but I'm not sure why. I also read that FAT16 is capable of making 2GB partitions. And I would've though that the BIOSes in ao 486 and PCem I've used were already capable of handling that too.

@jca: any chance you could zip and upload this (empty) 1GB Hdd of yours somehwere? I'd like to test it vs my files. Even though in theory seems we are making them in the same way. If you have a minute you could post the step by step how to make it again, I will try to replicate.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

Here it is.

Attachments
Test.zip
(1.17 MiB) Downloaded 127 times
jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

If you want to make one yourself you can refer to viewtopic.php?p=64696#p64696.
Change usb0 to fat if you use an SD card.
Change count from 115 to whatever number of MB you want. I used 1000 which is slightly less than 1GB.
I placed a bootable MSDOS 6.22 for C and Test for D.
I did not place a system on D, just FORMAT D:.

akeley
Top Contributor
Posts: 1309
Joined: Mon May 25, 2020 7:54 pm
Has thanked: 422 times
Been thanked: 409 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by akeley »

Thanks for the vhd. I tried it with my files and I could only fill 442MB out of the 1022MB of its capacity. Then I tried copy some big files from a CDROM (like 10x100 MB files) and in this case I was able to fill it to the max.

So this means the problem lies with the small file/sector handling, possibly related to the issue with FAT I linked above. Unless I'm making some basic mistake or there is something strange with my setup - but the same thing is happening in PCem and ao486, so...

If anybody has an idea how to remedy this I'm all ears.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

I found this:
Max. number of files
FAT12: 4,068 for 8 KB clusters
FAT16: 65,460 for 32 KB clusters
FAT32: 268,173,300 for 32 KB clusters

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

I also found this:
It greatly depends on the cluster size you chose, too, and that varies from one file system to the next. Choose 64KB as cluster size, and you won't fit as many files as you could with 12KB, for example, because to lodge a 32KB file, you'd (theoretically) use a whole 64KB cluster. There's no definite answer to your question, because it's not purely mathematical. If your disk is a system disk vs a storage disk, it'll change parameters too, since there'll probably be some space saved for the system if the former. Method of formatting may change a lot of things too.

Take your disk which filled with 442MB, run CHKDSK and see what it says, it may give you some idea.

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

I checked with my disk and the cluster size is 16K, I could not make any calculation because it is empty.
For each file you will waste between 0 and 16K-1 bytes.
Take the worst case scenario and round it to 16K, multiply this by the number of files given by CHKDSK and see what it gives.
I made test with a 400M disk:
8K per cluster.
3550 files.
Total disk used 321.5M.
Max loss almost 28M.
I did not count directories.
That is around 11% max loss.

akeley
Top Contributor
Posts: 1309
Joined: Mon May 25, 2020 7:54 pm
Has thanked: 422 times
Been thanked: 409 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by akeley »

Thanks for looking into this. I've asked around and the consensus seems to be that this is indeed the "slack space" problem, which is excarberated when samll files come into play. And early games are all about small files.

Seems there is no real solution for this issue in this case, just to accept it. Perhaps I can use 2 images, seeing as at 500MB-1GB the clusters are smaller (16Kb).

akeley
Top Contributor
Posts: 1309
Joined: Mon May 25, 2020 7:54 pm
Has thanked: 422 times
Been thanked: 409 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by akeley »

The vhds created in PCem or in Linux here are not mountable in Windows 10 - presumably because of FAT16. Do you know of any simple/direct way of doing this (ie not via a VM or PCem)?

User avatar
kathleen
Top Contributor
Posts: 413
Joined: Fri Jun 26, 2020 4:23 am
Location: Belgium
Has thanked: 226 times
Been thanked: 134 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by kathleen »

akeley wrote: Mon Jan 09, 2023 6:58 am

The vhds created in PCem or in Linux here are not mountable in Windows 10 - presumably because of FAT16. Do you know of any simple/direct way of doing this (ie not via a VM or PCem)?

On my side, what I do to add/remove files to my PC-XT HDD, I use Power ISO which works very well (with FAT16 among others).
It even works if I mount the HDD when it is on the mister side (My Mister is connected to the network via an Ethernet cable and not by WiFi), just need to pay attention to do not "edit" the HDD once it is in use by the core of course.

かすりん

flynnsbit
Top Contributor
Posts: 552
Joined: Sun May 24, 2020 8:07 pm
Has thanked: 182 times
Been thanked: 308 times
Contact:

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by flynnsbit »

I use ImDisk and it gives you mounting abilities like windows does for those FAT16 images. https://sourceforge.net/projects/imdisk-toolkit/

jca
Top Contributor
Posts: 1911
Joined: Wed May 27, 2020 1:59 pm
Has thanked: 145 times
Been thanked: 454 times

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by jca »

I am under the impression that there is some confusion:
You can mount a FAT 16 vhd under Windows.
If you cannot mount it it is because it is not a Windows VHD. Windows VHDs have special information at the end of the disc which must be present.
On my system I created a 20MB VHD under Windows, I can upload/download it to/from MISTer and transfer files to/from MISter. I mount it as a secondary drive under PC/XT.

Farsil
Posts: 7
Joined: Thu Mar 16, 2023 9:42 am
Been thanked: 1 time

Re: Right Way to Create a Blank HDD With PCEm?

Unread post by Farsil »

For the record, I managed to create a working HDD image from within MiSTer with the following instructions. You need a bootable MS-DOS floppy disk:

1.
Open up a shell with F9

2.
Navigate to the desired path

3.
Run the following command:

$ dd if=/dev/zero of=image.vhd bs=516096 count=N

This will create a file named image.vhd which is about 2 * N MiB in size. The number 516096 was chosen to match the size of a cylinder that has 16 heads and 63 sectors (16 * 63 * 512 bytes per sector), because to my understading XTIDE likes to address disk images by using the maximum allowed amount of heads and sectors per cylinder but you can use whatever number you like, the final image will be count * bs bytes large.

4.
Open the PCXT core

5.
Mount the bootable MS-DOS disk in Floppy A: and image.vhd in IDE 0-0. For some reason, this won't work if the HDD is mounted as a slave.

6.
Boot from A: and run:

A> fdisk

Create a primary MS-DOS partition, use all the available size. fdisk will reboot.

7.
If you need the disk to be bootable, boot again from A: and run:

A> sys c:

(if it doesn't work, you may have to format c: /s)

Post Reply