Shared Folder Discussion

ZigZag
Posts: 99
Joined: Sun Jun 28, 2020 7:05 am
Has thanked: 1 time
Been thanked: 5 times

Re: Shared Folder Discussion

Unread post by ZigZag »

d0pefish wrote: Sat Dec 12, 2020 9:22 pm Hi - first post here :)

I took a break from mt32-pi to try and fix some of the bugs with the Amiga shared folder system....
Great, thanks. I've had a long break from MiSTer but there's lots to catch up on. So impressed by the amount of development this project has.
kolla
Posts: 188
Joined: Sat Jun 13, 2020 7:56 am
Has thanked: 17 times
Been thanked: 33 times

Re: Shared Folder Discussion

Unread post by kolla »

This is great!

I have a few uhm points I just wish to air :)
* a version string in MiSTerFileSystem would be useful, just a running number and the build date.
* DEVS:Mountlist is kinda old school, modern day (post 1.x?) is to use a DEVS:DOSDrivers/SHARE (well, I call it MR0) - maybe provide both options?
Michael1260
Posts: 36
Joined: Wed Sep 16, 2020 11:42 am
Been thanked: 10 times

Re: Shared Folder Discussion

Unread post by Michael1260 »

Nice , it fixes my problems with Voodoo-X ... can't load files from Shared folder. I have to copy them on another VHD. Now, it's working fine :-)

ps: probably this : Some applications would fail to load some files due to negative seek offsets being corrupted by unsigned conversion.
User avatar
d0pefish
mt32-pi Creator
Posts: 39
Joined: Fri Dec 11, 2020 6:38 pm
Location: Newcastle-upon-Tyne, UK
Has thanked: 27 times
Been thanked: 29 times
Contact:

Re: Shared Folder Discussion

Unread post by d0pefish »

kolla wrote: Sun Dec 13, 2020 11:30 am * a version string in MiSTerFileSystem would be useful, just a running number and the build date.
* DEVS:Mountlist is kinda old school, modern day (post 1.x?) is to use a DEVS:DOSDrivers/SHARE (well, I call it MR0) - maybe provide both options?
Sure, I actually thought about the second one myself. :)

I've had another idea; we can treat this driver as an AmigaDOS "handler" instead of a "FileSystem". FileSystems are paired with devices, hence the need for a "dummy.device" in the mountlist, but if we have a "MiSTer-Handler" instead, we can remove the dummy.device. This is how things like FTPMount and (I think) SMBFS work.

I've tested this idea and it seems to work nicely. Also, I've removed the BCPL stuff and the ASM/binary patching that happens at build time and it seems to work fine, at least on 3.x. I need to investigate why this was done in the first place, and also set up a 1.3 environment and do more testing.

Re: international filenames; I could attempt a solution for this but it's not straightforward. It'd be a lot easier if we could get iconv into the Linux system. iconv itself is part of the libc already, but I think the locale definitions/support files that make it work aren't present; we basically just need to be able to bounce back and forth between Latin-1 and UTF-8 safely and iconv is ideal for this. This would not magically give the Amiga side Unicode; it'd simply expand the current level of support (ASCII only) to full Amiga capabilities (Latin-1).

I don't currently know what's involved in (re)building the Linux rootfs to add extra libraries.
User avatar
Sorgelig
Site Admin
Posts: 877
Joined: Thu May 21, 2020 9:49 pm
Has thanked: 2 times
Been thanked: 211 times

Re: Shared Folder Discussion

Unread post by Sorgelig »

It would be good to keep linux filenames in Latin code page. If there some national names required then it should be moved to local Amiga disk through compatible archiver.
Keep things simple. Different retro system may have different code pages, so better to keep them locally.
User avatar
limi
Top Contributor
Posts: 619
Joined: Sun May 24, 2020 6:53 pm
Has thanked: 135 times
Been thanked: 418 times

Re: Shared Folder Discussion

Unread post by limi »

Yeah, I believe both the Linux and Amiga side use Latin-1 (ISO8859-1), but something goes wrong when copying files? Or maybe when unarchiving on a shared drive? I should see if I can make a minimal test case that shows it.
kolla
Posts: 188
Joined: Sat Jun 13, 2020 7:56 am
Has thanked: 17 times
Been thanked: 33 times

Re: Shared Folder Discussion

Unread post by kolla »

From what I recall, there is a “Control” mount option, which could be used to configure charset options. An option to set volume name would also be welcome.

(Or for that matter, a option to specify path on the Linux side (so one can have several mounts), though then I would actually much rather see superfast network link and just use existing network filesystem :) )
User avatar
d0pefish
mt32-pi Creator
Posts: 39
Joined: Fri Dec 11, 2020 6:38 pm
Location: Newcastle-upon-Tyne, UK
Has thanked: 27 times
Been thanked: 29 times
Contact:

Re: Shared Folder Discussion

Unread post by d0pefish »

The reason why we get failures is because attempting to create a file or directory on an exFAT file system with a Latin-1 encoded name without translation to Unicode is illegal.

Example:

Code: Select all

mkdir("f\xf6\xf6", S_IRWXU | S_IRWXG | S_IRWXO)
...(directory name "föö" encoded as Latin-1) will return EINVAL if the current rectory is /media/fat, but succeed if in, e.g. /root.
Sorgelig wrote: Tue Dec 15, 2020 2:24 pm It would be good to keep linux filenames in Latin code page.
Even if the Linux filesystem supported Latin-1, the user is then faced with annoying filenames, or even "mojibake", because the system is almost certainly expecting Unicode, and their terminal doesn't know how to correctly represent the characters:
Clipboard Image.jpg
Clipboard Image.jpg (20.51 KiB) Viewed 8226 times
This could get even worse if you are trying to manage these files over a Samba share or other remote access method that may need yet another conversion to work correctly. It's jut not practical.

I maintain that translating to UTF-8 and back again is the best experience for the user, and not much of a problem for us because converting between Latin-1 and UTF-8 is trivial.

If we don't want to add a Latin-1 locale to the Linux system and enable the use of iconv to perform the conversion via glibc, a couple of hand-written conversion functions takes care of this easily.

I've implemented some simple conversion functions and it seems to work very nicely. You can extract, e.g. BoulderDäsh on the Amiga side, and you can even see the correct names if accessing the Amiga shared directory through Samba:
Clipboard Image 2.jpg
Clipboard Image 2.jpg (17.13 KiB) Viewed 8226 times
If the Linux side contains file/directory name that cannot be converted to Latin-1 perfectly, it is simply ignored and won't appear on the Amiga side.

I've attached a new MiSTer executable (again, place on root of SD card) - you shouldn't need to update the Amiga driver.

Please test, and if we're happy, I'll send a pull request. :)
Attachments
MiSTer.zip
(390.61 KiB) Downloaded 190 times
bbond007
Top Contributor
Posts: 519
Joined: Tue May 26, 2020 5:06 am
Has thanked: 85 times
Been thanked: 198 times

Re: Shared Folder Discussion

Unread post by bbond007 »

d0pefish wrote: Mon Dec 21, 2020 2:25 pm Please test, and if we're happy, I'll send a pull request. :)
Downloaded...

Thanks for looking into this 8-)
User avatar
Caldor
Top Contributor
Posts: 930
Joined: Sat Jul 25, 2020 11:20 am
Has thanked: 112 times
Been thanked: 111 times

Re: Shared Folder Discussion

Unread post by Caldor »

Btw ByteMaverics made a version of the MiSTer filesystem that works for Kickrom 1.3 and WB 1.3:
https://misterfpga.org/viewtopic.php?f= ... 979cbb30e5

I have not tested it yet, but I suspect it might work as a replacement for the current, making it more compatible without losing features... but I might be wrong, the newer version is newer for a reason after all. But in some cases I have replaced some C files with 1.3 files to get a more compatible OS, which helped me make a bit of a hybrid OS that I could boot into WB 1.3 and WB 3.1 depending on what ROM I had loaded.
User avatar
d0pefish
mt32-pi Creator
Posts: 39
Joined: Fri Dec 11, 2020 6:38 pm
Location: Newcastle-upon-Tyne, UK
Has thanked: 27 times
Been thanked: 29 times
Contact:

Re: Shared Folder Discussion

Unread post by d0pefish »

Ah, you just reminded me I need to polish up the above Latin-1 stuff and send a PR.
I didn't realise the current release was not built for 1.3 - that wasn't the intention, so I'll check that too.
User avatar
Caldor
Top Contributor
Posts: 930
Joined: Sat Jul 25, 2020 11:20 am
Has thanked: 112 times
Been thanked: 111 times

Re: Shared Folder Discussion

Unread post by Caldor »

d0pefish wrote: Wed Mar 10, 2021 8:38 pm Ah, you just reminded me I need to polish up the above Latin-1 stuff and send a PR.
I didn't realise the current release was not built for 1.3 - that wasn't the intention, so I'll check that too.
I just did a test, and the one linked in the thread I linked does seem to work with WB 1.3 while it also works with WB 3.1.

Have not tested with with 3.9, but I suspect it works with that as well.
ByteMavericks
Posts: 53
Joined: Tue Oct 27, 2020 4:52 pm
Has thanked: 69 times
Been thanked: 11 times

Re: Shared Folder Discussion

Unread post by ByteMavericks »

The code for misterfilesystem in MisterFPGA github is being linked to KS2+ startup code - using the 1.3SDK and linking the associated startup sorted it. I built it using the Docker environment from Niklas Ekstrom's original Pi314 build environment... the only issue I had compiling was needing to deference a pointer to the name in the source.

I also don't see the need for the dummy device: it's not mandated to have a driver for a file system, so I could attempt to remove it if it helps?

Somewhat related, I have written a script to download and install the extras from the linux side - https://github.com/ByteMavericks/Minimi ... erDisks.sh . It needs a configured AmigaOS envrironment and will install Picasso96, Mister RTG, and the Share filessytem and mountlist.

Massively impressed with MisterFPGA Minimig, it is the Amiga I am using daily :D - thanks everyone involved! I'd love to contribute further.
ByteMavericks
Posts: 53
Joined: Tue Oct 27, 2020 4:52 pm
Has thanked: 69 times
Been thanked: 11 times

Re: Shared Folder Discussion

Unread post by ByteMavericks »

The only thing I have found that doesn't work is that info under 1.3 returns "not a dos disk" for share, and I can't quite see why...!
User avatar
d0pefish
mt32-pi Creator
Posts: 39
Joined: Fri Dec 11, 2020 6:38 pm
Location: Newcastle-upon-Tyne, UK
Has thanked: 27 times
Been thanked: 29 times
Contact:

Re: Shared Folder Discussion

Unread post by d0pefish »

ByteMavericks wrote: Wed Mar 10, 2021 8:55 pm I don't see the need for the dummy device: it's not mandated to have a driver for a file system, so I could attempt to remove it if it helps?
This is true, and in my local tree with the international fix I've already done this. :) The way this shared driver works is more like a handler than a filesystem, so a device is completely unnecessary.

I've also added a DOSDriver icon that on 2.x+ you can just drop into DEVS:DOSDrivers instead of using the MountList.
ByteMavericks
Posts: 53
Joined: Tue Oct 27, 2020 4:52 pm
Has thanked: 69 times
Been thanked: 11 times

Re: Shared Folder Discussion

Unread post by ByteMavericks »

Makes a lot of sense to have that icon for ks2+ - one thing I should do to the install script is use "join" rather than just copying things over: will sort that.
User avatar
Caldor
Top Contributor
Posts: 930
Joined: Sat Jul 25, 2020 11:20 am
Has thanked: 112 times
Been thanked: 111 times

Re: Shared Folder Discussion

Unread post by Caldor »

For me it actually works best under WB 1.3, not sure what I did wrong in 3.1, but I cannot open it as a disk. But I can in WB 1.3.

Btw. I have a special 1.3 kickrom that supports SCSI. But not sure that explains why it works for me as a drive. I usually replace icons for stuff when they are not supported by WB1.3 using DOpus.
ByteMavericks
Posts: 53
Joined: Tue Oct 27, 2020 4:52 pm
Has thanked: 69 times
Been thanked: 11 times

Re: Shared Folder Discussion

Unread post by ByteMavericks »

I too have patched the SCSI rom - there's a process that could do with a script, too...
User avatar
pepito
Posts: 65
Joined: Tue Jul 20, 2021 10:07 am
Has thanked: 5 times
Been thanked: 7 times

Re: Shared Folder Discussion

Unread post by pepito »

NML32 wrote: Sat Aug 29, 2020 2:58 pm
Darthcustom wrote: Sat Aug 29, 2020 2:42 pm I want to configure Shared folder in my Mister. I need an ADF to transfer the file MiSTer_share.lha to the Minimig running in Mister. Can anybody provide a link to the ADF file so that I can insert it in the floppy?

Thanks and regards
https://mega.nz/folder/PeZDnAwD#BfYi8JkN_WfVrIzxTTfMqQ
If you could re-upload this ADF, it will help me :D
thanks
User avatar
NML32
Posts: 288
Joined: Sun May 24, 2020 6:57 pm
Has thanked: 272 times
Been thanked: 43 times

Re: Shared Folder Discussion

Unread post by NML32 »

pepito wrote: Fri Apr 08, 2022 2:08 pm
NML32 wrote: Sat Aug 29, 2020 2:58 pm
Darthcustom wrote: Sat Aug 29, 2020 2:42 pm I want to configure Shared folder in my Mister. I need an ADF to transfer the file MiSTer_share.lha to the Minimig running in Mister. Can anybody provide a link to the ADF file so that I can insert it in the floppy?

Thanks and regards
https://mega.nz/folder/PeZDnAwD#BfYi8JkN_WfVrIzxTTfMqQ
If you could re-upload this ADF, it will help me :D
thanks
Re-Shared https://mega.nz/folder/yLo1xDQK#2r2yHOAF-D4VstE6PBOXjg
User avatar
pepito
Posts: 65
Joined: Tue Jul 20, 2021 10:07 am
Has thanked: 5 times
Been thanked: 7 times

Re: Shared Folder Discussion

Unread post by pepito »

CLEAN :!: 8-)
txs.
Malor
Top Contributor
Posts: 860
Joined: Wed Feb 09, 2022 11:50 pm
Has thanked: 64 times
Been thanked: 194 times

Re: Shared Folder Discussion

Unread post by Malor »

Has there been any thought about including the files for shared folder support on the WheelDriverAkiko.adf image? Assuming they fit, it wouldn't even take any additional space, and it would be a little easier to get fresh Workbench installs up and running. I could edit the image and put it up somewhere if there's any interest... I think it would take maybe ten minutes.
Post Reply