How can I let MiSTer support Chinese Fonts?
- Newsdee
- Top Contributor
- Posts: 830
- Joined: Mon May 25, 2020 1:07 am
- Has thanked: 98 times
- Been thanked: 209 times
Re: How can I let MiSTer support Chinese Fonts?
- aberu
- Core Developer
- Posts: 1162
- Joined: Tue Jun 09, 2020 8:34 pm
- Location: Longmont, CO
- Has thanked: 244 times
- Been thanked: 404 times
- Contact:
Re: How can I let MiSTer support Chinese Fonts?
https://github.com/MiSTer-devel/Fonts_MiSTer
The current framework only supports fonts like these.
https://en.wikipedia.org/wiki/CJK_characters#Encoding
The number of characters required for complete coverage of all these languages' needs cannot fit in the 256-character code space of 8-bit character encodings, requiring at least a 16-bit fixed width encoding or multi-byte variable-length encodings. The 16-bit fixed width encodings, such as those from Unicode up to and including version 2.0, are now deprecated due to the requirement to encode more characters than a 16-bit encoding can accommodate—Unicode 5.0 has some 70,000 Han characters—and the requirement by the Chinese government that software in China support the GB 18030 character set.
- jotego
- Core Developer
- Posts: 61
- Joined: Sun May 24, 2020 7:07 pm
- Has thanked: 24 times
- Been thanked: 205 times
Re: How can I let MiSTer support Chinese Fonts?
RBF files for my MiSTer cores in jtbin
Support new IP and core development here
Re: How can I let MiSTer support Chinese Fonts?
That and all the cores have the menu strings baked in as part of the RBF.
Ideally you'd move all the user displayable strings to some sort of a 'resource' file that contained all the translations for a given language. You'd probably want each core to be able to reference a 'private' resource file that could be distributed in the git repo for each core.
If you do it this way users could contribute resource files for a new language without requiring full rebuilds of Main_Mister and every core.
I don't think the work is impossible, it would just be very tedious to convert everything initially.
-
- Core Developer
- Posts: 537
- Joined: Sun May 24, 2020 9:30 pm
- Has thanked: 19 times
- Been thanked: 143 times
Re: How can I let MiSTer support Chinese Fonts?
while this would certainly be a nice thing to have, I wouldn't expect this support soon because of the amount of effort involved.
Re: How can I let MiSTer support Chinese Fonts?
On top of being a gigantic work, where would/should a resource file/part of the program be able to "run" from?
The fpga or the arm cpu?
If from the fpga does it have enough space left to implement this on every core on the mister?
If from the arm cpu does it have enough clock cycles left so input/network and what ever else is run on the arm side does not suffer?
For example a bit flashier gui according to sorgelig would make input/network suffer some slowdown really fast.
-
- Core Developer
- Posts: 537
- Joined: Sun May 24, 2020 9:30 pm
- Has thanked: 19 times
- Been thanked: 143 times
Re: How can I let MiSTer support Chinese Fonts?
- the filesystem (filenames).
- the display subsystem (both FPGA and arm)
- encoding format - there are only a few hundred different ways that these have been encoded over the years, with several thousand characters each - if Unicode were to be used, there would be UTF-8, UTF-16, and other variants to be concerned with. If you implement one, there will be requests for others and various conversions
- font (several tens of thousands of characters) at potentially multiple bitmap resolutions. Again, if you have one, there will be requests for others.
- storage (space on FPGA may be limited for things like this - it would at least double the requirement)
- then there are the localization concerns - menu items would be requested in multiple languages, when they aren't even in multiple roman-character formats
- at some point, there would be requests for top-to-bottom and left-to-right display
It is really endless.
As I said, anybody who has looked at this in the past 30 years knows that it is a huge subject, and even on the systems which currently implement it, it exists due to the work of thousands of people (much of which will not be open-source, or may not apply on this platform, requiring rewrite).
And even on systems which do support it, very few programs are actually written to make use of it. So, there are multiple confusing ways of using 1-byte or 2-byte encoding.
Of course, if somebody is ambitious enough to undertake this, I wouldn't want to stand in their way... but I know enough about what it required, to know that I am not that person.
- aberu
- Core Developer
- Posts: 1162
- Joined: Tue Jun 09, 2020 8:34 pm
- Location: Longmont, CO
- Has thanked: 244 times
- Been thanked: 404 times
- Contact:
Re: How can I let MiSTer support Chinese Fonts?
-
- Core Developer
- Posts: 48
- Joined: Fri Dec 11, 2020 2:39 pm
- Has thanked: 3 times
- Been thanked: 32 times
Re: How can I let MiSTer support Chinese Fonts?
It would be nice to support displaying cjk characters in filenames.
I have Japanese named snes roms which I dumped from my real carts collection and wish I could use them without converting them to roman characters.
I know it requires tons of work even without support for localization resources.
Just dreaming...
Menu items:
Ideally it would be good to have localized menu.
But I don't think I (Japanese) can get much benefits from localized menu so it may not pay off.
I feel there are not so many words or sentences which are hard to tell on menu.
And as for Japanese, most technical terms are untranslatable. They can simply be replaced fully or partially by Japanese-English in Katakana, which expresses a similar(but weird) sound.
Even converted to Japanese-English, those who don't know well about the feature will not be able to understand it.
From friendliness point of view, it would be better to have things like below in their language rather than to have localized menu, IMHO.
- Help docs
- Commentary articles/videos by enthusiasts
- User community
-
- Top Contributor
- Posts: 995
- Joined: Thu Dec 10, 2020 5:44 pm
- Has thanked: 309 times
- Been thanked: 228 times
Re: How can I let MiSTer support Chinese Fonts?
That said I'm sure some way of handling ROM names would be welcome if it could be figured out.
-
- Core Developer
- Posts: 48
- Joined: Fri Dec 11, 2020 2:39 pm
- Has thanked: 3 times
- Been thanked: 32 times
Re: How can I let MiSTer support Chinese Fonts?
But I gave it up to complete, although it was an interesting challenge
![Smile :)](./images/smilies/icon_e_smile.gif)
It is not perfect and universal. There're issues left that I'm not capable to solve
![Sad :(](./images/smilies/icon_e_sad.gif)
Please don't expect further work...
How I did it?
Filenames given from Linux were UTF8 in my environment.
When executing mount command, I found that /media/fat is mounted with iocharset=utf8.
Maybe that's why but I don't know where it is defined.
Anyway, char* for filename in Main_MiSTer is actually UTF8.
Main_MiSTer seems to be designed to handle one byte UTF8 which equals to ASCII.
So first I added multibyte UTF8 handling to PrintDirectory function in menu.cpp.
Next problem was font.
Luckily I found a free awesome Japanese font(Misaki Gothic) which is designed with 8x8px including 1px margins, the same as MiSTer bitmap font!
Rendering was done by freetype2 library with its Cache Sub-System.
In OsdWriteOffset function in osd.cpp, if the character consists of 2 or more bytes, a bitmap for the character is rendered instead of using original MiSTer bitmap font.
What's the issues?
1. Handling scroll
When filename is too long to fit in 2-lines, it starts scrolling when the menu item is focused.
This logic is on ScrollText function in osd.cpp.
It should also support multibyte UTF8, but it is too complex for me to mod.
Even modding PrintDirectory function in menu.cpp was a nightmare to support expanding to 2-lines for long filename.
2. Universal use
Rendering logic expects font to be designed with 8x8px with 1px margins.
Other font than Misaki Gothic cannot be used properly so it is not universal.
3. Size
Although Misaki Gothic is a great artwork, it is a bit hard to read Kanji(Chinese characters) compressed into 8x8px (actually 7x7px without margins).
10x10 to 12x12 px would be nice but it requires major work on OSD, which I don't want to touch.
4. Unknown bugs
There may be bugs caused by side effects that I'm not aware of.
It's not fully tested.
Changelogs are available in the forked branch below.
Hope someone will improve...
https://github.com/bellwood420/Main_MiS ... p-filename
- jotego
- Core Developer
- Posts: 61
- Joined: Sun May 24, 2020 7:07 pm
- Has thanked: 24 times
- Been thanked: 205 times
Re: How can I let MiSTer support Chinese Fonts?
This is already quite awesome.bellwood420 wrote: ↑Thu Mar 11, 2021 8:47 am I tried and succeeded in displaying Japanese filenames by quick and dirty tweaks.
But I gave it up to complete, although it was an interesting challenge
It is not perfect and universal. There're issues left that I'm not capable to solve![]()
Please don't expect further work...
About translating the cores, well, some time ago most arcade cores moved to the MRA file format so at least we have the DIP switches and button names in that file, which is plain text.
RBF files for my MiSTer cores in jtbin
Support new IP and core development here
-
- Core Developer
- Posts: 48
- Joined: Fri Dec 11, 2020 2:39 pm
- Has thanked: 3 times
- Been thanked: 32 times
Re: How can I let MiSTer support Chinese Fonts?
When it comes to handling MRA files, we need to make changes in support/arcade/mra_loader.cpp where there are a lot of string manipulations.
Low-level C/C++ is not suitable for manipulating multibyte UTF-8.
It will mess the codes up as seen in my changes, make maintaining hard and increase potential bugs
![Sad :(](./images/smilies/icon_e_sad.gif)
Using fixed width wide character(wchar_t) instead of multibyte UTF-8 would make manipulating easier as it can be used with library functions defined in wchar.h.
However, interfacing with tons of existing functions which expect ordinary char: one byte UTF-8(ASCII) becomes a huge problem with the way.
-
- Posts: 1
- Joined: Sat Dec 09, 2023 9:12 am
- Been thanked: 4 times
Re: How can I let MiSTer support Chinese Fonts?
bellwood420 wrote: ↑Thu Mar 11, 2021 8:47 amI tried and succeeded in displaying Japanese filenames by quick and dirty tweaks.
But I gave it up to complete, although it was an interesting challenge
It is not perfect and universal. There're issues left that I'm not capable to solve![]()
Please don't expect further work...
I have improved your mod by supporting 8x12px fonts.
It makes Chinese/kanji characters a lot more readable.
My “even dirty hack” is always display one text as two lines on osd.
- Display upper part of the characters with offset of 4px. It will make sure only 8x4 of the font is displayed on the first line.
- Display lower part of the characters with a new line in parallel. Full 8x8 px are being rendered as one line.
With 2 lines together, Chinese characters is very readable even on my arcade monitor.
Here is my fork: https://github.com/RickyL1213/MiSTer_8x12_Font
Re: How can I let MiSTer support Chinese Fonts?
redclover1213 wrote: ↑Tue Dec 12, 2023 12:03 pmbellwood420 wrote: ↑Thu Mar 11, 2021 8:47 amI tried and succeeded in displaying Japanese filenames by quick and dirty tweaks.
But I gave it up to complete, although it was an interesting challenge
It is not perfect and universal. There're issues left that I'm not capable to solve![]()
Please don't expect further work...I have improved your mod by supporting 8x12px fonts.
It makes Chinese/kanji characters a lot more readable.My “even dirty hack” is always display one text as two lines on osd.
- Display upper part of the characters with offset of 4px. It will make sure only 8x4 of the font is displayed on the first line.
- Display lower part of the characters with a new line in parallel. Full 8x8 px are being rendered as one line.
With 2 lines together, Chinese characters is very readable even on my arcade monitor.
Here is my fork: https://github.com/RickyL1213/MiSTer_8x12_Font
Do you have a layman's version of how to apply the font?