Skip to content

Various Unicode handling issues #176

@jmmv

Description

@jmmv

In #161, @zenria fixed various Unicode-related crashes, but a few issues remain when dealing with "wide-looking" characters (don't know what the right name for those is). The easiest way to reproduce is to type some Korean characters (load and play with the test utf8.bas file).

  • Rendering these in the web version looks very weird because the console expects all characters to have the same width as "A" -- but those characters are wider. This does not have an easy fix (and I'm not even sure I want to fix it, because then we break the "grid" abstraction of the console), unless we make all rendered characters wider, which would look terrible.

  • Navigating such characters in the terminal editor doesn't work well either. The characters are rendered but navigation moves within them and stops prematurely before the end of the line. I suppose this is due to the TODO about handling graphemes properly. I did a quick try to use the unicode-segmentation crate and replace uses of chars() with graphemes(true) in LineBuffer, but that didn't fix the issue.

  • Rendering of Unicode characters works well in SDL, but not all. I tried with things like á and ç and those show up properly, but Korean characters show up as the typical empty boxes. Don't know what's going on there because the font we use supports Korean and, by looking at the SDL crate, it's actually calling into SDL's UTF8 rendering features. I was going to add a test for this but didn't because it's broken. If/when fixing this, I suspect we'd encounter the same issues as with the web regarding different character widths... so I'm not even sure it's worth fixing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions