Change labs to have separate yaml file and pregen JavaScript for internationalization

[david-a-wheeler]

I propose a change to how the YAML data is handled for internationalization.

Currently each lab is a stand-alone HTML file that contains data in YAML format. The YAML format includes both natural language text (e.g., hints) and other information (e.g., regular expressions). The HTML file representing the lab brings in a library to implement the lab, and that library includes a run-time library that can process YAML (JavaScript doesn't have a built-in mechanism for this).

Having the YAML data embedded within the HTML is simple in some ways, and it can work for internationalization and localization, but it also causes problems. If every translated lab has its own YAML, the different YAML data will quickly go out of sync with the other. That is not great. It'd be better if there was one YAML for all translations of a lab. That way all the non-text data would not be duplicated, and there wouldn't be the risk of things going out of sync. Text that needed to be translated would be obvious, and the locales distinguished with a suffix (e.g., text vs. text_ja).

We can't just create a .yaml file and load the yaml file at run-time. The problem is that this approach prevents local use and local testing. JavaScript has operations like fetch that can fetch an adjacent file (like a .yaml file), but these only work on http/https, not on file. You can invoke browsers with special options to disable this, but that's not a practical test method. We must make it easy to use the labs in a local environment, at least for testing.

However, JavaScript can load other JavaScript files. A small build-time script could convert YAML (including JSON, a subset of YAML) into a JavaScript program that basically looked like info = followed by a JSON representation of the YAML format. This conversion would become a separate build step, executed before the lab is used. This would be typically during website deployment, or during development it'd occur after making a change to the YAML file during (so you can see the result).

Pros:

• A given lab would have a single shared YAML among all its translations, eliminating duplication of non-text and the risk of translations going out of sync.
• We can eliminate the run-time library for processing yaml. That would greatly simplify things, by removing a big run-time library. It would speed startup too; because we wouldn't need to load the library, then use it to process the data. Instead, the JavaScript would be ready-to-go.
• The processing would happen before any user interaction, eliminating some security concerns. We'd still need such a library, but it would occur before an attacker had a chance to do anything, and any language would do.

Cons:

• This creates an extra processing step on website deployment. That's no big deal, really.
• During development you need to remember to run the tool after updating the yaml so you can see the result. Basically "must compile before seeing new result". No doubt people will forget sometimes.
• People who download the files & run them locally will need to run this extra step. That's extra work for them. We could store the generated JavaScript files in the repo if that's a problem for them, but that risks using old versions.

[david-a-wheeler]

Note: An alternative: generate the JavaScript from the YAML once, and make the new JavaScript the editable format. That would eliminate the preprocessing step, but create its own problems.

1. The JavaScript could be a call to convert the YAML. Ugly hack, though, and it still means you need a YAML processor at runtime.

2. The YAML could be converted into a JavaScript that's basically JSON. One problem: regexes are really hard to understand in this format (you have to at least escape all the backslashes). That difficulty is likely to result in many subtle bugs.

So I don't think those are the way to go. Comments welcome.

[david-a-wheeler]

Better idea: generate the JavaScript from the YAML once, and make the new JavaScript the editable format. In addition, for regular expressions, do NOT use simple '...' or "..." strings (which work poorly for them). Instead use raw strings, that is, String.raw. That would ELIMINATE the problem of hard-to-understand regexes. It could also accept regexes (we'd have to add code for that), but the problem with doing that is that our format isn't the same by default (so that would be confusing.

This would be the best of all worlds. No preprocessing or runtime processing is required. To edit things, you directly edit the JavaScript. The single JavaScript file is shared among translations, making it easier to keep all translations up-to-date.

It does mean switching from a true data format to a programming language. However, that also allows us to define constants and reuse them.

Comments welcome.

[david-a-wheeler]

A start at this transition is implemented here: #728