This minor but essential NarraFirma release fixes three categories of bugs.
No new story collections!
In the last update (v1.6.0) I introduced a show-stopper of a bug that meant you couldn’t create the first story collection in a project. Sorry, that was sloppy of me. I tested the new dialog in a project that already had story collections in it, and I forgot to think of the new-project situation. It is fixed now.
Multiplayer bugs!
We recently had the opportunity to have a whole bunch of people use NF at the same time. A whole bunch of updating bugs popped up. It was frustrating! But so very useful. I would like to apologize to the users who encountered these bugs, and to thank them for their patience and understanding.
The worst of the bugs came about due to a misunderstanding of how mithril handles screen updates during user editing.
There is a critical assumption in the version of mithril we are currently using in NF (0.2.0). When a mithril element gets a redraw message, it redraws itself using its value function, which draws data from the datastore. The current value of the text box (that is, whatever you were typing) is overwritten by the datastore value.
If you are using the oninput method to save changes to the datastore, this is not a problem, because the oninput method fires on every keystroke. So every character you type is immediately saved to the datastore, and differences don’t pile up.
However, we didn’t want to use the oninput method in NarraFirma text boxes. Saving to the server database on every keystroke would multiply the number of messages so much that the NF server would slow to a crawl. Plus, NF databases would be orders of magnitude bigger. So we decided to use the onchange method, which only fires when you leave the field or hit Enter. This is why, when you are typing into a text box in NF, you have to hit Enter or Tab, or click outside the box, to save what you have typed to the server.
We have always been using the value function to set an element’s displayed text. But in a multi-user situation, the value function might be called by an update message while the user is in the middle of editing a text box. In our session last week, people were having texts disappear in front of their eyes.
This was a complete shock to us, because we should have seen it before. We have tested the multi-user aspect of NF many times over the years. But I think we never typed a lot of text into a field, over a long period of time. So we never saw these interruptions happening. When you are typing “test123” into text fields over and over, you never get a chance to see what happens when you spend a whole minute puzzling over what you are going to write there.
Actually, one user did tell me that they were seeing multi-user collision bugs four years ago. I assumed that they were talking about two people working on the same field at the same time. So I set up a conflict-handling system to handle that situation. It worked! But only for that one situation. There must have been other conflicts that I continued to not see, probably because I continued to test by typing in things like “test123.” That user never told me about any more multi-user conflicts, so I thought the problem was fixed. But looking back, maybe what actually happened was that they finished their project and stopped setting up the long-edit conditions under which the other bug was showing itself.
This onchange-overwrite problem was discussed on the mithril Github issues list way back in 2016. The mithril user in that discussion, like us, was using the onchange method in a situation in which updates were coming in from elsewhere. Their workaround was to use the config function to “keep a separate ‘work-in-progress’ state property and another ‘real’ property on the side.” That was a great idea. We implemented it (just now, mind you, not eight years ago) and it works.
Some readers of this blog will be surprised to hear that we are still using mithril 0.2.0, which came out several years ago. The thing is, we need to set aside a nice big chunk of time to make all of the careful changes required to upgrade our mithril version to the current version (2.2.2). It sounds like mithril is much nicer now, and we are eager to do this, but we just never have that much time free. Hopefully we will find a way to get to it someday. In the meantime, mithril 0.2.0 still works fine, except for this one problem, which is now fixed. (The config function goes away in later mithril versions, so we will have to change how we do text-box updates during that refactoring.)
We did see some other updating bugs in our multi-user session. They were all failures to update various screen widgets when new data came in (like when you were annotating stories and another user added a new answer for an annotation question). Those are fixed now too.
Invisible mismatches!
Another new bug came up recently. A user had entered answers for a fixed-list question, and (probably without noticing) had included extra white-space characters (spaces and tab characters) in the answers. (This was probably from pasting the answers in from somewhere else.)
NF doesn’t care if you have a lot of extra crap in your answers – or at least I thought it didn’t. In reality, there was a hidden assumption in the filtering and display lumping features of NF. To use those facilities, you enter a bunch of answer names, and NF parses the text you enter, trimming white-space characters as it goes.
Do you see the problem? The answers you specify in the filter or lumping command cannot match answers that have extra spaces in them.
So, after some thought, I fixed the problem on both sides. NF now trims out whitespace characters when it compares data during the filtering and display-lumping processes (that’s for legacy data). And it will no longer allow you to enter fixed-list answers with leading or trailing white-space characters.
The new restriction on white-space characters could affect your legacy data.
- If you have a choice question that has leading or trailing white-space characters in its answers,
- and it is connected to a story form that is used in a story collection that has stories in it,
- and you change the list of answers for the question, NF will force you to remove the extra characters.
- Then, if you update the story form in the collection, the answers that are saved in your stories may no longer match your new trimmed answers.
I think this sequence of events is unlikely to happen to anybody, because probably few people have leading or trailing white-space characters in their answer lists. But if this does happen to you, you can fix the mismatch.
- Export your stories to a CSV file.
- Use a spreadsheet or text editor to remove the extra characters (in the answers to that question only) via search-and-replace.
- Import the stories into a new story collection, using the updated story form.
If you run into this problem and have trouble solving it, feel free to ask me for help (cfkurtz at cfkurtz dot com). I may need to ask you to send me your project so I can fix the data for you. (I always delete projects again as soon as I have fixed them.)
I would like to say a big thank you to the users who told me about (and had to deal with) these bugs. As always, if you find any bugs, please tell me on the GitHub issues page.