This very minor NarraFirma release  does nothing but fix some warnings I got from WordPress when I released version 1.6.8.

As always, if you find any bugs – or find anything in NF confusing or hard to use – please tell me on the GitHub issues page.

This very minor NarraFirma release  adds a notes field to each question category (about each group’s status, ability, expectations, and feelings) on the “Describe participant groups” page.

As always, if you find any bugs – or find anything in NF confusing or hard to use – please tell me on the GitHub issues page.

This very minor NarraFirma release  adds a bit more functionality to the story-form building and translation facility. Specifically, you can now set (and translate) the “Does not apply” slider label in general, not only for specific questions.

As always, if you find any bugs – or find anything in NF confusing or hard to use – please tell me on the GitHub issues page.

This smallish NarraFirma release adds some quality-of-life improvements and fixes some small bugs.

Display lumping + renaming + hiding

In the Catalysis part of NarraFirma, display lumping (on-the-fly data manipulation) is proving to be quite useful. However, I recently realized that it can be used for more than just lumping together similar answers. So I extended it slightly.

1. You can now rename an answer to a choice question. This might be useful if you realize that some of the answers you used in your survey (or interview script) are too long for (or too confusing on) your graphs. For example, if an answer was “I’m not sure about that” you might want your graphs to say “Unsure.”
2. You can now hide an answer that was rarely chosen. For example, if you asked people something like, “How often do you [do some thing]?” and 5 people out of 500 chose “never,” you can draw your graphs without that answer.

I did think about the fact that people could use these new functions to distort what people say. But there are so many ways to distort what people say that I don’t think withholding this functionality would deter anyone doing that if they wanted to. After all, you could just export the data to a spreadsheet, change it there, and import it again. Also, this functionality could be quite helpful in situations where the collected data is too messy to be useful without some cleanup. The general rule is: in participatory work, if you use any form of data manipulation, in any project, using any software, you should always be transparent about what you did and why.

Less verbose story cards

My original idea for story cards was that choice questions would provide context by displaying all possible answers for each story, with the chosen answers marked in bold. However, for some choice questions (those with many and/or long answers) a show-it-all display confronts participants with too much text to wade through.

So I have implemented an option where you can show or hide non-selected answers in your story cards.

• Showing non-selected answers – Feel about: happy sad relieved frustrated hopeful hopeless
• Hiding non-selected answers – Feel about: happy hopeful

I also made the between-answer display character(s) customizable, so your answers lists can say “happy/hopeful” or “happy, hopeful” or “happy and hopeful,” or anything else you like.

Custom annotation order

When you are annotating stories, you might not think of annotation questions in the same order as you would like to answer them. So I added an option to the “Write annotation questions” page to specify the order of annotation questions on the “Annotate stories” page. While I was doing that, I added the option to create headers above groups of questions (to make the annotation process easier).

New correlations report

As I use NarraFirma on projects, I keep wanting new reports (“if only I could see this”), so I “scratch the itch” and build them, then use them in my local copy of NF. Later, if I think other people might want to see the same reports, I clean them up and add them to NF. (I’ve been doing that for a long time.)

I recently worked on a project using NF, and I wanted (so I created) a summary correlations report. I think you might want to see that report too, so I kept it. NF can now spit out (to CSV) a table that summarizes the significant positive and negative correlations in all subsets of stories with all answers to all choice questions. This is a good report to glance over when you want a quick idea of where your correlations lie – before you go through your (possibly thousands of) subset correlation graphs.

If you are doing a project with NF and there is a report you would love to see, tell me about it on the GitHub issues page. Depending on what sort of report it is and how it fits into NF’s existing architecture, I might be able to build it for you (maybe quickly) and add it to NF. Of course, whether I will have the time to do that will depend on many other things (which are mostly out of my control). But please do reach out if this happens to you, because I would like to hear what would make NF work better for you.

Spot check -> Review

I decided to rename the “Spot-check graphs” page to “Review graphs.”  I did this because recently (well, recently in the glacial way my mind works) it came to my attention that some NF users have been using that page not to spot-check graphs for completeness during story collection (as I thought they would) but to look for patterns in their data without going through the catalysis process.

I had not thought of that use for that page. But I can see that the full catalysis process might not seem worth doing on very small projects. In the last NF update (which happened after I heard about this) I improved the spot-check/review page. This time I thought I’d rename the page to address the confusion people must have over what it can be used for.

By the way, I did think a little about building a sort of dashboard page that (like the survey) could be accessed by project participants who do not have access to the entire project. That’s a harder task than it may seem. Just in case anyone wants to know why, I’ll explain.

Every PNI project needs to have a control point somewhere in the collection process. This is because people often volunteer information you asked them not to provide, such as their phone numbers or the phone numbers of other people. (You would be amazed how often people do this.) As a result, all PNI projects must go through a phase in which personally identifying information is “scrubbed” out of the stories and other data. This has to be done before the stories (and other data) can be shown to any participant groups. This is why the code that runs the NF survey is completely separate from the code that runs the NF project management interface. They are basically two different pieces of software.

So a NF dashboard that is visible to project participants would have to have some way to show people only the stories and data that had been reviewed and marked as safe to share. The data architecture of NF doesn’t currently have any way to mark stories (or story collections) as safe to share. We would have to add a way to enter and store those markings. That’s not hard to do, but we would also have to find a way to deal with legacy data that has no such markings. Dealing with legacy data has been a big part of my work on NF for almost a decade. I’m proud of the fact that the very first NF projects can still be read and used without any translation issues. I want to keep that record clean. I think probably the best way to do that would be to assume that data without markings cannot be shared, but I would like people to be able to mark legacy data as shareable using some process.

Also, building a participant dashboard would require us to write a whole new set of separate access and display scripts, both client and server, that are similar to the survey scripts but that require a new intermediate level of permission (and a permission-granting interface that also deals with missing permissions in legacy projects). We could do all of this if we had the time and funding to do it, but right now we don’t. It’s a good someday idea, though. Who knows, maybe someday we will get a nice big grant so we can make NF work even better for everyone. :-)

Bug fixes

I also made lots of little tweaks to the software as I used it and talked to users about it. For example:

• I cleaned up the table of links on the application’s home page to make it easier to see and access your story collections. I also cleaned up some of the page descriptions on section pages.
• On the “Explore patterns” page, I moved the export buttons (there are now three) from the bottom of the page (where they interfered with writing observations) to the top. I also made them smaller and moved them to the right, where they will hopefully not be as distracting as they were.
• I fixed a few small display bugs, where interface elements appeared where they were not supposed to be.
• I fixed the fact that I previously forgot to include the write-in answer label in the translation dictionary (thank you to the helpful user who noticed that mistake and brought it to my attention).

As always, if you find any bugs – or find anything in NF confusing or hard to use – please tell me on the GitHub issues page.

This minor NarraFirma release fixes one small bug that applies only to older projects.

Too many remarkable patterns

A helpful user told us about a strange problem. On the “Explore patterns” page of NF, when they double-clicked on a pattern, NF was setting the “remarkable” flag of that pattern to “yes” – on its own – and they could not set it back again.

What caused the problem was a bit of code we wrote to update legacy data back in 2019. It was supposed to add a “remarkable” flag to any pattern that had at least one observation but no “remarkable” flag. That’s because, prior to version 1.4.0 of NF, there was no remarkable flag.

The problem was that this update was creating a circular interaction because it was (accidentally) being done while the page was being redrawn. Changing the data caused the page to redraw, which changed the data, causing the page to redraw, and so on. Double-clicking was not the only way to get the cycle going; it could also happen by clicking the arrows under the table. But double-clicking made the cycle more visible, which is probably why we never noticed it before. (Double-clicking has no actual meaning in NarraFirma, so we’ve never done it.)

To fix the problem, we changed the code so the updating only happens when the user changes the “remarkable” flag by hand. This does mean that people who have legacy data (created before NF 1.4.0) will have to manually update their remarkable flags. But that is probably few people at this point, and now it won’t happen “on its own.”

Thank you, user! And as always, if you find any bugs – or find anything in NF confusing or hard to use – please tell me on the GitHub issues page.

This minor NarraFirma release adds one nice-to-have thing, but mostly fixes little bugs.

Hiding “not through the story” graphs

When I’m helping people work on catalysis, I often advise them to save time by ignoring graphs that don’t go “through the story.”

What does that mean? Well, say I asked people to share stories about a topic, and I also asked how old they are and where they live. This means that, among all the graphs generated by NarraFirma, there will be a graph of age versus location. That graph has nothing to do with the stories people told. Once in a while a graph like that will be useful, but not often. Not-through-the-stories graphs are worth a quick look, but I wouldn’t put a lot of time into considering them, since they are not why you did the project.

So I have added an option, on the “Configure catalysis report” page, to hide graphs that involve no questions about stories. You might find this option useless. However, if you happen to have a lot of questions about people, especially if you threw in a lot of demographic data you happened to have handy, hiding not-through-the-story graphs could save you some time.

Reset messes

When you reset a project, NF makes a backup copy in case you change your mind later. (We really don’t want you to lose your data.)

In the WordPress version of NF, I had recently (like a few years ago) found out that our scheme of adding the date to the backup table name was breaking the MySQL database, because MySQL table names have a character limit. So I changed NF so the backups just put a “b” in place of the “j” (for journal) that was usually in the name.

When I did this, I had assumed that the “rename” MySQL command would overwrite any existing table with the same name. It doesn’t. It throws up an error. So if you happened to reset the same project twice, NF would refuse to do it.

I have now changed this so that any NF reset backup table – for any project in your NF installation – is called

[your WordPress prefix]narrafirma_reset_backup_[date/time]

So in the unlikely event that you need to recreate a NF project from a reset backup, you will need to find the table by its date, not by the project name. Since (as far as I know) this has never actually happened, I think it is acceptable to remove the project name from the table name to make space for the date and time.

Note that the date/time in the table name is in UTC, not your local time zone.

No both/and

People never use software the way its developers think people will use it.

Funny story: One of the first pieces of software I wrote for money was a database system for a former professor. He told me what he wanted, and I built it. Then we met so he could look at what I had built. He sat down at the computer, put his hands on the keyboard, and somehow accidentally typed – in perfect sequence – the three keys that would immediately exit the entire database system. He said, “I broke it!” and I think my mouth dropped open. Anyway, after that, I added a special prompt to make sure the system would ask the user if they really did want to exit the system, just in case that ever happened again (though it probably never did).

NarraFirma has more users than ever, and they keep doing things I never do. Recently a user typed in a question short name with a forward slash in it, something like “Preference/choice.” It seems obvious – now – that people might do that, but I never thought of it before. The problem is that it creates a conflict.

The NF module that looks things up, the “valuePathResolver,” uses a forward slash to separate data identifiers in a lookup sequence. For example, when you are on the “Configure catalysis report” page, and you click on a type of graph to show, the code for that page saves your choice with the “valuePath”

/clientState/catalysisReportIdentifier/graphTypesToCreate

Meaning: ask the “clientState” module (which remembers what’s going on in your browser client) what the currently selected catalysis report is. Then save your choice in that report under the field name “graphTypesToCreate.” That’s why you can see the same types of graphs when you come back to the “Explore patterns” page again, even if you closed your browser in between.

The surveying code does not use the valuePathResolver, so a question with a slash-containing short name does not cause any data to be lost. But the table widgets throughout NF can’t find the data. They all use the valuePathResolver, and it chokes on the slash. (It thinks the question name must be the first part of the name, before the slash.)

I decided not to change how the valuePathResolver works, at least not yet. I didn’t write it, and it’s used all over the system, and I don’t want to create any new bugs. Instead, I wrote an extra validation step. You can no longer include a forward slash in any short name identifier. It’s not a beautiful solution, but it should fix the problem. If we ever get enough time to transition the badly-designed short-name-identifier system into a more robust all-UUID-identifier system, this conflict will go away.

As I told the user who discovered and then helped me track down this problem – and I want to say a huge thank you to that user – the “price” of free software is that its users and its testers are the same people. I would love to hire a dedicated tester to find other conflicts like this, but I don’t think anybody would work on it for the same salary I’m getting (nothing). So there will continue to be surprises like this – hopefully fewer and fewer as we go.

Clearer import & export

While I was messing with the Reset functionality, I decided I hated the Project admin / Import & export page, so I cleared it up a little.

Smaller bugs

I also fixed a few more bugs that were reported by users (or that I happened to notice while fixing other things).

• In the CSV import process, there was a bug while reading the specification for a scale question in which the “Does not apply” label was being ignored.
• Also in the CSV import process, you might have seen a browser error if one of the headers in your story-collection CSV file was not what NF was expecting to see. I’ve now moved that error message to its proper place in your “check import” readout in the browser console.
• The HTML page title for printed story cards was an ugly long UUID instead of the project name.

As always, if you find any bugs – or find anything in NF confusing or hard to use – please tell me on the GitHub issues page.

This minor NarraFirma release adds bulk export of graphs and statistics. It also fixes a few bugs.

Graphs without catalysis

On the “Explore patterns” page, you can now export all of the graphs and/or statistics for all of the patterns in the list, even if you have not written any observations for them. This is for situations in which you want to use NF to generate graphs and statistics, but you don’t want to use it to build catalytic material. Or you want to show someone the graphs you have found before you start writing your observations.

I also fixed a few small bugs, as usual, including a bug that might have been slowing down the “Explore patterns” page for you.

As always, if you find any bugs – or find anything in NF confusing or hard to use – please tell me on the GitHub issues page.

This minor NarraFirma release makes a few small quality-of-life improvements and fixes a few bugs.

Print your catalysis report in bits and pieces

On the “Print catalysis report” page, you can now save a CSV (spreadsheet) file with just the observations and interpretations you wrote, for pasting into whatever software you want to use to prepare your catalytic material. Paired with the option to spit out just the graphs associated with your observations, this should make creating your catalysis materials even easier.

Note that this new bits-and-pieces option ignores any clustering you do inside of NF. You might or might not care about that.

If you would like to write your catalysis reports in NF (perhaps to use its clustering interface), I recommend that you give pandoc a try. It works like a charm to convert NF’s HTML reports into any format you like.

Just-in-time help

When you’re looking at graphs in NF, it is not immediately obvious that you should click and drag on them to select stories. So I added a little bit of text below the graphs on the “Spot-check graphs” and “Explore patterns” pages to let you know that you can do that.

Another thing that happens sometimes when you are working with patterns on the “Explore patterns” page is that you forget the longer version of a question on the graph. So now, in the “Things you can do” list under each graph, there is an option called “Show survey questions for this pattern.” If you choose that option, then click “Do it,” NF will pop up a dialog that gives you the full text of the question, along with its (non-lumped) answers.

I also fixed a few little bugs.

As always, if you find any bugs – or find anything in NF confusing or hard to use – please tell me on the GitHub issues page.