Why are there no triangles in NarraFirma?

If you have been doing (or learning about doing) story work, you may have seen graphs in which storytellers were asked to describe a story by placing a point inside a triangle with three labeled vertices. NarraFirma does not support this data collection method. Why? Because I believe it is a mistake.

There are three reasons triangles don’t work, and there are three corresponding conditions that, when satisfied, can make them sort-of work.

1. Triangles don’t fit the way people actually talk.

When you want to ask people about their experiences — possibly about things they feel vulnerable about — you need them to feel safe, free to shape the conversation, and listened to. Forcing people to use a strange, arcane, complicated, artificial method of conversing makes most people feel intimidated, challenged, and evaluated — not safe, free, or listened to.

It is true that some highly-educated intellectuals find triangles interesting. But that group represents only the tiniest minority among the universe of people who tell stories in story projects. The great majority of people do not find triangles interesting. They find them confusing, frightening, and insulting. This is a big problem, because the most important thing in story work is not computers or your time or expertise. It’s the time, attention, and motivation of the people who are telling the stories. Participation is ambrosia, the nectar of the gods, and you don’t want to spill one drop of it.

When people feel confused, frightened, and insulted, they don’t stop telling stories. They just tell different stories. That’s the worst thing about triangles: they make you think you got authentic stories when you didn’t. You didn’t get ground truth; you got camouflage. Triangles make beer seem like ambrosia.

I was once told by a person who works in development that they watched someone try to use triangle questions to talk to (poor, illiterate) farmers in the field. They said that they could see the farmers’ faces fall as they went from curious and intrigued to angry and insulted. The farmers shut down and stopped responding as soon as they saw what they were being asked to do. They went from being excited that someone actually wanted to hear what had happened to them to dully obedient as they realized that they were being forced to go through yet another useless, meaningless exercise.

The farmers in that project told stories, but they didn’t tell the stories they wanted to tell. The sad thing is that whoever did that project probably thought they got good stories. But they didn’t. All they got was whatever the farmers thought would make them go away the fastest.

The person who told me that story told it in the context of explaining why they came to me for help with a new project (also with poor, illiterate farmers). I told them what I tell everyone: that any human being can answer the question “how much” or “how big” with their hands, and nobody is insulted by such a question. It’s how people have been talking to each other for thousands of years.

So that person and I designed a set of questions with choices and scales. The local-language interviewers on the project asked the farmers to answer the scale questions with their hands — hands touching for one end of the scale, hands as far apart as they could reach for the other end of the scale. The interviewers looked at the farmers’ hands and wrote down numbers from zero to ten to represent their responses. The farmers were engaged, forthcoming, and grateful. Their stories were thoughtful and revealing. The patterns that arose provided useful insights that helped everyone.

This is an extreme example, but I’ve seen similar results — people who had been excited shutting down and doing the absolute minimum to get out of the situation — in other contexts as well.

Working with triangle questions is like trying to carry water with a sieve. You may think you have found something, but you have lost far more than you have found. If you don’t work with the ways in which people actually speak, you can never hope to understand what they have to say.

2. Triangles generate confounded, useless data.

I once helped a client with a fairly large project that used triangles (against my advice). Out of curiosity, I looked at patterns in how participants used the triangles. Here’s what I found.

  • One quarter of the participants placed their dots only at the vertices of the triangles. They converted the question into a single-choice a/b/c question.
  • One quarter of the participants placed their dots only along one side of the triangles, ignoring the opposite vertex. They converted the question into a scale. The problem is, different people chose different sides of the triangles, and they chose the same side for all of the triangle questions they answered, regardless of meaning.
  • One quarter of the participants moved all of their dots just a tiny bit from the center of the triangle (where the dot started), probably just to be able to move on and finish the survey. They didn’t answer the question at all.
  • One quarter of the participants placed all of their dots all over the triangle, as intended.

As you can imagine, when all of these patterns were overlaid, the data showed . . . pretty much nothing. The project was only saved by the fact that, as we were designing it, I insisted on using at least some scales. And that’s where the patterns were.

There are always useful patterns in scale data. There is rarely anything useful in triangle data. I’ve been told by at least three people that they’ve seen the same patterns in the use of triangles (vertices, one side, pretending to answer).

3. Triangles are statistically problematical.

Statistics are essential tools when you want to talk about the meanings of patterns in data. You can’t talk about meaning until everyone can agree on whether the patterns in the data actually exist. This is why data that can be statistically analyzed is so much more useful than data that can only be “eyeballed” — that is, stared at.

Historically, there were no statistical tests that applied to triangular data. Recently there has been some new development in the area of “compositional” statistics that deal with triadic (ternary) data. Apparently there are now routines in the R statistical library to do this. I don’t know how many people are using compositional statistics for triangle data. I suspect that most of the people who are using triangles are not using statistics. From what I’ve seen, many people are just eyeballing the patterns in their triangles.

I have read some reports and presentations from projects that relied heavily on triangle data. Their insights have seemed to me to be weak and surface-level compared to the rich explorations I’ve seen in projects that did not involve triangles. Some of this probably came from people being confused and insulted (my point 1 above), and from confounding in the data (my point 2 above). But some of it also probably came from the impossibility of knowing where the actual patterns were.

Having said all of this, I believe that it is possible to use triangles in a story project, under these specific conditions.

  1. Triangles could work if people became used to them to the extent that they were not seen as a jarring, confusing, intimidating test, but became part of a normal conversation. This could conceivably happen if triangles were used repeatedly over a long period of time within a community or organization. They could become part of a group’s shared language of meaning. (This could take years, but it could happen.)
  2. Triangles could work if they were used in situations in which understandings could be reliably verified and corrected before any data was collected. I have seen people use triangles in participatory workshops where they had the time (and participants were willing) to verify that everyone involved truly understood what they were doing and were not simply pretending to understand because they were confused or intimidated. (Though some people will pretend for a very long time.)
  3. Triangles could work if whoever was analyzing the data looked up the new compositional statistics and used them. (This is the most likely condition to be achievable, since it relies only on time and expertise.)

In over twenty years of working in this field, I have never seen a story project meet all three of (or even any one of) these conditions.

Thus, in my considered professional opinion, triangle questions pose great risks while delivering few benefits. More importantly, they are totally unnecessary. I have helped people get amazing, authentic, insightful, inspiring results without ever using triangles. In my view, they are a distraction from the real work of engaging people, listening with respect to what has happened to them, and helping them work with their own stories.

This is why NarraFirma does not, and almost certainly never will, support the use of triangle questions.

Why can’t I create a multi-page survey in NarraFirma?

In most survey packages you can group your questions into separate pages, and survey-takers only see one page at a time. NarraFirma does not support this, on purpose.

Asking someone to tell a story (and then reflect on it) is a different category of interaction than asking someone to answer factual questions. This leads to different requirements, thus:

  • Individual questions on a factual form usually stand alone. But with a story form, questions often build on previous questions, and certainly on the story itself. So the story form has a need for coherence, or a continuity of context, that most surveys don’t. For example, when you can’t see your story anymore, it’s hard to answer questions about it; it doesn’t feel “present” to you.
  • Because telling a story is more revealing than answering factual questions, people tend to want to evaluate the entire form before they decide to participate. When they can’t see the whole form up front, they seem to tell more distant stories, to be safe. For example, when I plan to use a story form in a group session, I make sure people have a chance to look over the forms before they start sharing any stories, so they can see what they will be asked to reflect on. People need the same transparency on the web.
  • People filling in a story form tend to look back over their form when they’re finished, and sometimes change their answers, or even the story itself, because the questions make them rethink the experience as they go along.
  • When you ask someone to tell you a story, it’s important to avoid having them feel like they are being tested, judged, interrogated, or asked to perform. A typical web survey with separate pages and a progress bar may be familiar, but it tends to make people feel that they are ticking off tasks, not having a conversation. From what I’ve seen, this leads to “just the facts” stories and “am I doing it right” performance over reflection.

All of these things combine to make me want to give people the freedom to move about the entire story form quickly and easily. I know that most web survey forms have “back” buttons, but I’ve also noticed that a lot of people don’t use them. That’s why I decided to make NarraFirma story forms one-page affairs. I’ve seen people react to them like they do paper forms: they scroll down to the bottom and back up again very quickly, evaluating the situation they are in; they tell their story and answer the questions, and at the end they usually scroll back up and down some before they are ready to submit the form. I think this design gives people an experience that feels closer to the negotiations that go on during a conversation where stories are shared.

Of course, not everybody agrees with me about this. Some people think the one-page form doesn’t look as professional. Some think its length turns people away. I think if your story form looks too long, you’ve got too many questions, and/or you haven’t yet found the questions that people want to answer!

However, if you disagree with me on this point, you can collect stories using other surveying software and import them into NarraFirma through a CSV (spreadsheet) file.

Can I use NarraFirma as a front end (or back end) for story collection?

Yes. If you want to collect your stories using some other method (like SurveyMonkey or LimeSurvey or some other surveying method), you can import your stories into NarraFirma in CSV files. Look in NarraFirma’s help system for its “Guide to importing data.”

If you want to collect your stories using NarraFirma and export them to some other system (Tableau, R, a spreadsheet), you can export your data (on the “Export stories or story forms” page) in CSV format.

Does NarraFirma support story work that isn’t participatory narrative inquiry?

It’s hard to say. You might not use a lot of what is in the software, but it’s flexible, and nothing is required. On the other hand, there are many aspects of NarraFirma that assume you will be collecting stories and answers to questions about them. My advice is to look at the example NarraFirma project on this web site (click Try It) and see if you think it will work for what you want to do.

How many stories can NarraFirma handle?

Because the stories (and answers to questions) are text only (so far), NarraFirma is able to handle quite a few. We have stress tested it with more than 10,000 stories, and it worked fine, though slowly. With larger numbers of stories you will see slower performance (depending on what sort of computer it is running on). But with typical projects of 100-300 stories, and even several hundred stories, you should not see a problem.