If you have been doing (or learning about doing) story work, you may have seen graphs in which storytellers were asked to describe a story by placing a point inside a triangle with three labeled vertices. NarraFirma does not support this data collection method. Why? Because I believe it is a mistake.
There are three reasons triangles don’t work, and there are three corresponding conditions that, when satisfied, can make them sort-of work.
1. Triangles don’t fit the way people actually talk.
When you want to ask people about their experiences — possibly about things they feel vulnerable about — you need them to feel safe, free to shape the conversation, and listened to. Forcing people to use a strange, arcane, complicated, artificial method of conversing makes most people feel intimidated, challenged, and evaluated — not safe, free, or listened to.
It is true that some highly-educated intellectuals find triangles interesting. But that group represents only the tiniest minority among the universe of people who tell stories in story projects. The great majority of people do not find triangles interesting. They find them confusing, frightening, and insulting. This is a big problem, because the most important thing in story work is not computers or your time or expertise. It’s the time, attention, and motivation of the people who are telling the stories. Participation is ambrosia, the nectar of the gods, and you don’t want to spill one drop of it.
When people feel confused, frightened, and insulted, they don’t stop telling stories. They just tell different stories. That’s the worst thing about triangles: they make you think you got authentic stories when you didn’t. You didn’t get ground truth; you got camouflage. Triangles make beer seem like ambrosia.
I was once told by a person who works in development that they watched someone try to use triangle questions to talk to (poor, illiterate) farmers in the field. They said that they could see the farmers’ faces fall as they went from curious and intrigued to angry and insulted. The farmers shut down and stopped responding as soon as they saw what they were being asked to do. They went from being excited that someone actually wanted to hear what had happened to them to dully obedient as they realized that they were being forced to go through yet another useless, meaningless exercise.
The farmers in that project told stories, but they didn’t tell the stories they wanted to tell. The sad thing is that whoever did that project probably thought they got good stories. But they didn’t. All they got was whatever the farmers thought would make them go away the fastest.
The person who told me that story told it in the context of explaining why they came to me for help with a new project (also with poor, illiterate farmers). I told them what I tell everyone: that any human being can answer the question “how much” or “how big” with their hands, and nobody is insulted by such a question. It’s how people have been talking to each other for thousands of years.
So that person and I designed a set of questions with choices and scales. The local-language interviewers on the project asked the farmers to answer the scale questions with their hands — hands touching for one end of the scale, hands as far apart as they could reach for the other end of the scale. The interviewers looked at the farmers’ hands and wrote down numbers from zero to ten to represent their responses. The farmers were engaged, forthcoming, and grateful. Their stories were thoughtful and revealing. The patterns that arose provided useful insights that helped everyone.
This is an extreme example, but I’ve seen similar results — people who had been excited shutting down and doing the absolute minimum to get out of the situation — in other contexts as well.
Working with triangle questions is like trying to carry water with a sieve. You may think you have found something, but you have lost far more than you have found. If you don’t work with the ways in which people actually speak, you can never hope to understand what they have to say.
2. Triangles generate confounded, useless data.
I once helped a client with a fairly large project that used triangles (against my advice). Out of curiosity, I looked at patterns in how participants used the triangles. Here’s what I found.
- One quarter of the participants placed their dots only at the vertices of the triangles. They converted the question into a single-choice a/b/c question.
- One quarter of the participants placed their dots only along one side of the triangles, ignoring the opposite vertex. They converted the question into a scale. The problem is, different people chose different sides of the triangles, and they chose the same side for all of the triangle questions they answered, regardless of meaning.
- One quarter of the participants moved all of their dots just a tiny bit from the center of the triangle (where the dot started), probably just to be able to move on and finish the survey. They didn’t answer the question at all.
- One quarter of the participants placed all of their dots all over the triangle, as intended.
As you can imagine, when all of these patterns were overlaid, the data showed . . . pretty much nothing. The project was only saved by the fact that, as we were designing it, I insisted on using at least some scales. And that’s where the patterns were.
There are always useful patterns in scale data. There is rarely anything useful in triangle data. I’ve been told by at least three people that they’ve seen the same patterns in the use of triangles (vertices, one side, pretending to answer).
3. Triangles are statistically problematical.
Statistics are essential tools when you want to talk about the meanings of patterns in data. You can’t talk about meaning until everyone can agree on whether the patterns in the data actually exist. This is why data that can be statistically analyzed is so much more useful than data that can only be “eyeballed” — that is, stared at.
Historically, there were no statistical tests that applied to triangular data. Recently there has been some new development in the area of “compositional” statistics that deal with triadic (ternary) data. Apparently there are now routines in the R statistical library to do this. I don’t know how many people are using compositional statistics for triangle data. I suspect that most of the people who are using triangles are not using statistics. From what I’ve seen, many people are just eyeballing the patterns in their triangles.
I have read some reports and presentations from projects that relied heavily on triangle data. Their insights have seemed to me to be weak and surface-level compared to the rich explorations I’ve seen in projects that did not involve triangles. Some of this probably came from people being confused and insulted (my point 1 above), and from confounding in the data (my point 2 above). But some of it also probably came from the impossibility of knowing where the actual patterns were.
Having said all of this, I believe that it is possible to use triangles in a story project, under these specific conditions.
- Triangles could work if people became used to them to the extent that they were not seen as a jarring, confusing, intimidating test, but became part of a normal conversation. This could conceivably happen if triangles were used repeatedly over a long period of time within a community or organization. They could become part of a group’s shared language of meaning. (This could take years, but it could happen.)
- Triangles could work if they were used in situations in which understandings could be reliably verified and corrected before any data was collected. I have seen people use triangles in participatory workshops where they had the time (and participants were willing) to verify that everyone involved truly understood what they were doing and were not simply pretending to understand because they were confused or intimidated. (Though some people will pretend for a very long time.)
- Triangles could work if whoever was analyzing the data looked up the new compositional statistics and used them. (This is the most likely condition to be achievable, since it relies only on time and expertise.)
In over twenty years of working in this field, I have never seen a story project meet all three of (or even any one of) these conditions.
Thus, in my considered professional opinion, triangle questions pose great risks while delivering few benefits. More importantly, they are totally unnecessary. I have helped people get amazing, authentic, insightful, inspiring results without ever using triangles. In my view, they are a distraction from the real work of engaging people, listening with respect to what has happened to them, and helping them work with their own stories.
This is why NarraFirma does not, and almost certainly never will, support the use of triangle questions.