(2021-02-24) Matuschak Ratcheting Progress In Tools For Thought

Andy Matuschak: Ratcheting progress in tools for thought. There are some people trying to develop tools for thought, but there isn’t yet a meaningful field around tools for thought. The difference is that a field is about ratcheting: developing a growing shared corpus of general knowledge and methods which allow projects to meaningfully build on each other, across researchers and across years, on and on in an upward cycle. (compounding)

Michael Nielsen and I have suggested that the most powerful tools for thought express deep insights into the underlying subject matter. Creating them involves what we call “insight through making,” in which powerful subject-matter ideas enable new systems, and observations of those systems enable new insights, and so on. (bootstrapping)

we can see these practices in the “golden era” design work of computer-powered tools for thought: Douglas Engelbart’s NLS, PARC’s Alto, Ivan Sutherland’s Sketchpad, and so on. Unfortunately, if we look at the contemporary proto-field, we’ll find that most people interested in tools for thought (myself included) are not reliably performing all these steps—which has left our struggling field without a functioning ratchet.

Common failure modes

One failure mode, particularly common in academia, comes from lacking a serious context of use

Startups and tech businesses are powerful venues for tool development. They’re generally not trying to push the field of tools for thought forward. But we might hope that they happen to push it forward anyway. Unfortunately, a few common patterns prevent most tech industry efforts from contributing to a ratcheting field.

Perhaps the most common pattern of all is that people in the tech industry focus mostly on building systems. Those systems are usually expressions of technological or market insights, rather than of fundamental insights about a subject domain or about cognition.

Another common pattern for tech companies is that they’re founded on the premise of some powerful insights, insights which motivated the founders to start a business.

This is great! But the object of all this iteration is rarely “generalizable subject-matter insight,”

Changes to the foundational theory of the product are usually (and often rightly) “off limits”, or just not even salient for product teams. I think good tools-for-thought research often focuses on transcending and discarding the current system, asking “how should we build the next system?”. But good business usually don’t throw out their core product and build a meaningfully different one every few years.

Dissemination may be another challenge for tech companies contributing to this field

Some of my favorite work in tools for thought comes from idiosyncratic Twitter tinkerers. (tinkering)

The most common pattern seems to be: a bricoleur identifies some powerful idea about a representation and designs a prototype, but then fails to engage seriously with observing and deriving insight from the systems they’ve built.

I think there’s also a cultural gap here, a missing research practice of careful, diligent observation and synthesis.

Designing research insight production into the system

it’s incredibly difficult to do

the system has to be shaped in a way which allows you to ask the questions you want to ask. But often you can’t even identify the right questions to ask before you see the system in operation

Sometimes interesting answers come by accident.

For instance, when interviewing readers about their experiences, we were surprised to discover that memory effects aside, the regular review sessions meaningfully changed how people related to the material. It caused them to think of themselves as “doing quantum computing” in a much more serious way than they would if they’d just read some essay on one afternoon.

But many insights can’t be explored through passive observation and open-ended interviews. (customer discovery)

How exactly do the narrative and the retrieval practice interact?

Different answers to these questions would point to substantially different paths for the evolution of the mnemonic medium.

To answer these questions, the system has to be designed in a way which produces the necessary observations. Or you have to manipulate the system with an experiment, which may be difficult if you didn’t initially architect your system with those questions in mind

Ben Shneiderman, a pioneering human-computer interaction researcher, offers this charming schematic for research project design in The New ABCs of Research (ISBN:0198758839). He calls it the “two parents, three children” pattern.

*The challenge is similar to what learning scientists must do in designing educational interventions. In Principles and Methods of Development Research, Jan van den Akker offers a beautiful distillation of what a unit of progress looks like in that field (thanks to Sarah Lim for the pointer):

[Educational design] principles are usually heuristic statements of a format such as: “If you want to design intervention X (for the purpose/function Y in context Z), then you are best advised to give that intervention the characteristics A, B, and C (substantive emphasis), and to do that via procedures K, L, and M (procedural emphasis), because of arguments P, Q, and R [(theoretical emphasis)].”*

The key thing it does is to explicitly connect the dots between a grounded theoretical claim, the implied design approach, and the desired outcome.

Or, to take another example, I know many of my readers are fans of outline-based text editing. This morning, inspired by a message from patron Ethan Plante, I went looking for academic work on the theoretical or empirical foundations of outline processors. I was shocked how little I could find. So, if you’re experimenting with building outline processors, or “block-based” tools, or whatever, some questions to be answered: what effects do these alternative writing primitives have on composition? on thinking? on reading?

I wrote this post to clarify my own challenges, so it’s necessarily coming from a place of frustration

I want to close on a more positive note, by pointing to a few contemporary examples of projects which complete the full cycle I’ve described

Bret Victor’s projects are the classic modern example

Ink and Switch, the industrial research lab, did a thoughtful and well-documented series of experiments with freeform multimodal tablet interfaces for supporting creative thought

They built a number of interactions to support their model, identified opportunities and limitations with that system, and designed a new system based on those insights called Muse. That project put inking front-and-center and produced general ideas about designing ink-centric interfaces with no chrome. The research project is now a product company (co-founded by patron Adam Wiggins).

Piotr Wozniak, the contemporary founder of spaced repetition, is another great example... he’s been iterating on those ideas for decades now

Evan Wallace at Figma developed and documented a new primitive for representing and editing vector paths, motivated by practical problems with existing vector pen tools, which more directly (naively?) expose the underlying Bezier curves. He shows how this new representation makes certain common operations easier to perform.

In the sphere of para-academic Twitter tinkerers, I want to applaud Omar Rizwan’s experiments with TabFS. That project expresses a deep insight of Omar’s: that a shortcut to end-user programming may lie in extending the architecture of operating systems up to application-level objects—like browser tabs. In many ways, this project is an extension of Plan 9, but with a powerful injection of worse-is-better folk/craft philosophy.