Legibility of Visual vs Natural Language Interface

Kortina, 7 Nov 2017

One of my favorite examples of how I personally use Fin is, “can you book me for Sean tonight?”

Only by understanding my current context (location, day of the week, etc) and remembering the history of my previous interactions can Fin understand that I want it to visit a specific website and book a specific class with a specific yoga instructor at a specific time.

This is of course awesome when it works, but it feels totally broken (1) when the backend system cannot understand the intent and has to clarify meaning with numerous back and forth speech exchanges or (2) when the backend system does not have the power to execute the command once understood.

The feeling of “brokenness” is a result of an implicit promise of the natural language interface: by not communicating any limits to users, it says it can handle anything.

Visual interface works around this problem by both (1) grounding our expectations and communicating to us its limits and the scope of acceptable inputs through visual hierarchy — a finite number buttons, menus, single function apps we can choose from — and (2) guiding us through the work of translating an intention and our context into an explicit command legible to a software system.

Navigating visual interface feels like tedious and burdensome micromanagement, however, because we must constantly re-communicate a lot of detail and context that is ultimately inconsequential to the results we desire. A more capable system (like a person) could learn these details from a single interaction with us.

Natural language interface promises to relieve us of all of the communication overhead and work with us in a flow state of collaboration like a trusted teammate whose shared context and adept skill allow for terse and/or non-verbal (and, therefore, extremely efficient) communication.

If you’re interested in designing and building systems that are capable of learning and remembering all of the implicit context about users that would allow software to understand cryptic user intentions and deliver on the promise of natural language interface, lmk: kortina@finxpc.com.