The impossibility of communicating desired intent

Communication relies on interpretation of the message by the recipient

Communication of desired intent can never be fully achieved. It would require a mind-meld between two individuals or between an individual and a machine.

The meaning (the semantics) propagated in a codified message is determined by the interpretation of the recipient, and not by the desired intent of the sender.

In the example on the right, the tree envisaged in the mind of the sender is not exactly the same as the tree resulting from the interpretation of the decoded message by the recipient.

To understand the practical ramnifications of interpretation, consider the following realistic example of communication in natural language between an analyst, a journalist, and a newspaper reader:

Communication of desired intent and interpretation

1. intent

  • Reiterate that recurring system outages at the big four banks are to be expected for at least 10 years whilst legacy systems are incrementally replaced
  • Indicate that an unpredictable and disruptive change will likely affect the landscape in banking within the next 15 years
  • Explain that similarly, 15 years ago, no one was able to predict that a large percentage of the population would be using Gmail from Google for email
  • Suggest that overseas providers of banking software or financial services may be part of the change and may compete against local banks
  • Indicate that local banks would find it hard to offer robust systems unless they each doubled or tripled their IT upgrade investments

2. interpretation

  • Bank customers must brace themselves for up to 15 years of pain
  • The big four banks would take 10 years to upgrade their systems and another five to stabilise those platforms
  • Local banks would struggle to compete against newer and nimbler rivals, which could sweep into Australia and compete against them
  • Local banks would find it hard to offer robust systems unless they each doubled or tripled their IT upgrade investments

3. intent (extrapolated from the differences between 1. and 2.)

  • Use words and numbers that maximise the period during which banking system outages are to be expected
  • Emphasise the potential threats to local banks and ignore irrelevant context information

4. interpretation

  • The various mental models that are constructed in the minds of readers who are unaware of 1.

Adults and even young children (once they have developed a theory of mind) know that others may sometimes interpret their messages in a surprising way. It is somewhat less obvious to realise that all sensory input received by the human brain is subject to interpretation, and that our own perception of reality is limited to an interpretation.

Next, consider an example of communication between a software user, a software developer (coder), and a machine, which involves both natural language and one or more computer programming languages:

Communication of desired intent including interpretation by a machine

1. intent

  • Request a system that is more reliably that the existing one
  • Simplify a number of unnecessarily complex workflows by automation
  • Ensure that all of the existing functionality is also available in the new system

2. interpretation

  • Redevelop the system in newer and more familiar technologies that offer a number of technical advantages
  • Develop a new user interface with a simplified screen and interaction design
  • Continue to allow use of the old system and provide back-end integration between the two systems

3. intent

  • Copy code patterns from another project that used some of the same technologies to avoid surprises
  • Deliver working user interface functionality as early as possible to validate the design with users
  • In the first iterations of the project continue to use the existing back-end, with a view to redeveloping the back-end at a later stage

4a. interpretation (version deployed into test environment)

  • Occasional run-time errors caused by subtle differences in the versions of the technologies used in this project and the project from which the code patterns were copied
  • Missing input validation constraints, resulting in some operational data that is considered illegal when accessed via the old system
  • Occurences of previously unencountered back-end errors due to the processing of illegal data

4b. interpretation (version deployed into production environment)

  • Most run-time errors caused by subtle differences in the versions of the technologies have been resolved
  • Since no one fully understands all the validation constraints imposed by the old system (or since some constraints are now deemed obsolete),  the back-end system has been modified to accept all operational data received via the new user interface
  • The back-end system no longer causes run-time errors but produces results (price calculations etc.) that in some cases deviate from the results produced by the old version of the back-end system

In the example above it is likely that not only the intent in step 3. but also the intent in step 1. is codified in writing. The messages in step 1. are codified in natural language, and  the messages in step 3. are codified in programming languages. Written codification in no way reduces the risk of interpretations that deviate from the desired intent. In any non-trivial system the interpretation of a specific message may depend on the context, and the same message in a different context may result in a different interpretation.

Every software developer knows that it is humanly impossible to write several hundred lines of non-trivial program code without introducing unintended “errors” that will lead to a non-expected interpretation by the machine. Humans are even quite unreliable at simple data entry tasks. Hence the need for extensive input data validation checks in software that directly alert the user to data that is inconsistent with what the system interprets as legal input.

There is no justification whatsoever to believe that the risks of mismatches between desired intent and interpretation are any less in the communication between user and software developer than in the communication between software developer and machine. Yet, somewhat surprisingly, many software development initiatives are planned and executed as if there is only a very remote chance of communication errors between users and software developers (coders).

In a nutshell, the entire agile manifesto for software development boils down to the recognition that communication errors are an unavoidable part of life, and for the most part, they occur despite the best efforts and intentions from all sides. In other words, the agile manifesto is simply an appeal to stop the highly wasteful blame culture that saps time, energy and money from all parties involved.

The big problem with most interpretations of the agile manifesto is the assumption that it is productive for a software developer to directly translate the interpretation 2. of desired user user intent 1. into an intent 3. expressed in a general purpose linear text-based programming language. This assumption is counter-productive since such a translation bridges a very large gap between user-level concepts and programming-language-level concepts. The semantic identities of user-level concepts contained in 1. end up being fragmented and scattered across a large set of programming-language-level concepts, which gets in the way of creating a shared understanding between users and software developers.

In contrast, if the software developer employs a user-level graphical domain-specific modelling notation, there is a one-to-one correspondence between the concepts in 1. and the concepts in 3., which greatly facilitates a shared understanding – or avoidance of a significant mismatch between the desired intent of the user 1. and the interpretation by the software developer 2. . The domain-specific modelling notation provides the software developer with a codification 3. of 1. that can be discussed with users and that simultaneously is easily processable by a machine. In this context the software developer takes on the role of an analyst who formalises the domain-specific semantics that are hidden in the natural language used to express 1. .


2 thoughts on “The impossibility of communicating desired intent

  1. Pingback: No one is in control, mistakes happen on this planet « Jorn Bettin

  2. Pingback: The story of life is language « Jorn Bettin

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s