Question the Source: A Questionable Code History

A model which binds together code (git) history with a question-centered decision history.

Dated
Publish
Revise
Reading time
~16 min read

Abstract

Over a software’s development life cycle (SDLC) many decisions shape the history of the source code.

Maintaining a record of decisions is ambiguous and tedious. However, naive modification is susceptible to a runaway effect (known as architectural erosion or compounding technical debt) where change becomes increasingly expensive, risky, unsafe, or stops.

This paper proposes to mitigate this effect by joining the source code history with a history of decisions. The "tandem history" joins two spaces: (1) the source space of versioned source history and hierarchical code artifacts; (2) a question-centered decision space (a directed acyclic graph, often tree shaped). The spaces are bound together at two levels of granularity: commit and line levels. The two histories must progress in tandem in reponse to changing circumstances. Participants observe and act within history through a protocol.

We present an event-based protocol, including core libraries, CLI, GUI, and instructions for use in regular development and onboarding of established code bases.

The protocol acts as a cognitive forcing function: links between lines of code and prior decisions act as inertial resistance; source changesets must be reconciled with the prior decision state.

1. Introduction

Software's exact representation is code. Should we entertain less precise representations? If so, what are worthwhile representations and their trade-offs? Finally, what cost is acceptable for the maintenance of an additional representation?

One approach to answering these questions is to observe the process of creation. In the process, many decisions occur prior to the creation of code.

This paper assumes these underlying decisions are a worthwhile representation of software. From this assumption, we propose a model and evaluate ergonomics of encoding those decisions.

A software system is defined by the code's structure, but equally of the decisions which produced that structure - shaping the architecture, tooling, deployment, and so on. Some decisions may be revised quickly with little consequence. Other decisions remain stuck with compounding effects.

Code itself is well supported by mature tooling. It can be tracked, versioned, tested, reviewed, deployed, and reverted.

Decisions are not as easily supported; decisions are of a subjective nature and without a formal definition; unlike code, a decision cannot be compiled, executed, tested, and then deployed. In practice, a decision may be accepted while tolerating 1. incomplete, ill, and/or stale context, and 2. fragmentation across isolated artifacts.

In other words: code is reliably preserved in a standardized manner, yet methods for preservation of design decisions vary and are often passive or incomplete.

This "decision-implementation" gap matters because software development depends on the ability to make and revise decisions under changing conditions. Those conditions are often poor: incomplete information, moving requirements, time constraints, shifting ownership, fatigue, and on.

The proposed method attempts to improve the conditions for decision making by modeling decisions as active elements which are tightly integrated with source code. Integration of decisions and source code occurs through the authoring of a singular history where decision history is joined with source code history at two levels of precision.

2. Background

2.1. Inspiration

This work began as a curiosity during a course on software engineering. While setting up a repository and initial scaffolding for a group project, I wanted to communicate the thought process behind the decisions I was making, so I documented my considerations in Architecture Decision Records (Architecture Decision Records (ADRs)). Usually only a single ADR is created for the accepted choice of a decision. However, as an experiment, I also made ADRs for the options I did not choose. This prompted me to wonder about how ADRs might be structured differently. I imagined a decision “environment” as an active space linked directly to the source code. Version control systems provide the “unit of change” as an anchor between code and this decision space.

2.2. Motivation

The motivation behind bothering with this work comes from a few places. One is to allow creative exploration of an idea and see it through. Another is the aspiration of designing a tool which encourages and facilitates group decision making. A third is a response to automation of decision making.

3. Methodology

The proposed methodology revises and combines existing methods. The “decision space” is a variation on Question-Options-Criteria (QOC). This QOC-based decision space evolves alongside the source code history at each commit. Additionally, within a commit, links are made from each line of code to an accepted answer.

3.1. Architecture

This section describes the software architecture of this project.

3.1.1. Tandem History

The name “tandem” indicates that multiple histories move in sync.

A high-level architecture diagram showing the Tandem History containing multiple Participants and Narratives. A Narrative contains a single Decision Space and a single Source Space.
Figure 1: High level architecture diagram of the Tandem History
3.1.1.1. Participants, Interactions & Narratives

Within the Tandem History are participants and narratives. A narrative is defined by participants’ interactions recorded in the journal. The narrative is where source history and decision history are joined. There may be many narratives for a codebase, however, in general, there is one commonly accepted narrative. An example of differing narratives is: a detailed versus condensed narrative.

3.1.1.2. Books, Chapters & Paragraphs

A narrative is organized in books, chapters and paragraphs. A book contains a sequence of chapters. A chapter contains a sequence of paragraphs. Paragraphs contain a set of participant interactions. Paragraphs also validate coverage of lines of code in source history (discussed in “Responsibility Links”).

3.1.1.3. Binding

A paragraph brings together source history and decision history via “binding”. A paragraph binds to a sequence of states in source history and a set of interactions in decision history.

  1. Source History

    The source code history is represented as a discrete sequence of states (or, commits, snapshots, etc). The current implementation supports one sequence (e.g. main branch) which all narratives reference.

  2. Decision History

    Decision history is represented as a journal of participants’ interactions. Decisions are modeled as questions. While all narratives reference the same source history, the decision history may subjectively vary.

3.1.2. Source Space

The source space is represented as a simple sequence of states of the source code.

3.1.3. Decision Space

The decision space is a graph structure primarily composed of questions and answers.

3.1.3.1. Questions

Questions are the nodes in the graph and contain possible answers and the accepted decisions.

3.1.3.2. Answers: Slices & Stances

Answers live within a question node and serve as a source for edges to connect to other questions.

Answers are broken into two abstractions: slices and stances. The purpose is to allow the exact answer (stance) to be revised, while still belonging to the same general direction (slice).

A slice is the “general direction” of an answer. A slice serves as a container for answer revisions.

A stance is the exact position of an answer. To revise an answer, a new stance is created within the slice.

  1. “Base” Answer

    The methodology defines a concept of a “base” answer which must be defined for every question. The base answer is not “empty” or “null”, but instead represents the position “do nothing” and the impacts of that position. The purpose of the base answer is to provide an origin point with respect to other answers.

3.1.3.3. Connections

Edges within the decision space are directed. The source is always an answer (stance) and the destination is another question.

There are two types of edges: "leads to" and "constrains".

  1. Leads To

    Answers may lead to more questions.

    image2.png
    Figure 3: Diagram representing a stance leading to another question
  2. Constrains

    Answers may also constrain the decision space of other questions.

    Currently, there is no enforcement of the "constrains" edge.

3.1.3.4. Decision Points

For a question, decision points are explicitly created. At a decision point, participants may individually accept an answer (a stance).

A new decision point must be created, to accept a revised or different stance.

alt text
Figure 4: Diagram representing a participant accepting a stance at a decision point
  1. Accepted Answers

    After a decision point is created, participants can then accept an answer for that question.

3.1.4. Observation Space

The purpose of the observation space is to supply supporting evidence for an interaction. A participant may “ground” a target with an observation. Observations may be a simple description, URL, a file, etc.

3.1.5. Meta-interactions: Explanations & Annotations

The methodology includes a concept of explanations and annotations. Explanations give supporting context at the time of recording an interaction. Annotations annotate previous interactions after the fact. Explanations occur at the interaction level, while annotations, like observations, may annotate any target.

3.1.6. Journal

Participants author the tandem history through interactions which are recorded in an immutable journal.

3.1.6.1. Command & Query Interface

Interactions have two kinds: commands or queries. Queries are “reads” and commands are “writes”. Query interactions only return a value to be read. Command interactions may emit zero or more events.

3.1.6.2. Event-Based

The methodology utilizes an event-based approach. An event represents a successful mutation to a narrative. For example: an event may describe the creation of a question, an accepted answer, a responsibility link, etc. The graph representation is then built from the sequence of recorded events.

3.2. Usage

3.2.1. User Interfaces

3.2.1.1. Command Line Interface

A CLI exposes commands for initializing a new history, running tutorials, as well as performing individual participant interactions. The CLI may also be driven conversationally by an agent.

3.2.1.2. Graphical Interface

A visual representation of the methodology is extremely helpful in understanding the method itself but as you are authoring history. A web based viewer is provided. The web viewer can replay history on an individual interaction basis or narrative organizational basis (book, chapter, paragraph).

3.2.2. Usage Patterns

3.2.2.1. Question First

Ask questions first, consider options, accept answers, then author the code. After making commits, bind commits to the paragraph, and lastly, assign responsibility from accepted answers to lines of code.

3.2.2.2. Code First

Write the code first, then backfill questions and answers to assign responsibility coverage to lines of code.

3.2.2.3. Simultaneously

Switch between the decision and source space as new questions naturally arise or answers are revised.

3.2.3. In Practice

Below are examples of how this methodology might fit into a daily scenarios.

3.2.3.3. During A Conversation
  1. Actively

    Pause to deliberately form questions and answers as they arise in the conversation.

  2. Passively

    Converse as usual but infer questions and their answers in real time or afterwards.

4. Evaluation

4.1. Qualitative

4.1.1. Toy Scenarios

4.1.1.1. “Double-it”

An example scenario of designing a library which can double a number to give a minimal demonstration.

4.1.2. Real World Use

This methodology has been used in the following use cases.

4.1.2.1. This Project Itself

The methodology was used during the development of this project’s own source code. From this experience, I must say that the process of formulating the question graph does indeed introduce quite a bit of friction. Coming up with questions, proposing possible answers… then linking accepted answers to cover every line of merely the README took a lot of time and effort (that being said, the readme contained quite a few concepts). I might compare it to the effort of deciding on appropriate member names, doing research, documenting a process, etc.

Before line responsibility coverage, binding commits to paragraphs forces one to reconsider the order in which behavior is introduced. Grouping by paragraph does enable flexibility of commit size, however by design demands that nearby commits are related, which in practice is often not the case. Even though I was conscious of this as I was authoring the code, I very often found myself re-writing [git] history. Re-writing history is acceptable for unpublished code, but generally forbidden otherwise.

4.1.2.2. Example Boilerplate Repositories

I curated a set of starting questions and minimal source boilerplate as an open-source repository to be forked. The purpose of the example repository is: 1. demonstrate the methodology, 2. promote an accessible starting point, 3. identify common sets of questions.

4.1.2.3. Financial Audit Software

This methodology has been trialed in a three person team for development of a financial auditing software. During the initial information gathering phase of an audit, the software helps coordinate documentation requests between a firm and the client being audited.

4.1.2.4. Existing Repository Onboarding

To evaluate onboarding an existing repository, a general set of instructions have been developed to serially process repository commit history and directly linked context (issues, Pull Requests (PRs), etc). The goal is to infer a condensed educational history which introduces the code expressed as a series of questions.

4.2. Quantitative

While this method is of a subjective nature, we can quantitatively evaluate using it.

4.2.1. Cost

In the sections below, cost is compared with and without using this method.

4.2.1.1. Interaction Method
  1. Manual

    This section describes an evaluation of manual use of the CLI.

    1. Baseline (git)
    2. Additional `inistory` commands
  2. Conversational

    This section describes an evaluation of conversational use where a language model facilitates the CLI.

    1. Baseline (git)
    2. Additional `inistory` commands

5. Limitations

The proposed method is relatively untested outside of the development of this project and demands further evaluation in additional scenarios and specifically over long time horizons. Until further evaluation, no claims to its usefulness can be made and it may merely be suggested as an educational tool.

5.1. Bypassing Intended Use

It is possible to work around the intended use by creating meaningless decisions and mapping them to lines of code. The method cannot enforce otherwise. Just as you may write bad code, you may ask bad questions and make bad decisions.

5.2. Published Trunk Dynamics

In practice, published history (the main trunk) is never revised unless absolutely necessary (e.g. a leaked secret must be unpublished). Instead, mistakes are addressed by subsequent fix commits.

It may be possible to use this method in a trunk based workflow, though the current design implies that the entirety of source history is to be revised while the question space contains the immutable record. In practice, it may be possible to partition history into: an immutable trunk, a mutable middle section, and a dumping ground tip.

5.3. Cost, Cognitive Load

Adopting any tool has costs. This method and tooling is additionally designed with intentional friction. It remains to be evaluated if these up front costs pay off over time.

5.4. Proposed Structure vs Real World Mismatch

The proposed data structures and interaction protocol may not match real world team dynamics. Specifically, the model of questions and answers may not match a team’s decision making process.

6. Future Work

6.1. Education

6.1.1. Maintaining an Open-Source Set of Example Repositories

As a form of documentation while also a point of adoption, I would be interested in maintaining a set of high quality “boilerplate” templates. These templates would also aim to illustrate design decisions.

6.2. Machine Learning

6.2.1. Small Language Models as Facilitators

For conversational use, I would be curious to explore small models as facilitators.

6.2.2. "System One" Structured Decision Models

6.2.2.1. Constraints Enforcement

The constrains edge is currently ceremony. I would be curious to try a System One model which "decides" if a proposed answer is valid or invalid with respect to the constraints from previously accepted answers.

6.2.2.2. Responsibility Assignment

Currently, responsibility assignment is manual through the GUI or CLI. I have also evaluated it when driven via a Large Language Model (LLM) agent. I would be curious to feed a line or hunk into a decision model where it returns a noul for each accepted answer.

6.2.3. On-boarding Existing Repositories

There are many ergonomics to explore in how this may be semi-automated.

6.3. Future Versions & Continued Development

6.3.1. Configuration: Fields, Scoring Schemes

In the current implementation, an answer (a stance) defaults to only two fields: “Position” and “Impact”. However, answer fields are configurable. Additionally, answers can be scored in many ways.

6.3.2. Deployment Permissions: Participant Abilities

An individual participant’s acceptance of an answer to be used as a signal in a deployment pipeline.

6.3.3. Organization / Multi-project Support

Support for organizational needs such as project templates, referencing other projects’ narratives, etc.

6.3.4. Real-Time Collaborative Decision Making Interface

A prototype was created to extend the methodology to a real-time group decision making scenario. One aim of the methodology is to encourage collaboration: anyone is capable of asking a question.

6.3.5. Typed Stacked Pull Requests

In the method, `paragraphs` serve as the binding site with a segment of commits, thus similarly shaped to a stacked pull request. I am curious about the idea of a “typed” stack of pull requests where each commit in a segment obeys a pattern with properties.

6.3.6. Compare-and-Swap as Reviewable Change Units

Curious about experimental merge queues that reconstruct history instead of appending to the end.

7. Conclusion

Still lots of work to do. The method is being “dogfood-ed” and I am running evaluations and reconstruction of existing repos this week as encouraged by my professor!

References

These exist! They’re in emacs, they will be exported here.

todo refs for sections 1, 2, …

More examples are available from the Chicago Manual of Style: http://www.libs.uga.edu/ref/chicago.html#authordate (opens in a new tab)

8. Acronyms

ADR Architecture Decision Record 1, 2, 3, 4, 5

LLM Large Language Model 1

PR Pull Request 1

9. Glossary

Cognitive forcing function Something that makes you think. More specifically, something that disrupts your reflexive reaction (System 1) and slows you down to think more deeply (System 2). 1, 2

Site settings

Reading progress
Reading progress