Tuesday, May 10, 2022

Less obvious advice on User Stories

Much have been written already about the craft of writing User Stories and this post is not meant to repeat or replace any of the existing writings on the subject. On the other hand I know there is a dead zone in there and it is not easy to find advice on certain practical aspects of writing well-crafted Stories. I also feel I've accumulated enough of the less obvious knowledge of that craft that sharing it is likely to fill in some of the gaps. This post is targeted at the audience with a good amount of experience at writing User Stories already: Product Owners, Scrum Masters and Agile Coaches.

I'm putting down my thoughts in a few sections. First of all, we will examine the difficulties of writing User Stories for Cyber-physical products and Web Services, where the notion of a user does not fit well into what we try to achieve. Then, we will see why developing User Stories that break customer or marketing requirements can be a good thing (as crazy as it may sound). In the third section, I'm making a case against the widespread criticism of using the well known "as a user" in User Story Statements. Finally, I have some advice related to User Story structure - what parts a well-defined Story consists of and in what order to define them.

 

User Stories in Web Services and Cyber-physical products

I worked on multiple products where features being added were not directly perceivable to the users. A few examples:

A device in a network infrastructure, such as in a telecommunication systems, where only the System (and not the device alone) delivers a capability perceivable to the user - for example, the user is able to make a call over an LTE network.

A device deployed as an IoT device in an ecosystem of other devices and applications that performs an action perceivable to the user - for example, a device that opens a door lock in a building.

A Web Service that retrieves data from a backend system and responds to a request with a JSON object - for example, a service that retrieves a real time quote for a given ticker symbol.

In all of these cases many, or most of the User Stories don't align well with the widely known template: As {user type| user role | persona}, I want (...)

 


Typically, the scenario here is that a product gets a stimulus from the environment (e.g. from the network, but it also could be a GPS signal or a temperature sensor, etc.) and needs to respond with again, either a network packet or some other physical action (such as opening / closing an electric circuit, applying voltage to a motor or turning on a LED). We deal with similar situations when working on web services that expose APIs to the outside world, but the service itself is not directly used by any human users. This poses some difficulty and sometimes people try to cope with it by bending the "As a user" template and desperately trying to apply it. Instead, a better approach is to treat system components as Actors, as in the following examples:

  • Upon receiving a request for {type-of-data | params}, the {product} sends out a response containing {response-data}
    • this could be a typical Web Service case with REST API
  • When {product} receives a {network-message} from {other-system-component}, it stores the {input-data} persistently and performs {action-description}
    • in a multi-component cyber-physical system, our product interacts with other system components - typically other physical devices that communicate over a network
  • When {stimulus} is not received by {product} for longer than {timeout-value}, the {product} logs a critical malfunction to the internal log and reboots
    • this is a fairly typical heartbeat scenario, often employed in cyber-physical systems such as telecommunication systems
  • The {product} will periodically send (every 500 ms) a {message} over {link-name}
    • as above, but could also be expanded with an ACK
  • If the input power goes below {power-level}, the {product} will disable {high-power-features}
    • typical of IoT or personal devices, such as smartphones

  • If license expiration is detected during login, the {product} blocks all functions and redirect the user to the license renewal screen

In these examples, our Product is an Actor and at the same time it may be just one of the components (other Actors) in a bigger, cyber-physical system. Or it may be a Web Service or an IoT device that interacts with its environment (a network, but also physical environment - temperature, GPS signal, sunlight, etc.)

The last example is interesting in that it could be a regular web or desktop application, so there is definitely a notion of a user and the user directly perceives product's behavior. Why don't we use the usual "As a user" in that case? Well, we could, but... I just feel it isn't natural in this example, because the truth is, the user wants to use the product. The expired license is an obstacle. The user does not really want all of the functions to get blocked, so it would really sound unnatural to say "As a user, when my license expires, I want all features to become unavailable", wouldn't it? For this reason, this scenario feels more product-centric to me and I prefer treating the Product as an Actor, rather than the user.


User Stories that make the product [temporarily] not adhering to requirements

Building a product iteratively (which is not the same as incrementally) is the key to enabling early feedback, reducing risk and delivering features efficiently. Iterative development enriches the product with new functions and characteristics in each pass. Extensively used, the iterative approach naturally evolves into an advanced method where some of the Stories being developed make the product temporarily not adhering to customer's or marketing requirements - this is the Ri stage in terms of Shu Ha, Ri stages (守 破 離). I know, it sounds a little bit cryptic, so let's go through some realistic scenarios to understand what I'm actually getting at. The three scenarios below are examples of building something that seemingly breaks the requirements (and for a good reason) and then progressing towards a correct, production-ready solution. They are presented by the type of the progression:

  • From wide to limited
    • a capability is delivered in a wide-open form and will be then gradually trimmed to its desired shape 
    • Examples:
      • a new feature is made accessible for all users, with no regard to user roles; later on, access to the feature is constrained so that only the users with specific role(s) can use it
      • a connection to a service or data resource is established without final security measures (for example, basic auth is used instead of OAuth 2.0); later on, the desired authentication method is added to the solution.
  • From technically simplified to production-grade
    • a capability is delivered with major technical simplification, in order to capture the end-to-end scenario flow; this often means employing the walking skeleton approach
    • Examples:
      • data that is retrieved does not come from a target resource (such as a DB or a service), but from a static text file; later on, the target data flow is added
      • a data caching solution is developed without optimizations and some pieces of information presented to the users are refreshed as infrequently as every hour; later on, the solution is optimized and a near-real-time status of the data is achieved
  • From specific to exhaustive
    • simplification is achieved by developing only towards a specific set of conditions; later on, the conditions are broadened to cover all desired scenarios; this also marries nicely with the walking skeleton approach
    • Example:
      • generate a money lend request with all of the input options hard coded, except the amount; later on the input options are gradually added to cover all lend request options

When we are in the middle of the progression from the current state (e.g. data comes from a text file) to the future state (e.g. data is retrieved from a DB) the solution as a whole may not be usable at all. For example, a lend request with a fixed lend time most likely isn't a production-ready solution. Well, sometimes that's the way to go. Let's look at this scenario a bit closer:

  • let's suppose we are not able to deliver a production-ready solution in one Sprint, but we are still able to produce something valuable within one Sprint. We are able to deliver an end-to-end flow that can be presented to our stakeholders. The stakeholders can exercise the product in a QA environment, knowing that it has its limitations, and produce valuable feedback.
  • within the next two Sprints we are able to add new functions that are built on top of that and together they make the solution production-ready
  • in order to be able to do this, ideally we want to be able to hide the in-progress solution behind a feature flag, so that it remains disabled in production until it is production-ready.

Let's suppose the whole work on a production ready solution takes three consecutive Sprints. We are not going to benefit from the business value delivered, until Sprint 3 finishes. Does it mean we are not getting anything useful after Sprints 1 and 2? No, not at all. First of all, we are generating new knowledge by making two passes through what we are building. The team will likely learn things such as technical constraints, refactoring opportunities, testing challenges and many other. Secondly, as I mentioned already, the working software available after Sprints 1 and 2 is (even though not production-ready yet) can be internally used to elicit feedback from stakeholders, designers, customer reps and people in similar roles. And it goes without saying, all the time it is a tested piece of software, integrated to our codebase, it goes all the time through our CI pipeline, etc. This is a real-life situation you may find yourself at pretty often and that's the a really good approach to deal with it.


The defending of the user term in User Stories

I remember when the well known template

As a user, I want <capability> in order to <goal>

started circulating in various articles and books. At that time, it was a really good guidance on how to approach defining the User Story Statement. Then I learned about a variation of this template which was even better, because it focused attention on the user goal:

In order to <goal>, as a <role> I want to <capability>

What I can very often see today is that people frown upon using the term "user" in the User Story Statement. It's almost as if they deliberately try to spot this term in order to pronounce it a schism.

What I want to say here is that I feel the plain "user" term does not always deserves so much criticism. I understand the need to be precise about who exercises a given capability of the product, but I can also point out scenarios where the plain "user" term is good enough, especially if augmented with some detail inside the Story. For example:

  • Some products truly have just "a user" and there is no consideration of user roles or other attributes whatsoever. A more typical example could be "a user" and "a user with paid subscription". Consider a gym app where you can be a user with or without a subscription. A good amount of app's capabilities is available to both types of users, so for these, saying "As a user" is completely fine. For capabilities accessible only to users with a subscription, we can add this detail either i) in the User Story Statement: As a user with subscription OR ii) in the Detailed Description of the Story (the term Detailed Description is explained later on): (...) and if the user has a subscription, then (...)
  • Other products have a set of roles that can be assigned to users in multiple combinations. Let's consider a case of a dozen roles or more (a realistic example from my experience). Using all possible combinations of roles or defining Personas for each combination and then using them in the User Story statement would be highly impractical in that case. I believe it is much better to accept "a user" in the User Story statement and add the logic related to roles in the Detailed Description, for example while explaining the scenarios: (...) but if the user also has the Administrator role, then enable (...)
  • Finally, the plain "user" term is also perfectly fine when we are precise about the user's intent outright in the User Story Statement: As a user whose credit card validity expired (...)
Next time you spot the "as a user", please don't be too quick to criticize it.

User Story Title, Statement and Detailed Description

If we follow "As a user" or any other template, our backlog may be soon filled with Stories that all seem alike. My own backlog used to be like that until someone showed me a better approach - namely, that we need a Title to tell the Stories from each other and be able to easily move about them, change their order, statuses and Sprints.

The Title should be short (a few words) so that when the team looks together at the Sprint backlog or Product backlog, the title unambiguously identifies each Story. Consider these examples:

  • Title: Cart item deletion
  • User Story Statement: As a user, I want to be able to delete an item from the cart.
 
  • Title: Quarterly sales report download restriction
  • User Story Statement: As a user with Regional Sales Manager role, I want to be able to download the quarterly sales report.
 
  • Detailed Description: (...) users without RSM role are not able to download the report.
  • Title: E-invoice option
  • User Story Statement: As a user receiving paper invoices, I want to be able to activate the paperless e-invoice option.
 
Credit: I learned the trick with the Title / Statement from Aneta Wojcik.

Start with the Acceptance Criteria, then do the rest

Well-crafted user User Stories are built on good understanding of the end goal. Therefore, one more tip I can provide on defining the User Stories is to start with Acceptance Criteria. Don't start from the User Story Statement and don't dwell into the details too soon - doing so may fool even the smartest professionals. Try defining one or more Acceptance Criteria that can be used to verify whether a Story has been delivered. If you start from Acceptance Criteria, you will notice that it influences the way you write up all the other parts of the User Story. Wait, so what are the other parts? They are all in the list below:

  1. Define Acceptance Criteria
  2. Write up the User Story Statement
  3. Write up the Title
  4. Write up the Detailed Description

The Title (3) can really be done in any order. You can do it before or after working on Acceptance Criteria and change it as your understanding of the Story evolves.

The Statement (2) should really be done after the Acceptance Criteria (1).

The Detailed Description (4) deserves some additional notes, because we have not introduced this term properly yet. Like was said earlier, the Title is just a few words. The User Story Statement is a one-liner. There are typically from one to a few Acceptance Criteria statements. Even summed up, it is often not enough to convey all of the necessary details, so it is good to put them together as a Detailed Description paragraph. This is what the paragraph will typically contain:

  • description of scenarios that do not obviously follow from the User Story Statement, such as consideration of user roles, involved conditions for something to happen, etc.
  • explanation of relation or comparison to other User Stories that build capabilities together, extend each other, build on top of each other, etc.
  • testing remarks, such as edge cases, notes about particularly difficult or otherwise special test environment conditions, etc.
  • any technical remarks from the Development Team (typically added as a result of a Backlog Refinement meeting, but could be added anytime) related to the existing code, patterns used, essentially anything the Development Team considers helpful.

All of these attributes: the Title, the User Story Statement, the Acceptance Criteria and the Detailed Description can change as a result of the Backlog Refinement meeting, but you will see the Detailed Description getting changed most often. Actually, the Development Team should be able to edit Detailed Description on their own, to capture any useful technical findings.

 

Summary

We've gone through some interesting cases that have the characteristic of being rarely mentioned in books or articles, but happen quite often in real life. We've seen some examples from Web Services and Cyber-physical systems where the notion of a user vague and we've learned how to deal with that by treating the Product as an Actor. We've seen how we can move about iterative delivery by working on Stories that seemingly break the requirements and we can realize gains from this approach and then in the end it all comes out nicely and we keep the stakeholders happy. I've also presented my take on User Story structure, which is how I do it and you may have your own way of doing it, but I still recommend that you start from doing the Acceptance Criteria. I also couldn't resist writing about the famous "as a user" and I actually used the plain user myself a few times throughout the post :-)
 
 

Pictures used:

[1] Ruben's 1536 rendition of Vulcan, the Roman counterpart of Hephaestus, the Greek God of Craftsmen, photographic reproduction

[2] My own hand-drawing of Product stimulus and response

[3] "The Spanish Inquisition" Monty Python's sketch photo by Eduardo Unda-Sanzana was used under cc-by-2.0 license

See also