When not to use BBD (yet)

By | 11 July, 2012

What is BDD

This blog is not about explaining BDD in detail, but here’s a short description. BDD is a way to build software against tests that are defined beforehand. BDD allows you to make sure your whole application is working, not just parts of it. It’s basically an outside-in way of testing, as apposed to unit testing which is focussed on the inner parts of the application.

What BDD is not

BDD is not a way to discover your application’s requirements or use cases. It’s a way of documenting the requirements and use cases in an executable way. Even the Wikipedia article on BDD gets it wrong here, so it’s a common misconception.

The pitfall I’ve come across is always starting your user story implementation with BDD. In the cases where you know how your application should behave that’s fine, but when the behavior still needs to be fleshed out BDD can get in your way. The confusion is between problem description and solution design, BDD is solution design while you might confuse it for problem description.

So let’s take an example:

Suppose your customer wants to have a view on the customers orders. The orders are stored in a database, but not necessarily in a user friendly structure. The user also isn’t very familiar with all the data that is stored in the order, because some of the data comes from external systems. Now, when you start asking your user for BDD examples, you will probable fail. You can’t get examples for statuses, missing information, multi-line orders etc. because the user simply doesn’t know what he/she wants exactly at this stage.

So the usual course of action is to either discuss some examples of the data, but in this example the we ‘just build something ™’. We asked the users to interact with the data for a few weeks by just showing the raw data structure. The users could relate the stored orders to other sources of information, their work process and past systems that stored customer orders. Only after that experience could they tell us how they wanted to see the data, what features they would like, what manual steps they’d like automated. And only now could we actually create a BDD scenario.

If you’d started with BDD, you would have been describing user goals and scenario’s that they couldn’t really support. Since you’re not building to solve their problems (because you don’t know them yet) how can you create BDD scenario’s? In that case your BDD tests will just be system integration tests, not value driven scenarios. So skip BDD in this case until you’ve learned a little bit more.

Lean Startup Approach – paper mockups

In the Lean Startup reminded us all of finding out what problem you’re solving before you start solving it. The goal in the Lean Startup is learning about your customers problems, desires, needs etc. The trick is to find the fastest (and cheapest) way to learn so that you can solve the right problem effectively. Sometimes writing code is the easiest way to start learning, get something out there. Now this kind of code is similar to the post-it notes in a customer brainstorming session: you throw them away.

Wrapping up

So to summarize, if you know how the application should behave use BDD. Often when a user just wants the same features he/she has seen before on another application or website, the behavior is clear. If application behavior is not clear, make sure the problems/needs are discovered and you’ve agreed on the application design before starting BDD.

A drawing that explain the two scenarios:

Application behavior is known, use BDD

Discovering through building, use BDD later

6 thoughts on “When not to use BBD (yet)

  1. Cesario Ramos

    Interesting link with the lean startup.
    I use BDD as a technique to drive development. Next to that I use requirements workshops to discover and understand customer needs.
    So I would call your ‘Idea -> Build -> Reality’ a requirement workshop where you can use any kind of discovery / modeling technique.

  2. Liz Keogh

    “BDD is not a way to discover your application’s requirements or use cases.”

    Looks like I’ve found a new way to use BDD, then.

    “Even the Wikipedia article on BDD gets it wrong here, so it’s a common misconception.”

    Oops. That’ll be mostly my fault.

    “BDD is solution design while you might confuse it for problem description.”

    I thought that was pretty much why Dan replaced “test” with “should” in the first place – because people were thinking of “test” as being related to the solution, and TDD-done-well was actually about problem exploration. http://dannorth.net/introducing-bdd/

    Here are the two main conversational patterns I use with BDD, to explore and discover requirements: http://lizkeogh.com/2011/09/22/conversational-patterns-in-bdd/

    I am interested to learn where you’re learning your BDD from.

  3. Machiel Groeneveld Post author

    Thanks for your comment and useful articles. There is no big difference in our understanding of BDD, good to read that you’re having success with it, so have I. However, any tool/language/process that describes the system is a design thing. Of course in most cases your user will be able to explain what they want in terms of systems, databases, screens and e-mails. However, that’s not always the case. In that case you need to start with the problem analysis first and BDD doesn’t seem to support this in its current form.

  4. Liz Keogh

    I think your understanding of BDD may be a bit limited. BDD *is* a problem analysis tool. Conversations around different scenarios help us uncover scenarios and outcomes we haven’t thought of (especially if you involve testers too).

    Ideally we wouldn’t be phrasing the scenarios in terms of systems, databases, emails and screens; we’d just be having conversations around the capabilities we’re delivering. If you’re using BDD as a mechanism for describing interactions with a screen then you’ll probably find that your scenarios are too fine-grained, horrendous to maintain and of little interest to the business.

    You can have the conversations for multiple levels of analysis. This blog post might help too – http://lizkeogh.com/2012/07/07/examples-in-the-large/ – using scenarios at scale, well before we start actually looking at individual features.

  5. Machiel Groeneveld Post author

    We’re getting more abstract, but capabilities is still in the realm of system design. We could call BDD a problem analysis tool, it’s just not the only tool. If your BDD spec contains something like ‘ensure a rejection message is displayed’ you’re designing the interaction of the system. My point was that sometimes your users can’t give you the spec on that level but need to interact with some software or prototype before someone can decide that that’s what the system needs to do to support the users goal.

  6. Liz Keogh

    “We could call BDD a problem analysis tool” – This is contradicted by a lot of your post.

    “it’s just not the only tool” – Of course not. I find it quite a useful tool, though, and would hate for people to avoid putting it in their toolbox because they read your blog post first.

    BTW, “ensure a rejection message is displayed” is still far lower-level than I would expect. I would probably expect “then it should tell me it’s invalid”, or something less specific. Note that the word “should” rather than “must” or “will” allows you to question the requirements – should it? Really? – so this is how you can find the places where the uncertainty you mention exists.

    A capability, to my mind, is something like “A user should be able to book a holiday”. You could do that with a piece of paper and a pen, or a visit to a travel agent. Normally I can’t tell from my scenarios what mechanisms we’re using to get those capabilities, so we’re still just analyzing the problem. Have a read of this and see what you think. http://dannorth.net/2011/01/31/whose-domain-is-it-anyway/

Leave a Reply

Your email address will not be published. Required fields are marked *