Rationale

What is a version number? It’s a label (doesn’t have to be numeric) that identifies one particular snapshot of your software. Usually there is one overall version number for the whole system, even when components of that system have their own versions. Using this label, anyone can reason about the state of the code for that label. Users might say, in version IE 9.0.1.2 there is a bug when I try to print. The developers will know exactly which state the code was in for that version number and should be able to find and run that version (and fix the bug). Summary: you need to label revisions of your code to refer to it later.

Numbers are silly

So far, so good. Version labels are useful. But in stead of labeling the software with a date of release, the name of the latest feature, most developers use numeric labels. And not just a sequence number (1, 2, 3), but fancy numbers with dots! Giving your application a version number like 15.0.12.21.h makes it (and you) look really complicated and smart, don’t you think? Have you seen the Chrome version numbers? Anyway, let’s go into these version numbers more deeply. A typical application, let’s say a website for creating blogs, releases these versions:

• 1.0
• 1.1
• 1.2
• 2.0

What does that mean? Well, convention says:

• first ever release
• small change
• small change
• big change

That’s it, nothing more to it. But the numbers look more professional, don’t you think? Although list 1 and 2 have the exact same meaning. And was the change from 1.2 to 2.0 was really that big? We don’t know. What this developer deems big, might not be so big and worthy of a ‘major’ version bump for others. So why use numbers at all? Why not use descriptive names like:

• ‘first basic YourBlog release’
• ‘fixed bugs in login’
• ‘fixed bugs in page-saving’
• ‘redesigned UI’

My Application is an API

Maven: WTF?

Maven doesn’t help in making people steer clear of the number fetish. When you create a new maven project. The initial version for you module is: 0.0.1. Yes, not one dot, but two! You’re going to be doing some serious API-building business. No, you’re not.

What does that mean? By convention, 0.0.1 means ultra-pre-beta-alpha release. It’s probably just prints ‘hello world’ in a console. So you start building and maven increases your version with every release you make. 0.0.1, 0.0.2, 0.0.3. You’re not making much progress, so after a few weeks you make the bold move and change it to 0.1, your first pre-beta-alpha release! Or is it, perhaps it’s already live, in production and making money. When do you switch over to 1.0? When all the bugs have been fixed? The first time you go live? 1.0 means done, right? Stable, right? Yes, it does. But your website is never done, your GUI will always have issues so there is no 1.0!

I find when building whole systems, there is never a clean cut release. The first public release is not much different from the one before or after. So stop fiddling with the numbers.

Just to get back at the API argument, when you release an API to the world, there will be a 1.0. It’s that contract again, meaning: this API is stable and ready for production use, it has undergone testing and everything. This number means something because your communicating it explicitly, it’s in the product you ship (like commons-lang-1.0.jar). Your website just has a url, your GUI an icon, no version.

Version-less coding

Some code doesn’t need version numbers at all, it’s just code. If it’s checked into source control (on a particular branch) it’s usable. No 0.1 release or 0.1-SNAPSHOT. Just the code. If I want to make change that breaks stuff, I’ll create a separate branch. Maven doesn’t allow this, it’s basically duplicating what my source control system can do better (track revisions). For libraries, I might want this, but for my main project, I don’t. One of the reasons these version numbers are still so prevalent, is that Maven requires a version number. Starting with 0.0.1 will start you off thinking you need a complex version numbering scheme.

The build number alternative

I propose a complete alternative to using version numbers. It’s simple: build numbers and labels. Every time you build your system, for instance in your CI tool (like Hudson) it gets a build identifier, let’s say a timestamp. That number might be set in your source control as a label or the revision of the code might be stored alongside the build number, either way the build number will be a reference to a specific state of your code, but also a specific attempt to build that code. Sometimes you have to build the same code multiple times to get a good release (your build procedure might have issues). Now the build numbers are just a sequence. You can label releases that are actually going out for release, so you can refer to them later. Using the build number, you can even pick up the exact artifacts from that build.

• 2011-01-01-1543 – ‘added new content type’
• 2011-01-02-1218
• 2011-01-02-1543
• 2011-01-02-1743 - ‘improved render time’
• 2011-01-03-1109

Most CI tools (like Bamboo, probably others too) even support labeling a build.

One extra benefit of this, is that your code doesn’t have to contain version numbers. I think it’s a smell that the version of the code, is in the code. The version is something external to your code, your source control system has to deal with revisions, not the code itself. You’ll see that a lot of web projects don’t have a version at all, it’s just the code.

So, using labels in stead of dot-numbers, everyone will know what the version entails but you don’t have to worry about the numbers anymore. So, let’s make it easier!

Most teams working with an Agile process like Scrum use a task board (physical or digital). Even many teams that only do a bit of Agile or Scrum will work with a task board. It’s a core practice in Agile software development, not specific to any methodology. So, it’s common practice.

So like all things in Agile it should be subject to scrutiny once in a while. A fair question is, why have a task board at all? Just because you’re doing Agile(tm)? No one benefits from blind adoption, so ask yourself this question. Not to get rid of your task board, but to find out if it really fits your purpose and to find room for improvement.

The previous board(s)

The kanban board:

One team had a kanban board, that had many lanes. It represented roughly our process. But since the team had 3 team members (2 developers and 1 tester), the team had some trouble keeping track of what story actually needed work. Also the meaning of the lanes became vague. For instance after the tests for a user story had been defined, it went into Implementing (make the tests pass), but what if more tests had to be defined along the way? Was this testing or implementing? Also some stories didn’t require formal acceptance, so the column was skipped. Some user stories popped up during development, so didn’t go through the explore & plan columns but also weren’t picked up immediately. So, in short, not very helpful.

The lanes were a result of months of tuning, so they did have meaning at some point in time, for some kind of work, but not the work the team was doing now. We did appreciate the visibility of the phase a user story was in and the wip limits.

The simple task board:

The other team had a task board which was almost the complete opposite of the kanban board. It didn’t even have a ‘done’ column! It was the bare bone task board with a twist, trying to keep the pull between the tester and developers by using two task boards, one for the developers and one for the testers. The main issue here is that it wasn’t clear what the exact tasks were that were being worked on. And as the cooperation between developers and testers tightened, having two boards didn’t always make sense. When we tried adding sticky notes (green ones) to track the smaller tasks within a user story, it became quite messy, it was hard to see which task belonged to which user story.

Having a board retrospective

Picking a theme for a retrospective is a good tip from the Agile Retrospectives book. For us the theme is sometimes the task board. When you hold a retrospective with your team, to discuss whether your task board really fits your needs, the format we used last time was:

1. Define the goals for your task board (what would you miss when it’s gone?)
2. Evaluate whether the current board meets those goals
3. Brainstorm on new board formats that better meet the goals
4. Decide how to change the board

Board goals

So in our last task boar retrospective, the team defined the goals for the board. Based on the team’s experience and day to day needs, the main goals for any task board were defined as:

1. See what’s left for a user story
2. See what’s left for an MMF/Sprint
3. See what we’re working on now
4. Visualize loss of focus

The New Board

We brainstormed on new board designs. We introduced the ‘swim lane’ concept and tried to use it to fulfill our goals. At the end of the meeting we came up with this (the arrows shows a swim lane):

Wrapping up

So we’ve had many different boards in the past and many retros discussing the use of the board. We started with Scrum, switched to kanban and increased the number of lanes. Finally we ending up with a board that supports the goals we discovered along the way and  now defined explicitly. It has two basic principles of kanban (pull & wip) strongly embedded while giving the team visibility of what’s ahead.

Perhaps you’ve had similar retrospectives or discussions? Do you have explicit goals for your board? I’m very much interested in your story regarding the development of your task board, so please share your experience!

When exploring a new medium, four interesting questions to ask are:

- What does it improve or enhance?
- What does it neglect, displace or make obsolete?
- What does it bring back from the past that has been neglected recently?
- What does it overturn? (When pushed to its extremes, a technology fosters side effects that are opposite to the effects that were first expected, both positive and negative).

Now you could ask yourself these questions regarding the iPad, which is a new medium for conveying information and is quite visible an extension of its user. Using it as a knowledge base, navigating, helping us communicate with others it extends our natural abilities.

Back to the questions, let’s give them a try, perhaps your answers would be different, that would be a great start of a discussion of what the iPad actually means. Think sociological effects, not just purely functional. What helps is to image everyone having an iPad and/or using it for everything, what would be different in such a world? Answers can be both positive and negative.

What does the iPad improve or enhance?

• It increases the time spent behind a computer (in the park, in bed, on the train etc.)
• It enhances people’s ability to interact with data and software, it helps computerize society

What does the iPad make obsolete?

• Some people thought it would make the laptop or net-book obsolete. I’m not too sure about that, but you might say it makes the laptop less favorable for watching movies or pictures. But that’s is not a sociological effect in itself.
• Books, encyclopedias, cook books, news papers.
• It makes staying at home or at a desk obsolete (when consuming data)

What does the iPad bring back from the past?

• The touch interface obviously brings back the tactile experience which other computers don’t have. Before the age of computers, dealing with information and entertainment was much more tactile, we had books and magazines and games were physical as well (board games). The iPad brings back interaction using our touch senses.
• It brings back the book, but improved. Perhaps reading and engaging in stories will get an impulse.

What does the iPad overturn?

• You could imagine that if everyone stopped using PCs and laptops and only iPads, users would consume only bite-sized data and though the iPad is an information-sharing device, if it’s the only thing you use, you might stop producing data. In effect limiting the information sharing.
• Can’t think of more, this one is hard…

Hopefully this has raised your interest in McLuhan and made you think a bit more about what sociological effects a new medium (like the iPad) has, often they have a far more profound impact on our society and way of thinking than we initially presume.

Currently I’m thinking about different ways to set up a good set up for creating/changing websites. When your team consists of HTML/CSS developers on one side that know about user interaction, graphics and layouts and Java developers on the other side that excel at making data available and dealing with the complexities of 3rd party communications. The challenge is to set up the team and architecture for an effective workflow, letting everyone do what he/she does best.

The context is a company that has a lot of data and wants to make this data accessible through multiple channels and sites. The user experience is a very important, so tuning the websites on the HTML level is an ongoing activity. Expanding to new devices and websites is one of the (technical) objectives.

The way to set up the components and what abstractions to use is not obvious. After speaking to some peers in the industry, I’ve distilled three main options to choose from. Combinations are possible, but I’ll describe the simple case. I haven’t decided which is best, though and I might be overlooking some crucial things. My main goals for any solutions are:

• Zero round-trip while developing HTML/CSS/JS
• Independence of HTML developer when creating new ways to display the same data
• Smooth work-flow from concept to implementation.

JavaScript All the Way

With this approach JavaScript takes care of all interactions of the website. A good example is the Google Start page, which contains very little HTML, but once you enter a search term, the content completely changes, using JavaScript

The pre-rendered HTML is very minimal, providing the structure that you fill with JavaScript. The biggest advantage is, provided you can create a clean remote API for JavaScript to access, total independence of your HTML development. As long as you want to display the same data, new websites can be created with few changes required on the back-end. Since the HTML doesn’t have to be server generated, it can be plain HTML all the way. Although that simplifies things there are some downsides as well. Google being the biggest issue; the content generated by JavaScript is not parsed by Google, so your site will not do well in the search engines. Getting the data to your JavaScript means you’ll have to provide an easy http accessible API (using JSON or REST etc.), it might not be trivial to come up with a clean API, basically you’re adding a physical layer to your architecture.

Pros

• Largely independent from back-end
• Zero roundtrip when changing HTML
• There is only one version of the HTML (no conversion to templates)

Cons

• Google cannot index content generated by JavaScript
• Complex workflows that keep state are harder to maintain/debug in JavaScript
• An extra layer of
• Performance and security
• Re-use is limited

Server Side Scripting

Here you don’t let JavaScript fill your combo-boxes, but it’s a dynamic language like PHP/Python/Ruby/ASP that renders them (let’s assume PHP). The HTML is generated by a language that intrinsically supports a fast (save&refresh) development cycle. Although these scripting languages are powerfull enough to build your entire application, you risk not separating the data from the presentation or trying to build things that are too complex (PHP-hell). So you will probably have your PHP access some (remote) API that handles the complex things. There is a bit more flexibility in terms of connecting your PHP to the back-end API and because the PHP is run on the server security is less of an issue.

Pros

• HTML can be indexed by Google.
• Short roundtrip when changing PHP
• PHP code runs on a trusted server
• Simple enough for HTML developers

Cons

• Risk of PHP layer to grow too big
• Extra runtime environment needed (Apache)
• HTML only works after PHP parsing

The Java World

This is (for me) the most common model. Some Java Web-framework handles the user request, invokes a controller and passes the result to some HTML-template engine which renders the Java objects into HTML. The real difference here is that you have direct access to Java objects from the HTML template, so when you can model and access your data in Java, you can use these objects directly in your HTML. For Java developers this is very natural. Your web application generates HTML, whilst the developer can work and reason about Java objects almost all the way, the HTML is considered the VIEW part of the framework. One issue with this, is when the HTML layer is doing more than just render the data. Integrating interactive behavior in your HTML is handled by some frameworks quite extensively but at the cost of proprietary abstractions of both JavaScript and the interactions between front and back end. Although this works, it seems to be mostly geared towards Java developers that don’t want to deal with real HTML and JavaScript. Good for Java developers, unnatural for HTML developers.

Pros

• Data access and HTML generation in one machine/JVM, easy and fast
• Short development time for Java-heavy applications

Cons

• Extra channels and websites will need to be modeled in the server framework
• Templates are not HTML, only work when run inside the Java framework
• Longer development round-trip, but needs full Java set-up with war overlay

Wrapping up

So what’s the best? Of course it depends. Did I miss any pros or cons? What do you use to get maximum productivity from both your back-end and front-end developers.

References

• 7 reasons I switched back to PHP after 2 years on Rails

Deming is world famous, especially in the Agile/Lean world. During his life Deming taught top management how to improve design (and thus service), product quality, testing and sales. His name is synonymous to quality improvement and his philosophies are still seen as visionary.

One of  Deming’s principles is: “Cease dependence on inspection to achieve quality. Eliminate the need for massive inspection by building quality into the product in the first place.” Sounds good, right? In stead of testing every last inch of your product, you increase it’s intrinsic quality which is more sustainable, has a direct benefit for the customer and eventually lowers cost.

Now how does that relate to software development? In software development we do quite a bit of inspection, not only by peer reviews but mostly by testing. We test documents, units of code, modules of code, systems, user interfaces, services etc. We test functionality, speed, error handling, ease of use, fit for purpose etc. etc. My interpretation of what Deming would say to us: focus less on creating more tests, focus more on product and process quality.

Let’s start out with the obvious, tests are useful. Even when you’re 100% Deming compliant™ you’ll still have tests, especially automated tests. But I’m not arguing to stop testing, the point I’m making is that tests lure you into a false sense of security and are not a guarantee for improving quality. A symptom of that is in software development QA people (often called testers) seem mainly concerned with ‘checking’ and spend very little time on ‘mistake proofing’.

Mistake proofing

John Hunter says tests are a form of prevention. They are a form of Poke Yoke, Poke Yoke means “mistake-proofing”. His reasoning is: with a test in place, you can’t introduce a bug in the software. But this is not true, the test detects the bug, but doesn’t prevent the bug from entering the code. A better example of Poke Yoke in software development is using high level language constructs, these prevent mistakes that are easily made with low level languages. The difference is that the mistake-proofing is done at the level of human actions where the mistakes occur and prevents the mistake being made. A test doesn’t prevent a mistake being made, it just signals that the product contains (at least) one bug. So even if your test servers as a Poke Yoke, it’s only detecting, not preventing.

Prevent Construction Errors

Tests and other forms of inspection have one fundamental limitation. I’ll try to clarify this with an analogy:

Suppose you’re a gardener who has to keep a big lawn. The worst enemy of the gardener is the mole and its molehills. Now suppose you’ve discovered a molehill. You don’t want those on your lawn. So you can remove it and place a heavy object (a slab of stone) where the hole was dug. You’ve made sure the mole can’t create a molehill at that particular place, but you haven’t prevented the next molehill from appearing.

After talking to your software tester friend, he helps you to find a way to strategically put the slabs in place before the mole arrives, preventing a lot of the molehills from appearing. You’ll soon end up with a garden covered in stone slabs and it keeps your lawn fairly green. However, the obvious question is, why aren’t you trying to get rid of the mole?

In software development, I’ve seen too few team getting rid of the mole. Too few discussions on the real cause of bugs. I would expect a Deming compliant team™ asking: “What preceded writing that particular bit of code that led to the bug? What would make it impossible to create this bug again? How can we increase quality across the board to prevent these kind of bugs, not just this single one?”

Prevention vs. Inspection

Donald Reinertsen says “Inspection makes economic sense if it is more cost-effective than prevention.”  Of course, this is a truism and automated tests are a relatively cheap way to do inspection. When inspection is your major instrument for assuring and improving quality, there’s a risk of test-addiction. Suppose you have lots of tests, but in production the customers finds a bug. What do you do? You create a test for it and that might not be the best way to handle the bug. If the actual cause is low software quality due to overly complex design or sloppy coding, it’s not enough to create a test.The opposite could even be true: the number of tests per quantity of software (LoC) should decrease over time because quality increases to such a level tests are no longer needed in large numbers.

Decrease the necessity of testing

In software development we need to fight the mole more often than patching the molehill. Do a root cause analysis with the team to uncover the cause of the bug and change your process, change they way you do design, introduce patterns, use pair programming, take a training course or do whatever prevents the mistake from occurring again, than will not only prevent that particular bug from re-occuring, it will prevent a whole slew of bugs because overall quality is higher. Your customer will thank you for it.

No it’s not. It’s a product development method (or framework). The good thing is that projects always have a product development part, so Scrum can be embedded in your project. But beware of the  ’Scrum organizational model’ which is geared mostly to product development companies. If your organization chooses to have projects for all new development, you’ll need to embed Scrum in a project management model.

What is a project?

A project has a start, a middle and a finish. If I tell you this project will take approximately one year, an interesting process starts, most people assume there will be a single release at the end. Even if the project will be done with an Agile process, it will still be ‘steering’ to that single release. All work is planned backward, burndowns are drawn to show you’re going to make (or miss) the deadline. Anyway, most of us know what a project is.

What is product development?

The origins of Scrum point to a different model. The term product backlog says it all, it’s not ‘project’ backlog, but product backlog. The default case for Scrum assumes you’re working on releases of a product, which has a virtually infinite lifespan like MS Windows, Atlassian Jira, Patientkeeper etc.

Scrum is a method for making development part of your entire business. Or better yet, if development is an integral part of your business, use Scrum. You rarely start projects because new features (large of small) are just put on the backlog. Developing software is what you do. This differs from organizations that do projects to not only develop software.

Can we do projects with Scrum?

Using Scrum for projects is surely possible, but you can’t use Scrum for projects without some project management practices like tracking budget, planning resources and loads of extra communication. For instance, if you’re using Prince2 and Scrum, you’re mixing two models, the ongoing development model of Scrum and the special initiative (a.k.a. project) model of Prince2.

A pure Scrum implementation should be embedded in a big vision, when you’re working on a product that vision is clear, product managers are hired for that purpose. On the other hand, in a project the variations are much more extreme, the choices of technology, budget, features etc. are much harder to make. In many cases projects are done by people who aren’t familiar with the context and history of the company. So expect a preparation phase where the team at least gets acquainted with the environment.

The thing to watch out for when implementing Scrum is that you don’t try to force the product development model onto your organization. If you use projects, accept that there is a ramp-up at the start of the project. And you might need a different process for different phases of the project. For instance, when your project is a one-off special campaign launching project, you can’t expect the whole organization to react quickly (just-in-time) to your impediments. It’s a matter of economics: any process changes they make for your project will be obsolete by the time your project finishes, which is not always worth the effort.

So what are you saying?

You can use Scrum in many contexts, it fits product development companies like a glove. Organizations that are project-driven can also benefit from Scrum, but not without some extra project practices. Expect a phased process and accept that in a project some impediments are simply not worth solving. Much of the frustration that goes along with Scrum implementations boils down to forcing the product development model (dogma) on your project organization. It’s like forcing a cube in a triangular hole, you’ll get frustrated and it won’t fit, no matter how hard you try. A project management method can be the adapter that makes Scrum fit.

Het indammen van inhuur en drastisch verlagen van tarieven heeft gevolgen. De markt zal zich aanpassen aan deze verandering in de vraag. Een kort overzicht van wat dit op korte termijn kan betekenen, ervan uitgaande dat er geen structurele verandering komt in de vraag:

Wat ik nog niet kan bedenken is, wat voor effecten heeft dit op langere termijn? Wat gebeurt er als de vraag weer aantrekt? Is er dan voldoende interne capaciteit opgebouwd om de nieuwe projecten te bemannen? Of zal er opnieuw een beroep worden gedaan op externen en begint het liedje weer van voor af aan? Graag hoor ik wat jij hiervan denkt.

Open Source

Hippo is open source. This has a number of benefits, such as a lower risk of vendor lock in. Hippo’s commercial success is a result the Dutch government’s preference for open source software and Hippo’s professional, local and Dutch support, a unique combination.

For the developer open source projects have the benefit of community support. Obvious examples such as Apache and a CMS like WordPress show that open source can lead to solid products with tremendous support (books, articles, plugins, forums).

Alas, Hippo still has a fairly small community. Their mailing list is mostly used by it’s own employees. The amount of plugins is small and mostly created as spin off from commercial projects. Perhaps Java is one of the limiting factors because hosting Java is expensive, which prevents individual developers to use Hippo for their own blog or website (a limitation WordPress or Joomla don’t have). Without a large community, customers would still have only Hippo to maintain their code because their knowledge of the system is not free.

User Experience

The user interface is another Achilles’ heel. The competition is also appalling in this respect (Tridion, Sharepoint), but user experience is on the rise because of it’s high competitive value. WordPress, for example  has a large amount of goodwill simply because of the good user interface for editing content.

However, it’s not just the user interface, it’s also the type of information supplied. A professional content editor would like to know how people rate the content, where it’s used, how often is it read, what content is obsolete etc. Hippo does not provide this and thus cannot be used to share knowledge and be the single content repository for the customer. There is great opportunity here to beat systems like Sharepoint.

Writing plug ins

The documentation for contributing to Hippo is quite minimalistic. There is one tutorial, which didn’t work when I tried it because it was based on an old version. There are other weird things about Hippo’s documentation. The site contains quite a few dead links, not very reassuring. An even weirder issue is that the site changes while you’re reading it. The tutorial I read on Friday morning was replaces by a tutorial on Friday afternoon. It’s like the site is in constant motion and you never know when a page might disappear. It seems like testing is not part of Hippo’s publishing process and the CMS itself doesn’t have functionality to check dead links on the site.

Creating add-ons means you’re extending the CMS, which means you cannot drop in your plugin into a running CMS, you need to bundle it with the application, in essence making the add-on an integral part of the CMS. Any change in the internal API or storage system will result in rewriting add-ons. Thus the learning curve for add-ons is steep because you have to  know the entire architecture of Hippo before creating them. I would prefer a plugin architecture that makes it easy to drop in functionality and creates visible results quickly.

If you want community contributions, plugins should be easy to create, loosely coupled to the underlying architecture and easy to exchange and install. Google and Yahoo have created such great APIs that developer will use them just because they’re fun and easy.  And although a good plugin architecture is always hard to master, Eclipse has great tooling and support for writing plugins.

Wrap up

Hippo has a great crew and  it’s organizational structure (people, vibe) resembles a company like TomTom. I’m convinced Hippo can create great products and service. With this Forge Friday it has made a serious attempt to enlarge the developer community. But it needs to step up its effort if it wants a real community and a solid product. Open source without a community is just as useless as closed source products because the customer still depends on the vendor for maintenance. And a CMS without serious tools for managing content and its meta data is not worthy of managing any other content than pages on a website.

My tips for Hippo:

• Create a plugin architecture that makes it fun and easy to add functionality to Hippo.
• Make testing an integral part of the CMS (such as an automatic dead links checker)
• In future versions build the user interface of the CMS from the ground up with the content editor in mind. Make it the slickest tool to inspect, report on and publish company content (hire an information specialist).

After explain the usual method I’ll give you some examples which you can probably identify with. The next post will discuss the small solutions and their gripes.

To clarify, in this post ‘fixing problems’ means fixing them in real life, not just analyzing the problem. The process that leads to big solutions is something like this:

1. Get a clear overview of the situation
2. Simplify the situation
3. Fix the problem if it hasn’t fixed itself already (more on this later)

To illustrate let’s imagine the problem is picking all the green circles from this pile:

Now, the usual process:

1. Get an overview: place all the circles in a line:

2. Simplify the situation: Group the circles by color:

3. Fix the problem: it has almost solved itself as the green circles are on a separate pile. Now you might notice your mind altering between seeing all the different circles and seeing 4 piles. Just  by sorting the circles by color the problem was fixed. Isn’t that neat? And now we can reason about the piles in stead of all the different circles, which makes life easier still.

Abstractions

A pile is an abstraction of the circles on that pile. In most situations it’s far easier to talk abstractions than the actual things you’re dealing with because otherwise the amount of detail would be overwhelming. That’s why we like big solutions, when restructuring the context of the problem, it becomes easier to reason about. For instance if everyone uses the same operating system it’s easier to reason about PCs in your workplace because you can forget about the detail of the operating system.

To make abstractions, there has to be some visible patterns in the situation, otherwise it’s just a big mess (like the single pile of circles). To get to a point where simple abstractions can be made, people go very far in restructuring the situation, often destroying value in the process. One important thing to realize is people don’t favor big solutions, big solutions are a result of our desire to abstract and simplify, it’s a natural tendency with adverse effects (in some cases).

This is the issue with big solutions. It’s not a bad strategy to deal with complicated problems, but big solutions don’t solve the problem very quickly, they start paying of usually months or even years later.

When simplifying the context of the problem, you could for instance introduce a new method: throw out all existing social structures and procedures and replace it with a full implementation of a method that is ‘consistent and correct’ throughout all layers of the organization. The big problem here is destroying the existing structures that worked very well or were even specially tailored to the domain. The biggest issue with restructuring is the new structure will have problems of its own. So by fixing one problem you could be introducing several others that come along with your new method.

I’ll give you some other examples of big solutions from my recent experience:

• IT department is slow and expensive, we’ll replace the entire IT infrastructure with an off-the-shelf product.
• Our process is too slow and complicated in places, we’ll redesign and document all business processes and put them all in a BPM tool.
• The use of one technology doesn’t fit certain problems in our application, we’ll refactor all modules in all applications to use a new technology.
• We’ve got a big pile of legacy systems, let’s define a new architecture (SOA) and demand all projects to conform to that architecture from now on.

I also tend to come up with big solutions. These big solutions, when you first image them, always have sense of elegance, simplicity and control. When struggling with the current system architecture I come up with things like: “If we could only replace the entire back end with technology X” or “If we could only get 6 months to make tests for all the legacy systems”.

But while you’re restructuring, training people in new methods and drawing up conceptual charts you’re not actually fixing anything. You’re laying ground work to fix the problem in one big sweep. It might actually work too, the big solutions are not wrong per se, they just distract you from the problem and a smaller solution would have had a far higher return on investment.

Do you explicitly think when to use a big or small solution? How effective are big solutions? In my next post I’ll talk about ‘small’ solutions and how they are a great alternative when you find yourself drawn to the big solution.

One major topic of any process optimization is the management of bottlenecks. One intriguing statement is made, it urges to go beyond the popular idea that ‘the capacity of the bottleneck controls system flow’. It’s not only the bottleneck, it also very important to look at the flow through the bottleneck.

The example given in the book discribes a traffic situation where three lanes are reduced to two. Combining two lanes into one causes turbulence and you don’t want this turbulence at the bottleneck, which is why the narrowing process starts miles before the actual bottleneck.

Although this is better to leave the narrowing totally unmanaged, in practice it often doesn’t work like this. On a recent car trip I found there is a better way of controlling flow which works much better than the regular solution, I’ll explain.

First the basic problem found very often on the Dutch highway, three lanes need to be narrowed to two:

The regular solution is to urge the car driver to start ‘turbulenting’ a few hundred meters before the third road ends. This usually doesn’t happen, what does happen is people cram their car into the bottleneck, usually with high speed. Their motivation is to slip through (escape) the bottleneck as quickly as possible. This decreases flow sometimes to a standstill.

An alternative solution, I recently experienced, is much more focussed on flow. This solution takes a two step approach, it splits the three lanes into two before combining them. This way the driver in the middle lane has two options when merging with another lane, to the left or right lane. This spreads the turbulence more evenly over the two other lanes. It also makes it undesirable to drive up to the road block as fast as possible because you haven’t ‘escaped through the bottleneck’ yet because the real bottleneck is still a few hundred meters ahead.

I’m not sure if the road workers made this arrangement on purpose to increase flow, but it’s a brilliant way of dealing with a bottleneck and minimizing it’s impact.

The relevance to organizations is to keep in mind that widening a bottleneck might not have the highest value, optimizing flow through the bottleneck can also reap huge benefits.

Suppose you have too few testers in your team. One thing you can do is hire more testers. But instead you can make sure the flow of work through the testers is optimal. In stead of the tester testing a feature as a whole after (or even before) it’s developed, involve the tester a few short times during development. This way he/she can detect testing issues before the full test and spread the ‘testing turbulence’ over a longer period of time, reducing the impact of tester-shortage. No extra testers, but increased flow. This is how I prefer to tackle bottleneck issues in my team, when increase in capacity is not at hand.