This is part II of a series on agile architecture. You can read part I here.

In the previous installment I provided a definition for software architecture and raised the apparent friction between the up front design implied by software architecture and the YAGNI approach and deferred requirements prompted by agile development in  the large. This installment take a look at an additional angle of the problem which is the difference between design and architecture (while architecture is a type of design I cam calling design the code or close abstractions of it   that don’t yet fall under “architecture” as defined in the previous post). The difference between the two, as the title suggests, is that while design can be emergent, Architecture, unfortunately needs to evolve.

Unless you’ve been living under a rock, you’ve probably already know about Test Driven Development (TDD) or its cousin BDD. If you’ve actually used TDD, you’ve probably noticed the impact it has on the actual code. Writing code only to cater for defined tests coupled with refactoring that keeps the design tight we get to a result that, more often than not, simpler than going through the more traditional design first (you can take a look at Uncle Bob Bowling Kata for a simple, albeit synthetic , sample)

In fact, a more accurate expansion of the TDD acronym is Test Driven Design. Yes the tests are there to make sure the code adheres to the specified requirements. However, the iterative process (test,code & esp. refactor) makes the design emerge . The emergent design effect, works very well with the goals and tenets of Agile and Lean methodologies as it helps eliminate wasteful future-proofing  code that we don’t need; eliminate waste of extra component (due to pre-design) etc.

This sounds great, so it is very tempting to take that  to the architecture level, after all Architecture is a type of design, wouldn’t it be efficient to do TDA (Test Driven Architecture) as well. I think that in theory it might be possible. However as the old adage goes “Theory and practice are the same – at least in theory”. So in practice, the problem is that architecture is global and have solution-wide consequences. Time and size (of work) constraints make us want to set the playfield for the solution as soon as possible. Architecture decisions are hard to postpone on the one hand and have extensive influence on the other. E.g. a buy vs. build decision; 3-tier vs. SOA; RDBMS vs a NoSQL solution etc. if we try to have the architecture grew the way “regular” design can the ripple effects will be devastating to the development process.

What we have to do instead is evolve the architecture. Start with something that is in the ball park and then when we know more about the problem make changes, evolve the architecture over time. This is beneficial since requirements also tend to change over time so if we find a way to evolve the architecture we can deal with that as well.

In the next part I’ll try to give a couple of examples on how this can actually be done i.e. how you can evolve an architecture

 

 

 

 

*A better name is Test Driven Design but that’s not how it stated, as far as I know anyway

 

We are going to use some of our test code in production. Yes you read it right test code in production. Here are the details
In our system, among other things, we support visual search in video calls. i.e. an end user calls the system, points the camera at something she is interested, and (hopefully :) ) gets relevant information. Basically the system is made of several resources (image extraction, identification etc.) that collaborate via an event broker. We have a blogjecting watchdog that makes sure everything is up and running and we have applicative recovery service to handle failures.
The watchdog makes sure resources/services are up, resources report their liveliness and wellness so we know more about the resources than the fact that they are up. However, we still need a way to make sure that resource instances  can collaborate to provide the service.

Enter our automated acceptance tests. Part of our development effort included building a test runner for automated tests scenarios, e.g. load tests, verifying algorithms correctness etc. One of these tests is the smoke test (run after each successful build) which includes a sunny-day scenario of a video call- as described above. What we're going to do now is build on the test runner and the sunny day scenario a "keep-alive" tester that will periodically make test calls to the system (depending on the current load etc.) and make sure that everything is still working correctly.


So there you have it, an unexpected benefit of automated acceptance tests, who would have thunk it :)

 

The year is almost done so I'd thought it would be a good time for a short retrospective into what I blogged here. The 13  posts below are the ones  I liked best this year. Turns out these posts touch on a lot of different subjects: requirement, software management, agile development, architecture, SOA and programming.

• Use Cases vs. User Stories
• The Layered Architecture Style
• Sticky notes vs. the computer
• Sleep tight, don't let the test bugs bite
• Stateless services - The state is out there
• Code readability - Documentation vs. Refactoring
• SOA security reminder
• Why the Database as a Service is a bad idea
• Think Hollistically
• Architecture - It is always a tradeoff
• Architect Soft Skills
• Broken build white noise
• SOA Anti-Pattern : The Knot

 

Even if you don't write your tests first on a regular basis, you probably want to adopt that at least for debugging.
All the developers I know want to repro a bug before they make any changes to a the code (I know I do). When you write your test before you debug you do just that and you get several other benefits since you can

1. Make sure that the bug is indeed reproducable
   
2. Easily (and repeatedly) step through your code to pinpoint the bug
3. Verify that the bug is fixed
4. Get a regression test to make sure the bug isn't reintroduced
   

The only question now is what is the difference between a bug and a new feature?

One of my favorite things about UserVoice -- which we use for Stack Overflow -- is the way it intentionally blurs the line between bugs and feature requests. Users never understand the difference anyway, and what's worse, developers tend to use that division as a wedge against users. Nudge things you don't want to do into that "feature request" bucket, and proceed to ignore them forever. Argue strongly and loudly enough that something reported as a "bug" clearly isn't, and you may not have to to do any work to fix it. Stop dividing the world into Bugs and Feature Requests, and both of these project pathologies go away. (coding horror)

But that's something for another post :)

Powered by ScribeFire.

 

Back in may Andrew Binstock wondered "is the popularity of unit tests waning?" Andrew lists 5 indications he observed to support his conclusion:

1. Commercial tools are waning
2. Few OSS products
3. Unit testing is not taught by major Java instructors
4. Few books get published on unit testing
5. xUnit alternatives are completely invisible.

I am not sure I agree with all these observations, nevertheless,in a followup article in Java World Andrew continued to ask "Is unit testing doomed?".
Andrew says that two issues with unit testing are that they seem to offer little value and that the evangelism of unit testing in general and TDD in particular also causes antagonism (I can say from personal experience with introducing unit testing to long time C and C++ developers that, indeed, the initial responses are rather cold)

Andrew also quotes former CEO of Agitar (Jerry Rudisin) saying that "unit testing hasn't taken off as mainstream practice". A survey ran by Scott Ambler for Dr. Dobb's also found that "The more difficult practices, such as TDD and executable specifications, have lower adoption rates than easier practices such as daily scrum meetings." (although another survey by VisionOne found that within companies where agile practices are used 77% use unit testing and about 49% use TDD)

I recently read another interesting post on the subject, this time by Dan North about the "end of endotesting" explaining why the record-reply idium of most mocking framework is not intuitive and more easy to use mocking frameworks like mockito (his example) or Moq and Mocha (my examples) are better.

Now, yesterday I read Roy Osherov's post "Goodbye mocks, Farewall stubs". Roy makes similar observation on the adoption (or lack therof ) of unit testing practices. Roy believes the number one reason holding back unit testing is the learning curve associated with it and suggests we simplify the tools and terminology (and gives a example similar to Dan's mentioned above)

What we see is that both in the Java and the .NET worlds unit tests adoption hit a few snags. The way I see it there are event  additional barriers to wide adoption of unit test and TDD but I'll talk about that in part II where I'll also try to talk a little about what can/should  be done

Powered by ScribeFire.

 

Awhile ago I asked "who tests the test":

" One question I don't hear asked too much is "who tests the tests?" - after all we are writing all this additional code - if we write so many bugs in our production code that we need tests - what are the chances the test code is clean?"

I would add that this is especially true considering that the same person that writes the code also writes its unit tests

One answer I had was making sure to keep the triad of tests->code->acceptance tests. so that each two test the third. This is especially good since, usually, the acceptance/integration tests are not written by the developer who wrote the code.

One answer I got was that tests are usually simple enough to be self evident and when tests contain complex constructs you should probably test them as well.

Basically, tests can have two types of problems

• Not testing the right thing
  
• Not testing the thing right

Code+unit tests +acceptance seem to take care of the first type of problems. The claim that tests are simple should take care of the second type of problems.
However I think that it isn't always true, especially when it comes to using the tests for regression. consider for example, the following two incidents I had  where the the problems were actually with the tests:

1. We have a component that interacts with resources in different URIs, so when resources wake up they register themselves with that component. Resources register two URIs -> one for control and one for interactions. We've added a business rule that both URIs should be on the same host (naturally a new test for that was also added but that not the point) suddenly a bunch of the old tests started failing. Sure enough, I took a look at the new code, found a bug, fixed it, but the tests were still in the  red. In the end it turned out one of the Uris that is registered during the tests had an extra "L" (locallhost instead of localhost) so the tests failed with the new rule - as they should..

2. After writing a piece of code to "just work" it had to be made multi-threaded. I needed to write a test that would make sure the component can handle multiple concurrent requests. Sure, I thought, I'll just run few or even all of the other tests in parallel and be done with it. The only problem was that the other tests were written to be isolated (i.e. new instance per test) running them in parallel had no effect what-so-ever.

What can we do to help us locate problems and bugs with tests?

• remeber that Test code is code - so as you write more tests you should also refine the tests design and refactor the code accordingly. For instance in the first example above, it was easier to diagnose the problem since the faulty setup was done by a single method used by alll the failing  tests (Single Responsibility Principle @ the method level).
• Test-First - One of the reasons test-first is beneficial is because you can actually see the test fail before you implement the code to make it work. This helps you make sure that the test actually tests the behavior you want. When the code is already in place it is harder to make sure if the test works because of the change or not. In the second example running all the tests in parallel everything was still green, which triggered me to write specific tests for checking for multi-threading issues.
  

What do you do?

 

One question I don't hear asked too much is "who tests the tests?" - after all we are writing all this additional code - if we write so many bugs in our production code that we need tests - what are the chances the test code is clean?

The current answer I have is that the code, the tests and the acceptance tests all test each other so if one fails we'll spot the problem in at least one of the others. I hope that this  it is a good enough answer... :)

What do you think?

 

Having just read Doron's Yaacoby's post on some of the insights he got  from reading Tom & Lister's Peopleware, I'd thought I'd repost a couple of posts I wrote about a year ago in my Dr. Dobb's Journal blog (with a few edits):

I just finished reading Software Conflict 2.0: The Art and Science of Software Engineering, by Robert L. Glass.

This is a reprint (with few new retrospective additions) of his 1990 book. While the technology mentioned in the book is outdated, many of the author's ideas and views are still valid. The book is a collection of short articles on various subjects, and one of the more interesting articles is about the cognitive side of design.

Glass explains that research done showed that design includes the (obvious) steps of understanding the problem, decomposing it into goals and objects etc. The essence of design, however, are the mental steps taken by designers:

1. They construct a mental model of proposed solution
2. They mentally execute the model (i.e. simulate the model to see if it solves the problem)
3. If the model isn't good enough (e.g., too simple) replay the simulation to find what wrong and remodel
4. Repeat 1-3 until the model seems to solve the problem

Glass also said that people tend to start with a model that worked on a similar problem and that good designers perform this process extremely fast. As I was thinking how one can train himself to get better at performing this task, it occurred to me that I heard something similar  somewhere... While not the exact process this is very reminiscent of TDD - only TDD makes the process explicit. In a way we can say that TDD will not only help make your design better it would also  trains you to design better altogether.

Robert's book has several other views still relevant today. While it can seem odd that a 16-years old book contains relevant thoughts looking at my bookshelf I see there are quite a few other books (some of which are event older) which contain essential information and brilliant ideas that are still very relevant today. Here are just a few examples:

• Peopleware Second Edition by Tom Demarco and Timothy Lister (Second edition 1999, original 1987). Development should be centered on people and not on methods

• The Mythical Man-Month: 20th Anniversary Edition by Fred Brooks (20th anniversary edition 1995, original 1974). "No silver bullet" don't just add people to a late project it will only slow it down and many more.

• Code Complete by Steve McConnell (Second Edition 2004, original 1993). A great book on coding.
  
  
• An Introduction to General Systems Thinking (Silver Anniversary Edition) by Gerald M. Weinberg (silver Anniversary Edition 2001, original 1975). Observations on thinking processes
  
  
• About Face and About Face 2.0 (3.0 edition 2007, 2.0 Edition 2003, original 1995). UI mistakes and guidance, the second edition also details a scenario based approach to user experience design.
  

Getting back to TDD or at least the idea of test first. It seems (via Rob Keefer pomiet blog)  Gerald Weinberg talked about it more than 35 years ago in  Psychology of Computer Programming :

"By pursuing this test example to the point where he understands the problem, he will not only learn the one thing he did not know, but perhaps will learn others as well, for test programs such as this are often better learning instruments than are production programs.

As a matter of good practice, the test program should be constructed before the 'fix' is made to the production program. In the first place, there will be an all too human tendency to forget about the problem once the production program is working correctly, so we must impose a little discipline on ourselves. Possibly more important, however, is the chance that by the mere act of constructing the test case we shall discover the problem."

Anyway, with so much good advice lying around for years (11-40+) and the fact that only about 30% of the projects are successful (on-time;on budget; on scope) I think one question we should all ask ourselves is -- don't we ever learn?