This blog has now moved to arnon.me.

Everything already here, will stay alive (that's what permalinks are all about) but new posts will appear on the new site (unless I'll be tempted to post them here:) ). Also some of the internal links will be updated.

if you are subscribed to the feedburner feed - you probably don't even see this message, otherwise please update your reader to point to the new feed
 

You’ve probably read/heard that yesterday Google held a press event on making the ChromeOS the’ve been working on an open source project. Actually they’ve announced ChromeOS back in July, but now that the source is available it is making more waves. This is a very logical move for Google, and even though I think that they will need to enable some local/offline capabilities before it would be a viable option.

As usual* for Google, they are not the first in this space (If you want to get a webbook today you can try out litl, which looks like a very nice,  or the GOS cloud on the Gigabyte M912)  - but Google’s size and position in the market makes this interesting for everybody.

Anyway, I am almost thinking about a career change to analyst as I’ve been says that’s what they’ve up to since they released Chrome in Sept. 2008. when I published Google Chrome -The browser is the new Desktop. Here’s an excerpt:

Now,in my opinion, Google makes a bold move to change the rules and re-define the playground - if webapss need to run on the desktop, let's make the browser the new desktop.
What makes me say that? because it is focused on application (see the comics),because the browser runs each tab in its own process, because it has a process monitor, because it is a link on the google home page...

From the chrome "OS" point of view we can look at javascript,HTML etc. as the IL (bytecode in java speak) on which the application run. This makes cross-compilers like GWT and the good side of MS Volta (vs. the bad side) the next abstraction layer. I expect these will be more significant in the future

On the same note, it is probably good time to provide a link to a presentation on the future of PCs I’ve created in 2007, which highlights some of the trends that are more apparent today in moving to the cloud

---

* the most obvious example is of course search (altavista, hotbot etc. have all been there before..), a more recent example is the free GPS turn-by-turn where Waze (an Israeli company by they way) is already there but Google’s announcement is considered disruptive

 

I didn’t blog for the past month – well, I didn’t really had time to breath either :). However as of last week we (xsights) moved our system from demos and testing into production with our first client – Maariv, israel’s second largest newspaper.

In case you don’t know what xsights does – we hyperlink the real world using the mobile as your mouse (sort of like barcodes e.g. microsoft tag – but without the barcode;) )

Anyway, our first product (actually service) is for printed newspapers/magazines (see video above) and we are now in the dry-run phase were we make sure all the interfaces (specifying content, links etc.) work ( With the official launch planned in a few weeks) – but already each newspaper coming out has a few links which are marked by a small “mem” : for instance, Today’s (independence day) paper has 6 different links (in the Sports, signon and Asakim). Currently only us and Maariv’s employees use the system but we thought it would be nice to start getting some more feedback from more objective parties so if you are interested in trying new technologies before others, live in Israel, read Maariv and have a video call  capable 3G phone drop me an email (arnon at xsights dot com). Note that there is no software to install, we use the video call capabilities of the phone (we are working on a client for phones that don’t support video calls like the iPhone etc. but that’s another story)

Other related good news  (I hope that’s how you’d see that anyway) is that I feel I can go back to my regular blogging schedule. Some of the posts in queue include nano-services anti-pattern, aggregated reporting pattern, blogjecting watchdog pattern and two interesting questions I received which are related to it, post on twitter integration for reporting  etc.

 

I decided to do another article collection post. As I mentioned in the last post that I am extremely busy these days  as we're preparing to launch the first public pilot of our service. On top of that I cannot disregard the ROI (views vs. hours invested) in previous collection post I made on "10 papers architect should read". It isn't that I don't have other articles and papers with 40K+ reads but compiling a list takes much less time than, say, writing an article on what is SOA, explaining the fallacies of distributed computing, a mini-book on Use cases - not to mention the effort that goes into writing  SOA patterns/anti patterns book (Saga, Edge Component, The Knot etc.).
But enough about me...

• The mythical man-month (1975) Fred Brooks - it is the second chapter of a book with the same name. A classic paper where, I think, for the first time, the ideas that some tasks need to be done sequentially and can not be solved by adding more man-power or in Fred’s words “The bearing of a child takes nine months, no matter how many women are assigned”. It is also famous for stating that adding more people to a late project will only delay it further.
• Why the Vasa Sank: 10 problems and some antidotes for software projects (2003) by Richard E. Fairley and Mary J. Wilshire -it isn't free but worth it (though you can read the gist of it for free in "Why the Vasa Sank: 10 Lessons learned" by Richard E. Fairley) – The tale of the Vasa ship is a very nice analogy for developing software project. I first heard it from Ivar Jacobson several years ago. This article presents it very well.
• The Cathedral and the Bazaar (1997) Eric S. Raymond - the classic essay on open source vs. closed source models for developing software
• Software Maintenance is a Solution, Not a Problem - Robert L. Glass – it really is when you think about it.
• IT doesn't Matter (2003) - Nicolas Carr – A thought provoking article on the place and importance  of IT – Carr basically states that software is getting to a point where it will become a utility like electricity and in that will loose it strategic place. In a sense we’re seeing more of this direction today with all the hype and move to cloud computing.
• Chapter 3 of the "Report of the Defense Science Board Task Form on Defense Software" (2000) – various – An extensive analysis of software intensive projects. Besides revealing interesting facts on the ever growing importance of software (eg. 80% of the functions of F-22 are performed in software vs. 8% in the 1960’s F-4) it also acknowledge the advantages of iterative lifecycle over waterfall even in large, safety critical,complex  projects. 
• Observations on Balancing Discipline and Agility (2003) Barry Boehm and Richard Turner – provide a few rules of thumb for assessing a project’s suitability for agile methods (e.g. team’s level, criticality etc.)
• Succeeding with "Agile Fixed Price" part I & part II Pascal Van Cauwenberghe  - a two part paper with practical advice to employing/injecting agile methods into fixed price contracts.

 

Well, actually I joined sometime ago - but I now actually started using it  - @arnonrgo

 

I don't usually blog about promotions and vendor programs - but this is something we joined and if you are a startup using/considering Microsoft's technologies you ought to hear about.

The program is called Microsft Bizspark and essentially it gives you free licenses for all the development team for the porpuse of designing, developing, testing and demoing the application - that's a huge save in development costs (we're using tools like Mingle and subversion but the program even includes a license for team foundation server if you fancy that)

Furthermore, if you, like us, develop a software as a service (SaaS) solution you are also eligible for free production license

To be eligible you have to

1. be a startup that develops software
2. privately held
3. be in the business for less than 3 years
4. and have less than 1 million USD revenues

check out the program guide for more details

Powered by ScribeFire.

 

I didn't intend to comment on this post from Jeff Atwood where he claims we (developers) are typists first and programmers later but since I qouted his blog on my previous post I reconsidered.

I'd have to say that this isn't one of Jeff's most intelligent posts.
 How many lines of code do you think developers write per day? let's google us some numbers:

Here is one interesting qoute

"Let's see if, quantitatively, there's any truth to the perception that the code velocity (net lines shipped per developer-year) of Windows has slowed, or is slow relative to the industry.  Vista is said to have over 50 million lines of code, whereas XP was said to have around 40 million.  There are about two thousand software developers in Windows today.  Assuming there are 5 years between when XP shipped and when Vista ships, those quick on the draw with calculators will discover that, on average, the typical Windows developer has produced one thousand new lines of shipped code per year during Vista.  Only a thousand lines a year.  (Yes, developers don't just write new code, they also fix old code.  Yes, some of those Windows developers were partly busy shipping 64-bit XP.  Yes, many of them also worked on hotfixes.  Work with me here.)

 

Lest those of you who wrote 5,000 lines of code last weekend pass a kidney stone at the thought of Windows developers writing only a thousand lines of code a year, realize that the average software developer in the US only produces around (brace yourself) 6200 lines a year.  So Windows is in bad shape -- but only by a constant, not by an order of magnitude.  And if it makes you feel any better, realize that the average US developer has fallen in KLOC productivity since 1999, when they produced about 9000 lines a year.  So Windows isn't alone in this.  [KLOC data comes from “Worldwide IT Trends & Benchmark Report 2001”, produced by META Group (now acquired by Gartner)]"

Here is another one from Jeff Atwood himself

But what I really want to focus on here is how you measure a project's size. What's big? What's small? McConnell is using lines of code (LOC) as his go-to measurement. Here's a table that illustrates the relationship between project size and productivity:

Project Size	Lines of code (per year)	COCOMO average
10,000 LOC	2,000 - 25,000	3,200
100,000 LOC	1,000 - 20,000	2,600
1,000,000 LOC	700 - 10,000	2,000
10,000,000 LOC	300 - 5,000	1,600

some of the best numbers I've found:

"

Alien Problems:

problems about which I know nothing, those have seen them for first time:

12 lines of usefull code per hour :((, includes thinking on it 

Some what, I know problems:

problems, that I know about but have not solved before.

33 lines of usefull code per hour, including thinking

Ah, I have done that before:

63.5 lines of useful code per hour"

These numbers shows us the most of the time spent on projects is doing other stuff- as in not typing programs.

but let's say you code 1000 lines of code per day. if you code for 4 hours in that day you would write 4-5 lines per minute - how many words per minute do you think that amounts to? unless you are writing COBOL I would say 3-5 words per line. what sort of typing speed do you need for that?

Note that " lines of code" is not a very good measurement of productivity. e.g. refactoring a design may generate lines of code for you (e.g. extact method) or you may remove lines of code (applying principles like DRY) or you can generate code from templates /wizards)
As John D. Cook says:

"I heard of a study recently that concluded inexperienced and experienced programmers write about the same number of lines of code per day. The difference is that experienced programmers keepmore of those lines of code, making steady progress toward a goal. Less experienced programmers write large chunks of code only to rip them out and rewrite the same chunk many times until the code appears to work. "

In any event,in my experience, you just get better in typing just because you spend all that time at the keyboard - it is defenetly not a goal in itself, just a nice side-effect.

Powered by ScribeFire.

 

Since the last post turned out kind of lengthy (3000+ words) I thought it would be more comfortable to read as a PDF whcih you can  download directly here or  from the Papers, Presentations and Articles section of my site
Also it is probably a good opportunity the "Architect soft skills" is the fourth and last part of the series of posts:
 

• A few Pithy Thoughts on Architecture
  
• Think Holistically
  
• Architecture - it is always a tradeoff
• Architect Soft Skills
  

Powered by ScribeFire.

 

Introduction

There's a lot of discussion about the hard skills software architects needs to have; for example, see one example at the Software Engineering Institute (SEI). Architects need to be familiar with a wide range of technologies, methodologies, understand the software lifecycle, have design experience, and some say an architect must write code, and so on and so forth. Indeed, the hard skills are important, very important. However it doesn't stop there. There are also several soft skills that you need to master if you are to be a good architect.

I believe that the minimal skill-set for an architect should include capabilities from the following areas:

• Leadership. Influencing others to accomplish tasks and following your guidance
• System thinking. Understand decisions and constrains in the wide scope pertaining to whole of the solution at hand. This includes the ability to abstract problems.
• Strategic thinking. Understanding decisions and constrains and their alignments to the overall business of the company.
• Organizational politics. Understand the environment you operate in and how it influences you.
• Communications. Making sure you get your point across.
• Human relations. Understand the "people" aspects or human factors and dynamics. This includes things like pragmatism, understanding team dynamics and personal dynamics

Let’s take a look at them one by one

Leadership

Solutions architects are the "technical managers" of projects. This means they are responsible that all the designs and code are aligned to the functional requirements and that the quality attributes are kept.

However, architects are seldom the direct managers of a development team - and even if the architect is the manager, you still need to inspire your workers. Tyranny? Well, that just doesn't work. You might think that establishing yourself as a technical authority will be enough (that's why they made you the architect in the first place, right?) but it isn't. You need to cultivate your leadership skills as well.

What is leadership anyway? Leadership is about exerting influence on people and increasing the chance that people will follow your vision and decisions. To do that you need to gain the respect of the teams you work with, communicate your vision and designs clearly and are trustworthy.

So how do you do that? Unfortunately, I don't have a definitive answer to that but here are a few things to think about which I think are useful:

• Provide direction. To lead you need to know where you are going and make the decisions that will get you there.
• Explain your decisions. Deus ex machina doesn't count. You work with intelligent people, they may not have your experience or the same depth of knowledge but they want to know why they are doing something.
• Listen to what others have to say. They may actually say something valuable you know :)
• Don't postpone decisions and don't avoid conflict. This will not make them go away. Do try to manage your conflict though.
• Motivate people. This can be done by things like mentoring and teaching, allowing people design freedom. (Letting others make the decisions in their relative fields/areas even if the solution they propose is not perfect.)
• Set an example. E.g., you can sit (pair with) other developers in the team to design/code important things together

Leadership is one of the most important soft skills. As I mentioned earlier, architects are usually not the managers but they do need to lead if they want to ensure the stakeholders’ needs (the system quality attributes) will indeed make it into the solution. To better grasp the quality attributes you’d need “System Thinking”

System Thinking

If you managed to make yourself a leader, it only increases your responsibility to actually know where to go. Two soft skills that can help you with that are System Thinking and Strategic thinking (which is described in the next section)

Microsoft Encarta defines system as "any collection of component elements that work together to perform a task." (You may want to take a look at some of the characteristics of systems by Donald E. Gray). When we get a problem, that is, a software solution that needs to be built, we tend to think about breaking it down into manageable "parts" (the subsystems/components/services/objects) that makes that solution.  This, however, will only take us so far if we don’t also employ "Systems Thinking".

System Thinking originated as a way to think about social systems but has emerged as a way for problem solving problems for systems in other practices. In essence it is about understanding that system parts that work together behave differently then each part alone ("The whole is greater than the sum of its parts"). Which is, by the way, the reason "loose coupling" is such a holy grail--as it helps reduce the parts interdependence and interactions, and thus simplify the system.

One important trait for understanding systems behavior and component interaction is the ability to create abstractions and models; that is, simplifications of the reality which contain enough detail to be useful (Another thing that is needed is the ability to communicate that to the different stakeholders which is another soft skill I'll expand upon). It is important to remember that mental models limit the perspective we have which is why we need to have several models and why it is beneficial to have more than one person working on a problem

As it happens this is also aligned with my definition of software architecture:

Software architecture is the collection of the fundamental decisions about a software product/solution designed to meet the project's quality attribute requirements. The architecture includes the main components, their main attributes, and their collaboration (i.e. interactions and behavior) to meet the quality attributes. Architecture can and usually should be expressed in several levels of abstraction (depending on the project's size).

This definition gives interactions and the environmental implications on the system as a whole the same weight as designing the parts themselves. If the architect doesn't (or can't) understand the effects of the components playing together the quality attributes (performance, availability etc.) of the system will suffer and the system will not operate as planned.

For an introduction to the subject, I recommend Gerald M. Weinberg's Introduction to General Systems Thinking. It lives up to its name.

The second part of understanding “where to go” is strategic thinking.

Strategic Thinking

While system thinking takes care of the system in its environment, strategic thinking is about understanding where the organization is heading and its long term goals so that the solution being developed is in sync with them. While I guess it is easy to see why this is important for internal projects, I think it is also important for product/solution companies.

Strategic thinking involves the techniques and thinking processes essential to setting and achieving the business's long term priorities and goals. Strategic thinking (understanding the problems) is the preamble to strategic planning -- but we'll leave planning to management and focus on the understanding which is important for the architects.

Ruth Malan and Dana Bredemeyer define the Strategic Thinking soft skill (which they call "Strategic Perspective"):

“[An architect who has strategic perspective (ARGO)] understands the industry, market, customers, competitors, suppliers, partners and capabilities of the business. Identifies opportunities and threats, and actively identifies trends and future scenarios. “

This is a good definition because it really explains why this skill is so important for architects. One of the architect's core roles is to understand what the different stakeholders’ need, then balance these needs to create a usable, robust solution. Their needs expressed as quality attributes are the driving force of the architecture.

Furthermore, if you think about all this "align business and IT" stuff we constantly hear about these days (especially in regard to SOA), it is evident that all the careful planning of how technology and software can help in getting that alignment is useless unless we really have a good understanding of where the business is going and what this alignment really is. Thus, the architect should really “get it”. The architect should first understand what the business is about and where it is going. Then, armed with this understandings and insights, she can translate them to technological and architectural decisions that ensure these needs are met.

In an ideal world, gaining these insights would be enough. However, the architects do not operate in a vacuum. Architects should also understand the organizational forces that can make the solution work (or break)

Organizational Politics

If strategic thinking helps you understand where the organization is going, Understanding Organizational politics helps you understand how e organization is working.

Consider the following anecdote; A while ago I co-architected a Naval Command and control system. One of the key elements of that system was a service bus component which we wanted to base on a commercial messaging middleware. We thoroughly explained why choosing messaging was the best choice for the to the project’s management. Nevertheless, another solution based on a proprietary (and fundamentally flawed) distributed objects middleware was constantly suggested and eventually made a constraint we must follow. It took us several iterations (and a lot of rework later) to prove that a messaging middle-ware was indeed the (much) better solution for that project). What happened here? Two experienced architects gave ton of good reasons justifying a technical decision, but somehow that decision was overruled, why?

Decision making, especially technical decision making, seems like such a logical process. You just look at the alternatives; analyze the merits vs. the problem at hand, and may the best option win. This works out well if you are the king (or work alone which makes you the king by default) -- otherwise there are other people and they won't necessarily agree with you. One reason for that may be they really have another solid opinion, in which case you need to negotiate with them , but that has to do with the leadership skill (we already mentioned) or communications skill (discussed later). The other reason for people to disagree is that they may have other interests and agendas, which run much deeper than the positions they externalize (i.e. their disagreement with you).

Organizations (and the larger they are, the more complex they get) tend to get to decisions by employing a system of rules which encompasses a lot of interests on top of the rational reasons to agree or disagree. Understanding Organizational Politics is about understanding these non-rational influences on the decision making processes.

To return to the anecdote above, it didn't take us too long to understand the real motivation there. It turns out both the project leader boss and a few others recommended buying that flawed component which also happened to cost a small fortune. As that component was already bought, it had to justify itself by being used everywhere. In this particular case the only way we found to reverse the decision was to prove that it was flawed and to minimize its infiltration into the project (so it will be relatively easy to remove it later).  In other cases there might be more cost effective ways to do achieve the desired result.

The first thing to do is understand where you are. This is one of the reasons the first step in the SPAMMED framework is to understand the stakeholders. If you manage to uncover the agendas and interests of the different stakeholders as well as the influence they may have on your project, it will at least help you pin-point your problems.

 The tricky part in knowing to how to navigate and influence the organization. Tools you can use are interpersonal skills, networking capabilities, schmoozing, and you also need excellent communication skills.

The point I am trying to make here is that even though technical people tend to regard organizational politics as dirty, you cannot afford to dismiss them. Organizational politics can have a severe influence on your project and actions. I didn't talk a lot about how to become more judicious political animal, I am probably not qualified enough to do that (you may want to check some of the resources below though)

More resources:

• David Ing mentions this in his "overly long guide to being a Software Architect"
• Ruth Malan and Dana Bredemeyer talked about it as well (see for example their paper on the architect's role or their competency card on organizational politics).
• David Dikel, David Kane and James Wilson cover some related organizational patterns in their "Software Architecture: Organizational Principles and Patterns"
• Mark Maier and Eberhardt Rechtin have a chapter on politics in their "Art of System Architecting, Second Edition"

As, I mentioned, one of the essentials to actually getting your points across is good communications skills.

Communications Skills

“What we’ve got here, is failure to communicate, some man you just can’t reach…”  - this can work for a warden in a movie or an opening to a song[1], but it is definitely not an excuse for an architect. The architect is a hub of communication between management, users, developers and what not - A failure to communicate in the case of an architect can mean a conceptual bug down the line if talking to a developer, increased costs and animosity if talking to a project manager or even cancellation of the project if talking to upper management.  

As a senior technical person you already know how to solve problem and translate your ideas into working code – However as an Architect working with teams you have to solve a few more soft problems. One barrier to cross is conveying your ideas to others – or presentations skills.

Presentations skills start with the way you create your slides (e.g. bullet points vs. telling a story) and go well beyond that into how you present, your stance, your interaction with the audience etc. Note that presentation doesn’t have to be a keynote address in OOPSLA it can be a by-the-white-board standup with a college. There are different focuses for each type of presentation but the principles are the same.

So assuming we covered presentation skills is that enough? – No, since explaining yourself will only set you off on a journey, you also need to engage in a dialog with the “others” so that you’d reach an agreement (That you are right, negotiate some middle ground or understand your errors). Thus the next component of communication skills is negotiation skills. Negotiation skills will let you defuse situation that would otherwise deteriorate into a bunch or raving lunatics shouting at each other and instead have a collaborative common problem solving experience. That may sound like BS but if you want to move things in positive directions you need to be prepared. If anything you should at least know when to stop (a.k.a. BATNA – Best Alternative to a Negotiated Agreement) but there are many more things you can do before that (see more resources below)

More resource

• Beyond bullet point – A refreshing  approach to building presentations
• Presentation Zen – more presentation tips
• Business Storytelling resources
• Negotiation/Persuasion - Anything based on "Getting to yes","Getting past no" and "The power of a positive no" should be fine.

 

Human Relations

Negotiations  skills, Leadership and  dealing with organizational politics are all, in a sense, aspects of Human Relations. Nevertheless Human Relations have a few additional  aspects which I think we, as architects, should be aware of. Basically there are three main components I want to discuss–team dynamics and personal dynamics and pragmatism.

The first aspect  of  human relations is team dynamics. There are several models explaining the various stages teams go through as they grow. I think the best knows is Bruce Tuckman’s model Forming-Storming-Norming-Performing.  The team members act and interact differently during these stages as a (technical) leader of a team or teams; you should be aware of these dynamics and behave accordingly. For instance mentoring and guidance are usually more welcomed during the Forming stage, while these efforts might be rejected during the storming one.

Looking at the team as a whole is one thing. However the developers and the rest of the stakeholders are all individuals. Each with his/her own personality, motivations and what not. Again there are many theories related to individuals from Maslow’s pyramid of needs (which talks about what motivate people) to Jung’s personality types which affects how people interact with each other’s e.g. software developers are likely to be Introverts, Sensing, Thinking, Judging/Perceiving (ISTJ/ISTP) so, for instance, they would not like to be told to do things that don’t make sense to them. In any event, I guess my main advice here is to be aware of some of these theories and pay attention to how they manifest themselves in situations you encounter. In a company I once worked for, I got a small tip from my direct manager. It turned out that when I was talking to people I didn’t hold in high regard – it well, er.. showed. This made them feel uncomfortable working with me but unfortunately none of us was going away. Being aware of the situation made me improve myself. For instance, I wouldn’t just cut away their speech when they tried to make a point or tried not to talk down at them, and hey, even listen sometimes J- This brings me to the last point I want to make on the subject – Pragmatism.

Pragmatism - the art of the possible.  As a leading technical figure (I assume that how you got to be an architect) you should be wary of architectural tyranny. Letting others have their way even if it that way is not the great (I am sure) way you’ve managed to come up with.  There’s a whole lot of clichés that can be thrown here like “there’s more than one way to skin a cat”, “better be smart than right” etc. so what? Sometimes even clichés are rightJ. As project span over a long time and you have to maintain good relations with most if not all the involved parties you should be watchful for absolute truths. Sometimes a little bit of pragmatism can go a long way towards making the atmosphere of the project calmer and nicer. If you take into account the person you are taking to or

 

Further reading

·         Team dynamics what to expect and do - Abhishek Agrawal talks about some of the models for team dynamics

·         Collaboration Explained -  A book by Jean Tabaka on team collaboration

·          

Summary

The architect’s role goes well beyond the technical skills. I hope that this short paper helped highlight some of the softer skills that are required (in my opinion) for an architect to be successful.

The goal of this paper was not to teach all the soft skills but rather to highlight tem and increase the awareness for them. I’d be happy to hear if you find any of the stuff here useful

 

I was demonstrating a POC I wrote to one of my colleagues and I showed him one of the integration demo where I have a two console application - a producer and a consumer who communicate by WCF. He was surprised to see  I get two separate console windows running from within Visual Studio.

So just in case you don't know that as well, all you need to do is to go to the solution properties (Right click on the solution->Properties) and under "Startup project" select "Multiple Startup project".
You can even decide which projects will run with a debugger and which won't :)

 

CrossTalk, the journal of defense software just published its Aug. 2008 issue. Which, incidentally is its 20th anniversary issue.
This issue includes a couple of very interesting articles one is called "Good Old Advice" by Alistair Cockburn and the other is called "What Have 20 Years Accomplished and What Is Left to Accomplish in the Next 20 Years?" by Gerry Weinberg.
Cockburn explains very eloquently something I also wrote about a year ago that the ideas and lessons expressed in literature of yore are not just valid today but they drive home the points of the "coolest" ideas we have today. Alistair does that by demonstrating how the principles behind the agile manifesto builds on the works of Fred Brooks, Barry Boehm, TomDemarko, Gerry Weinberg and many others.
We have known for decades that is is all about people , users, communications between them. Alistair summarize his article as follows (emphasis mine):

"

I derived the four values of the Agile Manifesto from much older, recognized, and highly reliable articles. The point I wish to make is that the authors of the Agile Manifesto did not throw out all previous experience as they wrote it. On the contrary, what they did was cherry-pick four of the most important issues from among hundreds. The appropriateness of their selection is seconded by decades of prior experience and data, as evidenced by the numerous articles referenced in this article.

For decades, we have learned, documented – and ignored – the same lessons:

• Attend to the individuals and their interactions.
• Put the mercilessness of running software to good use – write, and learn from the writing.
• Get inside the heads of your users and customers, and get them on your side.
• Plan coarse-grained long term and fine-grained short term, and find a way to replan quickly – because you’ll have to.

And, finally:

• Do not believe in any single process or methodology because each works only in a particular and limited context.

Those who do not learn from history are doomed to repeat it. Let’s stop pretending we don’t know everything and let’s stop repeating painful history by following this good, old advice."

Weinberg writes a short retrospective of the last 20 years. Here are some interesting quotes (again emphasis mine):

Individual managers, however, have certainly improved. The good news is that we now have many more excellent managers who can mentor new managers. The bad news is that because the field has grown so much, there simply are not enough experienced managers to go around. We still see people abused or poorly used by their managers. All of this in spite of the fact that we have much more information about how to work with people in software development – information that, sadly, is frequently ignored. Too many managers have failed to learn that the world will not end due to a late delivery or a defect in production. Nor have they learned to relax, breathe, learn from their mistakes, and try again.

and

The idea of certification is only one of the management myths that has not changed in a generation. We still have an excess of heroic environments, often leading to death march projects. Many managers still maintain the fallacious perception that quality can be tested into slapdash software. Some still try to develop software without being bothered by potential users of that software. And some still believe that process models are the answer to all of our problems.

These are the things I notice on my pessimistic days. On my optimistic days, I notice more managers realizing that it is the people who make a difference, that they can hire talent, but then must also build the relationships to build a team. Quite a few organizations are now using process models successfully, but as only one of the tools providing information for informed cultural changes to their software community.

Though looking from different angles, both articles somehow hash the same ideas like 'work iteratively", "b e wary of magic or single processes" etc. I do believe that what underlines both these articles and, indeed, software development itself is that at the end of the day software is developed by the people and for the people and we should not allow ourselves to forget it.

---

*with pardon to Abe for the misqoute
 

Unless you've been living under a rock, you probably already heard about the MS offer to acquire Yahoo. While it is Microsoft who is poised to acquire Yahoo (Microhoo) it seems that it is part of the greater move of Microsoft towards moving into Internet based service - all the "live" initiatives along with the "Software + Services" moniker.
I tend to agree with Nicholas Carr who recently published an article in the Financial Times where he talks about how Gates leaving MS marks the end of the desktop era. In fact looking for the above mentioned reference I saw another article Carr wrote for Forbes where he says it even more bluntly

"One important message is this: Software is becoming a media business. The Net is not only a universal medium, a distribution channel for words, sounds and images. It is also turning into a universal computer--the machine we use to run software and store data."

You might also want to check out  a presentation a few of my colleagues  and myself has prepared about half a year ago where we talk about the same phenomena - Called the "Future of Home Computing" (5.5Mb).


In this sense the Microsoft - Yahoo merger (if it will follow through) will result in Yahoosoft a company which is focused more on the internet aspects rather than the more traditional (some would say legacy) desktop aspects of Microsoft.

It is also interesting to note in this sense that while winning the web search (and the related ad-revenue) is something Microsoft is very interested in.  The eyes of Microsoft (and Google for that matter) are on the next battlefield - mobile search. While there are something like 305 Million broadband subscribers worldwide the number of mobile phones sold just in the last quarter of 2007 was larger (334 Millions) and the total of mobile phones worldwide is in the billions... An added bonus here is that Google is yet to take over this market (thought it of course moving in that direction with things like the android platform and location based search and services)

 

I've made a minor enhancement to the site a couple of days ago. I've decided against using the static and hard to maintain blog-roll (I currently have about 250 feeds, some of which are aggregators and linkblogs. )
Instead, I've added a the more dynamic and up-to-date  "What I am reading" section, which lists interesting (to me anyway) posts, articles, presentation etc. from other blogs and or sites. I've also made it available as a feed.

 

Not the usual topic for this blog, but it may save you a few hours of trying to find what went wrong with your office 2007.
I was opening word 2007 to try to edit a document when I suddenly noticed that the mouse doesn't work in the editor window. I could click the ribbon, buttons and menu but as I said I couldn't mark any text or click in the editing area. Not to mention that Word crashed everytime I tried to close it.

Naturally, the office diagnostic didn't find anything wrong so there I went trying reparing, uninstalling stuff I installed in the last few days, blaming the anti-virus, Vista SP1 RC and whatnot.

 I finally found the answer in http://support.microsoft.com/default.aspx/kb/921541 . The kb article suggest several options. Basically all I had to do is delete HKEY_CURRENT_USER\Software\Microsoft Office\12.0\Word\Options\Data and let Word rebuild it.

As the kb article says " If Word starts and works correctly, you have resolved the problem" :) (and saved yourself a few hours)

 

PaperLnx develops an advanced visual search solution for mobile handsets based on computer vision and image understanding technologies developed by Rafael. PaperLnx solves the cumbersome web surfing experience on mobile handsets by enabling end users to send captured images from their mobiles to retrieve relevant information for the object photographed.

We now have few open positions (in Israel) for the following profiles:

 Senior Developer

We are looking for a highly motivated, resourceful and intelligent developer. Good interpersonal and communication skills will be very appreciated. A Team player. Broad thinking and problem solving capabilities are also desired.

• At least 5 years of server side development with thorough understanding of Object Oriented principled and  understanding of architectural styles and design patterns.
• Experience in multithreading and in distributed systems.
• .NET/Ruby experience
• Integration with C++ components
• Experience with O/R Mappers such as nHibernate/ActiveRecord etc.
• Video processing experience/ familiarity with video related protocols (H.324m, H.323 etc.)  a plus
• Web experience (AJAX, CSS, ASP.NET) a plus
  
• Mobile Internet backend development a big advantage
• Understanding of architectural constraints (security, availability, scalability etc.) for internet scale platforms a big advantage
• Team leading skills – an advantage.

 Algorithms Developer

We are looking for an experienced algorithm developer or an outstanding MSC graduate with image processing concentration. A team-player interested in joining a young and dynamic firm.

• Experience developing image processing algorithms using Matlab
• Computer vision/Image processing experience a must
• Experience transitioning algorithms from Matlab to software (C++/C/C#/F#)  a big plus
• Familiarity with video compression protocols a plus
• experience with performance tuning and scaling optimizations
  

 
If you think you qualify and want to join a promising startup you can send your CV  or call  me at 052-3331027
 

 

In a post called "Ignorance vs. Negligence", Ayende blows some steam off on some of the so called "professionals" that he met along the way. You know ...those with a fancy title that don't know jack and design some of the nightmares we see from time to time (go read his post, I'll wait, promise). I see this myself all the time:

• The senior security expert who recommended something which isn't supported by the platform
• The senior architect who throw the system down to hell by basing all the system on a clunky asynchronous solutions that should only be used by a tiny portion of the application.
• The geniuses that built this wonderful code generator that generated code with so many dependencies and singletons that made the solution unusable
• The chief architect that created this wonderful performance hog, and then kept poking around to make sure we don't fix it too much.
• The architect that partitioned a distributed solution based on functions - so that each and every business process has visit go through all the tiers and components. The solution made the everything more complicated by few orders of magnitude (scale, synchronization, availability, performance what not)
• The architect that designed his own distributed transaction mechanism (basically duplicating COM+) - naturally with less than satisfactory results...
  

Etc.
Ayende says

"They all have a few things in common, they represent themselves as experts, senior, knowledgeable people. In all those cases, they have actively acted to harm the business they were working for, by action, inaction or misaction

I have no issue with people not knowing any better, but I do expect people that ought to know better to... actually do know better."

I don't think that this is negligence involved here- I think all of these people want to do the right thing, they probably believe they are right. They were probably also pretty good at their jobs which is what got them to their current position.
What they didn't learn is to "know that they don't know". This is a hard lesson to learn. I think hope I learned my lesson after the first time I tried to distribute a (naive) solution I was so proud of. At least, in my case, I stayed around for enough time to both see the results and learn how to fix the problem.
I think not staying around for enough time is one of the problems that causes this ignorance -  since starting out things usually look good enough and if by the time the problem rears its ugly head you are already in a new shiny job, they you don't know better.

Another problem that causes ignorance is not looking around and learning only from your experience. For instance, I am now interviewing a lot of people, and when I ask a question like "tell me about something interesting you recently read- a book, an article, a blog anything" - I usually get blank stares. Few people tell me about an article they read that is related to a problem they had and fewer still tell me about something without a direct relation to their work.  If you don't look beyond the keyboard you will never know better. learning only from your mistakes can be problematic - especially if we also consider the previous point (people don't stay around)

Ignorance is bliss they say, but I think ignorance has a lot to do with the crappy systems we see all around us and its one of the reasons writing software stays more of an art than a science or craft.

 

I received a number of inquires regarding PaperLnx following the help we rendered to Yediot in enhancing the quality of the video of Yitzhak Rabin's assassination (The link is to a site in Hebrew).

Our business is not video enhancement per se. What we do is use this and other similar proprietary technologies to provide a form of visual search for pictures taken on mobile phone cameras. "surfing" on a mobile phone is not a good user experience, typing URLs and search terms is cumbersome and lengthy. Using this image understanding capabilities we enable end-users to get at the relevant multi-media content for the object they are interested in (the object in the picture taken). This type of application are called "physical world connection" solutions. Naturally we are not alone in this space, but think that the technology we have gives us a competitive edge in the robustness of  our solution.

We are now at the stage where we the technology is pretty solid and we "only" need to turn this into a product, which is as I mentioned in another post, why we are looking for a few good men (and/or women) to join us.

 

Finally, I've quit my current position - to take a new job as VP R&D at (what I think is) a very promising start-up called PaperLnx.
 
In a nutshell we are going to build few solution in the "physical world connection". What's "phisical world connection" you ask? Well, here's a nice definition from "The Pondering Primate":

"When a display was added to the first mobile phone, a new media was created. Since then, Internet connection and a camera have been added that have created a new way to interact with the physical world.

Soon speech recognition will allow an additional way to browse the physical world too.

Every physical object will have a physical world hyperlink

That means every physical object will allow connection to a designated website and the mobile phone with it's psychical world browser will be able to surf the "real world", the physical one."

As can be expected from the "long tail" curve we are not alone in this space (the link above has a nice list of other companies).  However I think we have a competative edge... :)

Incidentally, I am looking for "a few good men" (or women) to join us. We need both developer and QA positions:

Senior Server Developer

We are looking for a highly motivated, resourceful and intelligent developer. Good interpersonal and communication skills will be very appreciated. A Team player. Broad thinking and problem solving capabilities are also desired.

• At least 5 years of server side development with thorough understanding of Object Oriented principled and  understanding of architectural styles and design patterns.
• Experience in multithreading and in distributed systems.
• .NET/Ruby experience
• Integration with C++ components
• Experience with O/R Mappers such as nHibernate/ActiveRecord etc.
• Video processing experience/ familiarity with video related protocols (H.324m, H.323 etc.)  a plus
• Mobile Internet backend development a big advantage
• Understanding of architectural constraints (security, availability, scalability etc.) for internet scale platforms a big advantage
• Team leading skills – an advantage.

Senior Web Developer

We are looking for a highly motivated, resourceful and intelligent developer. Good interpersonal and communication skills will be very appreciated. A Team player. Broad thinking and problem solving capabilities are also desired.

• 3 years web development experience is a must, 5+ years experience a big plus.
• Experience with web-related technologies Javascript, HTML, CSS, AJAX
• .Net/Ruby experience.
• Understanding of Object Oriented principles and REST architectural style.
• Experience with web MVC frameworks like MonoRail/RoR
• Experience with O/R Mappers such as nHibernate/ActiveRecord etc.
• Familiarity with SQL 2005/mySQL. Knowledge of SQL an advantage. 
• OLAP, datamart experience an advantage
•  Team leading skills – an advantage.

QA Person

We are looking for a highly motivated, creative and intelligent QA person. Good interpersonal and communication skills will be very appreciated. A Team player.

• Experience testing WEB  applications.
• 
  
• -Testing creativity. Understanding of Test related methodologies
• -Experience designing and controlling all the stages of the project's life cycle.
• -Experience with test environments such as FIT, FITNESSE or at least other automated tools- advantage.
• -Knowledge of SQL an advantage.
• -Dot-Net/Ruby/Python/Powershell - advantage.
• - Cellular web testing experience a big advantage
• -At least 2 years experience in the field

If you think you qualify, located in Israel and you are interested in changing  the world :), you can send your CV to me
 

Having just read Doron's Yaacoby's post on some of the insights he got  from reading Tom & Lister's Peopleware, I'd thought I'd repost a couple of posts I wrote about a year ago in my Dr. Dobb's Journal blog (with a few edits):

I just finished reading Software Conflict 2.0: The Art and Science of Software Engineering, by Robert L. Glass.

This is a reprint (with few new retrospective additions) of his 1990 book. While the technology mentioned in the book is outdated, many of the author's ideas and views are still valid. The book is a collection of short articles on various subjects, and one of the more interesting articles is about the cognitive side of design.

Glass explains that research done showed that design includes the (obvious) steps of understanding the problem, decomposing it into goals and objects etc. The essence of design, however, are the mental steps taken by designers:

1. They construct a mental model of proposed solution
2. They mentally execute the model (i.e. simulate the model to see if it solves the problem)
3. If the model isn't good enough (e.g., too simple) replay the simulation to find what wrong and remodel
4. Repeat 1-3 until the model seems to solve the problem

Glass also said that people tend to start with a model that worked on a similar problem and that good designers perform this process extremely fast. As I was thinking how one can train himself to get better at performing this task, it occurred to me that I heard something similar  somewhere... While not the exact process this is very reminiscent of TDD - only TDD makes the process explicit. In a way we can say that TDD will not only help make your design better it would also  trains you to design better altogether.

Robert's book has several other views still relevant today. While it can seem odd that a 16-years old book contains relevant thoughts looking at my bookshelf I see there are quite a few other books (some of which are event older) which contain essential information and brilliant ideas that are still very relevant today. Here are just a few examples:

• Peopleware Second Edition by Tom Demarco and Timothy Lister (Second edition 1999, original 1987). Development should be centered on people and not on methods

• The Mythical Man-Month: 20th Anniversary Edition by Fred Brooks (20th anniversary edition 1995, original 1974). "No silver bullet" don't just add people to a late project it will only slow it down and many more.

• Code Complete by Steve McConnell (Second Edition 2004, original 1993). A great book on coding.
  
  
• An Introduction to General Systems Thinking (Silver Anniversary Edition) by Gerald M. Weinberg (silver Anniversary Edition 2001, original 1975). Observations on thinking processes
  
  
• About Face and About Face 2.0 (3.0 edition 2007, 2.0 Edition 2003, original 1995). UI mistakes and guidance, the second edition also details a scenario based approach to user experience design.
  

Getting back to TDD or at least the idea of test first. It seems (via Rob Keefer pomiet blog)  Gerald Weinberg talked about it more than 35 years ago in  Psychology of Computer Programming :

"By pursuing this test example to the point where he understands the problem, he will not only learn the one thing he did not know, but perhaps will learn others as well, for test programs such as this are often better learning instruments than are production programs.

As a matter of good practice, the test program should be constructed before the 'fix' is made to the production program. In the first place, there will be an all too human tendency to forget about the problem once the production program is working correctly, so we must impose a little discipline on ourselves. Possibly more important, however, is the chance that by the mere act of constructing the test case we shall discover the problem."

Anyway, with so much good advice lying around for years (11-40+) and the fact that only about 30% of the projects are successful (on-time;on budget; on scope) I think one question we should all ask ourselves is -- don't we ever learn?

 

Yesterday, I finally finished all the chores and obligations for my MSc. in Information Technology.

Phew, what a relief - especially considering the last semester (summer) was very loaded as I studied 3 days a week .
Anyway, at least now I should have more time to work on my book (Chapter 6 was starting to feel a little neglected :)).

 

dasBlog 2.0 is out which is good news for those hosting with  medium trust like (yeah Go Daddy...). dasBlog 2.0 finally allows me to use some of of the new features like Tag Cloud, Akismet integrations, native feedburner integration instead of the hack I did earlier etc. So I immediately upgraded

I've took the chance to also move to  a nicer theme (thanks to John Forsythe and Jon Stovall for adding this theme to dasBlog). Lastly I've also moved the URI naming  to PascalCase URIs instead of the ugly GUID Permalinks. The old Permalink type URIs are of course still available as well (Even with my limited command of the English language, I've managed to figure out the idea behind the "Perma" in Permalink)

(PS  Technorati blog claim:Technorati Profile- you can disregard this line otherwise)

 

Just got back from Chicago, where I attended Architecture & Design World. The event was great. it had many interesting talks and sessions  which I'll blog about in  few of the following posts (Also I'll upload the slides from my sessions later this week). One session however, was very disappointing for me, Ivar Jacobson's Keynote on "Next Generation Process with Essential Modeling".

I was really looking forward for this keynote; After all Ivar is one of the "Three Amigos", father of use cases and all. Also I had the chance to attend a couple of his presentations several years ago and he had a very interesting and convincing appearances at those times. Unfortunetly this session was nothing like those previous events.
Ivar talked about his new methodology. The basic idea behind the essential software process is (in my opinion) correct. Instead of big bang prescribed processes - it is better to tailor a process from a set of practices to get something that fits the organization/project at hand. It might be a little unnerving to hear this from one of the people who brought (upon) us the RUP, but, I guess it is good to see people who are constantly learning.

The problem was that while the main idea can be summed up in one line (see above) Ivar went on fro the better part of the hour to explain how cool it was that his base set of practices comes on a set of que cards that can be sorted and arranged. He also went on to explain the game board (or something to that effect) where you can lay different cards to both describe your current process as well as the bunch of practices you want to use for your future process.
If that wasn't enough, when he (briefly) shown us the cards that talk about the architecture and architecture description practices the advice/guidance there was very mediocre and general
I expected much more from someone of the stature of Ivar Jacobson.
On the up-side this was only one keynote and the other sessions I listened to where much better - as I already said, I'll blog about few of them in upcoming posts.

 

Coming back from vacation I saw that Jeff Atwood (coding horror) wrote a post on "rethinking design patterns"
a few days ago. Jeff criticizes the GoF design patterns book
Jeff says his two main grips with the book are

1. Design patterns are a form of complexity. As with all complexity, I'd rather see developers focus on simpler solutions before going straight to a complex recipe of design patterns.
2. If you find yourself frequently writing a bunch of boilerplate design pattern code to deal with a "recurring design problem", that's not good engineering-- it's a sign that your language is fundamentally broken.

I agree that these can be a problem with using design patterns in general and the GoF ones in particular. I would add to this that I think that using design patterns just to be able to say you've used them is very wrond as well (this may sound obvious - but I have seen organizations where this was encouraged).

I don't think however, any of those problems are problems with the patterns themselves. This is not to say that the GoF book is perfect.Looking back at the GoF book I think that it isn't clear of problems. For instance, I don't think all the patterns stand the test of time. The most prominent example for that is the Singleton pattern, which I hardly recommend using these days. Singletons are problematic for testing and  create a tight coupling for a specific instance. There are better ways to create a mono-instance if that's what you need (I think others have posted about it in the past - but I can post about it separately if needed)
Also, newer languages sometimes provide better ways to tackle some of the design patterns solutions - see for example a recent post by Alex Miller where he talks about an alternative for Template Method.
I can even say, that not all the patterns are that useful (Flyweight comes to mind as an example).

Nevertheless, I still think that the GoF book is one of the most important books in computer science. First it is a seminal work which introduced the patterns thinking into software development. Today we have literally hundreds of patterns on all subjects and technologies. I think it is a very good think since the looking at a problem from a pattern perspective gives us more depth and understanding on both the problem and the solutions vs. other ways I've seen.
Even in itself, the GoF book is great, since many of the patterns are very valuable and can help us solve real problems. we just need to keep in mind that the sample implementation in the book is just that - sample. There can be more than one way to code a pattern and still gain the benefits (these are "design" patterns not "coding" patterns). A few weeks ago, someone asked a question on the visitor pattern in one of the forums I monitor. The guy needed to add an additional parameter to the Visit method and asked if it wasn't a violation of the pattern. I told him that there isn't such a thing as "violating a design pattern". The patterns are a means to an end not some coding codex we should keep. I think, that if we treat the design patterns as pieces of knowledge
rather then a holly script, they can really help avoid some stupid mistakes.

Thus, at the end of the day, I still think the Gof book is required reading for any new developer. But hey, that's just my .2 cents :)

 

A couple of quick  observations following the Events and temporal coupling post

Events, Current data and aggregated data all have Time-to-live aspects.

• Events value usually diminishes over time until the TTL reaches
• Current data usually have a constant value while their TTL lasts (until a new value is the current data) - unless we are talking about version data which is a  component of or a step  in the direction of aggregated data.
• Aggregated data  has the longest TTL, it is interesting to note that its value increases over time

Also while the Current data TTL is determined by the producer both Events and Aggregated data TTLs are determined by consumers

Yeah, I know these are not not earth shattering observations but I still think they are  interesting
 

 

It was more than 10 years ago that I got my first MVP award from Microsoft (I was probably the first Israeli MVP, because few years later Microsoft Israel awarded what they said werethe first Israeli MVP awards... :) )
Anyway, now, for the forth time, I got awarded again - this time it is in the Solution Architect category -
Thanks

 

I've been working in Rafael for a total of 5 years now and I guess my tenure here is nearing its end.  The latest company-wide reorg has brought with it the end of the biometric product line (or maybe just its evolution into something I don't really like - it isn't completely clear yet, but since I don't like both option it doesn't rally matters).

I am checking several possibilities of joining one of the consulting companies here in Israel - none of the options has fully matured as of yet but I believe that in the next month or so something will be finalized - meanwhile I am also open for other options (my CV is on the about page - hint, hint :) ).

In any event, between the book, DDJ, InfoQ and looking for a new job (not to mention my family) I sure have have my hands full - I guess now I understand why they say "may you live in interesting times" is an old Chinese curse :)

 

Yesterday I attended an SOA governance presentation by Brent Carlson. The presentation was basically an updated version of an article he authored in 2006 "SOA Governance Best Practices - Architectural, Organizational and SDLC implications" As a  tool vendor Brent has a lot of focus on the governance processes which I don't completely agree with (I prefer Jim Coplien's organizational patterns approach - see my post from last week). I also think the reuse figures he cites (registration required) are a little optimistic common place for what I consider the right granularity for services.
He also made a few points I  strongly agree with

• Brent talked about difference between the needs of run-time service repository (e.g. UDDI or an ESB) and a development time one. You need to address the services and their interactions during the development and you need to do that in a way that would be easy for the development teams. For example, one thing you want to log is usage, who is using the services since that will let you perform impact analysis when you have to make a change
  
• Building an SOA for an organization is an iterative process not a "big-bang" effort. This means you can't do just top-down design. you need to be pragmatic and also roll out working services.

The reason for this post however is the insight Brent gave regarding treating services as products rather than applications

Treating Services as products is  important because even if you don't believe that the SOA initiative should be  an iterative process once the move is  finished you would have quite a few services deployed in your organization. These services would integrate and interact with other services - some of which outside of your organization. You would also want to capitalize on flexibility claim that SOA makes and adapt your services to the changing business needs.
The challenges you face regarding updating and upgrading  functionality , anticipating consumer's needs, allowing consumers to get used to changes etc. are exactly the challenges product management techniques and principles come to answer

Treating services as products means a lot of things. let's look at a few examples: For one, it means predictable release cycles services like products get updated over time you want service users to be able to cope with this changes. Predictable release cycles means they can get organized in advance. Another aspect is the emphasis on backward compatibility e.g. orderly deprecation of features and version management,.One other thing  is introducing a "product manager". someone whose responsibility is to interact with customers, and potential customers, understand their needs and build a release road map for the services.

You might be used to doing some of that with applications   but thinking about services an products makes all this more explicit and that in itself is also important.

 

Yesterday I attended Jim Coplien's presentation on "Organizational Patterns - a Key for Agile Systems Development". Overall I think It was a very good presentation. Jim makes a few interesting claims, some of which are controversial within both in the traditional and the agile spaces
Few examples

• Process guidance (ISOs etc.) doesn't work - roles are stabler than processes, processes always change.
• Jim says that in order to make a change you need to make it at the organizational structure level. The processes will then support these changes
  
• TDD is evil - it is just an re-incarnation of bottom-up procedural design. It is better to follow "Design by contract"
• He says XP is not a good methodology (He thinks SCRUM is good)
• etc.

Additionally he talked about some of the organizational patterns he and Neil Harrison discovered studying organizations for more than a decade. You can  read the Top ten patterns on his site.

Jim covered 2 patterns that are related to software architecture: Architect controls product and Architect also Implements
Architect controls Product basically says that you should have an architect and that she should oversee that the direction of the project is flowing in the right direction.

Architect also implements - this pattern says that in order for the architect to broaden her leadership without sacrificing depth and pragmatics she must also participate in the implementation (beyond advising and communicating). Jim gave the example of the development of Borland's Quatro pro for windows in 1993 where the team's architect had a daily meeting (akin to scrum stand-ups) for synchronization and would then go and code with the developers. The Quatro pro team had 4 architects out of 12 persons that made the team.If a third of the development team is architect I'd say he is right -  My experience, however with most organizations I see is that you hardly have one architect per  project (sometimes you only have one for several projects). In these cases I hardly see the architect writing production code as part of the team since she would not have time to fulfill her architectural responsibilities. She must know how to code though and she must be able to prove her designs in code or be able to offer a candidate implementation if needed (I also wrote about that in the past see "Should architect's code" part 1, part 2, part 3

By the way if you are located in Israel, Jim will be here for a couple of weeks and he is giving a few courses like Agile Architecture, Patterns of Agile Project Management etc. You can find more information on pacificsoft's site

 

Back in January, I took part in an architect panel that Microsoft Israel organized. The panel was led by Ron Jacobs and it featured Udi Dahan, Assaf Jacoby, Coby Cohen, Dudu Benabou and myself. A few days ago Ron edited this recoding and turned it into a podcast in his Arcast series.

The panel's focus was on lessons learned from mistakes made in past project. Ego maniac as I may be :) -- even though you don't get to hear me much in the final edited version -- I think the podcast is worth listening to, as the panel raised some interesting points. You can download the podcast here (don't worry it is in English even though it was recorded in Israel)

PS

I am the first speaker after the introduction, in case you are wondering.

 

I'll be presenting a  90 minute class on SOA Patterns on the upcoming Architecture & Design world 2007 - which will take place in Chicago on July 24-27th.
If any of you happen to be there, I'd be very happy if you drop by and say hello :)

 

 

Back in January I opined that  moving to web applications was not the optimal solution to the real problem we have/had with desktop applications which was  installation woes. What we got was a poor UI without installation problems so we (software industry) started to resolve problems like we had when we moved from  terminals to graphical UIs etc. - 
So now we have Rich Internet Applications (RIA) - using technologies like AJAX - but they suffer from other problems  which again we've already been through

Well that was the topic of the post in January. Now I've stumbled upon an interesting/amusing  twist - called Adobe Apollo
Apollo let's you, yes you've guessed it, take your RIA applications and deploy them as desktop applications.  you can now take your HTML, CSSs , AJAX scripts pack them up as a single file (AIR) and lo and behold deploy them on the desktop. You even get these nifty start menu and desktop shortcuts :)

The reason not to dismiss this a complete waste of time - is that what we actually see here is another example of a trend to convergence  web and desktop UI architectures  and programming models. I say "another" because coming from the desktop direction Microsoft is doing pretty much the same thing. WPF brings the web-programming model  with its markup (XAML) and "code-behind" concepts to the desktop as well as pushing the same model to the browser with WPF/E .

The difference between Microsoft's and Adobe's solutions is that, Adobe is coming from the web-side and, as I said, Microsoft is coming from the Desktop side - both companies are striding toward the same goal -  and what we are left with,  is yet another technology war

 

All I can say is I want one

http://cellphones.engadget.com/2007/01/09/the-apple-iphone/

 

[based on a few posts from my DDJ blog]

Implementing Business Intelligence (BI) solution on top of Service Oriented Architecture (SOA) is not a simple feat. A recent survey by Ventana Research shows that "...only one-third of respondents reported they believe their internal IT personnel have the knowledge and skills to implement BI services.". There's a good reason for that since there an inherent impedance mismatch between BI and SOA which takes some effort to overcome. The purpose of this paper is to look to explain the problem as well as look at the possible solutions.

Service-Oriented Architecture is about autonomous loosely coupled components. These traits gives you lots of benefits such as greater flexibility and agility but it also means that services have private data. Data that you don't want to expose to the outside as exposing it will decrease autonomy and increase coupling. This is why services only expose data and processes via contracts rather then exposing their internal structure.

That is all fine until you start to think about business intelligence. The cornerstone of any business intelligence initiative is gathering, collecting and consolidating data from all over the place. Once you have the data, you can use tools to analyze it, data mine it, slice, splice, aggregate, and whatnot. Traditionally BI builds on ETL (Extract, Transfer, Load) which goes directly to the database of the involved sources.

And here lies the problem: On the one hand we have services that want to keep their data private, and on the other we have a datamart or warehouse that wants that data badly.

What are our options?

• If you go with traditional ETL, you introduce coupling into your service.
  
• If you only rely on contracts that were constructed for business processes you may be missing out on important data.
  
• If you build a specific contract that exposes "all" the data you are back at the point-to-point integration -- solving point-to-point integration is one of the reason we want SOA in the first place.
  

The second option seems to be the most reasonable choice of the three -- but it also has several problems. One problem is that the BI needs to know about all the contracts. The second was already mentioned -- important data might be missing. The third problem is that the BI system need to fetch data from the services which means it may miss out on data in the intervals between request. On the other hand, too frequent requests and you can congest your network easily as well as cause DOS on your own services.

Clearly we need a fourth option

In my opinion, the best way to tackle BI in SOA is to add publication messages into the contract. By "publication messages", I mean that the service will publish its state either in a periodic manner or per event to anyone who listening. This is a service communication pattern which I call "Inversion of Communications" since it reverse the request/reply communication style which is common for SOA.

To make the solution complete, you can add additional requests/reply or request/reaction messages to allow consumers to retrieve initial snapshots. Following this approach, you get an event stream of the changes within the service in a manner that is not specific for the BI. In fact, having other services react on the event stream can increase the overall loose coupling in the system - for instance by caching results of other services

Why is this better than the other three approaches? For one , you can get a good picture of what happens within the service. However the contract is not specific for the BI and can be used by other services to cache the service state (thus increasing their own autonomy), for reporting (you can see an early draft of the aggregated reporting pattern), and for BI purposes. By working against a steady stream of events, the BI platforms can Analise treands, keep history and get the complete picture they need.

The approach above is sometimes referred to as "Event Driven Architecture" (EDA) and while I (and others) see EDA as another facet of SOA, not everyone agrees. Gartner, for instance, sees EDA as another paradigm and SOA just for request/reply, or client/server. Recently, however, they published a paper that calls the approach described here as "Advanced SOA". I tend to agree more with the "advanced SOA" definition and don't see a contradiction with EDA and the SOA definitions. We are still using the same components and the same relations only adding an additional message exchange pattern into our toolbox.

A note on implementation: If you are implementing SOA over an ESB that is rather easy to implement as most ESBs support publishing events out of the box. Using the WS* stack of protocols, you have WS-BaseNotification, WS-BrokeredNotification and WS-Topic set of standards. If you are on the REST camp, then I guess you will need to implement publish/subscribe by yourself.

Once you have event streams on the network, The BI components grab that data scrub it as much as they like and push it to their datamarts and data warehouses. However, event steams can also enable much more complex and interesting analysis of real time events and real time trend data using complex event processing (CEP) tools to get real-time business activity monitoring (BAM)

You can also get post as as a presentation down loadable from the papers section on my site or directly from here. (The download is about 3MB.)

 

Welcome to chain-letters blogsphere style. There's this on-line tag game going around, I've been watching it spiraling around on many of the blogs I read and now Udi dragged me into this as well :)

So here goes - here are 5 things you don't know about me:

1. It only took me 14 years to get my BA degree in Computer Science. I began studying in 1990 in the Technion ,quit after 2 years and only bothered to graduate when I wanted to get a Masters degree

2. I was a Microsoft Foxpro MVP for 3 years in the 90s - about half of that time I was working on completely different platforms and tools (J++ and C++ on Windows and then J2EE on Solaris).

3. first met English in fourth grade (like most other Israeli kids at that time) - by the eighth grade I read my first real book - Shogun. I took me 3-4 month to get through the 1200 pages of the book, but I've been reading English since. In fact I hardly read Hebrew anymore.

4. I learned to program on the ZX81 - I remember the joy the first time I fully used the 1K memory it had, as well as the disappointment the followed thereafter when the instructor tried to add the 16K expansion which caused the machine to reboot.

5. I used to be a hobbyist bar tender. I still have more than 70 different bottles at home, with everything from Grenadine to a 25 years old Glenmorangie. I don't mix too many drinks these days, I mostly drink the Macallan.

I don't want to stay "it" for too long :), so on to tagging some other folks: Ohad Israeli, Andrew Johnston, Tad Anderson, Nancy Folsom and Ruth Malan. You are all it

 

Roy Osherov recommended this site today - but he also urged me to write more frequently.

This is probably a good opportunity to explain how posts are divided between my 3 blogs

• First theres the blog on Dr. Dobb's Journal. This blog is published on the "Architecture & Design" section of DDJ portal. I blog there about 3 times a week. Jon (my Editor @ DDJ) prefers a steady stream of blogs over longer posts which means that I break down large subjects (like OO principles, fallacies of distributed computing and the currently running series on the Architect's soft skills) into many parts.

• The second blog is a new one on Microsoft's Israel blogs site. The aim of this site is to bring Architecture content in Hebrew to the Israeli architects (As can be imagined, most of the technical content available is in English, I thought it was important to generate some content in Hebrew as well)

• The last blog is this one. My current plan for this blog is as follows

• • Cross posting selected posts from the DDJ site
  • I am posting here complete articles made by editing and aggregating multi-part blogs posts (again such as the fallacies etc.)
  • Pointers to presentations and articles I publish
  • In the near future, I'll start posting bits of my upcoming SOA patterns book (I am currently writing chapter 3). I've already documented 8 patterns (of more than 50 patterns and about 30 anti-patters). I plan to publish here at least some of the patterns here for review (I am still crossing the t's and dotting the i's with my publisher but I expect this to be finalized soon)

so Roy, does seven (7) posts in seven days (including 3 on Ms Israel site, 3 on DDJ and this one) qualify as posting often enough? :)

 

I've added a new section on the site www.rgoarchitects.com/Papers to allow easy access to all the papers, presentations and articles I published (and will be publishing e.g. I'll add a paper on architect soft skills in a month or so etc.)

 

 

The October issue of Dr. Dobb's Journal is available and with it my article on the SPAMMED Architecture Framework .

 

 

Dr. Dobb's has begun a special (6 issues) E-Zine  called  "Dr. Dobb's Requirements Development" and I am happy to say that Issue #1 includes the first part of an article by me on the subject of use case modeling.

The issue also includes articles by Joe Marasaco (author of "Software Development Edge"), Andrew Stellman & Jenifer Geene (authors of "Applied Software Project Management") and Karl E. Wiegers (Author of "Software Requirments" 2nd edition.)

You can get the EZine by registering on www.ravenflow.com to the bi-monthly Requirement Develompment magazine

 

 

Over the last few months I've posted a series of blogs on DDJ that cover the basic Object Oriented principles (e.g. Single Responsibility Principle, Don't Repeat Yourself, Inversion of Control etc.).

I've assembled all the posts into a single whitepaper which you can get here.

Also you can download the same (plus a little more) material as a powerpoint presentation.

 

 

[Crosspost from my DDJ blog]

When talking about multi-tiered architectures, we need to remember that the tier boundary is significant. The tier boundary is where distribution happens and if you remember the "fallacies of distributed computing", you know not to take that lightly.

A tier is a physical boundary (versus an Edge in an SOA which is a logical boundary, for example) and the implications are numerous. For instance, you need to consider:

• Trust--who do you let in?
• Security--what do you send out?
• Performance--you need to serialize to pass the boundary, and remote data is expensive to fetch.
• Availability--what happens if you crash?
• Manageability--can anyone see what's your state? Help you recover?
• Temporal coupling--can you afford to make synchronous calls?
• and many similar questions.

Yet many times people think passing a tier is as simple as passing a logical layer. I should know. I made this stupid mistake more than 15 years ago in one of the first distributed systems I designed. I planned this beautiful separation of the UI controls from the business logic (I didn't know it was called "MVC" and that someone else had figured it out ages ago, so I was pretty proud of myself). When you clicked on a button you just used metadata to say that BL should catch it. I had all this wonderful "infrastructure"that handled passing the call to its destination.

But then we wanted to take this n-layer application and put the BL in an "application server" which will handle multiple clients. Oh--now we need to move events over the wire , handle calls from multiple unrelated clients, pass a lot of data back and forth, and what about security... you can imagine the fiasco.

Thus, as Niels Bohr once said, "An expert is a person who has made all the mistakes which can be made in a very narrow field." But you don't have to make the same mistakes. Just remember that a tier is a natural boundary. You know what? You should probably even want to consider it the edge of a cliff at the end of your application--and be careful not to fall down.

 

[crosspost from my DDJ blog]

In a comment to my previous post on Architecture vs. Design, Yoni said:

It seems you are categorizing technical issues as architecture and logical issues as design. I think Martin Fowler's definition of "Making sure important things remain decoupled and easy to change" transverses both categories and is easier to follow.

I have a few things to say about this.

First, I don't categorize technical things as "architectural" and logical ones as "design." What I do say is both "architecture" and "design" are types of design where with one you focus on the wider aspects of the solution and quality attributes, while with the other you focus on local and functional aspects.

I don't see how the definition Yoni brings up is a better way to distinguish between the two? Who is to say what is important and what is not? Isn't decoupling an important trait at all level including so called "detailed design" level (e.g. utilizing dependency injection at the class level will give you better testability). Moreover, decoupling is important but sometimes you need to trade that to be able to satisfy a more prioritized quality attribute (if you want to meet a projec's quality, budget and schedule targets); see my definition of what software architecture is.

Another thing is that it doesn't matter that much where the line between architecture and design passes. The distinction between architecture and design is a semantic one that reminds us that the design of a system needs to be done in several levels of abstraction (provided the system is not too trivial). We need to abstract certain aspects of the system in order to be able to grasp the big picture. You cannot (well, I can't anyway) think at a 100 man-years project. You can only think about it at the class level and understand how everything will work together. Again, architecture is there to remind us to focus on a level of abstraction that lets you deal with non-local decisions and to make sure quality attributes are met if we cross the line to design -  No biggie.

 

[edited version of post I made on Dr. Dobbs Portal]

Back in April I  provided a definition for "architecture" as one of my first posts on DDJ. I also promised I'll talk about the distinction betwen Architecture and Design. Well this time is now.

When I try to think about it. I see two base criteria to distinguish between the architecture and design:

• Design deals with local decisions, where architecture is broader. For instance, you "design" the interfaces for your classes, but you "architect" the division into tiers.
• Design is mostly about the functional requirements, while architecture is mostly about quality attributes. You design how a specific workflow will fulfill a certain use case, but you architect the solution to the system's availability.

It is probably quite evident that this distinction only provides blurry borders between architecture and design; for example, when you have a multi-tier solution and you "architect" the UI and say it will implement MVP pattern. Can this be considered local decision and thus design or is this the overall decision (for the UI) and thus architecture?

The way I see it the exact cross-point from architecture to design is not that important. The point in talking about two distinct activities in the development process is to maintain separation of concern. you need to handle both to make sure a solution will actually work whether you do a little design while architecting or do a little architecture while designing really doesn't matter. Also architects should be involved in both activities anyway...

 

Last week I published a 3 part article on O/R mapping on my blog @ Dr. Dobb's Portal. The paper describes the benefits and costs of using O/R mapping as well as recommend when O/R mapping should be used.

Here it is as a single whitepaper: Architecture Dilemmas - OR Mappin.pdf (228.78 KB)

 

Well, not right away - but I just read he will be leaving MS by 2008. I don't know whether it will be good or bad for Microsoft in the long run but whatever the outcome will be it will definitely be the end of an era.

 

 

I have amassed more than 30 patterns related to SOA (e.g. SOA Patterns - Decoupled Invocation and SOA Patterns - Aggregated Reporting which I previously published here). I have patterns around security, availability, scalability, composition, adding a UI etc. Some of the patterns are original (I think) and some are based on other peoples work.

 

I am trying to decide whether or not it would be worthwhile putting all these patterns into a book. Writing a book is a very time consuming task (or so I am told) - so I thought I'd run a quick poll between the readers of this blog to see how many of you would be interested in reading (and buying) this book if it will get published.

I know this is not a representative crowd - but it can give me a (very) rough idea on the interest in such a book.

 

Please  send any comments (comments like "forget it, no one would ever want to read anything you write" are also ok) to soa@rgoarchitects.com (or leave a comment here)

 

Thanks in advance - Arnon.

 

I gave a peresetation on CMMI today.

The main points of the presentation are

• CMMI focuses on process
• It builds on the premise that improving the process imporves the quality of the product
• CMMI integrated several others CMMs including Software CMM
• CMMI is a framework that lets you define the process - you need to show you cover the CMMI process areas for certification
• Newer methdologies (Agile) focus on people rather than process.
• However considering CMMI is a framework you can map agile processes to CMMI
• You would want to do this (mapping) if you want to introduce agility into CMMI organizations
• Another reasons to mix both approaches is sometimes therese a need to use formal processes but you still want some agility for sub-projects.

 

Scott Ambler has a new article comparing most of the leading development methodologies. He also tries to  recommend which methodology fits which kind of project (e.g. for a commercial of the shelf products use EUP/RUP, ISO 12207, TSP/PSP or Data Driven Approach).

The article serves as a nice overview of available methods - however Scott doesn't explain the reasoning on why he think a particular methodology fits (or doesn't) a certain type of application which is a pity. Furthermore, I think Scott is missing the point a little by neglecting organizational, cultural other people related reasons for choosing a methodology. For example if all your teams are versed with RUP, you would most likely "force-fit" it to your new COBOL project rather than choose a better fit methodology.

Also, I am not sure I agree with the all his mapping - the most notable example is  mapping XP for  safety critical projects. To get a DO-178B (the certification required by the FAA for aviation software) you need to have the following documents (DO-178B has 5 levels and not all documents are needed for all levels):

DO-178B Documents:

DO-178B Records:

Plan for Software Aspects of Certification (PSAC) Software Development Plan (SDP) Software Verification Plan (SVP) Software Configuration Management Plan (SCMP) Software Quality Assurance Plan (SQAP) Software Requirements Standards (SRS) Software Design Standards (SDS) Software Code Standards (SCS) Software Requirements Data (SRD) Software Design Description (SDD) Software Verification Cases and Procedures (SVCP) Software Life Cycle Environment Configuration Index (SECI) Software Configuration Index (SCI) Software Accomplishment Summary (SAS)

Software Verification Results (SVR) Problem Reports Software Configuration Management Records         *list copied from LynxOS site

I don't think that this level of formality is a good fit for XP

 

 

[crosspost from DDJ]

Reading the comments on my previous two posts on whether architects should code (here and here) as well as the comments on Johanna Rothman's posts (here, here and here) leads me to a few observations:

The first apparent thing is that the issue is a very loaded. Some people believe it is essential for architects to code, while others (like me) believe that their time is better spent on other issues. (That said, it seems that a small majority of the commenters think architects should code as part of the development team--at least for feedback purposes if nothing else.)

There is a wide consensus (me included)that architects should know how to code and have extensive experience in coding. It is also agreed that architects should be involved in the project--that is, not just drop off the architecture, then disengage.

I still believe that when the project is big enough (that is, big enough to warrent more than one team working on it) the project is better served by the architect getting involved in all the teams, rather than participating as a developer in one of them. If you are an architect and develop as part of the development team you are (or should be anyway) committed--meaning you need to deliver the piece of code under your responsibility at an acceptable quality level as other developers. Which is exactly why you would be less likely to deliver on your responsibilities for the total quality of the project. I assume some of the differences in opinion can be attributed to disagreement on what software architecture is , at least when compared to design).

I also think those who think architects must code see the architect as some sort of a lead developer again. I don't buy that. The architect's role is much broader than that (see also this post by Kevin Seal, which also discusses this issue). I see a holistic view of the architect role, which is making sure the product is delivererable. This may translate to the architect coding a module or two, but it can also translate to a lot of other things. Examples from my experience as an architect include preparing initial cost estimates, iteration planning, helping debug and testing, solving installation problems, analyzing requirements, conducting design and code review, design, and prototyping (yes, that's coding but as I said in the previous posts, that's not writing the production code and this is not having to meet deadlines etc.).

I also liked a comment by Graham Oakses on one of Johanna's posts :

My experience is that an architect is pulled between three poles--the product, the team and the client. The product pole pulls you towards managing the "conceptual integrity" of the design. The team pole pulls you towards mentoring people, helping them build skills, etc (which may mean consciously letting someone write code that you could do much better yourself). The client pole pulls you towards translating between the technical and the client domains (which is often where you get pulled into powerpoint). You need to trade these poles off differently on every project...

To sum up, the answer to "should architects code? " is like so many things in life is--it depends.

 

While I still hold my view on the Current State of Software Factories - at least one company (EDS) is repoting  (in an MSDN article) that they've built a software factory for their domain entities [found via Steve Cook]. Nevertheless this is still a generic generator (i.e. not a factory for ordering system or something similar- but rather something like  a DSL for O/R Mapping).

Arnon

PS

I also wonder why do they generate unit tests fot the generated code - assuming they properly tested their templates the generated code should just work ...

 

Roy Osherove blogs about some of the questions that were discussed in the architect's panel in the Recent TechEd Israel

 

I've been thinking a lot about some of these questions lately  (and not just because I helped  draft the questions for the panel :)) specifically about when and how to introduce agile methods. One problem, which Roy points out, of fixed price/time projects (which unfortunately in Israel are pretty much the norm.). Another problem is with organizations such as the one I work for which have CMMI level 3 

(or more) certification (not to mention ISO 9002) which makes it really hard to introduce agility.

 

I stumbled upon this presentation  which analyzes the CMMI compliance of agile methodologies (I've started to try map SCRUM to CMMI 3 to get SEPG (Software Engineering & Practices Group) off of my back)

 

Another interesting approach I found is AgileTek's Agile+ methodologywhich is a mix and match approach that claims to be the best of both worlds (I am not sure I am 100% convinced but it is worth a look)

 

Lastly you can look at this interesting presentation by Barry Boehm and Richard Turner which talks about When to use which approach.

 

[Crossposted from my DDJ blog]

About the same time I wrote the post on whether architects should code, saying that architects should be able to prototype but shouldn't be part of the dev team (in the sense that the architect shouldn't get coding tasks that results in production code), Johanna Rothman wrote a blogpost that claimed architects must code .

Two days ago she posted a more detailed explanation of her view. I agree with most of the points she made:

1. Architects need to participate in the project; that is, not be some outsider who just drops her architecture on the team and leaves).
   
2. The best way to test a design is to code and run it.
   
3. It is beneficial for architects to know to code.
   
4. It is important that architects understand the implications of their decisions on the code and developers.

I don't see how architects taking coding tasks serves the greater good, versus their monitoring teams that code and making sure all aspects of the architecture actually fit the problem and work. Again, this may work on smaller projects, but probably not on larger ones.

You may also want to look at two related posts I made in the past
SAF Architecture Evaluation: Evaluation in Code talks about some of the ways architecture can be validated in code.
SAF Deployment: What to do when the architecture seems stable? talks about the architect's involvement in the project when they think the architecture is "finished".

A couple of points regarding the analogy Rothman uses--that is, architects who design bathrooms for hotels. Building architects are seldom a good analogy for software architects (I once used it as well). However, there are far too many differences (maybe I'll blog about that sometime in the future).

This brings me to the second point. This analogy doesn't serve Rothman's point well since building architects never actually participate in laying down brick or installing bathrooms. The fact that hotel bathrooms are not comfortable means that this quality was low on their priorities. In any event, verifying if a bathroom is usable--you don't have to install it just use it. (If you do take the analogy, you don't have to code it just stick around to see what's going on

 

[crosspost from Dr. Dobb's Portal]

Test Driven Development (TDD) is, in a nutshell, writing a unit test up front--making it fail, making it work , refactor, and repeat until the product is finished. (If this is new to you, read more at testdriven.com )

So with TDD you get a bunch of unit tests that are also proven as regression tests. That's pretty cool.

TDD also lets you work in small increments while maintaining the working code. That's even cooler.

And lastly TDD has a very good influence on design:

1. It encourage loose-coupling. When you want make something testable you want to remove the dependencies it has so you can test it by itself.
2. It makes you think about the interface of the unit under test--how is the interface going to look?
3. It makes you think about how the unit under test would be used--for example, the behavior of what you are writing (or designing).

Sounds great to me. I think TDD is a great way to do the detailed design. You specify the results (interface + behavior), then implement that design. One thing I don't buy though is that TDD alone will produce an "emergent design" for the whole system. The way I see it is that you have to do some design up-front (assuming your system is not a trivial one) since TDD, being a coding technique, keeps you working at sea-level.

There's also a fundamental matter of scale--it might be possible in theory to start that 100 man-year project as a single object, then refactor it in baby-steps until you'd get the perfect system. I believe that if you don't work at a higher level of abstraction (vs. code), you will not be able to partition the system in a reasonable time. This was true when we moved from assembly code to higher level languages which enabled us to write much more complex software--and it is true today as we need to answer the ever-changing requirements of modern enterprises.

To sum up, TDD is good for testing and it is a good design methodology for the detailed design level. It can be used to drive the overall design on smaller project--but on larger systems we need additional methods and tools to cope with the overall design and architecture.

 

Scott Bellware attacks Microsoft's cluelessness of modern development methodologies and tools. By talking about the 3 (in?)famous typecasts (Mort, Elvis & Einstein personas) used by Microsoft to model developers.

 

While I agree with a lot of the points Scott makes I think that (unfortunately) there are individuals and organizations that are suitable for to the Mort-Elvis-Einstein approach -  Where people are not as smart or competent as Scott is and can't handle agility. Also there are situations  where agility cannot be practiced -e.g.  clients insist on fixed price projects and waterfall-ish milestones where, against our better judgment, we were forced to do a lot of up-front planning (that had to be reworked later…), safety-critical system etc.

 

I much prefer the direction Scott took in an earlier post where he talked about a missing persona - Hugo the Agilist. I think Microsoft will be making a grave mistake if they will not pay attention to the needs of the growing community of developers that prefer agile methods and practices.

 

Dr. Dobb's has recently launched a new portal site, where they want industry experts to post their views - well, I guess it is not limited to experts only, as they've also offered me to write there - I'll be writing the blog on software architecture and design.

 

I am going to post there on a daily basis (read: ~5 posts a week). Naturally they are going to be shorter posts that will try to highlight and comment (hopefully) interesting things related to architecture and design (my thoughts, other peoples posts, news etc.).

I am still trying to decide on the balance between this blog and the new one, but  I guess most longer posts will go here (though I may cross-post them) and that whitepapers and presentations will continue to be posted here. Also note that the new blog has a wider spectrum as it also talks about design.

 

You will find my new blog "If you build it…will they come" here.

 

One of the roles of the software architect is to act as a mentor/coach. Reviewing some of the designs in one of my projects' teams it seems the time was ripe for doing just that. Thus, last week I gave them a presentation on the basics of good OO design  - which I thought might also be of interest for other people (you can download a copy here - 312KB).

 

The presentation starts with the  7 deadly sins of software design:

• Rigidity – make it hard to change
• Fragility – make it easy to break
• Immobility – make it hard to reuse
• Viscosity – make it hard to do the right thing
• Needless Complexity – over design
• Needless Repetition – error prone
• Not doing any

 

It is interesting to note that just yesterday I read an interesting piece on what makes good design (i.e. looking from the positive side) by James Shore (found via Sam Gentile) 

 

 

The main part of the presentation demonstrates the 5 basic design principles (drafted by people like Robert C. Martin , and Barbara Liskov ):

• OCP open-closed principle - a class should be open for extension but closed for modifications
• SRP single responsibility principle - a class should have a single responsibility
• ISP interface segregation principle - there should be separate interfaces for different consumer types
• LSP Liskov substitution principle - basically design by contract - a sub-class should fulfill the same expectations its suparclass set
• DIP dependency inversion principle - classes should depends on abstractions, class consumers should depend on abstractions and abstractions shouldn't depend on details.

 

These principles are the basis for  some of the techniques widely used today - few examples include:

Inversion Of Control - builds on OCP

Dependency Injection - a mechanism to allow DIP

Contract First - building on LSP,DIP

 

At the end of the day following these principles helps managing classes dependencies, increase overall loose coupling and cohesion thus increasing the overall quality of design. It sometimes amazes me how using just a  few simple rules can improve maintainability, flexibility and usefulness of designs so much.

 

I recently found MSF 4.0 is out  - yep, it is still in two flavors…

 

MSF Agile (or officially "MSF for Agile Software Development Process Guidance") and MSF 4 CMMI (MSF for CMMI Process Improvement Process Guidance)

I consider MSF Agile as a lightweight process, however I prefer SCRUM as an Agile project management  process.

 

I find the CMMI version more interesting - Both for  cases where Agile is not a good fit (also see  Agile vs. Plan driven development  )

and/or for organizations that need to have CMMI certifications (such as the one I currently work for). MSF 4 CMMI covers level 3 pretty well and also has some guidance moving on to the next levels.

 

Definitely worth a look :)

 

Scott Bellware writes about Microsoft missing an Agilist Persona (in addition to Mort, Elvis and Einstein. I pretty much agree with Scott's views - MS's lack of understanding of the Agile crowd is evident in "fiascos" like the TDD article  on MSDN or even MSF Agile, which is  relatively a light process but still very far from processes such as  XP, SCRUM and the like.

 

Personas is a very interesting concept, usable as a communication aide, for development teams - as a means to help maintain user focus.

 

The CHAOS Chronicles 3.0 by Standish group* (www.standishgroup.com)  cite "User involvement" as second most important success factor for development project success  (after executive management support). Indeed many agile methodologies also encourage high customer involvement in development process

 

During my professional career, I had the chance to work for both product companies and solution companies.  - Why is that relevant you ask? Well, when you work for a solution company you usually have tangible, real-life, customers to work with. You can walk them through early usability prototypes, you can consult with then on problematic requirements, you can have them on site for instant feedback etc. etc.

 

Things are more problematic  when you work on "shrink wrapped" products - now your customers are much more abstract , and elusive, yes you can still hold focus groups etc. but your you can't have that day-to-day interaction with end-users and customers - enter Personas.

 

I first heard about Personas several years ago, when I read Alan Cooper's "The Inmates are running the Asylum". The book demonstrates some bad  designs for software-based products and then introduces an approach (Personas...) to help avoid these problems (He elaborates more on the approach in "About Face 2.0").

 

Personas are basically a way to define archetypes of users. Unlike Use Case Actors which represent a role in the system, Persons try to high-light the characteristics of real users. The idea is to come-up with representative model of a user and to give it a full-bio and characteristics  so as to help the development team understand motivations and relate to actual users. The model for absent users is the reason this technique is very  important for product companies. If you don't have real users come up with abstract ones representing them. Alan Cooper introduced Personas as a means to help designers in the initial phases of product design. Microsoft extended the use of Personas as a communication aide to the full range of the development team (designers, developers, testers, managers, marketers etc.) - which brings more benefits. I highly recommend reading the very  interesting paper "Personas: Practice and Theory" by John Pruitt and Jonathan Grudin which  relates Microsoft's experience on the subject.

 

While Personas are very important for product companies, it can also be important for solutions development especially on larger projects where only few members of the development team get a chance to interact with customers and user.

When you cannot have the customer on site (or the customer representative doesn't give you the full picture of all user types of the system) you can use Persona-Scenarios as a way to augment user stories (or in other circumstances as an alternative to actor-use cases)

 

---

  

*The Standish Group has been collecting metrics from thousands of projects since 1994. They analyze success and failure factors and publish them on yearly basis. For example, while our ratios as an industry are getting better over the years - as 2004 only about third of the projects achieved their goals both  on budget and on time...

 

 

Forget that stupid agile methods and all that iterative junk - Waterfall 2006 is here http://www.waterfall2006.com

or as the "report of the DEFENSE SCIENCE BOARD TASK FORCE ON DEFENSE SOFTWARE 2000"   sums nicely:

"About 90% of the time, the [waterfall] process results in a late, over-budget, fragile, and expensive-to maintain software system. A typical result of following the waterfall model is that integration and testing consume too much time and effort relative to the other software development activities. Most waterfall projects, expend over 40% of their effort and schedule in integration and testing."

Oh well, maybe we should stick with what we know after all :)

(Thanks Grady Booch and David Ing blogs for the conference link)

 

 

I recently read Weblog Usability: The Top Ten Design Mistakes by Jacob Nielsen  [found via Jeff Tash's IT Scout blog]

The first 2 mistakes Jacob discusses are  missing author photo and bio - since these are relatively easy to fix, I went on and mended the situation. I've added a photo  on the sidebar and I've posted a short bio in a new "about me" section.

 

Avoiding the other 8 mistakes require more effort - but I think (read: hope) I have most of them covered.

 

I'll take this opportunity to make a quick note on the architecture posts. I see that there's a lot to say about architecture modeling (oh my, what a surprise). I want to cover SAF at a certain level before I get lost in the details - so if you found SAF interesting thus far, watch out for posts on Mapping, Evaluation and Deployment soon :)

 

I just read Paul Kimmel's very interesting article on "truisms" of software development called "Un-Dynamics of Software Development, or, Don't Bite the Flip Bozo" . The article is written in an amusing/cynical tone, however a lot (if not all) of it is really true -

"Flipping the Bozo bit " by the way is "to make a mental note that a particular person is a bozo and everything they say in the future should be ignored or looked upon as the meanderings of a slightly annoying, occasionally amusing child or a drunken uncle"

Paul comments that the mean time for someone in the crowd to flip the bozo bit on you when you start speaking is 10 seconds.

The article also mentions stuff like

• When someone says the schedule is going to be missed, they are never lying.
• If a manager says I am not technical, be prepared to spend a lot of time explaining things to them so they can make decisions they shouldn't be making.
• Managers hire experts and ignore them all the time.

[found via Mitch Barnett's blog on software industrialization - which provides for interesting reading by itself (focusing with Software Factories and DSLs and general software development). his company developed a Biztalk appliance, which you may want to checkout if you are using Biztalk*]

---

* I have my views and some reservations regarding Biztalk - but I guess that a subject for another post :)
 

Udi Dahan talks about the merits of agile methods vs. (heavy) process methodologies (countering a post by Joel Semeniuk on CMMI).

I can't say I totally agree with either (what else :) ) - There are cases where agile methods are a better git and there are cases where you'd be lost without a plan. While agile methods optimize for change (which is indeed a grim reality we all live with) plan driven methods optimize for complexity. Barry Boehm and Richard Turner detail the various people related issues that can make you choose one over the other. The diagram below (taken from that article) sums it up nicely.

It is always important to strike a balance between the level of process employed and the project at hand. That's why even the more heavy processes (like RUP or MSF for CMMI) are tailorable. I guess a lot of organization don't bother to take the effort to tailor the processes to their need and rely on the "out of the box" experience which is not tuned to their needs. and thus suffer less than optimal results.

Another important issue is tool support - If you are going to employ a more plan-driven (heavy) process you really want it to be supported by your tools to help alleviate that "document oriented development" feeling Udi mentioned in his post. This is where this upcoming release of MSF shines (especially vs. the previous release of MSF).

 

I just upgraded the blog from to DasBlog 1.8 (RC), I also took the chance and changed the theme for the site.

I guess this is a good opportunity to thank Scott Henselman and Omar Shahin for all the time and effort the put into maintaining and upgrading this software

 

I've added a new area on the site for the SPAMMED Architecture Framework (SAF)

www.rgoarchitects.com/saf

There's nothing much there (yet...) except links to the blog entries on the SPAMMED process, however, I am going to add there presentations, whitepapers, a workshop etc. in the future (some of these are already under development)

 

The presentation for the paper mentioned in the former post can be downloaded from here
 

First the disclaimer :)

I wrote the paper below about 2 years ago, summarizing my experience with requirements engineering using use cases. The problem was that it got to be too long to be a magazine article and too short for a book plus it needs a ton of editing.

Nevertheless, Now that I started blogging and considering that I think it still has some very useful information to anyone trying to make use cases work in a mid sized or large project - here it is for your viewing pleasure:

Methodology for building Use Cases for large systems.pdf (206.64 KB)

[fixed link]

 

Introduction