Last month we saw Ajax was making a move to the desktop with the aid of Adobe Apollo. Earlier this month I talked about it again when Sun announced JavaFX and after  Microsoft officially announced Silverlight.
I guess it was inevitable that Google would also join the fun of blurring the lines between desktop applications and web applications. And indeed I just read that Google released a beta of Google Gears.

What does Google Gears do? well, in essence it lets you develop occasionally disconnected web applications by providing an API to store application resources and data locally as well as Workerpools  that allow you to run tasks in the background (sort of like a collection of processes that host javascripts and  can communicate by messages).

What all these means is that it wouldn't be too long before the the architectural choice between web clients or desktop clients will not be that important it will be more about making a technology choice on the  preferred development environment and tooling as well as designing robust occasionally disconnected applications regardless of that technology. Just look at  the architectural guidance Google provides for utilizing Google gear.  You'd see that the components that they identify are very similar to the high-level components you are likely to design with any occasionally disconnected desktop application...

 

Yesterday I attended Jim Coplien's presentation on "Organizational Patterns - a Key for Agile Systems Development". Overall I think It was a very good presentation. Jim makes a few interesting claims, some of which are controversial within both in the traditional and the agile spaces
Few examples

• Process guidance (ISOs etc.) doesn't work - roles are stabler than processes, processes always change.
• Jim says that in order to make a change you need to make it at the organizational structure level. The processes will then support these changes
  
• TDD is evil - it is just an re-incarnation of bottom-up procedural design. It is better to follow "Design by contract"
• He says XP is not a good methodology (He thinks SCRUM is good)
• etc.

Additionally he talked about some of the organizational patterns he and Neil Harrison discovered studying organizations for more than a decade. You can  read the Top ten patterns on his site.

Jim covered 2 patterns that are related to software architecture: Architect controls product and Architect also Implements
Architect controls Product basically says that you should have an architect and that she should oversee that the direction of the project is flowing in the right direction.

Architect also implements - this pattern says that in order for the architect to broaden her leadership without sacrificing depth and pragmatics she must also participate in the implementation (beyond advising and communicating). Jim gave the example of the development of Borland's Quatro pro for windows in 1993 where the team's architect had a daily meeting (akin to scrum stand-ups) for synchronization and would then go and code with the developers. The Quatro pro team had 4 architects out of 12 persons that made the team.If a third of the development team is architect I'd say he is right -  My experience, however with most organizations I see is that you hardly have one architect per  project (sometimes you only have one for several projects). In these cases I hardly see the architect writing production code as part of the team since she would not have time to fulfill her architectural responsibilities. She must know how to code though and she must be able to prove her designs in code or be able to offer a candidate implementation if needed (I also wrote about that in the past see "Should architect's code" part 1, part 2, part 3

By the way if you are located in Israel, Jim will be here for a couple of weeks and he is giving a few courses like Agile Architecture, Patterns of Agile Project Management etc. You can find more information on pacificsoft's site

 

Udi Dahan writes that ".NET/Java Interop is not a reason for SOA". Udi writes that companies that  need to integrate two technologies turn to web-services and that

"The only problem is that in order for things to work right, they really must have a chatty interface, and flow transaction context between these “services”, and all the other things I describe as anti-patterns"

Udi is right that if you don't rethink and remodel your systems you will (probably) not  have an SOA as you are likely to find your self implementing  anti-patterns such as the ones he mentions.

However, using Web-services does not automatically mean that you are doing an SOA. If you don't think about moving to SOA you can still opt to use web-services as a remoting  or RPC technology to connect two systems. The advantage over the other proprietary products Udi mentions is that web-services are a standard technology. This will work well or fail is orthogonal to the technology choice. It depends on the architectures of the systems you integrate. If you need to flow transaction between the systems you'd also need that even if you cross-compile one of the applications in the other environment.

Another thing I don't agree with is the word must Udi uses. First, while it is likely that older systems has chatty interfaces it is not a must. The designers of the legacy system may have thought about the consequences of distribution without regard to SOA. Also you can still wrap an existing system with a service contract (using web-services or any other technology) and not get to chatty interfaces etc. However that means that the wrapper should have some substance or business logic inside it to mask the old system's behavior this is especially important  if you are thinking about moving to SOA and you take into consideration that the business will not just halt and wait there until you are done. You have to think about interim solutions, such interim solutions can include wrapping a legacy system with an Edge Component and a SOA facade (a pattern I call Legacy Bridge) while you move in the grader direction of a full blown SOA.

 

It has been awhile since I last published a pattern draft - but I guess it is better late than never.
The saga pattern deals with manginc complex interactions between services without the use of atomic transactions, which as I mentioned in the past are not a good idea (see "Transactions between services? No, No, No!" and "Some more thoughts on Cross-service transactions" )

You can download the draft for the Saga pattern from here.
I'll also add a link to it from the SOA Patterns book section (where you can also download the other pattern drafts I published)

By the way, I am not happy with the current  sketch (the pattern illustration) in this pattern, so it will probably change in later drafts. I would be happy to hear any suggestions you have for improving it.

 

I just read Shy Cohen's (via Nick Malik) article in Microsoft's Architecture Journal entitled "Ontology and Taxonomy of Services in a Service-Oriented Architecture" Shy provides a list of what he calls service types. He identifies two major types  bus services and application. He then continues to sub-divide them, the Bus Services are divided into communication and utility services and the Application services  are divided into entity, capability, activity and process services. I have to say it was quite alarming to see this coming from someone who had deep involvement in defining Windows Communication Foundation Indigo.

Where do I start?

well, for one, it seems completely fails to make the distinction between Services as in Service Oriented Architecture and Services as in capabilities or features an infrastructure provide. The "Communication services" are for the most part capabilities that a service infrastructure (such as an ESB) provides. Not Services you would define in an SOA initiative And then there's the matter of service granularity and the difference between Remote objects and SOA for instance, the example  Shy gives for  a "method" (his word) on a Customer service (entity service):

"An example of a domain-specific operation is a customers service that exposes a method called FindCustomerByLocation that can locate a customer's ID given the customer's address"

why would a service return a customer ID? This is the kind of call you would make on an object you hold a reference to not some remote "Something" that also want to authorize your call and may reside in a different company. This kind of thinking is what made remote objects fail. Gregor Hohpe explained that nicely in a paper  called Developing software in a Service Oriented World

The Transparency Illusion.  Distributed components promised to hide remote communication from the developer by making the remoteness "transparent". While the basic syntactic interaction between remote components can be wrapped inside a proxy object, it turned out that dealing with partial failures, latency, and remote exceptions could not be hidden from the developer. It turned out that 90% transparency was actually worse than no transparency because it gave developers a false sense of comfort.

As a side note, Gregor recently gave a presentation that covers this paper at JavaZone which you can watch online at InfoQ

Returning to Shy's article let's take a look at another quote:

Capability Service may flow an atomic transaction in which it is included to the Entity Services that it uses. Capability Services are also used to implement a Reservation Pattern over Entity Services that do not support that pattern, and to a much lesser extent over other Capability Services that do not support that pattern.

I already explained why cross-service transactions and especially flowing transactions is not a good idea in SOA so I won't do it again here - but you can read about it both here ("Transactions between Services? No, No, No!") and here ("Cross-Service Transactions"). Also I truly hope Shy didn't mean .NET data sets when he said "In some cases, typically for convenience reasons, Entity Service implementers choose to expose the underlying data as data sets rather than strongly-schematized XML data. Even though data sets are not entities in the strict sense, those services are still considered Entity Services for classification purposes." In any event the whole decomposition of Services into fine grained "capability", "Activity" and "process" takes no account of the fact the SOA is a distributed architecture...maybe Microsoft is not affected by the  fallacies of distributed computing ?

---

*ad nauseam (latin)- to the point of disgust

 

Yesterday, we had a meeting of a few architects and Memi Lavi gave a nice presentation on Architecture usability. The notions of architecture and frameworks/software infrastructure  are a little mixed in this presentation, but in essence Memi is right to say that among the quality attributes the architecture has to address you should have modifiability and architecture usage scenarios like "adding a complex form will take less than 2 weeks" or "wiring a new component into the workflow will take less than one day" etc. Reducing  friction* of frameworks is very important. I don't think it is the most important though. since if you could only have one of  the quality attributes you need to support (performance, correctness, architecture usability, framework friction, system usability, availability, budget etc.) I am not sure you would always pick architecture usability.
Driving home, I thought that essentially software architecture is just

The set of compromises, trying to maximize the effect of a limited set of  prioritized quality attributes

we can't have it all - so we need to prioritize our list of quality attributes and focus on the most important ones.

---

*Friction (from Ryan via Udi): "friction" is a (subjective) measure of how much the tooling gets in your way when trying to solve a specific-case problem. I've come to evaluate frameworks based on two rough metrics: how far the framework goes in solving the general case problem out of the box and how little friction the framework creates when you have to solve the specific-case problem yourself. When a framework finds a balance between these two areas, we call it "well designed."

 

Pat Helland is back in Microsoft (after a two years vacation in Amazon)  and more importantly he also restarted blogging. I only met him in person a few times - but he is definitely one of the few persons really worth listening to - especially when it comes to distributed computing. Not only does he make interesting observations he is also capable of explaining them in a crisp and interesting manner.  Indeed, it didn't take too long (his second post) before he blogged some valuable content. The post is called Memories, guesses and apologies (go read it).

Pat talks about how the notion of time in a distributed environment is subjective and you can really know what happened before what and what we can do about it (I really think you should  just go read it :) ).
Another related aspect of the phenomena Pat mentioned is that taking a snapshot in time, the chances of having a single unified truth in a distributed system degrade in a proportional manner to the system's load. I  had a chance to work on a few systems where some of the sites had either occasionally connected or connected over  low bandwidth networks. This situation makes the whole notion of guessing the state and compensating and/or apologizing for wrong conclusions much more explicit than in always connected high bandwidth system. Nevertheless, latency still exists even in connected systems and and you should really be weary of assuming a universal truth - unless you can stop the businesses  long enough to allow complete synchronization.
As I mentioned a few days ago, we can't afford to have cross-service transactions (I also think we can't afford too many distributed transaction in non-SOA architectures, but this is a especially true for SOA) which makes things even worse in this sense. One thing we can do in an SOA to achieve distributed consensus is to run a Saga. Saga, which is a long running conversation between services, is probably one of the most important interaction patterns for SOA.
You know what? instead of trying to explain it here in a haste i'll just publish the pattern draft - I'll try to do that before the end of the week.

 

Johanna Rothman writed about "letting go of BDUF" (Big Design Up Front). One statement she makes is that you can't predict what the architecture needs to be. I can't say I agree with that, since many times you do know something about the project and you do have prior expereice that can give you enough confidence to lay some ground rules. I called this Just Enough Design Up Front or JEDUF for short.

Another statement Johanna made, which I wholeheartedly agree with, is that you should let the architecture evolve. Evolving an architecture sounds very compelling but it is not a simple feat. Architectural decisions tend to have system wide implications which means that changing one too late in the game you'd get a lot of rewrite and/or refactoring to do
.
My strategy to solve that conflict is to:

1. Set the first one or two iterations as architectural ones. Some of the work in these iterations is to spike technological and architectural risk. Nevertheless most of architectural iterations are still about delivering business value and user stories. The difference is that the prioritization of the requirements is also done based on technical risks and not just business ones. By the way, when you write quality attribute requirements as scenarios makes them usable as user stories helps customers understand their business value.
2. Try to think about prior experience to produce the baseline architecture
3. One of the quality attributes that you should bring into the table is flexibility - but be weary of putting too much effort into building this flexibility in
4. Don't try to implement architectural components thoroughly - it is enough to run a thin thread through them and expand then when the need arise. Sometimes it is even enough just to identify them as possible future extensions.
5. Try to postpone architectural decisions to the last responsible moment. However, when that moment comes -make the decision. try to validate the architectural decisions by spiking them out before you introduce them into the project

These steps don't promise that the initial architecture sticks, but in my experience it makes it possible to minimize the number of architectural decisions but still have a relatively solid foundation to base your project on


 

In the article mentioned in the previous post, I talk about adding EDA to SOA and how you can use Complex Event Processing (CEP) to process the event streams and infer the trends and enhance the understanding of what happen inside your business. All the tools I knew about were Java tools I knew about were Java tools but now I've found out (via Nauman Leghari's blog) that there's also a .NET CEP engine and it is even open source - It is called NESper and like many other tools it is a port of a Java tool.
I am not sure how good it is - but at least I'll have an interesting evening today :)