I got a couple of emails with questions reagarding  my previous post on Volta. So here's another go at explaining why dynamic-tiering is not a good move - this time in technicolor.

Let's start with a simple illustration. The diagram below represents a typical local component(A) in its environment. As a component that works locally, it has access to other local components which it interacts with. These can be objects it created by itself or objects that where injected to it. The likely design for local components is to have a chatty interaction - After all objects can talk to instances of other objects quite easily.

 

In a post called "Ignorance vs. Negligence", Ayende blows some steam off on some of the so called "professionals" that he met along the way. You know ...those with a fancy title that don't know jack and design some of the nightmares we see from time to time (go read his post, I'll wait, promise). I see this myself all the time:

• The senior security expert who recommended something which isn't supported by the platform
• The senior architect who throw the system down to hell by basing all the system on a clunky asynchronous solutions that should only be used by a tiny portion of the application.
• The geniuses that built this wonderful code generator that generated code with so many dependencies and singletons that made the solution unusable
• The chief architect that created this wonderful performance hog, and then kept poking around to make sure we don't fix it too much.
• The architect that partitioned a distributed solution based on functions - so that each and every business process has visit go through all the tiers and components. The solution made the everything more complicated by few orders of magnitude (scale, synchronization, availability, performance what not)
• The architect that designed his own distributed transaction mechanism (basically duplicating COM+) - naturally with less than satisfactory results...
  

Etc.
Ayende says

"They all have a few things in common, they represent themselves as experts, senior, knowledgeable people. In all those cases, they have actively acted to harm the business they were working for, by action, inaction or misaction

I have no issue with people not knowing any better, but I do expect people that ought to know better to... actually do know better."

I don't think that this is negligence involved here- I think all of these people want to do the right thing, they probably believe they are right. They were probably also pretty good at their jobs which is what got them to their current position.
What they didn't learn is to "know that they don't know". This is a hard lesson to learn. I think hope I learned my lesson after the first time I tried to distribute a (naive) solution I was so proud of. At least, in my case, I stayed around for enough time to both see the results and learn how to fix the problem.
I think not staying around for enough time is one of the problems that causes this ignorance -  since starting out things usually look good enough and if by the time the problem rears its ugly head you are already in a new shiny job, they you don't know better.

Another problem that causes ignorance is not looking around and learning only from your experience. For instance, I am now interviewing a lot of people, and when I ask a question like "tell me about something interesting you recently read- a book, an article, a blog anything" - I usually get blank stares. Few people tell me about an article they read that is related to a problem they had and fewer still tell me about something without a direct relation to their work.  If you don't look beyond the keyboard you will never know better. learning only from your mistakes can be problematic - especially if we also consider the previous point (people don't stay around)

Ignorance is bliss they say, but I think ignorance has a lot to do with the crappy systems we see all around us and its one of the reasons writing software stays more of an art than a science or craft.

 

Jeremy D. Miller writes that he prefers user stories better than use cases - and I basically agree with him. Mostly because user stories are finer grained which lends to make them more "estimateable"  and manageable then use cases are.

However, having used both, I think there's one thing use cases do better then user stories which is eliciting related scenarios. When you write a use case that covers a  business scenario the use case template makes you think about variations and exceptions. When you try to think of a theme of user stories if you're not careful to do that in a structured manner you may miss some.

One important point regarding use cases is that you don't have to treat the use case a single monolithic block. When you work with use cases it doesn't have to mean that you sign-off use cases in a waterfall-ish manner- that is something that has to do with your development methodology.
You can develop use cases incrementally i.e. only do some of the use cases per "release". You can also elaborate use cases   iteratively by only elaborating/identifying some of the scenarios in each iteration (in a manner similar to user stories). I have successfully use both incremental and iterative use case elaborations  on several projects and I find it very useful (I even wrote paper summarizing my use cases experience a few years ago (warning - more than 60 pages with the examples and all..)

In fact, even today I find it useful to use both tools where I would usually start with identifying a user story and writing that down. Then map it to a use case (either existing  or a new one) - where a use case roughly equals a user story theme. As I mentioned previously I can them identify more user stories and add them to the product backlog. The use case stays a skeleton with links to the user stories and helps provide more context to individual use stories (ok, so that isn't a use case in the traditional sense, but I find it useful anyway).

Generally speaking, it is important to note that while it is tempting to equate a user story with a scenario in a use case, that isn't always true. sometime a user story would be a step in a use case and sometimes a story is shared by several use cases - since a fine grained feature that delivers business value is can be used in many business processes which is what use cases are oriented to


As a side note, if you have a project where you do have to write all the requirements up-front (as if they won't change anyway...) then I find that use cases are way superior than IEEE 830 style requirements ("The system shall...")  - but hey, that's another story :)

 

Back in April I wrote how Adobe AIR (then it was still called Apollo) marks the beginning of the invasion of the web clients into the desktop. Later I wrote about the Java and .NET counterattacks (JavaFX and Silverlight) and then I wrote about Google's answer when Google Gears was announced.

Well Mozilla "Prism" demonstrates that even simple steps can help make this transition.
The main idea behind Prism is to "integrate web-applications into the user desktop experience". Behind this fancy statement we have a very simple solution - the ability to add a a desktop/start/quicklaunch shortcut to any web application (or page for that matter)and have that show in a window that is configurable so that it doesn't waste pixels on irrelevant stuff for the applications (like navigation buttons, address bar etc.) - which makes it better then just adding a shortcut yourself. Simple and elegant. Here's what my Google reader looks like with Prism:

If you want to start using it, you can just download the prototype for Mac OS X, Linux, and Windows.


 

Earlier today Microsoft announced it newest SOA initiative codenamed Oslo Here are a few observations I have on this announcement.

Let's start with "what is it?" Well it isn't an "it" per se, since Oslo is a bunch of initiatives within the Microsoft offerings.

For one, it is some of the libraries within .NET 4.0 -- specifically the next versions of WCF and WF.

Secondly, it is a bunch of designers and tools that will be part of Visual Studio (beyond VS 2008).
The most interesting component of Oslo will be a new repository to allow version management of models and services. I guess it is safe to say it will be built upon Team Foundation Server (or a subset of which which will be used by both products).

The last part of the puzzle is of course V.Next of Biztalk and something currently branded as "Biztalk Services '1'". As far as I know Biztalk sells pretty, but I think it is both too bloated (e.g., think about the hardware needed to run this in high-performance solution) and builds on the wrong architecture (hub vs. bus). I hope Microsoft makes major updates this time (Biztalk 2004 to Biztalk 2006 mostly innovated around the business activity monitoring. While that's important I think more work on the engine was/is due).

Biztalk services would offer an implementation of some of the SOA patterns I talk about -- service host, workflodize, etc. -- to provide an infrastructure for building services. The relation between "Biztalk 6" and "Biztalk services 1" is not clear from the information provided by Microsoft; hopefully this is just a branding issue and not a tight relation between the products.

On the upside, one of the key persons working on this is Don Ferguson who, before joining Microsoft, was chief architect for IBM's software group. About a year ago I had the chance to hear him talk about SOA and all I can say is that Don is someone who really knows his stuff.

PS: It's amusing to see the press release talks about "model-driven" approach rather than software factories, but I guess that's just nitpicking.
 

Let's assume I convinced you that some projects need architects (see part I). Convinced, you go and hire an architect. now what?

Let's start by looking at  "architectural decisions" - which is sure sounds like something we'd want an architect to do. I read once (I think that was Martin Fowler) that an architectural decision is a decision that in hindsiight you wished you made right. if we look at a formal definition of software architecture (say from IEEE 1471) we see that the architecture embodies the fundamental decisions about the system its components, their relations and their properties. Using this definition an architectural decision is a fundamental decision about the system (which pretty much explain why we want to make them right etc.)

Well, here are two observations on what I've said thus far. One is that we would want to postpone architectural decision as much as we can, since changing them will cause us a lot of headache. The problem is that in order to postpone an architectural decision we need to build flexibility into the system which is an architectural quality in itself - which might not be the top of the list if we prioritize it vs. other architectural qualities we need.

The second observation is that if we "refactor" the pretty language out from both of these definition - we can see that an architectural decision is basically a guess, hopefully that's an educated guess but it is a guess none-the-less. and as Albert Einstein once said it is  hard to make  predictions - especially about the future.

This is why architects  breadth of knowledge - which helps explain the architect training program I posted about a few weeks ago (see Architect training program Part I and Architect training program Part II). Another aspect is experience. And to get a wider perspective it can be helpful if this experience includes other roles besides developer such as project manager or business analyst etc. Another important component is domain knowledge and understanding of the business.

Using all these you (as an architect) may come up with a reasonable architectural decision (e.g. use MVC pattern) and a design to match it and that's it.

Well, actually, not quite since as I said earlier it is still a guess. Remember  an architectural decision (and any design for that matter) is a mirage no matter how beautiful the power point slide looks (or white board or UML sketch etc.)

Alas, power point compilers are still in the making. Which means that as an architect, you must be able to prove your point in writing - that is coding. While you are at it, you also need to know a thing or two about the technology you are using because it too has an architecture, features etc. which can have a significant effect on the end result. (You can read a little bit more on this in the "Architecture Deployment" paper I published a while ago).

The result of trying to postpone architectural decisions, ever changing requirements along with adding details as we unfold the  architectural abstraction level to a working system, is that the architect can't just appear at the inception of a project and disappear afterwards - they need to stick around for the game. This is especially true if you want to have an evolving architecture

An architect needs to do more than "architectural decisions". There are also additional reasons why the architect should have continuous interaction with the rest of the development team. However that will have to wait for Part III. :)

 

I received a number of inquires regarding PaperLnx following the help we rendered to Yediot in enhancing the quality of the video of Yitzhak Rabin's assassination (The link is to a site in Hebrew).

Our business is not video enhancement per se. What we do is use this and other similar proprietary technologies to provide a form of visual search for pictures taken on mobile phone cameras. "surfing" on a mobile phone is not a good user experience, typing URLs and search terms is cumbersome and lengthy. Using this image understanding capabilities we enable end-users to get at the relevant multi-media content for the object they are interested in (the object in the picture taken). This type of application are called "physical world connection" solutions. Naturally we are not alone in this space, but think that the technology we have gives us a competitive edge in the robustness of  our solution.

We are now at the stage where we the technology is pretty solid and we "only" need to turn this into a product, which is as I mentioned in another post, why we are looking for a few good men (and/or women) to join us.

 

I am currently interviewing people for developer positions for my startup (PaperLnx) and as I presented my plans to use multiple languages to one of the candidates he immediately retorted with "why on earth would you want to do that?"

Why indeed?
 
As I mentioned in the previous post about F# and Erlang it seems to me that the age of "one size fits all" is ending  in a lot of areas. I think that it is also happening for languages.

When .NET was announced, one of the messages by Microsoft was that .NET is the platform and that you can choose any language you want to develop in it. Microsoft also worked with partners to back up this statement and  release a lot of languages for the platform (e.g. Eiffel#, Fujitsu NetCobol  etc.) vs. Java which was a single language with a "write once, run everywhere" vision. Microsoft itself released 2 manages languages VB.NET C# and managed extensions for C++. The C++ managed extensions were very awkward to work with  and only people  who really had to use it. the non-MS languages seemed to be niche languages for legacy integration and not much more. VB.NET (we're talking .NET 1.0 here) seemed like something to try to distract VB heads while Microsoft was pulling the rug beneath their legs. Which basically left C# as the only viable language to develop in if you were serious about .NET (at least in my eyes). The "many languages" vision seemed like something to hide the (obvious) fact that Microsoft had to come up with an answer to Java (especially since around the same time Sun also sued Microsoft over the J++ and JVM implementations)

5-6 years later the situation is a little different. For one, we see that the JVM is not just about Java anymore. Just like the original .NET vision, the JVM has become a platform supporting many different languages Jython, JRuby, Scala, Groovy etc.
The .Net platform is also getting more interesting  languages like  Boo or   Gardens Point Ruby.Net . .NET is also being extended by Microsoft itself (with IronPython, IronRuby, F# ).

Two interesting things we can see are one, suddenly the languages are cross platform. e.g. I can program in python and deploy it natively, .NET or on a JVM - so in a way you get some platform independence.

The other and more significant issue (in my eyes) is that the languages available on each platform are different enough in meaningful ways - to a point where it is getting worthwhile to use more than one language in projects. Here are few examples from around the blogsphere that demonstrate this:

• Neal Ford shows an example using the (excellent) Mocha mocking library (in JRuby for testing Java classes. So the dynamic properties of Ruby help make the testing simpler for the statically typed Java.
• The BOO Build System (via Ayende) - which is about building a DSL for builds (a la rake). again we see taking advantage of the properties of a dynamic language this time to help with .Net languages.
• (not) Dennis Byrne shows how you can call Erlang from Java - which demonstrate taking advantage of the concurrency features of Erlang from Java.

When it comes to the difference between general purpose languages (i.e. Java, C#, VB.NET) we mainly see syntactic nuances. When it comes to dynamic languages vs. static languages vs. functional languages etc. the differences are more meaningful. What has changed now is that both platforms and open standards (e.g. REST over HTTP/WS-*) enable better integration and increase the motivation to use the "best tool for the job" approach over "the everything looks like a nail" approach .

I think the time for language pluralism is ripe - In my solution this would probably translate to a mix of C/C++, F#, C# and Ruby - What about your projects?

 

A couple of thoughts on functional programming in general and on F# and Erlang in particular

Erlang - is it worth it?
I read Armstrong's Erlang thesis and other Erlang related material with great interest. After all Erlang claims some very compelling architectural benefits in the areas of scalability and availability.  However one thing that prevents me from digging too deep into it is the utter ugliness of the language vs. for example, ruby which is very compelling. 
i also tend to agree with Otaku that at least today you can achieve this tauted resilience with other languages as well.

On the other hand we start to see some really interesting things built on Erlang like RabbitMQ which is an AMQP implementation (AMQP - The Advanced Message Queue Protocol), MNesia persistence engine . Yaws web server etc. (you can see here one guy that even likes Yaws + MNesia better than Ruby + Rails)

Well, I am still on the fence  for this one, but I guess I will take a deeper look at Erlang eventually..

By the way, while other functional languages also have irritating syntax (e.g. Scheme ) - some look more intereseting (like F# or OCaml)

 which brings me to the next tidbit...

Functional Languages moving mainstream.

Functional programming is an old paradigm (see for example this paper from 1984 "Why Functional Programming Matters" explaining the advantages of functional programming over structured programming..).

For a decade or even more and even today the object oriented paradigm has ruled the software development world. In a way, and as part of the end of the "one size fits all" paradigm (I also mentioned this trend here) we also see more pluralism for languages so we get more dynamic languages vs. static typed one and we find a place for functional languages and not just object oriented ones. (I'll expand more on that in another post).

Anyway, Yesterday S. Somasegar (MS VP of DevDiv) announced on his blog that F# will be "productized" :

"We will be partnering with Don Syme and others in Microsoft Research to fully integrate the F# language into Visual Studio and continue innovating and evolving F#. In my mind, F# is another first-class programming language on the CLR. "

Microsoft is of course not the center of the universe, but when a company like Microsoft chooses to bring something closer to main stream it is a significant move which, in my opinion, shows that functional programming is getting more traction.

 

One question I don't hear asked too much is "who tests the tests?" - after all we are writing all this additional code - if we write so many bugs in our production code that we need tests - what are the chances the test code is clean?

The current answer I have is that the code, the tests and the acceptance tests all test each other so if one fails we'll spot the problem in at least one of the others. I hope that this  it is a good enough answer... :)

What do you think?

 

Well it seems setting up a new company can keep you busy at times - which is my official excuse :) for the quiet last week. Hopefully I'll have a little more free time this week

Note: I am talking in this post about roles and capabilities. i.e. when I say architect I mean someone who has the capacity to be an architect (e.g. as demonstrated in the architect training suggestion I made) and takes that role. On a specific project you may have a person that performs multiple roles such as an architect and project manger or an architect and developer while  other project warrant one or more full time architects.

Not all projects need architects. There, I've said it. Not all projects need architects and I am not talking here just about trivial projects. There are cases (maybe even many cases) where you can get by with what I call "off-the-shelf" architecture - maybe with a few adjustments that any master developer (i.e. seasoned and experienced developer) can handle. For instance a lot of web-sites can do pretty well by using Model-View-Controller (or a derivative of that) along with a simple O/R mapper such as active-record. In fact a lot of them do when they use a framework like Rails that made these architectural choices for them*. Another example is the vanilla 3-tier architecture provided by vendors (such as this one by Microsoft). Yes, when you take something off-the-shelf the result might not be optimal but that doesn't mean it isn't sufficient. You just have to be aware for the tradeoffs...

Another point is that "design" is not an exclusive architect thing. A developer is not a good developer unless she also known about  proper design. mastery of technical details of the language without understanding the wider context of design will just help you code a lot of crap faster.

Having said that we need to consider a few issues - When do you need architects? What do they Do? What's their relation and interaction with the developers ?

When do you need architects?

It is sometime a fine line between a project that can get by with an "off-the-shelf" architecture and one that needs an architect. It would be nice if we could have something like a litmus test that would tell us if architects are needed or not. I don't have one. The closest thing I could get to this is something I call the SCLR test (pronounced scaler). SCLR stands for Size, Complexity & Limited  Resources.

• Size - well if you are going to have something  estimated at 1000 man-years or dozens of teams. I think it is pretty obvious that you can't just use something  which isn't made to fit. If anything there's a need to divide the work between the teams in a way that would make sense so that you wouldn't get a big-ball of mud. There's also a lot of need to coordinate the efforts and keep the big-picture inline. Personally I think that it doesn't  have to be a huge project to warrent some architects involvement. Since as Fred Brooks notes the number of interactions grows exponentially as we add more people.  In my experience trouble starts even with more modest numbers - more than 4 or even 3 different teams working concurrently is probably a good number to start thinking  about architects
• Complexity - There are many signs for complexity in a project. The vision statement can provide a hint. "Let's design the software to support the next mars mission", "best CRM platform ever"  - an ambitions project will not make-do with "average" architecture. Size (which  I already mentioned) is also a sign of complexity, while previously I talked about size of the project, the size of data, users etc. is also relevant when we're thinking about complexity . A lot of external interfaces is another sign. Integration doesn't seem very complicated, until you actually try to pull it off. When you have to do a lot of that in a project that's complex. And there are many other signs
• Limited Resources - Naturally every project has limited resources, but limited resources should be considered as a sign for architect involvement if the resources are extreme. When resources are extremely limited the tradeoffs that have to be done are more meaningful, which is why wed want people who can help with that (i.e. architects). For instance in a projects I worked on in the past we had a lot of availability and performance requirements on one hand but only so many "U"s in the rack and even limited electricity to make all this magic happen. This turned something which otherwise was a relatively standard IT project into something a lot more challenging.
  

Assuming I manage to convince you that some projects can't just choose one of the available blue-prints and need some more work - the next step would be to convince you that architects are better suited to solve this than developers. I'll try to do that in the next post on this series where I'll explain what (I think) architects do and their place in the development team

---

* Rails has more than just MVC and Active-Record but that isn't an important point for this discussion

 

Pete Lacey has a post called "What is SOA?" where he defines SOA as follows:

"

• Network Oriented Computing (NOC): An approach to computing that makes business logic available over the network in a standardized and interoperable manner.
  • Service Oriented Architecture (SOA): A technical approach to NOC that has a non-uniform service interface as its principle abstraction. Today, SOAP/WS-* is the chief implementation approach.
  • Resource Oriented Architecture (ROA): A technical approach to NOC that has an addressable, stateful resource as its principle abstraction. Today, REST/HTTP is the chief implementation approach.
• Business Service Architecture (BSA): An unnecessary term (also not an architecture) that tries to make the obvious something special. Aka, business analysis. Aka, requirements gathering"

I am sorry but I beg to defer.

The first thing to note (again) is the architecture vs. architecture style differentiation I mentioned in a previous post (You can see a similar definition by Stuart Charlton) Here is a quick reminder :

Software architecture is the collection of the fundamental decisions about a software product/solution designed to meet the project's quality attribute requirements. The architecture includes the main components, their main attributes, and their collaboration (i.e. interactions and behavior) to meet the quality attributes. Architecture can and usually should be expressed in several levels of abstraction (depending on the project's size).

An Architectural style is a blue print that can be used when you desing an architecture. An architectural style defines some of the components and thier attributes as weel as place constraints on how they can interact.

My claim is that SOA is an architectural style for distributed computing which puts extra emphasis on the interface (and hence gets the easier interoperability). Ok, if SOA is indeed an architectural style, we should be able to define it as a set of components, interactions and attributes. Well, I already did that a while ago (in a paper called "What is SOA anyway?"). And while it may not be perfect, I think it is a reasonable definition all the same:

"SOA is an architectural style for building systems based on interacting coarse grained autonomous components called services. Each service expose processes and behavior through contracts, which are composed of messages at discoverable addresses called endpoints. Services’ behavior is governed by policies which can be set externally to the service itself. "

You can see the above mentioned paper for a little more detail on each of the components.

ROA, in my opinion, is just a re-branding of REST so that it would be easier to discuss it as an architectural style and not connect it to the HTTP implementation - which is what  a lot of REST proponents are doing.

By the way, as I pointed out before, there are a few other important architectural styles that are related to distributed systems like Event driven architecture, Spaced based architecture, peer-to-peer etc.

As for "Business Service Architecture" - I personally like to think about that as "SOA initiative" as in the strategic decision to try to implement an SOA in an organization while trying to achieve the more nebulous traits like business and IT alignment etc. (which is why it is nether architecture nor architecture style)

 

In a recent post Steve Vinoski said:

"Frankly, if I were an enterprise architect today, and I were genuinely concerned about development costs, agility, and extensibility, I’d be looking to solve everything I possibly could with dynamic languages and REST, and specifically the HTTP variety of REST. I’d avoid ESBs and the typical enterprise middleware frameworks unless I had a problem that really required them (see below). I’d also try to totally avoid SOAP and WS-*."

It is easy to dismiss this as just another yahoo who goes against conventional wisdom until you remember that Steve spent more than a decade working in Iona in leading roles like Chief Engineer of product innovations and helped develop some of the middleware standards for OMG and W3C.

Well, I guess that's becoming an epidemic  now :)  just recently we had Michael Stonebreaker talking about the RDBMS demise, Pat Helland talking about life beyond distributed transactions.  and now Steve on ESBs.

That trend aside, I think Steve is doing throwing the baby out with the bath water. The dream of a single infrastructure for an enterprise is ludicrous enough (Remeber Peter Deutsch and the "The network is homogeneous" fallacy). but if you drop the "E" from the ESB moniker you get a valuable middleware which is very usable in many situations and not just legacy system integration. For instance one thing that is missing form "HTTP variety of REST" implementation is reliable messaging. location transparency is  harder to solve with HTTP etc.

Another problem I have with the current approach of Steve is that he is replacing one dogma (EBSs are good) with another (ESBs are bad use Ruby, REST) - this is not a healthy approach. The solution should match the problem, that's probably the primary reason why we need architects after all

 

In the previous post on the subject I promised to expand a little more on the suggested content for "Distributed Systems Architectures Workshop" so here a short drill-down:

Even though most of the time should be spent on working, designing and evaluating architectures there's probably a little room for theory .

Module 1 The basics (probably not more than half a day)

What's software architecture
The software architect role
Activities
Scenario based architectural design
(documenting software architectures)
Agile SDLC and architects

Module 2 Distributed Systems background

Understanding the Fallacies of distributed computing

Distributed architectures styles - it is important to understand the different architectural styles that can be used to implement distributed system - whithin this topics like clustering, computation and data grids, messaging , publish subscribe etc should also be discussed

• Client-server - The most basic distributed architectural style. It is based on the  N=1 premise and isn't fit for most of today's challenged. However it is still an option for some types of projects.
• Pipe and Filters - not necessarily a distributed style, but it can be applied in distributed space
• N-Tier - That's actually a moniker to anything where N>2 but usually it pertains to 3-tier architecture (front-end, server, database) or the internet 4-tier version (client, webserver, application server, database). 
• Event Driven Architecture
• Service Oriented Architecture
• REST 
• Space-based architectures - like JavaSpaces  and its implementations like Blitz (open source) and Gigaspaces (commercial)
• Peer-to-Peer - you know that's what all those file sharing tools use

Distributed Consensus

• 2 phase commit - used by XA and COM+ distributed transactions
• 3 phase commit - considered a non-blocking protocol (vs. 2PC which is a bloging protocol) 
• Paxos commits
• Sagas
• Eventually consitent (BASE) - Basically Available Scalable/Soft state & Eventually Consistent. An alternative to distributed transactions used by a lot of internt-scale companies (see a post I made on ebay's architecture )

Module 3 - workshop - most of the days should be focused on actually working to design architectures.
I would think that this would be handles best by working  in groups. e.g. having each group focus on one architecture style.
 
The groups would be given a scenario which covers some architectural concern (integrity, performance, scalability, availability etc.)  and would try to design strategies to handle the scenarios within the constraints of the architectural style. Present that to the other groups and then have a facilitated discussion on the pros/cons of each strategy. The  scenarios should be based on a large enough story to allow meaningful architectures to emerge (e.g. you can see the  10 scenarios in my SOA Patterns presentation )


Any comments or other ideas for what's needed for this kind of a workshop are welcomed

 

It seems that even the smartest people can get the difference between architcture, architecture styles and technology wrong
For instance Anne Thomas Manes points out the Roy Fielding makes this mistake in his REST and Relaxation presentation by mixing an architectural style with technology:

 "Roy is equating SOA with web services. Although a lot of folks use web services to implement services, that's simply an implementation decision"

But then procede to make the exact same mistake 

"So when watching Roy's presentation, replace the term "SOA" with "WS-*", and the discussion will make a lot more sense."

REST is an architectural style you can implement it with WS-* which is a technology. It is not the most natural way to use WS-* standards but it is doable.

Looking at the same context (i.e. Roy Fielding's presenation) Steve Jones makes a similar mistake confusing Architecture and Architecture style.

My definition for software architecture is

Software architecture is the collection of the fundamental decisions about a software product/solution designed to meet the project's quality attribute requirements. The architecture includes the main components, their main attributes, and their collaboration (i.e. interactions and behavior) to meet the quality attributes. Architecture can and usually should be expressed in several levels of abstraction (depending on the project's size).

An Architectural style is a blue print that can be used when you desing an architecture. An architectural style defines some of the components and thier attributes as weel as place constraints on how they can interact.

For instance, the REST constraints (taken from Anne's post mentioned above) are:

"Uniform Interface:

• Resources are identified by only one resource identifier mechanism
• Access methods (actions) mean the same for all resources (universal semantics)
• Manipulation of resources occurs through the exchange of representations
• Actions and representations are exchanged in self-describing messages

Hypertext as the engine of state:

• Each response contains a partial representation of server-side state
• Some representations contain directions on how to transition to the next state
• Each steady-state (page) embodies the current application state"

Architecutre Styles can be combined to create new architectural styles. Roy Fielding demonstrates this in his famous dissertation  where he demonstrate how REST is a composition of several styles such as  Client/Server, Layered system, Stateless etc. As another example (which a lesser degree of precision) I take about enhacing SOA with EDA in "bridging the gap between BI and SOA"

The last piece of the puzzle is technology. Technology (in the software context) are set of tools provided by a vendor to enable and support building software solutions. As I've said here numerous times, technologies has their own internal architectures (as they are software solutions themselves) which is why different technologies support different architectural styles and why the alignment of the technology with the architecture chosen for your solution is important.

Yes this post is all about semantics - but clear meanings are important to prevent confusion, at least in my opinion anyway

 

A side effect of my decision not to become an independent consultant at this time means that I have to shelve some of the projects I was considering. One of these projects was to create a training program for software architects which I was discussing with a couple of training centers here in Israel.
Since It seems I am not going to promote it, I thought I'd share what I think a training program for .NET/Java architects should look like in the hope that someone would find it useful and do promote it (or parts of it)

Soft Skills
The way I see it architects needs a bunch of soft skills to be able to perform their roles.
Here is the list I identified in the past (by the way, I began a series of posts on each of these skills and never got to finish it - maybe it is time that I will :) )

• Leadership. Influencing others to accomplish tasks and following your guidance
  
• System thinking. Understand decisions and constrains in the
  wide scope pertaining to whole of the solution at hand. This includes
  the ability to abstract problems.
  
• Strategic thinking. Understanding decisions and constrains and their alighments to the overall business of the company.
  
• Organizational politics. Understand the environment you operate in and how it influences you.
  
• Communications. Making sure you get your point across.
  
• Human relations. Understand the "people" aspects or human
  factors and dynamics. This includes things like negotiation, pragmatism
  etc
  

I am not sure if you can teach all of them, but few courses that can help (in my opinion) include:

• Negotiation/Persuation - Anything based on "Getting to yes","Getting past no" and "The power of a positive no" should be fine.

• Presentation Skills - While getting the architecture and technology right is what matters, if you can't explain it to the different stakeholders you're toast.

• Strategic Planning - This has to do with the vision thing I expect architects to manifest. Note that having a vision should not be confused with future-proofing a solution. future-proof means excess work not needed. Having a vision is knowing where you want to end - it can still be perfectly valid to completely re-write your applications along the way

Project Management
While the architect is not the project manager (mostly anyway), I think understanding the constraints coming from the project management point of view is very important. Since most environments call for a mix of agile and formal disciplines (hey, you've got to be pragmatic). I would train architect both in SCRUM and RUP (or some other formal methodology)

Also while not all environments needs this I would give an 2 days overview of important standards. The first would be IEEE 1471, which defines a standard for documenting software architectures. I would also teach ISO 90003 and CMMI.

It should be noted that the ISO 90003 is much better than the previous incarnation (ISO 9003) as it basically lets you define what you want to do to cover the different areas. The standard just helps you make sure you think about the various parts of project management (requirements, environment etc.). For instance I demonstrated how key areas of 90003 can be mapped to SCRUM to get it approved on my last project.


Languages, Design  and Patterns

I would want the .Net/Java architect training program to include at least 2 of the following languages:
Ruby, Scala, Erlang, F#, Python , Groovy ,OCaml
The reason for this is that these languages have different design goals than .NET and Java so learning them gives you additional perspectives and broaden your horizons for other ways of thinking (even if you don't use them in your project directly). You might have noticed that there's no .NET or Java training here. The reason for that is that's a prerequisite as far as I am concerned. You should master at least one

Object Oriented principles - hopefully aspiring architects already know this. However, I often see people who discovered some of the principles by themselves but haven't heard about all of them.
I am talking about principles such as Liskov Substitution  Principle , Open Closed Principle, Single Responsibility Principle , IoC containers, Don't Repeat Yourself  and YAGNI (I summarized my opinion on most of them in this paper)

The next step is to cover some design issues like Domain Driven Design, UI Design, Database modeling, Database alternatives l(after all the database is dead, right? )

Advanced design patterns - When most people hear the term design patterns they think about the GoF patterns. There are however literally hundreds of design patterns. Some of them are even worth learning :) . For instance there are patterns for concurrent and parallel systems like Proactor, Reactor Half-sync/Half-Async etc; Workflow patterns like Cancelling partial Join, Recovery Action etc; SOA patterns (ok, so I am still working on that :) )



Architecture Workshops
Another important part of the training, in my opinion, is to do some workshops and actually try to apply some of the material covered.

• Architecture Evaluation - workshop 2-3 days - It is probably worthwhile to delve a little on scenario based evaluation techniques such as LAAAM and ATAM. While I prefer evaluating architecture in code, the scenario based thinking is very valuable for eliciting architectural requirements
• Distributed Systems Architectures workshop - I'll expand on this in the next post

Lastly, there are also a few miscellaneous subjects like architect 101, the SPAMMED architecture framework , Agile architecture, Behavior Driven Design , common frameworks (though hopefully this would  not be needed ) like Spring/Spring.Net, Hibernate/NHibernate, iBatis  etc.


PS

Note that there are a few architect training programs available out there
One is offered by the Software Engineering Institute (SEI) and includes a 6 courses. SEI program seems to be focuses on formal sides of architecture as it includes courses on documenting software architecture and ATAM (You can see an old presentation I have on ATAM here)
Dana Bredemeyer  also offer architect training. Dana offers several workshops that cover the software architecture profession.
TOGAF (which is more of an enterprise architecture framework) offers both a certification and courses
Lastly, IASA is considering creating a software architect program and has a few courses in development
If you know any others I'd be happy to hear about them

 

You may have read about the guy who after spending 2 years on a Ruby on Rails project switched back to PHP. Without getting into the debate of whether Ruby on Rails is better than PHP or wether Ruby is overhyped (probably - but that doesn't mean it isn't any good either.By the way  it is the same with SOA, but I digress)

Reading his post I saw a few quotes that raised a red flag for me such as:

"But at every step, it seemed our needs clashed with Rails’ preferences. (Like trying to turn a train into a boat. It’s do-able with a lot of glue. But it’s damn hard. And certainly makes you ask why you’re really doing this.)"

and

#2 - OUR ENTIRE COMPANY’S STUFF WAS IN PHP: DON’T UNDERESTIMATE INTEGRATION
By the old plan (ditching all PHP and doing it all in Rails), there was going to be this One Big Day, where our entire Intranet, Storefront, Members’ Login Area, and dozens of cron shell scripts were ALL going to have to change..

and

Speaking of tastes: tiny but important thing : I love SQL. I dream in queries. I think in tables. I was always fighting against Rails and its migrations hiding my beloved SQL from me.

What these quotes say really is that this guy doesn't know about "Technology Mapping 101". Here is what I wrote about  technology mapping*  about 2 years ago (incidentally that's about the same time this guy started his Ruby adventure :) )

"Technology mapping can have a significant impact on the ability to actually implement the architecture. The wrong mapping can make adhering the architectural guidelines very cumbersome and sometimes nearly impossible."

Every technology or framework has its own architecture. This architecture poses constrains and makes certain things easy (like using the ActiveRecord pattern in Rails) and certain things harder (like not using O/R Mapping ) so, for instance, on the case mentioned above a better technology mapping might have been RBatis (iBatis for Ruby/Rails) which lets you map SQL statements to objects. It is important to note (in Rails case) that one of the core tenets for Rails is preferring convention of configuration when you don't do that you have to work hard(er) - as you are working against the framework

Another problem with the technology mapping here was his point #2. It is a pity he only saw it in after the fact. It can be justifiable to switch everything but you've got to allow this change to occur iteratively. While I generally dislike the software architecture = building architecture metaphor, using it here does make sense: building software for an existing business (vs. greenfield development) is like building a new intersection. You have got to think about building all the detours that would allow the traffic (business) to continue to operate, sure it might run slower in the intermediate phase but it can't stop altogether.

So again, Once you have an architecture that fits your business, take a look at the technology options you have. try to choose the best fit. Whatever you choose - take a look at the implications of that technology and think about the tradeoffs you may find that you either have to adjust your architecture or change the technology altogether - if you don't you might find you waited 2 years of development effort or even more..

---

* Technology Mapping is one of the steps of a set of activities I identified as needed to make sure your have a solid architecture. The activities goes by the acronym SPAMMED and you can read about them more in this article and/or this presentation

 

There were a few threads about whether SOA is about the technology or not.

In my opinion SOA and Architecture in general  are never about the technology - technology is important but it is just one variable in the equation. What we are looking for is a way to satisfy as many of the business needs as we can under all the constraints we face.

For instance, a few days ago I got a question in my email box from someone calling himself coldplay. While I don't think the band has somehow got itself interested in Event Driven Architecture, the question itself looked interesting enough. Here is the situation:

Current Setup
Lets take a e.g of a Inventory Stock Reorder point exception with in heterogeneous apps environment(No-SOA and integrations)...
The exception definition was built into the source apps and when the stock dropped below reorder....event registered and led to a exception. Exception was further handled by a rules based engine and a workflow notification raised ..

Planned Setup
Same e,g as above .. Post SOA implementation.. Inventory management is composite service built by orchestrating collaborative services from SAP and Oracle...which have different data model supporting them...
The exception definition requires to be defined outside the native Oracle apps and might have to get some event related information from SAP web service also .. to arrive to a conclusion as to whether this really is a Exception or not ..


Possible Technical approaches:

• Data Persists somewhere in the processing the Exception
• Data doesnt persist
• In mem database used..


My Question now :

1. What do you advise to be used in EDA?  which would reduce network round trips, decrease apps server loads from the above 3 technical outlooks.
2. What is diff between in-mem db and usual processing of apps logic by a apps server

I feel :

• Data persistence would lead to larger commit times and reduce operational efficiency
• If the data doesnt persist... and all validations are executed on the fly... dont you think the current apps servers would die processing ... or if its processing capacity is increased .. is it going to be economically viable alternative.

To be quite honest I can't really answer the questioned asked because the question lacks the business context -
what are  the implications if events are  missed or lost? What's the acceptable latency that would allow the business to operate properly ? if the Oracle bits and SAP bits need a lot of data from each other - then maybe the whole service partitioning is wrong and the services are not cohesive enough? How many business events are expected anyway? How often? and the list goes on.

Once you answer the business questions you can look at the available technology portfolio and ask whether you would want a in-memory Database or maybe a datagrid would be a better option? and even then the decision is not just technology driven since when you do cost/benefit analysis you need to take into account purchasing costs, operational costs, skillset of the dev team, time to implementation etc.

This is not to say that if you choose a technology that isn't aligned with your architecture you should reconsider the architecture (or technology). Also since each technology product brings into the table its own architecture (with its own constraints and decisions) you probably need to verify the architecture once you make technology choices. but still, at the end of the day it is the business needs that sits in the driver sit, the rest is just tagging along for the ride.

 

Finally, I've quit my current position - to take a new job as VP R&D at (what I think is) a very promising start-up called PaperLnx.
 
In a nutshell we are going to build few solution in the "physical world connection". What's "phisical world connection" you ask? Well, here's a nice definition from "The Pondering Primate":

"When a display was added to the first mobile phone, a new media was created. Since then, Internet connection and a camera have been added that have created a new way to interact with the physical world.

Soon speech recognition will allow an additional way to browse the physical world too.

Every physical object will have a physical world hyperlink

That means every physical object will allow connection to a designated website and the mobile phone with it's psychical world browser will be able to surf the "real world", the physical one."

As can be expected from the "long tail" curve we are not alone in this space (the link above has a nice list of other companies).  However I think we have a competative edge... :)

Incidentally, I am looking for "a few good men" (or women) to join us. We need both developer and QA positions:

Senior Server Developer

We are looking for a highly motivated, resourceful and intelligent developer. Good interpersonal and communication skills will be very appreciated. A Team player. Broad thinking and problem solving capabilities are also desired.

• At least 5 years of server side development with thorough understanding of Object Oriented principled and  understanding of architectural styles and design patterns.
• Experience in multithreading and in distributed systems.
• .NET/Ruby experience
• Integration with C++ components
• Experience with O/R Mappers such as nHibernate/ActiveRecord etc.
• Video processing experience/ familiarity with video related protocols (H.324m, H.323 etc.)  a plus
• Mobile Internet backend development a big advantage
• Understanding of architectural constraints (security, availability, scalability etc.) for internet scale platforms a big advantage
• Team leading skills – an advantage.

Senior Web Developer

We are looking for a highly motivated, resourceful and intelligent developer. Good interpersonal and communication skills will be very appreciated. A Team player. Broad thinking and problem solving capabilities are also desired.

• 3 years web development experience is a must, 5+ years experience a big plus.
• Experience with web-related technologies Javascript, HTML, CSS, AJAX
• .Net/Ruby experience.
• Understanding of Object Oriented principles and REST architectural style.
• Experience with web MVC frameworks like MonoRail/RoR
• Experience with O/R Mappers such as nHibernate/ActiveRecord etc.
• Familiarity with SQL 2005/mySQL. Knowledge of SQL an advantage. 
• OLAP, datamart experience an advantage
•  Team leading skills – an advantage.

QA Person

We are looking for a highly motivated, creative and intelligent QA person. Good interpersonal and communication skills will be very appreciated. A Team player.

• Experience testing WEB  applications.
• 
  
• -Testing creativity. Understanding of Test related methodologies
• -Experience designing and controlling all the stages of the project's life cycle.
• -Experience with test environments such as FIT, FITNESSE or at least other automated tools- advantage.
• -Knowledge of SQL an advantage.
• -Dot-Net/Ruby/Python/Powershell - advantage.
• - Cellular web testing experience a big advantage
• -At least 2 years experience in the field

If you think you qualify, located in Israel and you are interested in changing  the world :), you can send your CV to me
 

Another REST related post - this time I want to share a couple of observations I had after reading (Roy's presentation from RailsConf 2 days ago via Pete Lacey) and listening to Roy T. Fielding's presentations.


The first point has to do with a question which is sometimes raised whether you can do REST without HTTP. i.e. can you have a RESTful architecture  if you don't use the http protocol and further more not using the http verbs (GET/PUT/HEADER etc.) or  as the unifier interface. I talked about it a while ago and I think you can. listening to Roy's talk  it seems that, at least in http architect's opinion the answer is yes as well.

Another point that occurred to me, watching Roy's talk, which is related to the "REST magic" post I wrote a little over a week ago. The use of a uniform interface is tauted by REST proponents (and Roy himself) as coupling reducing formula. After all if you use a uniform interface you are not coupled to the particular semantics of any resource/service you already know the capabilities (actually the maximal capabilities) they offer. What ensues is that instead of using a lot of verbs (ReserveRoom, UpdateOrder etc.) you use a lot of nouns (/rooms/, /orders/order1 etc.)

This works extremely well on the "human" web where my browser can navigate to any-ol'-site without any prior knowledge of what's the site about. When I navigate to Amazon I can buy stuff, when I navigate to New York Times I can read stuff etc. The problem here is the browser is really dumb about what's going on. I, as a human using the browser, understand the context from the content (well, most of the time anyway;) ) so the browser can remain decoupled.  However when you translate it to the "programmable" web you usually don't have some mighty AI engine examining the response to understand the context - instead what you do is trade the verb coupling, which with WS-* web-services would be defined in a contract, you are now coupled to the nouns ( this is not to say the nouns aren't discoverable - since they are due to the hypertext or document orientation communication REST encourages). The end result is pretty similar to what you get when you use verb based contracts your software still needs to understand (where "understand" means some level of coupling) what it is doing with the "other" services. not to mention that you still need to understand the content of the message (sorry- response) to do anything useful with it.

In any event, while loose coupling is very desirable, we also need to remember that the only way to truly achieve complete decoupling is to not connect components. So some coupling is always needed if we want to produce meaningful systems.

What do you think?

 

If there's one reason to go to ApacheCon 07 in Atlanta, then it's probably Roy T. Fielding's "a little REST and Relaxation"

Here is the abstract:

"Representational State Transfer (REST) is an architectural style that I developed while improving the core Web protocols (URI, HTTP, and HTML) and leading them through the IETF standardization process. I later described REST as the primary example in