A few weeks ago Google held a developer day here in Israel. Yesterday I got notice that the presentations and slides are now online (all the presentations are in English)

• I thought that the talk on AppEngine (by Jon McAlister) is worth watching - Google's offering in the cloud computing arena (some of the competition Azure is up against for you  MS buffs out there :) ).
• Thomas Steiner introduction to GWT 
  
• Chewy Trewhella talk about mushups (with examples of using Google's services)
  
• and event the Keynote is nice (starts around the 11th minute of the video) on opennesss and Google's offering
  

 

Back in may Andrew Binstock wondered "is the popularity of unit tests waning?" Andrew lists 5 indications he observed to support his conclusion:

1. Commercial tools are waning
2. Few OSS products
3. Unit testing is not taught by major Java instructors
4. Few books get published on unit testing
5. xUnit alternatives are completely invisible.

I am not sure I agree with all these observations, nevertheless,in a followup article in Java World Andrew continued to ask "Is unit testing doomed?".
Andrew says that two issues with unit testing are that they seem to offer little value and that the evangelism of unit testing in general and TDD in particular also causes antagonism (I can say from personal experience with introducing unit testing to long time C and C++ developers that, indeed, the initial responses are rather cold)

Andrew also quotes former CEO of Agitar (Jerry Rudisin) saying that "unit testing hasn't taken off as mainstream practice". A survey ran by Scott Ambler for Dr. Dobb's also found that "The more difficult practices, such as TDD and executable specifications, have lower adoption rates than easier practices such as daily scrum meetings." (although another survey by VisionOne found that within companies where agile practices are used 77% use unit testing and about 49% use TDD)

I recently read another interesting post on the subject, this time by Dan North about the "end of endotesting" explaining why the record-reply idium of most mocking framework is not intuitive and more easy to use mocking frameworks like mockito (his example) or Moq and Mocha (my examples) are better.

Now, yesterday I read Roy Osherov's post "Goodbye mocks, Farewall stubs". Roy makes similar observation on the adoption (or lack therof ) of unit testing practices. Roy believes the number one reason holding back unit testing is the learning curve associated with it and suggests we simplify the tools and terminology (and gives a example similar to Dan's mentioned above)

What we see is that both in the Java and the .NET worlds unit tests adoption hit a few snags. The way I see it there are event  additional barriers to wide adoption of unit test and TDD but I'll talk about that in part II where I'll also try to talk a little about what can/should  be done

Powered by ScribeFire.

 

Great news. Two of my friends and fellow DDJ bloggers, Eric Bruno and Udi Dahan have agreed to join my (now ours) SOA Patterns book which will be published by Manning.

Both Udi and Eric are competent and experienced architects who have experience designing SOAs . On the technology side -  Udi (“The software simplest”) specializes in .NET development e.g. his nServiceBus framework – which is a very good example for an endpoint-ware ServiceBus (vs. middleware ServiceBuses which is what most ESBs do).And  Eric, on the other hand, is a Java and C++ expert . Eric is the author of Java Messaging (one of the best books on JMS and web services ) and has also has a lot of experience in Financial systems. Together, the three of us bring a lot of real-life experience of building large and complicated system into this project.

The current game plan is for Eric to focus on the SOA pitfalls (“anti-patterns”) part of the book, Udi to provide a “putting-it-all-together” chapter , and for me to cover what’s left. I am sure however, that their experience and insight will also help make the other parts of the book (even? ) better.

If you are not familiar with the book - you may want to take a look at the first chapter and/or some of the published patterns like Saga, Service Firewall, Gridable Service, Edge Component and (a very early draft of) Aggregated Reporting pattern . Also you can take a look at the slides from my "SOA Patterns" presentation at Dr. Dobb's Architecture & Design world last year, which illustrates some additional patterns

 

I am currently interviewing people for developer positions for my startup (PaperLnx) and as I presented my plans to use multiple languages to one of the candidates he immediately retorted with "why on earth would you want to do that?"

Why indeed?
 
As I mentioned in the previous post about F# and Erlang it seems to me that the age of "one size fits all" is ending  in a lot of areas. I think that it is also happening for languages.

When .NET was announced, one of the messages by Microsoft was that .NET is the platform and that you can choose any language you want to develop in it. Microsoft also worked with partners to back up this statement and  release a lot of languages for the platform (e.g. Eiffel#, Fujitsu NetCobol  etc.) vs. Java which was a single language with a "write once, run everywhere" vision. Microsoft itself released 2 manages languages VB.NET C# and managed extensions for C++. The C++ managed extensions were very awkward to work with  and only people  who really had to use it. the non-MS languages seemed to be niche languages for legacy integration and not much more. VB.NET (we're talking .NET 1.0 here) seemed like something to try to distract VB heads while Microsoft was pulling the rug beneath their legs. Which basically left C# as the only viable language to develop in if you were serious about .NET (at least in my eyes). The "many languages" vision seemed like something to hide the (obvious) fact that Microsoft had to come up with an answer to Java (especially since around the same time Sun also sued Microsoft over the J++ and JVM implementations)

5-6 years later the situation is a little different. For one, we see that the JVM is not just about Java anymore. Just like the original .NET vision, the JVM has become a platform supporting many different languages Jython, JRuby, Scala, Groovy etc.
The .Net platform is also getting more interesting  languages like  Boo or   Gardens Point Ruby.Net . .NET is also being extended by Microsoft itself (with IronPython, IronRuby, F# ).

Two interesting things we can see are one, suddenly the languages are cross platform. e.g. I can program in python and deploy it natively, .NET or on a JVM - so in a way you get some platform independence.

The other and more significant issue (in my eyes) is that the languages available on each platform are different enough in meaningful ways - to a point where it is getting worthwhile to use more than one language in projects. Here are few examples from around the blogsphere that demonstrate this:

• Neal Ford shows an example using the (excellent) Mocha mocking library (in JRuby for testing Java classes. So the dynamic properties of Ruby help make the testing simpler for the statically typed Java.
• The BOO Build System (via Ayende) - which is about building a DSL for builds (a la rake). again we see taking advantage of the properties of a dynamic language this time to help with .Net languages.
• (not) Dennis Byrne shows how you can call Erlang from Java - which demonstrate taking advantage of the concurrency features of Erlang from Java.

When it comes to the difference between general purpose languages (i.e. Java, C#, VB.NET) we mainly see syntactic nuances. When it comes to dynamic languages vs. static languages vs. functional languages etc. the differences are more meaningful. What has changed now is that both platforms and open standards (e.g. REST over HTTP/WS-*) enable better integration and increase the motivation to use the "best tool for the job" approach over "the everything looks like a nail" approach .

I think the time for language pluralism is ripe - In my solution this would probably translate to a mix of C/C++, F#, C# and Ruby - What about your projects?

 

In the previous post on the subject I promised to expand a little more on the suggested content for "Distributed Systems Architectures Workshop" so here a short drill-down:

Even though most of the time should be spent on working, designing and evaluating architectures there's probably a little room for theory .

Module 1 The basics (probably not more than half a day)

What's software architecture
The software architect role
Activities
Scenario based architectural design
(documenting software architectures)
Agile SDLC and architects

Module 2 Distributed Systems background

Understanding the Fallacies of distributed computing

Distributed architectures styles - it is important to understand the different architectural styles that can be used to implement distributed system - whithin this topics like clustering, computation and data grids, messaging , publish subscribe etc should also be discussed

• Client-server - The most basic distributed architectural style. It is based on the  N=1 premise and isn't fit for most of today's challenged. However it is still an option for some types of projects.
• Pipe and Filters - not necessarily a distributed style, but it can be applied in distributed space
• N-Tier - That's actually a moniker to anything where N>2 but usually it pertains to 3-tier architecture (front-end, server, database) or the internet 4-tier version (client, webserver, application server, database). 
• Event Driven Architecture
• Service Oriented Architecture
• REST 
• Space-based architectures - like JavaSpaces  and its implementations like Blitz (open source) and Gigaspaces (commercial)
• Peer-to-Peer - you know that's what all those file sharing tools use

Distributed Consensus

• 2 phase commit - used by XA and COM+ distributed transactions
• 3 phase commit - considered a non-blocking protocol (vs. 2PC which is a bloging protocol) 
• Paxos commits
• Sagas
• Eventually consitent (BASE) - Basically Available Scalable/Soft state & Eventually Consistent. An alternative to distributed transactions used by a lot of internt-scale companies (see a post I made on ebay's architecture )

Module 3 - workshop - most of the days should be focused on actually working to design architectures.
I would think that this would be handles best by working  in groups. e.g. having each group focus on one architecture style.
 
The groups would be given a scenario which covers some architectural concern (integrity, performance, scalability, availability etc.)  and would try to design strategies to handle the scenarios within the constraints of the architectural style. Present that to the other groups and then have a facilitated discussion on the pros/cons of each strategy. The  scenarios should be based on a large enough story to allow meaningful architectures to emerge (e.g. you can see the  10 scenarios in my SOA Patterns presentation )


Any comments or other ideas for what's needed for this kind of a workshop are welcomed

 

A side effect of my decision not to become an independent consultant at this time means that I have to shelve some of the projects I was considering. One of these projects was to create a training program for software architects which I was discussing with a couple of training centers here in Israel.
Since It seems I am not going to promote it, I thought I'd share what I think a training program for .NET/Java architects should look like in the hope that someone would find it useful and do promote it (or parts of it)

Soft Skills
The way I see it architects needs a bunch of soft skills to be able to perform their roles.
Here is the list I identified in the past (by the way, I began a series of posts on each of these skills and never got to finish it - maybe it is time that I will :) )

• Leadership. Influencing others to accomplish tasks and following your guidance
  
• System thinking. Understand decisions and constrains in the
  wide scope pertaining to whole of the solution at hand. This includes
  the ability to abstract problems.
  
• Strategic thinking. Understanding decisions and constrains and their alighments to the overall business of the company.
  
• Organizational politics. Understand the environment you operate in and how it influences you.
  
• Communications. Making sure you get your point across.
  
• Human relations. Understand the "people" aspects or human
  factors and dynamics. This includes things like negotiation, pragmatism
  etc
  

I am not sure if you can teach all of them, but few courses that can help (in my opinion) include:

• Negotiation/Persuation - Anything based on "Getting to yes","Getting past no" and "The power of a positive no" should be fine.

• Presentation Skills - While getting the architecture and technology right is what matters, if you can't explain it to the different stakeholders you're toast.

• Strategic Planning - This has to do with the vision thing I expect architects to manifest. Note that having a vision should not be confused with future-proofing a solution. future-proof means excess work not needed. Having a vision is knowing where you want to end - it can still be perfectly valid to completely re-write your applications along the way

Project Management
While the architect is not the project manager (mostly anyway), I think understanding the constraints coming from the project management point of view is very important. Since most environments call for a mix of agile and formal disciplines (hey, you've got to be pragmatic). I would train architect both in SCRUM and RUP (or some other formal methodology)

Also while not all environments needs this I would give an 2 days overview of important standards. The first would be IEEE 1471, which defines a standard for documenting software architectures. I would also teach ISO 90003 and CMMI.

It should be noted that the ISO 90003 is much better than the previous incarnation (ISO 9003) as it basically lets you define what you want to do to cover the different areas. The standard just helps you make sure you think about the various parts of project management (requirements, environment etc.). For instance I demonstrated how key areas of 90003 can be mapped to SCRUM to get it approved on my last project.


Languages, Design  and Patterns

I would want the .Net/Java architect training program to include at least 2 of the following languages:
Ruby, Scala, Erlang, F#, Python , Groovy ,OCaml
The reason for this is that these languages have different design goals than .NET and Java so learning them gives you additional perspectives and broaden your horizons for other ways of thinking (even if you don't use them in your project directly). You might have noticed that there's no .NET or Java training here. The reason for that is that's a prerequisite as far as I am concerned. You should master at least one

Object Oriented principles - hopefully aspiring architects already know this. However, I often see people who discovered some of the principles by themselves but haven't heard about all of them.
I am talking about principles such as Liskov Substitution  Principle , Open Closed Principle, Single Responsibility Principle , IoC containers, Don't Repeat Yourself  and YAGNI (I summarized my opinion on most of them in this paper)

The next step is to cover some design issues like Domain Driven Design, UI Design, Database modeling, Database alternatives l(after all the database is dead, right? )

Advanced design patterns - When most people hear the term design patterns they think about the GoF patterns. There are however literally hundreds of design patterns. Some of them are even worth learning :) . For instance there are patterns for concurrent and parallel systems like Proactor, Reactor Half-sync/Half-Async etc; Workflow patterns like Cancelling partial Join, Recovery Action etc; SOA patterns (ok, so I am still working on that :) )



Architecture Workshops
Another important part of the training, in my opinion, is to do some workshops and actually try to apply some of the material covered.

• Architecture Evaluation - workshop 2-3 days - It is probably worthwhile to delve a little on scenario based evaluation techniques such as LAAAM and ATAM. While I prefer evaluating architecture in code, the scenario based thinking is very valuable for eliciting architectural requirements
• Distributed Systems Architectures workshop - I'll expand on this in the next post

Lastly, there are also a few miscellaneous subjects like architect 101, the SPAMMED architecture framework , Agile architecture, Behavior Driven Design , common frameworks (though hopefully this would  not be needed ) like Spring/Spring.Net, Hibernate/NHibernate, iBatis  etc.


PS

Note that there are a few architect training programs available out there
One is offered by the Software Engineering Institute (SEI) and includes a 6 courses. SEI program seems to be focuses on formal sides of architecture as it includes courses on documenting software architecture and ATAM (You can see an old presentation I have on ATAM here)
Dana Bredemeyer  also offer architect training. Dana offers several workshops that cover the software architecture profession.
TOGAF (which is more of an enterprise architecture framework) offers both a certification and courses
Lastly, IASA is considering creating a software architect program and has a few courses in development
If you know any others I'd be happy to hear about them

 

Another great presentation at Architecture & Design world was Neal Ford's presentation on Domain Specific Languages (DSLs) . As the title suggests Neal gave examples both in static languages (Java) and Dynamic ones (Groovy, Ruby).

One interesting observation Neal made was that humans tend to create DSLs in real life whenever they (ok, we :)) have any non-trivial interaction or behavior. Neal gave sevaral examples such as the Starbuck's order taking ("Venti Iced Decaf with whip...") , musicians and a few others.

The next important point was contrasting ("classic") APIs and DSLs. The main difference is that the context is implicit and not repeated

A key  technique for building DSLs Neal mentioned was Fluent Interfaces. Fluent Interfaces means modeling the API so that lines of code are readable English-like sentences. The fluency comes from the easier readability by the interface user.

Fluent Interfaces, now that's a novel idea - what would a fluent interface look like, hmm, wait, I have an idea. Here are 3 samples that come to mind

DIVIDE x BY z GIVING y ROUNDED

INSPECT data REPLACING ALL "foo" BY "bar

READ someFile AT END SET eof TO TRUE

If you haven't guessed  the statements above  are in ...Cobol (by the way pardon the caps that's Cobol conventions..)
So ok, it isn't a new idea, but it is interesting to see it is making a comeback
Anyway one area where we see a lot of fluent interfaces emerging is configuration (mainly as an alternative to those lengthy XML files). For instance the following is an excerpt of configuring  Restlet components  (taken from my Edge Component pattern paper):

Builders.buildContainer()
            .addServer(Protocol.HTTP, portNumeber)
            .attachLog("Log Entry")
            .attachStatus(true, "webmaster@mysite.org", "http://www.mysite.org")
            .attachHost(portNumber)
            .attachRouter("/orders/[+")
            .attach("/getAll$", getAllRestlet).owner().start();
            .attach("/getLast$", getLastOrderRestlet).owner().start();


Note that this example also uses another fluent interface/DSL technique which is method chaining.

Dynamic languages make it even easier to write DSLs since they provide a lot of extension capabilities (see my previous post on OCP in Ruby), are less strict about types, allow reopening classes etc. 
Oneexample for a Ruby DSL is RSpec which is a framework to support Behavior Driven Development (BDD) in Ruby - The example below shows an excerpt for defining specifications for an eight-ball game

require 'eight_ball'

describe Eight_ball do
    before(:each) do
        @eight_ball=Eight_ball.new
    end
    .
    .
    .
    it "should lose if 8-ball sinked in pocket other than called" do
        [1,2,3,4,5,6,7].each ( | val | @eight_ball.sink(:player =>"Player1", :Ball=> val)
        @eight_ball.call(:player => "Player1", :pocket => :upper_left)
        @eight_ball.sink(:player => "Player1", :Ball => 8, :pocket =>:middle_left)
        @eight_ball.game_status.should == :ended
        @eight_ball.player_status("Player1").should == :lost
    end
end
 
By the way Joe Ocampo built a very nice port for rbehave (another Ruby BDD framework) to .NET 3.5 by extending NUnit which has a very Ruby-like syntax
      

Anyway, the DSLs demonstrated by Neal provide a very good example of the difference between dreaming big and actually doing stuff in the small. The counter example for that are "Software Factories". As I wrote here about a year and half ago

Software Factories is not a new idea  - see for example "Software reuse: From Library to Factory" by M. L. Griss  (published in 1993(!)) which talks about "Software Factories" and "Domain Specific kits": components, frameworks, glue languages etc.  The current Microsoft  incarnation of Software Factories takes a similar approach focusing on Domain Specific Languages, Frameworks but also adding important aspects like multiple viewpoints, patterns and designers. The idea is that  building on modern technologies, as well as learning from the mistakes from sister approaches to code generation (OMG's MDA, in case you are wondering) will enable us to build something that is useable.

Microsoft seems to be taking some steps in the right direction (GAT is probably the best example). Nevertheless there is still a long way to go before we can realize the dream of "factories" for vertical applications

Unlike small code based DSLs - the modeling based approaches of software factories, MDA etc. aim too high and thus provide much less value or suffer too much from the generation gap (the code generated is too generalized or far off from the actual need of the solution). Another problem with software factories/ MDA DSLs is the modeling (i.e. diagrams) - they say a picture is worth a thousand words. This is true if you treat models as sketches you can raise the level of abstraction by as much as you want and convey ideas with less clutter. However when you need to make the model very specific so it would allow code generation - you get to a stage where it is more convenient to do it in code and rely on generated or pre-built DSL or framework

Lastly you can  download Neal's presentation (in PDF form) from his site