A side effect of my decision not to become an independent consultant at this time means that I have to shelve some of the projects I was considering. One of these projects was to create a training program for software architects which I was discussing with a couple of training centers here in Israel.
Since It seems I am not going to promote it, I thought I'd share what I think a training program for .NET/Java architects should look like in the hope that someone would find it useful and do promote it (or parts of it)

Soft Skills
The way I see it architects needs a bunch of soft skills to be able to perform their roles.
Here is the list I identified in the past (by the way, I began a series of posts on each of these skills and never got to finish it - maybe it is time that I will :) )
  • Leadership. Influencing others to accomplish tasks and following your guidance
  • System thinking. Understand decisions and constrains in the
    wide scope pertaining to whole of the solution at hand. This includes
    the ability to abstract problems.
  • Strategic thinking. Understanding decisions and constrains and their alighments to the overall business of the company.
  • Organizational politics. Understand the environment you operate in and how it influences you.
  • Communications. Making sure you get your point across.
  • Human relations. Understand the "people" aspects or human
    factors and dynamics. This includes things like negotiation, pragmatism
    etc
I am not sure if you can teach all of them, but few courses that can help (in my opinion) include:

  • Presentation Skills - While getting the architecture and technology right is what matters, if you can't explain it to the different stakeholders you're toast.
  • Strategic Planning - This has to do with the vision thing I expect architects to manifest. Note that having a vision should not be confused with future-proofing a solution. future-proof means excess work not needed. Having a vision is knowing where you want to end - it can still be perfectly valid to completely re-write your applications along the way


Project Management
While the architect is not the project manager (mostly anyway), I think understanding the constraints coming from the project management point of view is very important. Since most environments call for a mix of agile and formal disciplines (hey, you've got to be pragmatic). I would train architect both in SCRUM and RUP (or some other formal methodology)

Also while not all environments needs this I would give an 2 days overview of important standards. The first would be IEEE 1471, which defines a standard for documenting software architectures. I would also teach ISO 90003 and CMMI.

It should be noted that the ISO 90003 is much better than the previous incarnation (ISO 9003) as it basically lets you define what you want to do to cover the different areas. The standard just helps you make sure you think about the various parts of project management (requirements, environment etc.). For instance I demonstrated how key areas of 90003 can be mapped to SCRUM to get it approved on my last project.


Languages, Design  and Patterns

I would want the .Net/Java architect training program to include at least 2 of the following languages:
Ruby, Scala, Erlang, F#, Python , Groovy ,OCaml
The reason for this is that these languages have different design goals than .NET and Java so learning them gives you additional perspectives and broaden your horizons for other ways of thinking (even if you don't use them in your project directly). You might have noticed that there's no .NET or Java training here. The reason for that is that's a prerequisite as far as I am concerned. You should master at least one

Object Oriented principles - hopefully aspiring architects already know this. However, I often see people who discovered some of the principles by themselves but haven't heard about all of them.
I am talking about principles such as Liskov Substitution  Principle , Open Closed Principle, Single Responsibility Principle , IoC containers, Don't Repeat Yourself  and YAGNI (I summarized my opinion on most of them in this paper)

The next step is to cover some design issues like Domain Driven Design, UI Design, Database modeling, Database alternatives l(after all the database is dead, right? )

Advanced design patterns - When most people hear the term design patterns they think about the GoF patterns. There are however literally hundreds of design patterns. Some of them are even worth learning :) . For instance there are patterns for concurrent and parallel systems like Proactor, Reactor Half-sync/Half-Async etc; Workflow patterns like Cancelling partial Join, Recovery Action etc; SOA patterns (ok, so I am still working on that :) )



Architecture Workshops
Another important part of the training, in my opinion, is to do some workshops and actually try to apply some of the material covered.

  • Architecture Evaluation - workshop 2-3 days - It is probably worthwhile to delve a little on scenario based evaluation techniques such as LAAAM and ATAM. While I prefer evaluating architecture in code, the scenario based thinking is very valuable for eliciting architectural requirements
  • Distributed Systems Architectures workshop - I'll expand on this in the next post


Lastly, there are also a few miscellaneous subjects like architect 101, the SPAMMED architecture framework , Agile architecture, Behavior Driven Design , common frameworks (though hopefully this would  not be needed ) like Spring/Spring.Net, Hibernate/NHibernate, iBatis  etc.


PS

Note that there are a few architect training programs available out there
One is offered by the Software Engineering Institute (SEI) and includes a 6 courses. SEI program seems to be focuses on formal sides of architecture as it includes courses on documenting software architecture and ATAM (You can see an old presentation I have on ATAM here)
Dana Bredemeyer  also offer architect training. Dana offers several workshops that cover the software architecture profession.
TOGAF (which is more of an enterprise architecture framework) offers both a certification and courses
Lastly, IASA is considering creating a software architect program and has a few courses in development
If you know any others I'd be happy to hear about them


 
Tags: .NET | Agile | Design | Everything | Java | OO | Software Architecture

You may have read about the guy who after spending 2 years on a Ruby on Rails project switched back to PHP. Without getting into the debate of whether Ruby on Rails is better than PHP or wether Ruby is overhyped (probably - but that doesn't mean it isn't any good either.By the way  it is the same with SOA, but I digress)

Reading his post I saw a few quotes that raised a red flag for me such as:
"But at every step, it seemed our needs clashed with Rails’ preferences. (Like trying to turn a train into a boat. It’s do-able with a lot of glue. But it’s damn hard. And certainly makes you ask why you’re really doing this.)"
and
#2 - OUR ENTIRE COMPANY’S STUFF WAS IN PHP: DON’T UNDERESTIMATE INTEGRATION
By the old plan (ditching all PHP and doing it all in Rails), there was going to be this One Big Day, where our entire Intranet, Storefront, Members’ Login Area, and dozens of cron shell scripts were ALL going to have to change..
and
Speaking of tastes: tiny but important thing : I love SQL. I dream in queries. I think in tables. I was always fighting against Rails and its migrations hiding my beloved SQL from me.

What these quotes say really is that this guy doesn't know about "Technology Mapping 101". Here is what I wrote about  technology mapping*  about 2 years ago (incidentally that's about the same time this guy started his Ruby adventure :) )


"Technology mapping can have a significant impact on the ability to actually implement the architecture. The wrong mapping can make adhering the architectural guidelines very cumbersome and sometimes nearly impossible."
Every technology or framework has its own architecture. This architecture poses constrains and makes certain things easy (like using the ActiveRecord pattern in Rails) and certain things harder (like not using O/R Mapping ) so, for instance, on the case mentioned above a better technology mapping might have been RBatis (iBatis for Ruby/Rails) which lets you map SQL statements to objects. It is important to note (in Rails case) that one of the core tenets for Rails is preferring convention of configuration when you don't do that you have to work hard(er) - as you are working against the framework

Another problem with the technology mapping here was his point #2. It is a pity he only saw it in after the fact. It can be justifiable to switch everything but you've got to allow this change to occur iteratively. While I generally dislike the software architecture = building architecture metaphor, using it here does make sense: building software for an existing business (vs. greenfield development) is like building a new intersection. You have got to think about building all the detours that would allow the traffic (business) to continue to operate, sure it might run slower in the intermediate phase but it can't stop altogether.

So again, Once you have an architecture that fits your business, take a look at the technology options you have. try to choose the best fit. Whatever you choose - take a look at the implications of that technology and think about the tradeoffs you may find that you either have to adjust your architecture or change the technology altogether - if you don't you might find you waited 2 years of development effort or even more..



* Technology Mapping is one of the steps of a set of activities I identified as needed to make sure your have a solid architecture. The activities goes by the acronym SPAMMED and you can read about them more in this article and/or this presentation



 
There were a few threads about whether SOA is about the technology or not.

In my opinion SOA and Architecture in general  are never about the technology - technology is important but it is just one variable in the equation. What we are looking for is a way to satisfy as many of the business needs as we can under all the constraints we face.

For instance, a few days ago I got a question in my email box from someone calling himself coldplay. While I don't think the band has somehow got itself interested in Event Driven Architecture, the question itself looked interesting enough. Here is the situation:
Current Setup
Lets take a e.g of a Inventory Stock Reorder point exception with in heterogeneous apps environment(No-SOA and integrations)...
The exception definition was built into the source apps and when the stock dropped below reorder....event registered and led to a exception. Exception was further handled by a rules based engine and a workflow notification raised ..

Planned Setup
Same e,g as above .. Post SOA implementation.. Inventory management is composite service built by orchestrating collaborative services from SAP and Oracle...which have different data model supporting them...
The exception definition requires to be defined outside the native Oracle apps and might have to get some event related information from SAP web service also .. to arrive to a conclusion as to whether this really is a Exception or not ..


Possible Technical approaches:

• Data Persists somewhere in the processing the Exception
• Data doesnt persist
• In mem database used..


My Question now :

1. What do you advise to be used in EDA?  which would reduce network round trips, decrease apps server loads from the above 3 technical outlooks.
2. What is diff between in-mem db and usual processing of apps logic by a apps server

I feel :

• Data persistence would lead to larger commit times and reduce operational efficiency
• If the data doesnt persist... and all validations are executed on the fly... dont you think the current apps servers would die processing ... or if its processing capacity is increased .. is it going to be economically viable alternative.

To be quite honest I can't really answer the questioned asked because the question lacks the business context -
what are  the implications if events are  missed or lost? What's the acceptable latency that would allow the business to operate properly ? if the Oracle bits and SAP bits need a lot of data from each other - then maybe the whole service partitioning is wrong and the services are not cohesive enough? How many business events are expected anyway? How often? and the list goes on.

Once you answer the business questions you can look at the available technology portfolio and ask whether you would want a in-memory Database or maybe a datagrid would be a better option? and even then the decision is not just technology driven since when you do cost/benefit analysis you need to take into account purchasing costs, operational costs, skillset of the dev team, time to implementation etc.

This is not to say that if you choose a technology that isn't aligned with your architecture you should reconsider the architecture (or technology). Also since each technology product brings into the table its own architecture (with its own constraints and decisions) you probably need to verify the architecture once you make technology choices. but still, at the end of the day it is the business needs that sits in the driver sit, the rest is just tagging along for the ride.


 
September 20, 2007
@ 11:20 AM
Finally, I've quit my current position - to take a new job as VP R&D at (what I think is) a very promising start-up called PaperLnx.
 
In a nutshell we are going to build few solution in the "physical world connection". What's "phisical world connection" you ask? Well, here's a nice definition from "The Pondering Primate":
"When a display was added to the first mobile phone, a new media was created. Since then, Internet connection and a camera have been added that have created a new way to interact with the physical world.

Soon speech recognition will allow an additional way to browse the physical world too.

Every physical object will have a physical world hyperlink

That means every physical object will allow connection to a designated website and the mobile phone with it's psychical world browser will be able to surf the "real world", the physical one."

As can be expected from the "long tail" curve we are not alone in this space (the link above has a nice list of other companies).  However I think we have a competative edge... :)

Incidentally, I am looking for "a few good men" (or women) to join us. We need both developer and QA positions:

Senior Server Developer
We are looking for a highly motivated, resourceful and intelligent developer. Good interpersonal and communication skills will be very appreciated. A Team player. Broad thinking and problem solving capabilities are also desired.
  • At least 5 years of server side development with thorough understanding of Object Oriented principled and  understanding of architectural styles and design patterns.
  • Experience in multithreading and in distributed systems.
  • .NET/Ruby experience
  • Integration with C++ components
  • Experience with O/R Mappers such as nHibernate/ActiveRecord etc.
  • Video processing experience/ familiarity with video related protocols (H.324m, H.323 etc.)  a plus
  • Mobile Internet backend development a big advantage
  • Understanding of architectural constraints (security, availability, scalability etc.) for internet scale platforms a big advantage
  • Team leading skills – an advantage.
Senior Web Developer

We are looking for a highly motivated, resourceful and intelligent developer. Good interpersonal and communication skills will be very appreciated. A Team player. Broad thinking and problem solving capabilities are also desired.
  • 3 years web development experience is a must, 5+ years experience a big plus.
  • Experience with web-related technologies Javascript, HTML, CSS, AJAX
  • .Net/Ruby experience.
  • Understanding of Object Oriented principles and REST architectural style.
  • Experience with web MVC frameworks like MonoRail/RoR
  • Experience with O/R Mappers such as nHibernate/ActiveRecord etc.
  • Familiarity with SQL 2005/mySQL. Knowledge of SQL an advantage. 
  • OLAP, datamart experience an advantage
  •  Team leading skills – an advantage.
QA Person
We are looking for a highly motivated, creative and intelligent QA person. Good interpersonal and communication skills will be very appreciated. A Team player.

  • Experience testing WEB  applications.

  • -Testing creativity. Understanding of Test related methodologies
  • -Experience designing and controlling all the stages of the project's life cycle.
  • -Experience with test environments such as FIT, FITNESSE or at least other automated tools- advantage.
  • -Knowledge of SQL an advantage.
  • -Dot-Net/Ruby/Python/Powershell - advantage.
  • - Cellular web testing experience a big advantage
  • -At least 2 years experience in the field

If you think you qualify, located in Israel and you are interested in changing  the world :), you can send your CV to me
 
Tags: Everything | General | PaperLnx

September 20, 2007
@ 12:25 AM
Another REST related post - this time I want to share a couple of observations I had after reading (Roy's presentation from RailsConf 2 days ago via Pete Lacey) and listening to Roy T. Fielding's presentations.


The first point has to do with a question which is sometimes raised whether you can do REST without HTTP. i.e. can you have a RESTful architecture  if you don't use the http protocol and further more not using the http verbs (GET/PUT/HEADER etc.) or  as the unifier interface. I talked about it a while ago and I think you can. listening to Roy's talk  it seems that, at least in http architect's opinion the answer is yes as well.

Another point that occurred to me, watching Roy's talk, which is related to the "REST magic" post I wrote a little over a week ago. The use of a uniform interface is tauted by REST proponents (and Roy himself) as coupling reducing formula. After all if you use a uniform interface you are not coupled to the particular semantics of any resource/service you already know the capabilities (actually the maximal capabilities) they offer. What ensues is that instead of using a lot of verbs (ReserveRoom, UpdateOrder etc.) you use a lot of nouns (/rooms/, /orders/order1 etc.)

This works extremely well on the "human" web where my browser can navigate to any-ol'-site without any prior knowledge of what's the site about. When I navigate to Amazon I can buy stuff, when I navigate to New York Times I can read stuff etc. The problem here is the browser is really dumb about what's going on. I, as a human using the browser, understand the context from the content (well, most of the time anyway;) ) so the browser can remain decoupled.  However when you translate it to the "programmable" web you usually don't have some mighty AI engine examining the response to understand the context - instead what you do is trade the verb coupling, which with WS-* web-services would be defined in a contract, you are now coupled to the nouns ( this is not to say the nouns aren't discoverable - since they are due to the hypertext or document orientation communication REST encourages). The end result is pretty similar to what you get when you use verb based contracts your software still needs to understand (where "understand" means some level of coupling) what it is doing with the "other" services. not to mention that you still need to understand the content of the message (sorry- response) to do anything useful with it.

In any event, while loose coupling is very desirable, we also need to remember that the only way to truly achieve complete decoupling is to not connect components. So some coupling is always needed if we want to produce meaningful systems.

What do you think?


 
If there's one reason to go to ApacheCon 07 in Atlanta, then it's probably Roy T. Fielding's "a little REST and Relaxation"

Here is the abstract:
"Representational State Transfer (REST) is an architectural style that I developed while improving the core Web protocols (URI, HTTP, and HTML) and leading them through the IETF standardization process. I later described REST as the primary example in my dissertation. Since then, REST has been used (and sometimes abused) by many people throughout the world as a source of guidance for Web application design. But is the REST that we hear about today the same as what I defined in my dissertation, or has it taken on the baggage that comes with an industry buzzword? This talk will provide a real introduction to REST and the design goals behind its evolution as the Web's arhitectural style. This is not about XML-over-HTTP as an alternative to SOAP, nor about "resource-oriented" frameworks that help simplify CRUD operations, but rather about the design goals and trade-offs that influence the development of network-based applications. I will also describe what happens when we relax some of the REST constraints, and how such relaxation is impacting the design of the waka protocol as a replacement for HTTP."
Now all I have to do is find an excuse for my boss... :)

There isn't a whole lot of information available  on WAKA  (that replacement for HTTP Roy mentions in the end of the abstract). Belwo are a few links I managed to find
And there's a few others but not as interesting (to me anyway). Well, as we see this WAKA thing is in the works for a long time now. Also replacing something as ubiquiteus as HTTP is not a small feat. But I guess if anyone can pull this off it would be Roy... As always, only time will tell

Edited (18/9): it seems that a recent version of Roy Fielding’s presentation  is available online on parleys.com (via Stefan Tilkov)



 
September 14, 2007
@ 11:41 PM

Having just read Doron's Yaacoby's post on some of the insights he got  from reading Tom & Lister's Peopleware, I'd thought I'd repost a couple of posts I wrote about a year ago in my Dr. Dobb's Journal blog (with a few edits):

I just finished reading Software Conflict 2.0: The Art and Science of Software Engineering, by Robert L. Glass.

This is a reprint (with few new retrospective additions) of his 1990 book. While the technology mentioned in the book is outdated, many of the author's ideas and views are still valid. The book is a collection of short articles on various subjects, and one of the more interesting articles is about the cognitive side of design.

Glass explains that research done showed that design includes the (obvious) steps of understanding the problem, decomposing it into goals and objects etc. The essence of design, however, are the mental steps taken by designers:

  1. They construct a mental model of proposed solution
  2. They mentally execute the model (i.e. simulate the model to see if it solves the problem)
  3. If the model isn't good enough (e.g., too simple) replay the simulation to find what wrong and remodel
  4. Repeat 1-3 until the model seems to solve the problem

Glass also said that people tend to start with a model that worked on a similar problem and that good designers perform this process extremely fast. As I was thinking how one can train himself to get better at performing this task, it occurred to me that I heard something similar  somewhere... While not the exact process this is very reminiscent of TDD - only TDD makes the process explicit. In a way we can say that TDD will not only help make your design better it would also  trains you to design better altogether.

Robert's book has several other views still relevant today. While it can seem odd that a 16-years old book contains relevant thoughts looking at my bookshelf I see there are quite a few other books (some of which are event older) which contain essential information and brilliant ideas that are still very relevant today. Here are just a few examples:

Getting back to TDD or at least the idea of test first. It seems (via Rob Keefer pomiet blog)  Gerald Weinberg talked about it more than 35 years ago in  Psychology of Computer Programming :

"By pursuing this test example to the point where he understands the problem, he will not only learn the one thing he did not know, but perhaps will learn others as well, for test programs such as this are often better learning instruments than are production programs.

As a matter of good practice, the test program should be constructed before the 'fix' is made to the production program. In the first place, there will be an all too human tendency to forget about the problem once the production program is working correctly, so we must impose a little discipline on ourselves. Possibly more important, however, is the chance that by the mere act of constructing the test case we shall discover the problem."

Anyway, with so much good advice lying around for years (11-40+) and the fact that only about 30% of the projects are successful (on-time;on budget; on scope) I think one question we should all ask ourselves is -- don't we ever learn?


 
Tags: Everything | General | OO | TDD

From time to time I read about the magic that is RESTful services and how they solve everything and anything like scalability, idempotency, simplicity etc. for instance in "RESTful Web Services" by Sam Ruby and Leonard Richardson they say
 "PUT and DELETE operations are idempotent. if you DELETE a resource, it's gone. If you DELETE it again, it's still gone..." (p.103)
or
"the safe methods, GET and HEAD, are automatically idempotent as well" (p.219)

Another example comes from Anne Thomas Manes who said

"The REST architectural style defines a number of basic rules (constraints), and if you adhere to these rules, your applications will exhibit a number of desirable characteristics, such as simplicity, scalability, performance, evolvability, visibility, portability, and reliability.

The basic rules are:
  • Everything that's interesting is named via a URI and becomes an addressable resource
  • Every resource exposes a uniform interface (e.g., GET, PUT, POST, DELETE)
  • You interact with the resource by exchanging representations of the resource's state using the standard methods in the uniform interface
"

I think such claims  are plainly wrong and misleading.
 
Don't get me wrong, I like the REST approach, since it encourages better service design - e.g. document oriented message exchange vs. the RPC like message exchange which the so called "WS-death-*s" (or actually the tools that support them) encourages.

It also encourages the above mentioned traits - however that's exactly the  point - REST encourages this thinking not solves scalability or other problems out of the box- you still need to design your services properly.

For instance if you follow Anne's rules you can still end up with a service which is stateful, that performs heavy distributed transactions against multiple databases and systems - i.e. a service that is neither simple, scalable or perfromant

DELETE will only be idempotent if the resource is idempotent (e.g. a specific version of a resource)  or the message is idempotent (e.g. requesting a deletion of a specific version) if you are deleting the "recent version" then it might have been recreated between your calls you are now deleting something completely different. heck, even a GET (read) message with a single reader can be made to be non-idempotent  if you decide to code something that alters the state of a resource significantly whenever it is read. When you have multiple readers and writers GET will not be idempotent "automatically" as two consecutive reads can give you two different representations as the resource might have changed (again unless the resources are idempotent)

REST is not different from other styles in this respect - for instance you can do Object orientation in C but working in an OO language encourages object orientation (the opposite is also true - using an Object Oriented language does not guarantee that you get an Object Oriented design)

At the end of the day, architects should still think about the design if they want to ensure the results matches the quality attributes they want to achieve - some environments/styles/tools will make some quality attributes more easy to achieve but nothing will solve the problems for you.



 
September 7, 2007
@ 02:07 PM
In the previous post on the subject I wrote that the RDBMS is dead. I didn't mean that it is dead dead, but rather that it isn't well build to meet some of the newer challenges like linear scalability, high availability etc.
Well, it is one thing hearing it from me - and it is another thing hearing it from someone like Michael Stonebraker.
Michael, was the main architect for the Ingres prototype project at UC Berkely just one year after Codd's paper and (9 years before Oracle was released  and more than a decade before the commercial version of Ingress was released).

Well that was in 1970 - in 2007 Michael recently wrote :
 
In short, the world of 2007 is radically different from the world of the late 1970s. However, none of the major vendors have performed a complete redesign to deal with this changed landscape. As such they should be considered legacy technology, more than a quarter of century in age and "long in the tooth".
Among the new needs Michael cites are intelligence DBMSs (needs a lot of relations), textual and semi-structural data etc. He also said (promoting his own product) that 2007 customers expect high availability, linear scalability.
Michael's main point is that specialization can provide significant performance enhancements vs. the one-size-fits-all approach of RDBMSs. He gives his product (Vertica) as an example for how a column oriented database (vs. the RDBMS row orientation) can outperform RDBMs by a factor of 50. Google's Big table is another example.

Interesting...


 
September 4, 2007
@ 11:03 PM
When I begun writing SOA patterns, the first version of chapter 1 was a general introduction to Service Oriented Architecture from the perspective of Software architecture. When the editors saw the patterns chapters they've felt the chapter wasn't focused enough on patterns so I rewrote it untill it finally molded into the current version.

Nevertheless, I think that the first version has value on its own providing some guidance on the influences on architecture and putting SOA in an architectural context. I will probably edit it a little over the next few days so that it would be standalone (i.e. disconnected from the book). Meanwhile you can download the original version from here



 
September 2, 2007
@ 10:02 PM
Jeremy Miller recently tried to "answer hard questions about Agile development". One question he didn't really answer was that of agile and fixed bids :
"I don't really think the Agile answer to a fixed bid is any different from any other process.  I do think that Agile practices and project management can give you far more control and feedback on the "Iron Triangle" of resources, time, and features"

Jeremy says that this would enable the team to fail "softly" (i.e with some functionality etc.).
I think (well, actually, I know since I did that) we can do better than just use Agile and hope for the best. For instance, here are a few  strategies I successfully used:

1. Pre-budget the whole project with rough milestones and estimates (works best if you have prior experience in the domain)
2. Coordinate expectations with the client and trade - the single project to many smaller ones. What I did was to get a blanket purchase order for the estimated amount and each smaller order/project  "ate" some of that budget until it was done.
3. Since the project was for a CMMI level 3/ ISO 90003 we also added a few documents to the backlog - and let the product owner prioritize them vs. deliverables. For instance the "Architecture document" was delivered after 6 iterations when the architecture stabilized
4 exchange requests instead of change request - new functionality comes instead of old promised one

Of course we also did the other SCRUM practices of releasing frequently, demonstrating progress, retrospective etc.

I would love to take credit for the ideas above - however, Pascal Van Cauwenberghe details most of them (and more) in two papers called "Agile Fixed Price Projects part 1: The Price is Right" and "Agile Fixed Price Projects part 2: Do you want agility with that" (go read them..)