I've been taking a look at ruby for a few month, I am finally getting
to a stage where (I think/hope) I can actually say something
intelligent about it. Last time I had an "aha moment" about a language
was the fist time I saw Java. C++ was oh-so-powerful, but Java was
(is) much more elegant and nice. Now Ruby changes the rules of the
game again. When I first heard about ruby I thought it was just a
fad, something that the cool kids are using but its just another
language. I getting more and more convinced that it isn't so. I am
trying not to get too "silvery-bulletey" here but working in ruby
seems to actually increase productivity. Let's
take Dependency Injection(DI) as an example. DI is one of the most
important and powerful tool I've learned in regard to Object Oriented
development. Instead of classes depending directly on other classes
classes depends on interfaces. And external classes (assemblers)
provide them with their dependencies. This allows for loose coupling,
increase testability and a lot of other such goodies (you can read a
concise explanation in a paper I wrote on OO principles or get a more thorough explanation in a paper Martin Fowler wrote on "Inversion of Control Containers and the Dependency Injection pattern".
The .NET and Java worlds are filled with a lot of frameworks to help
solve this elegantly. Spring (and Spring.Net) is probably the most
known one. How do you do DI in Ruby? in two words - you don't If I am to join the "Define DI in one sentence" challenge by Jim Weirich I would say that "DI
is a powerful and good workaround to the collaboration coupling
problem between objects which is best addressed at the language level" Why doesn't ruby need DI? Well,
I would say that it all starts at the basics. I remember when I learned
OO, I was told objects communicate using messages. I never really
understood why they call "method invocation" messages - it doesn't make
any sense. The point is that in ruby you really don't "call a method"
you "send a message"*. When you make a call like
someVariable.SomeMessage - the ruby interpreter doesn't really care
about the type of someVairable just that the object it holds (and
everything is an object) has some entry which can handle SomeMessage.
Let's start with a simple example
consider the following code:
class Foo
def bar
puts "Foo-bar"
end
end
class Foosa < Foo
end
class Baruser
def baruse(b)
b.bar #dependency
end
end
bu= Baruser.new
bu.baruse(Foo.new)
bu.baruse(Foosa.new) # sub-class
Well, nothing particularly exciting here. if you run this you get
foo-bar printed twice. That's very much like the dependency injection
you see in .NET or Java
It gets a little more interesting when we consider that the following classes would all work as well
class Foz
def bar
puts "Foz-bar"
end
end
class NoBar
def method_missing(methodname, *args)
puts "NoBar" if "bar" == methodname.to_s
end
end
class MakeBar
define_method(:bar) {puts "madebar"}
end
bu= Baruser.new
bu.baruse(Foz.new) # another type altogether
bu.baruse(NoBar.new) # a class that doesn't have bar method
bu.baruse(MakeBar.new) # a class where the bar method is created programatically
We can see from the examples that what ruby does is searching for a
handler for the bar message. The handler can be a method (symbol)
called bar or a generic handle like Missing_Method - ruby doesn't care
as long as the message get handled
I think that's pretty nice, we have a lot of flexibility on the
dependency side but the depending class still essentially gets the
dependency by injection (the call to baruse)
Well, ruby can help us flex the dependent side as well. The answer is
in the last example which uses (an overly simple an uninteresting
example of) meta-programming Consider the following code example using Mocha which is a Mock object library for ruby
Lets say we modify our Baruser class to the following.
class Baruser
def initialize
buildfoo()
end
def baruse
foo = buildfoo()
foo.bar #dependency
end
private
def buildfoo
Foo.new #dependency
end
end
Note that buildfoo is private - if we wanted to test this in.NET we
need to have Foo around. i.e. we can no longer test Baruser by itself
if we use Mocha in ruby we can do the following:
foo = mock('foo')
foo.expects(:bar).at_least_once.
bu = Baruser.new
bu.stubs(:buildfoo).returns(foo) # basically what happens here is that the instance we have is changed
bu.baruse
I can't believe they invented it
- You can get ruby for just the price of download and if you call
within the next 15 minutes we'll throw in a copy of gems free of charge
:)
In any event what we see here that using meta-programming and other
ruby constructs we can forgo using DI altogether - no wonder Neil Ford
defined DI as :
"Dependency Injection enables a vitally important but nevertheless
weak, limited, syntactically confounding, and dauntingly complex form
of one of the kinds of meta-programming that should exist in the
language." Not
to mention that the resulting code is much more elegant. which is
actually what I like best about ruby, the code is much cleaner (but
that's for another post)
Some closing thoughts
-
.NET 3.5 bring some of the ruby goodness to C# - but as the previous
post demonstrated it is just a move in the right direction but not the
whole thing
-
While I am on the subject of the previous post the whole interface vs.
class thing is, of course, a moot point in Ruby since there are no
interfaces. Interfaces, like DI, are another thing that is very
important in C# and Java and not needed in Ruby
-
I've read some complaints on Ruby's performance. performance is
important but there are two things to remember. First, the fact that a
solution is not the fastest doesn't mean that it isn't fast enough.
Second, I can still vividly recall the performance benchmarks for our
Java code before we got the first hot spot compiler installed. The
point is, that if it is important to enough people it will get better.
* I know, I know, Smalltalk had it since the beginning of time.
However in Smalltalk everything is an object, in ruby you can also
write plain scripts and (more importantly for me) - I never really took
more than a cursory look at smalltalk so I never saw that.
One of the interesting presentations I attended in Architecture &
Design world was "User Interface Principles in API design" by Elliotte Rusty Harold. Elliotte
started off by mentioning that developers are people too (I guess...)
and that designing an API we need to think about the people
(developers) who are going to use it. I totally agree, in fact, when
it comes to API design is one place I insist developers use TDD even if
they don't do it on their day-to-day development. Dogfooding their own
API is probably the only way to really ensure you get a usable API.
Elliotte thinks you should go further than TDD and write some sample
programs first. The challenges of API or library design come
from the fact that you cannot be sure about Elliotte mentioned some of
the fundamental principles of UI design that apply to API design. - The importance of consistency
- Simple is better (and for god sake please keep complexity internal) which translated to YAGNI and "when in doubt, leave it out")
- Smaller surface (API) is easier to use.
To
demonstrate the point of proper API design Elliotte compared the
complexity of JMidi vs. the elegance of JFugue. While in JFugue you
just use a couple of lines to play a few notes: Player player = new Player(); player.play("C D E F G A B")
JMidi required more than 20 lines of setting all sorts of things related to its internal structure. Elliotte
also had a lot of practical advice on varios aspects like maintenance,
specific advice for Java, specific advice for .NET etc. - Instead of me
reiterating it I suggest you just take a look at the presentation which is available on his site. One
point Elliotte made which I don't completely agree with was that you
should prefer classes over interfaces. we discussed this issue after
the presentation and I said that it is true for libraries but I am not
sure that it is always true for frameworks (see Frameworks vs. Libraries
for more details on the difference). in IoC scenarios interfaces allows
extending the framewok for use with user defined types. One real-life
example I offered was a plug-in framework we developed internally a few
years ago. This framework incorporated a fixed set of components based
on certain 3-rd party components (grid, tree etc.) - This worked very
nicely, until in one project we wanted to use other 3rd party
components (let's say a ribbon or another grid) which left us with
either waiting for the infrastructure guy to find the time to make the
changes, fork the infrastructure code, or build something new (we went
with building something new since working with end-users we found we
actually didn't need most of the capabilities the framework offers
like security/roles, role tayloring capabilities etc.) On the
other hand one argument for using classes over interfaces that Elliotte
is right about is that when you use interfaces and then make a change
in that interface. The chage breaks all clients. If however all the
clientes used a subclass they would automatically inherit the new
functionality. I thought I could work around this limitation in
.NET 3.5 (or whatever microsoft will end-up calling it) by using
extension methods so I did something like the following (tests etc. are
excluded for brevity) class Program { static void Main(string[] args) { testInterface tc = new testClass(); tc.DoSomthing(); tc.DoSomethingElse(); } }
interface testInterface { void DoSomthing(); void DoSomthingElse(string test); // compilation error.... }
static class testInterfaceDefaultBehaviour { public static void DoSomethingElse(this testInterface ti) { System.Console.WriteLine("Doing something else"); } }
class testClass : testInterface { public void DoSomthing() { System.Console.WriteLine("Doing something here..."); } } I was wrong.... Main does recognized the DoSomethingElse pretty well - however adding it to the Interface definition caused a compilation error. This is a shame since the MSDN documentation says that " In effect, extension methods make it possible to extend existing types and constructed types with additional methods." If it had worked that would have let us have something like method_missing in ruby in C#. Well, .NET 3.5 is still in beta so maybe it would change (not counting on that)
I won't say anything about my presentations (that's for others to say :) ). The point of this post is just to let you download them. So here they are: - SOA Patterns (2.14mb) - Takes a look at different strategies (patterns) to solve common SOA pitfalls
- Getting SPAMMED for architecture (4.56mb) - Takes a look at the activities architects can/should do when they think about software architectures. The presentation also covers architecture in agile projects.
Evan H asked a question about distributed transactions and services in the MSDN architecture forum:
Are distributed transactions (ie.. WS-Transaction)
a violation of the "Autonomous" tenant of service orientation? Yes or
No and Why? Kudos if you can address concurrency and scalability (in
an enterprise with multiple interacting services).
I
answerd this questions back in april when I wrote a couple of posts
that explained why cross-service transactions are a bad idea: cross service transactions and some more thoughts on cross service transactions. Roger Sessions also agrees with this view (well, it seems actually, he wrote about it well before I did :) ): When
the WS-Transaction specification was first proposed, back in 2002, I
wrote an article explaining why I thought the idea of allowing true
transactions to span services was a bad idea. I published the article
in The ObjectWatch Newsletter, #41: http://www.objectwatch.com/newsletters/issue_41.htm.
Nothing since then has changed my mind. Atomic transactions require
holding locks, and spanning transactions across services requires
allowing a foreign, untrusted service to determine how long you will
hold your very precious database locks. Bad idea. Just because IBM and
Microsoft agreed on something doesn't make it good!
The reason I am bringing this issue back is that Juval Lowy (who wrote the article that triggered my first post on the subject) has recorded an Arcast with Ron Jabobs.
Where he re-iterated the idea that "Transactions is categoricaly the
only viable programming model" and you should strive to use it whenever
you can. It seems Juval admits you sometimes need to use Sagas
(which he called "long running transactions" - you can see in my link
why I think that's a wrong name). He also agrees that you can also use
a transactionable transport and then only do internal transactions from
each service to the transport (a pattern I call "Transactional
Service"). However, at the end of the day, he still thinks you should
use WS-AtomicTransactions whenever you can. I agree that
transactional programming is important. I think it is the simplest
programming model (from the developers side). I would probably never
write an interaction with a database that is not transactional; I look
very favorably at initiatives for in-memory ACI (no Durability)
transactions such as the one Ralf talks about. Until we get to Distributed Transactions... First, we should note that transactions are not "the only viable" option.As Martin Fowler notes
Ebay seems to be doing fine without distributed transactions. Not only
that, they abandoned distributed transaction and went
"transactionless"because they needed one simple thing... Scalable
performance .
In most COM+ scenarios you have a single server or a few internal
servers where the distributed transaction happen - and even there you
should plan your transactions carefully if you want to get any kind of
decent performance. In SOA scenarios the situation is more complicated
as the distribution level is expected to be higher (even if you don't
involve services from other companies). More distribution means longer
times to complete transactions (especially if a participant can flow
the transaction and extend it). It also means increasing the chances of
failure (see Steve Jones series of posts on five nines for SOA).
In my opinion, the more distributed components you have the more you
want their interaction to be decoupled in time - i.e. the opposite of
transactions. Juval also said he doesn't buy the denial of
service problem I mentioned (supporting a transaction means you allow
locks - if an external party doesn't commit you retain the lock..).
Juval said he assumes that a solutions has both authentication and
authorization so this shouldn't be an issue. For one, I have seen too
many projects where security was something that was neglected or
quickly patched in at the latest moment - so I would hardly assume
security. Even with security on - you increase your attack surface. But
that's just the half of it. Even if all your service consumers have
good intentions - you still don't know anything about their code. SOA
is not like the "good old days" where you owned the whole application
- this means you cannot trust their security to be ample. Also you
don't know anything about their code quality. Services are likely (in
the general case) to be deployed on different machines, even if they
start co-located. I think that a Service boundary should be treated as
a trust boundary just like a tier boundary.
I strongly believe you should have reduced assumptions on what's on the
other side of the service's boundary - transactions are not reduced
assumptions
SOA and distributed transactions do not go hand in hand - it isn't just
autonomy at stake here. It is a problem for performance and scalability
and even security period.
To finish this post - I would also highly recommend looking at Pat Helland's paper "Life Beyond Distributed Transactions: an Apostate's Opinion" and a post he recently made called " SOA and Newton's Universe", where he explains more eloquently than I ever could why SOA is not a good fit for distributed transactions.
In addition to the drafts of selected patterns I publish on my site, you can now purchase my book via the Manning Early Access Program (MEAP). MEAP
means you can get chapter drafts as I write them and the complete book
when its done (ebook or printed). Here is Manning's explanation: "Buy
now through MEAP (Manning Early Access Program) and get early access to
the book, chapter by chapter, as soon as they become available. You
choose the format - PDF or ThoutReader - or both. By subscribing to
MEAP chapters, you get an opportunity to participate in the most
sensitive, final piece of the publishing cycle by offering feedback to
the author. Reader feedback to the author is welcome in the Author
Online forum. As new chapters are released, announcements are made in
the MEAP Announcement Forum. After all chapters are released, you will
be able to download the complete edited ebook. If you order the print
edition, we will ship it to you upon release, direct from the bindery,
weeks before it is widely available elsewhere.
By the way, this is probably also a good time to mention that I'll be speaking about quite a few of the patterns in Architecture & Design World 2007 which will take place this July.
There is
still a lot of work, but I already like to thank all the people in
manning that helped me get this far. especially to Cynthia Kane my
editor (hey, maybe now she'll give me more slack :) )
Ok, 'nuff blubbering, back to completing chapter 5...
Udi Dahan writes that " .NET/Java Interop is not a reason for SOA". Udi writes that companies that need to integrate two technologies turn to web-services and that
"The only problem is that in order for things to work right, they really must
have a chatty interface, and flow transaction context between these
“services”, and all the other things I describe as anti-patterns"
Udi is right that if you don't rethink and remodel your systems you
will (probably) not have an SOA as you are likely to find your self
implementing anti-patterns such as the ones he mentions.
However, using Web-services does not automatically mean that you are
doing an SOA. If you don't think about moving to SOA you can still opt
to use web-services as a remoting or RPC technology to connect two
systems. The advantage over the other proprietary products Udi mentions
is that web-services are a standard technology. This will work well or
fail is orthogonal to the technology choice. It depends on the
architectures of the systems you integrate. If you need to flow
transaction between the systems you'd also need that even if you
cross-compile one of the applications in the other environment.
Another thing I don't agree with is the word must Udi uses.
First, while it is likely that older systems has chatty interfaces it
is not a must. The designers of the legacy system may have thought
about the consequences of distribution without regard to SOA. Also you
can still wrap an existing system with a service contract (using
web-services or any other technology) and not get to chatty interfaces
etc. However that means that the wrapper should have some substance or
business logic inside it to mask the old system's behavior this is
especially important if you are thinking about moving to SOA and you
take into consideration that the business will not just halt and wait
there until you are done. You have to think about interim solutions,
such interim solutions can include wrapping a legacy system with an
Edge Component and a SOA facade (a pattern I call Legacy Bridge) while
you move in the grader direction of a full blown SOA.
I just read Shy Cohen's (via Nick Malik) article in Microsoft's Architecture Journal entitled "Ontology and Taxonomy of Services in a Service-Oriented Architecture"
Shy provides a list of what he calls service types. He identifies two major types bus services and application. He then continues to sub-divide them, the Bus Services are divided into communication and utility services and the Application services are divided into entity, capability, activity and process services.
I have to say it was quite alarming to see this coming from someone who had deep involvement in defining Windows Communication Foundation Indigo.
Where do I start?
well, for one, it seems completely fails to make the distinction between Services as in Service Oriented Architecture and Services as in capabilities or features an infrastructure provide. The "Communication services" are for the most part capabilities that a service infrastructure (such as an ESB) provides. Not Services you would define in an SOA initiative
And then there's the matter of service granularity and the difference between Remote objects and SOA for instance, the example Shy gives for a "method" (his word) on a Customer service (entity service):
"An example of a domain-specific operation is a customers service that exposes a method called FindCustomerByLocation that can locate a customer's ID given the customer's address"
why would a service return a customer ID? This is the kind of call you would make on an object you hold a reference to not some remote "Something" that also want to authorize your call and may reside in a different company. This kind of thinking is what made remote objects fail. Gregor Hohpe explained that nicely in a paper called Developing software in a Service Oriented World
The Transparency Illusion. Distributed components promised to hide remote communication from the developer by making the remoteness "transparent". While the basic syntactic interaction between remote components can be wrapped inside a proxy object, it turned out that dealing with partial failures, latency, and remote exceptions could not be hidden from the developer. It turned out that 90% transparency was actually worse than no transparency because it gave developers a false sense of comfort.
As a side note, Gregor recently gave a presentation that covers this paper at JavaZone which you can watch online at InfoQReturning to Shy's article let's take a look at another quote:
Capability Service may flow an atomic transaction in which it is included to the Entity Services that it uses. Capability Services are also used to implement a Reservation Pattern over Entity Services that do not support that pattern, and to a much lesser extent over other Capability Services that do not support that pattern.
I already explained why cross-service transactions and especially flowing transactions is not a good idea in SOA so I won't do it again here - but you can read about it both here (" Transactions between Services? No, No, No!") and here (" Cross-Service Transactions").
Also I truly hope Shy didn't mean .NET data sets when he said "In some cases, typically for convenience reasons, Entity Service implementers choose to expose the underlying data as data sets rather than strongly-schematized XML data. Even though data sets are not entities in the strict sense, those services are still considered Entity Services for classification purposes."
In any event the whole decomposition of Services into fine grained "capability", "Activity" and "process" takes no account of the fact the SOA is a distributed architecture...maybe Microsoft is not affected by the fallacies of distributed computing ?
*ad nauseam (latin)- to the point of disgust
In the article mentioned in the previous post, I talk about adding EDA to SOA and how you can use Complex Event Processing (CEP) to process the event streams and infer the trends and enhance the understanding of what happen inside your business. All the tools I knew about were Java tools I knew about were Java tools but now I've found out (via Nauman Leghari's blog) that there's also a .NET CEP engine and it is even open source - It is called NESper and like many other tools it is a port of a Java tool. I am not sure how good it is - but at least I'll have an interesting evening today :)
An article I wrote on Business Intelligence (BI) and Service Oriented Architecture (SOA) has just been published on MSDN. You can find it here http://msdn2.microsoft.com/en-us/library/bb419307.aspx. The article explains the SOA & BI mismatch and how to bridge it by adding EDA to SOA. (I bloged about it here before, but the article is more ordered and complete)
Back in January, I took part in an architect panel that Microsoft
Israel organized. The panel was led by Ron Jacobs and it featured Udi Dahan, Assaf Jacoby, Coby Cohen, Dudu Benabou and myself. A few days ago Ron edited this recoding and turned it into a
podcast in his Arcast series.
The panel's focus was on lessons learned from mistakes made in past project. Ego
maniac as I may be :) -- even though you don't get to hear me much in
the final edited version -- I think the podcast is worth listening to,
as the panel raised some interesting points. You can download the
podcast here (don't worry it is in English even though it was recorded in Israel)
PS I am the first speaker after the introduction, in case you are wondering.
After seeing Juval Lowy's article on WCF transaction propagation in the May issue of MSDN magazine. I posted " Transactions
Between Services? No, No, No! " in my DDJ blog. I've got a few comments which I thought warrant a post in their own-right.
The previous post was triggered by an article that promoted flowing
transactions (i.e. you perform a transaction against one or two services and
then one of the services calls an additional service and it joins the
transaction). It is important to say that I think transactions between services
should be discouraged regardless of automating extension of transactions.
Transaction propaqgation just makes the matters worse.
There might still be some edge case where you have to have an atomic
transaction from a service consumer to the service. I think that in the vast
majority of SOA implementations you shouldn't do that and I would think real
hard about the other options before allowing it in my architecture.In general I think cross-service transactions are an antipattern (and that's the way you'd find them documented in my SOA patterns book :) )
One of
the comments I received began with:
"Cross service transactions are a sure way to introduce coupling and
performance problems into your SOA." I'm not sure I agree with that thought.
Logically speaking, cross service transactions are a must. The question is how
to implement them. There are two mechanisms we can use for implementing TXs: (1)
ACID TXs; (2) Long-running TXs. The latter is preferable for the cases Arnon is
talking about (large geographical distances, multiple trust authorities, and
distinct execution environments). ACID TXs are more suitable for what Guy has
mentioned (DeleteCustomer service invokes the DeleteCustomerOrder service
internally). I agree with Arnon the a-synchronicity is preferable, but we all
have encountered use-cases where ACID-ness is required from a business
requirement level... [snipped]
One minor point in regard to this comment is that I don't like the term
long running transaction - there is a long running interactions between services
and I think the term SAGA describes them better. Sagas are made of a series of
business activities that flow back and forth between services to realize a
larger business process. Note that these interactions doesn't necessarily have
transaction-like behavior.
which brings me to the more important point of looking at the statement
"Logically speaking, cross service transactions are a must". I don't think so.
For instance, if a service that manages the inventory in a warehouse receives a
request for some items and later a cancelation of that request. The first
request can trigger the inventory service to order some more items from a
supplier. Whether or not the cancellation would cause a cancellation of the
order of the supplier depends on the business rules of the inventory service for
inventory levels for the items ordered. it might also depend on whether or not
the items have already been received etc. The cancellation (the "abort") of the
original request does not have to translate to an abort (or compensation) on the
request receiver. Furthermore if the service communications model is based on
the push model (e.g. using EDA with SOA) the cancellation notice would just be
propagated without regard to the inventory service -. It is the inventory
service's responsibility to understand the ramifications of this event and act
accordingly. Even the example given in the comment 'DeleteCustomer service
invokes the DeleteCustomerOrder service internally" is not a good candidate from
ACID transactions (there's also a problem of service granularity here - I'll
talk about it later). Since when the customer service decides to delete a
comment and request the Orders service to delete orders - there's a reasonable
chance that some of the orders are already paid for but not delivered. In this
case the customer cannot really delete the customer until all the paid orders
are resolved. Or maybe the order service is a facade to a night batch that does
the actual deletion. - I know I am just fantasizing with these examples but the
point is that the customer service has no knowledge on the order service or the
inventory service above except the messages supported in their contract. To
assume something about the internal behavior is problematic. Even if you know
about the internal structure on the onset, the whole idea of SOA is that the
services can evolve independently from each other...
Another thought triggered by the example in the comment originated by the
granularity of the services (DeleteCustomer service vs. a Customer Service that
also supports deleting customers) is that we should be really conscious to the
difference between other architectures like 3-tier client/server and SOA. SOA is
actually more distributed than 3-tier - we cross a distribution boundary every
time we pass a message from a service to a service and not just when we move a
massage from a client-tier to an application server. We add this distribution to
gain advantages in flexibility and agility. However, we should note that this is
a weakness of SOA (considering for example, that Martin Fowler's first law
of distributed object design is" Don't distribute your objects!") means we
should really pay attention to the way services interact with each other.
- The granularity of services - having a lot of fine grained services means
there will be a lot of interactions over the wire (even if you don't go out to
the network you still have to serialize/deserialize, follow the security policy
etc.) rather than internal interactions that much faster
- The Granularity of messages - The same considerations should also guide us
to try to create larger and fewer messages. for the example above . Instead of a
DeleteCustomerOrder message maybe something like an UpdateCustomersOrders
message that can hold a list of customers and orders and the status changes or .
by the way this would also support off-line clients better since they can
accumulate changes.
- The assumptions we can make on the other service's availability,
performance, internal structure, the trust we have for it etc. - We should try
to minimize the assumptions we make and concentrate on what can be inferred from
the contract. Remember that policies can change externally so the business logic
within a service cannot count on them being constant. this brings us back to the
issue of transaction. every cross-wire interaction increases the chances of
failure - in transactions one failure invalidates all the transaction is
invalidate. every cross-wire interaction within a transaction increases the
length of time we lock internal resources (even if we do trust all the involved
parties) - especially if that transaction can extend itself automatically. Also
as I've mentioned in the previous post the transactions also open the door for
denial of service attacks.
If we want to reap the benefits that are sold under the SOA moniker, like
flexibility and agility, we really have to pay attention to this extra
distribution and design our services differently than we would components in a
3-tier architecture - but hey, that's why they pay us the big bucks, right ? :)
I should probably also add that building SOAs is not a goal in itself.
We can build perfectly good solutions using other architectures - but if we find
that we do need SOA (or any other architecture for that matter) we have to pay
attention to the way we implement it to both keep its benefits and not harm
other quality attributes like performance, security etc..
Back in January I opined that moving to web applications was not the optimal solution to the real problem we have/had with desktop applications which was installation woes. What we got was a poor UI without installation problems so we (software industry) started to resolve problems like we had when we moved from terminals to graphical UIs etc. - So now we have Rich Internet Applications (RIA) - using technologies like AJAX - but they suffer from other problems which again we've already been through Well that was the topic of the post in January. Now I've stumbled upon an interesting/amusing twist - called Adobe ApolloApollo let's you, yes you've guessed it, take your RIA applications and deploy them as desktop applications. you can now take your HTML, CSSs , AJAX scripts pack them up as a single file (AIR) and lo and behold deploy them on the desktop. You even get these nifty start menu and desktop shortcuts :) The reason not to dismiss this a complete waste of time - is that what we actually see here is another example of a trend to convergence web and desktop UI architectures and programming models. I say "another" because coming from the desktop direction Microsoft is doing pretty much the same thing. WPF brings the web-programming model with its markup (XAML) and "code-behind" concepts to the desktop as well as pushing the same model to the browser with WPF/E . The difference between Microsoft's and Adobe's solutions is that, Adobe is coming from the web-side and, as I said, Microsoft is coming from the Desktop side - both companies are striding toward the same goal - and what we are left with, is yet another technology war
The Microsoft Name Machine is at it again - this time's poor victim is Atlas:
[from Joshua Flanagan's blog] " The Microsoft AJAX (formerly known as DHTML) technology, codename Atlas, will be officially named... ASP.NET 7.0!"
nuff' said
Over the last few months I've posted a series of blogs on DDJ that cover the basic Object Oriented principles (e.g. Single Responsibility Principle, Don't Repeat Yourself, Inversion of Control etc.).
I've assembled all the posts into a single whitepaper which you can get here.
Also you can download the same (plus a little more) material as a powerpoint presentation.
[Edited version of post in DDJ]
Since I have been blogging for about a year now on this blog and now also on the DDJ blog. I think it is time to try making something with more two-way communications.
Consequently, I am going to run a little experiment for a few weeks and see how it goes.
The idea is as follows: If you have an interesting architectural or design dilemma, drop me an email at ask@rgoarchitects.com I'll pick one issue per week and post (on the DDJ blog) the dilemma (anonymously) plus voice my opinion (and/or suggested solution)--and then everyone else can chime in with their comments and insight which hopefully will shed some light on the subject.
I'd be interested to hear both your opinions on this initiative and, of course, interesting dilemmas you are facing. Again, send your dilemmas to ask@rgoarchitects.com)
While I still hold my view on the Current State of Software Factories - at least one company (EDS) is repoting (in an MSDN article) that they've built a software factory for their domain entities [found via Steve Cook]. Nevertheless this is still a generic generator (i.e. not a factory for ordering system or something similar- but rather something like a DSL for O/R Mapping).
Arnon
PS
I also wonder why do they generate unit tests fot the generated code - assuming they properly tested their templates the generated code should just work ...
There
are 113 domain models over at eclipse.org site of all sorts of things.
I guess most models don't have any practical value (what will I do with a
metamodel of COBOL) but there are several interesting ones of things
like RSS, WSDL, and SDM
It is also interesting to note that the
models are expressed in several forms including MS DSL models
, UML
and images . The transformation from format to format was done by code.
(Found
via Steve Cook )
One
of the roles of the software architect is to act as a mentor/coach. Reviewing
some of the designs in one of my projects' teams it seems the time was ripe for
doing just that. Thus, last week I gave them a presentation on the basics of good OO design - which I thought might also be of interest for other people (you can download a copy
here - 312KB).
The
presentation starts with the 7 deadly
sins of software design:
- Rigidity – make
it hard to change
- Fragility –
make it easy to break
- Immobility –
make it hard to reuse
- Viscosity –
make it hard to do the right thing
- Needless
Complexity – over design
- Needless
Repetition – error prone
- Not doing any
It
is interesting to note that just yesterday I read an interesting piece on what
makes good design (i.e. looking from the positive side) by James Shore (found via Sam Gentile)
The
main part of the presentation demonstrates the 5 basic design principles
(drafted by people like Robert C. Martin , and Barbara Liskov
):
- OCP open-closed
principle - a class should be open for extension but closed for
modifications
- SRP single
responsibility principle - a class should have a single responsibility
- ISP interface
segregation principle - there should be separate interfaces for different
consumer types
- LSP Liskov
substitution principle - basically design by contract - a sub-class should
fulfill the same expectations its suparclass set
- DIP dependency
inversion principle - classes should depends on abstractions, class
consumers should depend on abstractions and abstractions shouldn't depend
on details.
These
principles are the basis for some of the
techniques widely used today - few examples include:
Inversion
Of Control - builds on OCP
Dependency
Injection - a mechanism to allow DIP
Contract
First - building on LSP,DIP
At
the end of the day following these principles helps managing classes
dependencies, increase overall loose coupling and cohesion thus increasing the
overall quality of design. It sometimes amazes me how using just a few simple rules can improve maintainability,
flexibility and usefulness of designs so much.
I
recently found MSF 4.0 is out - yep, it
is still in two flavors…
MSF
Agile (or officially "MSF
for Agile Software Development
Process Guidance") and MSF 4 CMMI (MSF
for CMMI Process Improvement
Process Guidance)
I consider MSF Agile as a lightweight process,
however I prefer SCRUM as an Agile project management process.
I
find the CMMI version more interesting - Both for cases where Agile is not a good fit
(also see Agile vs. Plan driven
development
)
and/or
for organizations that need to have CMMI certifications (such as the one I
currently work for). MSF 4 CMMI covers level 3 pretty well and also has some
guidance moving on to the next levels.
Definitely
worth a look :)
When
I first learnt about WPF (it was still called Avalon back then) I thought it
was strange that Microsoft is touting Smart Client development using WinForms
when the WPF/XAML development model is much more closer to the code behind
concept of ASP.NET.
Developers
with ASP.NET experience will have life much
easier in the WPF era - They'll be able to utilize the same skill-set
and thinking to build:
- Regular, run of
the mill, ASP.NET application
- AJAX
application (utilizing Atlas)
- XBAP (XAML
Browser Applications) - Smart clients running in the browser
- XAML
stand-alone applications
While the WinForm guys (and gals) will be left behind to try and learn the new programming model. Looking at the different
CTPs of WPF it seemed all of us have a lot to learn in order put it to full use
- so I guess no one will have it realy easy :).
Anyway, today
I read Charles Petzold's "WinForms: More Vital Than Ever" blog
post - it seems that WinForms will stay
with us for a while after all (ok, ok, so no real surprise there - I just
wanted to make a point). By
the way, it is especially interesting seeing this coming from
Charles considering he is one of the authors currently working on WPF
books
ANother interesting point is that Microsoft is going into a lot of trouble unifying all
the remoting and distribution
technologies into WCF on the one hand - but will leave us with 5 different
technologies for UI development (ASP.NET, WinForms, DirectX, Atlas and WPF) on the other.
I've just found out (via Gianpaolo's blog) that Roger Wolter (former PM of Service Broker) started blogging . He is going to focus on Data in a Service Oriented world. I had a chance to work with Roger for a short time, which was enough to notice that if anyone knows about data, it is him. I guess there is no surprise there, considering his past at Microsoft working on :SQL Server Service Broker, SQL Express, SQL XML Datatype, SOAP
Toolkit, SQLXML and COM Plus. His first post (after the obligatory "hello world" post) is about Service Broker positioning (vs. MSMQ, Biztalk and WCF) - Subscribed
It seems the PDC presentations are available here ...
(like everyone else ?) I've finished reading the C#3 Spec and the LINQ project overview.
with samples like:
public void Linq43() { List customers = GetCustomerList();
var customerOrderGroups = from c in customers select new {c.CompanyName, YearGroups = from o in c.Orders group o by o.OrderDate.Year into yg select new {Year = yg.Key, MonthGroups = from o in yg.Group group o by o.OrderDate.Month into mg select new {Month = mg.Key, Orders = mg.Group} } };
ObjectDumper.Write(customerOrderGroups, 3); }
Hey - that's de ja vous all over again :)
Actually it is (er..will be) much stronger than SQL - for one, you can use a single LINQ query to combine data from arrays and data and XML and whatever. Also combined with lambda expressions (next gen for anonymous methods ) you'd have even more power.
While I am on the subject of what's new in C# 3.0 - I also found Extension methods very interesting - it kind of lets you have a look and feel of multiple inheritance without the associated headaches
consider the following
namespace MultipleInheritanceSimulation { public static class Extensions { public static void DoSomething<T>(this T var) { //do whatever }
} }
And then:
using MultipleInheritanceSimulation;
namespace Tester { public class Foo { public void Bar() { MyClass x; MyOtherClass y; x.DoSomething(); y.DoSomething(); } } }
oh well,enough playing - back to work...
I've just read Klaus Aschenbrenner's Service Broker article on DevX. While the article provides a nice overview of the mechanics of service broker, I think there's a need to elaborate more on the essene of Service Broker (esp. vs. MSMQ) - so here's my 2 cents on the subject.
While MSMQ is a general purpose messaging engine, Service Broker is focused on (some would say limited to) transactional messaging -And in this area it does a better job than most if not all the other messaging solutions out there. Here are two examples:
- Message order - MSMQ transactional messages only guarantees ,message order inside the transaction - it does not guarantee order between transactions also messages from two transactions can be interleaved (see MSDN for illustrations). Service broker solves both these problems.
- Multiple readers - If you have a large message load and want to set up multiple readers. One problem that might happen (which Service broker solves) is of two related messages being handles by different readers at the same time - this can occur if you have a long running processes between services or if you utilize service broker for message exchange inside a service. Also in the realm of multiple readers - Service broker will do some form of load balancing the readers for you - if you are willing to let the reader be stored procedures.
Another important trait of Service Broker is that it seems it was built with service orientation in mind (and not just because of the name..). For one you have to define messages and contracts (which party can send which messages) in order to establish a conversation (even though you can trivialize it to send any message you want in any direction). Furthermore, you send messages on a conversation and not to a queue. This means that the sender doesn't really have to know who would get the message (well some layer has to know in order to set up the plumbing - but the business logic that decides to send a message is free from all this).
The other use for Service Broker is to introduce asynchronous operations into the database. It lets you create short transactions updating a single table or view and have the effects of the change ripple as a continuing transaction or transactions. for example you can update a view (that has a lot of underlying tables) and in its "instead of " triggers put the updated information in a queue. You can then read from the queue in another transaction and update the underlying tables. since everything is transactional (the send from the view's trigger, as well as the read by the updater stored procedure) you can be sure that the updates will occur (by the way, this view scenario is something that can be used for implementing insert-only databases, but that's a matter for another blog post :) ).
Arvindra Shemi posted a diagram with the main concepts of service brokers. Note that he also mentions there Monologs, which, as Pat Helland once told me (before he left Microsoft), was originally intended to be part of the product, but was dropped (postponed ?) somewhere along the way. It should be noted that you can implement the behavior of monologs using dialogs, however you need to do a some work to make that work. Another concept missing from that diagram is conversation group, which is a way to treat several conversations as if they were a single conversation.
One technical remark on the DevX article - The article shows the TSQL - it isn't too difficult to wrap these with C# code. I didn't check the latest beta of SQL but the samples of previous versions already had such an implementation (prepared by the service broker team).
[minor updates]
Architecture Description Languages (ADLs) sound like a great idea when you look at the "spec sheet"
ADLs represent a formal way of representing architecture ADLs are intended to be both human and machine readable ADLs allows simulation and analysis of architectures – completeness, consistency, ambiguity, and performance ADLs can support automatic generation of the system (as it is represented by the architecture)
There are quite a few ADLs out there xADL , ACME/ADML, SSEP, Rapide, Wright and the list goes on and on.
I am guessing most of you didn't hear about any of those (well ACME appears in all those Willy E Coyote vs. Road Runner cartoons but that doesn't count). I think that the culprit lies in the fact that most (if not all) these come from the academic world where the focus lies on the models (in terms of semantics, completeness, rigorness etc.) and not on the practical use and applicability to day-to-day issues. Another problem lies in the fact that main-stream tools used by the industry.
By the way, UML isn't considered an ADL for several reasons for example the weak integration between the different model that inhibits automatic analysis.
The reason I am bringing the issue of ADLs up is that looking at the new architecture designers in Visual Studio 2005 (the Application Designer , the logical data center designer ,the System Designer and the Deployment Designer) actually form a set of DSL (Domain Specific Languages) that together can be treated as an integrated ADL. Furthermore Microsoft also provides an SDK for the"System Definition Model" that is the underlying model behind all these DSLs which lets interested parties extend and build additional designers.
While this model is not complete in several aspects ( some designers like the logical data center designer are too limited or there aren't enough views to cover all the architectural description) it is a good starting point in bringing the ADL concept into more practical and usable form.
What if there was a new platform for .NET that has the following characteristics
- lets you run .NET 2.0
- supports web-services (something like IIS)
- integrated message-queue (a-la MSMQ)
- high-availability (better than clustering performance)
- strong management tools
well there is one - it is made by Microsoft and goes by the name SQL Server 2005.
SQL Server 2005 lets you run .NET 2.0 code in process (an interesting point here is that DB2 already lets you run .NET code and Oracle will also add this capability with 10g Release 2 but both do it out-of-process).
It also lets you expose stored procedures as web-services
It has a performant message queuing mechanism called service-broker. Service broker guarantees message ordering as well as solves the problem of multiple readers (what happens if one reader takes the order header and another takes the order line) and as I said it has good performance. by the way Roger Walter who is the group program manager for service broker is going to publish a book on the subject. Having worked with him for a short while (He was our "Redmond connection" on one of the projects I consulted) I can say without hesitation that he is the authority on service broker and a smart guy to boot :)
SQL Server 2005 also has (almost) instant fail-over capability - using Database mirroring you can have your (application?) server fail over in less than 5 seconds (compare that with clustering fail over times of 60 seconds or more). Note that each database can only fail over to one other database, but a server can be both the backup database of one server and the active server (vs. another server).
And you get all the management capabilities of SQL server (oh yeah, you also have a great DB there :) )
Taking all this into consideration there are (a limited set of) situations where you can actually use SQL Server as a hosting platform (instead of COM+ or IIS) for your application. True, this is not something I will generally recommend but I can think of one or two projects where scale-out was less of an issue where it could have been used (had it existed).
[thanks to Eliaz Tobias from MS Israel for the link]
The beta version of MSF 4.0 formal is available for download. This is an interactive process which is compliant with CMMI level 3.0 (see http://www.sei.cmu.edu/pub/documents/02.reports/pdf/02tr029.pdf chapter 7 - maturity levels) and supported by the tools (i.e. Visual Studio Team System 2005). This can be great news to organizations like the one I work for these days who are certified for CMMI 3.0 and can (hopefully) stay compliant with less bureaucracy.
Looking at this process from my perspective (i.e. as an Architect) it also looks interesting. It defines several roles for architects (e.g. in domain technical level as Subject matter experts and in the solution level as an Architect). The process suggested is tailored/aligned with VS2005 capabilities (and thus somewhat limited) however many of the steps are both viable and important. For example it has parts that do with quality of service requirements (what I call Quality Attributes in SPAMMED). My particular favorite step is "Assess Alternatives (LAAAM)" which I helped introduce ( :) ) in my previous job as an Architect for Microsoft Consulting Services
The Composite UI Application Block (CAB) is an interesting architectural solution for smart client development. It is essentially a plug-in architecture that brings the WebParts concept to the desktop.
The CAB has the following elements:
- WorkItems. These are the classes which represent use cases in your application and contain the business logic for those use cases.
- SmartParts. There are the building blocks of a Windows-based application - similar in concept to WebParts.
- Workspaces. These are helper classes that can display SmartParts with a uniform style.
- UIElements. These are elements such as toolbars and menu bars that are shared by SmartParts within the application.
- Support services. These include:
- Event broker service to manage the publishing and subscribing of events between SmartParts.
- State management service to hold shared state for areas within the application. This provides an option to encrypt the state before it is stored.
The CTP is now available on GotDotNet (needs .NET 2 beta)
Don't worry I didn't switch to blogging on Greek mythology (well, not yet anyway) this is still a technology blog.
Ajax is an old/new web development paradigm. It is basically a combination of several existing client-side technologies to achieve responsive and "rich" web applications (a good example for that is Google maps - just pan the map and you'd see what I mean). specifically Ajax builds on XHTML, cascading style sheets (CSS),Javascript, DHTML and XMLHttpRequest (for asynchronous communication with the server). By the way the reason it was dubbed AJAX is that it is an acronym for Asynchronous Javascript and XML.
Microsoft has downplayed Ajax importance for quite sometime - claiming that basically the technologies has been available for years in Internet Explorer. Nevertheless Microsoft has recently decided to join the fray and provide structured tools to help realize Ajax development on the .NET platform - The code name for this initiative is Atlas. Scott Guthrie from Microsoft published some of the details in his blog
Here are a few highlights of what Atlas will include:
- Client Side Framework - an extension for Javascript with UI controls, network stack, eventing etc.
- Server Controls - server side-controls that interact with the client side controls (based on async callbacks)
- Web-service integration
- Few Server-side and Client side Services - e.g. Profiles, Personalization, Authentication and Roles on the server and local browser cache on the client
It seems Microsoft will release a technology preview of Atlas at the PDC later this year.
Meanwhile, if you don't want to wait for Atlas - you can download Ajax.net - Ajax.Net will generate client-side scripts that interact with server-side methods decorated with [Ajax.AjaxMethod] attribute - it isn't everything, but it is a very nice start (and it is available today)
Microsoft has released the Connected Systems Toolkit. A very nice intorduction to the SOA concept (as Microsoft sees it). The toolkit (DVD) includes:
- a connected system's course (with 4 labs)
- an extensive sample application (with code and documentation)
- few whitepapers
- few presentations (ppt & video)
- some marketing hype (well, nothing is perfect :) )
The architect doesn't talk, he acts. When this is done, the team says, "Amazing: we did it, all by ourselves!" (17) (The Tao of software architect - Philippe Kruchten)
On the surface - When it comes to agile development the role of the software architect is a little more blurred.
The most obvious aspect where architects is for the technical architecture. An experienced technical architect, can greatly enhance any project by steering the designs into the "best" directions (under the chosen platform constraints). The technical architect can also promote reuse etc.
Additionally while the requirements change a lot and not fully defined, the quality attributes of a system are more stable - if you need performance, then you need performance!. More so, there are some qualities that are inherent to agile process - for example you want to put an emphasis on flexibility and maintainability - if your developed solution does not have these it is going to do refactorings. Thus, depending on the size of the project you may want to use an architect to help set the ground rules in the first couple of iterations.
Another area where architect involvement can be very beneficial is when you try to scale an agile project a good choice is to try to break the project to smaller loosely coupled projects (see this paper for example) - well, this is just what we (architects) live for...
By the way, few agile processes, define the architect role up-front, one such process is MSF 4 Agile. Note that MSF (Microsoft Solutions Framework) version 4 comes in two "flavors" one that is aligned with CMMI and the other a (much) more light version, the afford mentioned MSF 4.
How does the SPAMMED process fare with an Agile project - (surprisingly enough) I would say pretty well
First of the architect should be hands-on i.e. part of the development team (most likely the technical lead)
- Stakeholders, you would probably want stakeholders on-site for any architecture related meeting (just as much as you would want the customer on site for other activities)
- Principles would include things like TDD, Simple Implementations, Refactoring
- Quality attributes would hold Flexibility and maintainability and a couple or so of the important project qualities (performance, availability..)
- For the modeling you would chose very few views and would try to focus on ones that have manifestation in deliverables
A good example for this is the Application designer in Whidbey (see screen shot below) - the result of which is the projects structure for the solution

- Mapping doesn't really change
- Evaluation would be focused on proof of concepts and/or skeletal architecture.
- Deployment - well, being part of the team... you would notice if your decisions were wrong or circumvented
To sum up - yes Agile projects can and many times should use an architect to help it stay on track - hey, even XP has an architectural spike...

One of the most interesting things that MS is building these days is the set of tools around their software factories initiative.
Software Factories, is actually a methodology that builds on the concepts of Domain Specific Languages (DSLs) along with frameworks, patterns and guidance.
The idea is to bring the notion of domain modeling and DSLs from horizontal markets (e.g. the form designer in any modern IDE) to the everyday use of vertical markets (i.e. your next application) and help realize the promise of product lines. Many of the ideas in software factories are not new (see for example this article from 1993) - however, today,as the Jack Greenfield et al state in their book, a number of needed technologies has (finally) matured enough to make it feasible. Furthermore What's unique in this effort is that Microsoft is making the effort to back the ideas with tools that will enable architects and developers to put them into use.
A software factory needs 3 elements a schema, a template and tool to support them (a development environment). The schema is basically the recipe containing the overall information on how things should work together. The template is the set of DSLs, samples, frameworks that are used to create the factory and lastly we need the IDE that supports them (e.g. Visual Studio "Whidbey" - although, at least technically this can also be achieved in Eclipse etc.)
I am, personally, very interested in this initiative. We already use code generation all over the place (DAL, serializations, Data Entities etc) and employ a domain driven approach even today. I would love to have "user-friendly" tools that will let me as an architect achieve these goals more easily.
One final note - yes I am aware of MDA (Model Driven Architecture) but all the MDA tools I've encountered thus far are lacking in their usability. This, by the way, brings me to one caveat with software factories - it is not based on UML 2.0. I would rather have a DSL that is based on stereotypes, tagged values and OCL. This would allow for a specific view using the appropriate designer but also for backward compatibility (and a downgraded view) on other tools. I do agree though, that UML 2.0 has its own share of problems - but that's a matter for another discussion altogether
Just saw an entry @ Kevin Hammond's blog that when you download the Indigo/Avalon beta you have to read through to download the SDK rather then the redistributable - I have to admit that I've made the same mistake myself...
I almost forgot another commercial GRID implemrntation is Gigaspaces. Gigaspaces is interestig since it is actually based on Java standards JINI and JavaSpaces but also has support for C++ and .NET
GRID computing, in the sense of distributed computing with a focus on large-scale resource sharing and high performance computing, is starting to emerge as a viable direction for several computing problems (you can also see What's the grid )
Below are links to several projects that lets you create Grids using the .NET platform
-
-
OGSI.NET - Another open source implementation of the same standard
-
-
BFC - Base one Foundation Class - a commercial GRID implementation
Happy Gridding
|