May 8, 2009
@ 10:59 PM

Apropos the Blogjecting  Watchdog pattern,  In addition to blogging I recently added to our system the ability to twitter. I am using Tweet# from DimeBrain (thanks Mark Nijhof for the tip via twitter).

Tweet# makes using tweeter really simple (I included the code below in case you find it useful).

The tweeter sender is part of a PostOffice service (I thought that it would be problematic to present it as SpamServer which was its original name :) ).

image Update 11/05 Here it is working on our staging environment :)

A few points about our design in general that are interesting in this regards are

  • The PostOffice is a “Server” type service – we have 3 types of services: server which has one instance per node, channel which has multiple instances per node and algorithmic which has one instance per core
  • The PostOffice implements a pattern I call “Legacy Bridge” – which is basically an SOA version of an adapter+facade in OO terms. The post office supports the events (over WCF) mechanism we have in our system from one side  and connects to external systems (SMS, coupons and twitter) on the other. The PostOffice, basically contains an Edge Component which accepts the requests and funnels them to *Sender classes that interact with the external systems.
  • from contract design perspective – The events I added into the system are StatusEvent and AdminStatusEvent (and not TwitterEvent and DirectMessageEvent). this is better, in my opinion, as it carries the intent of what I want to achieve. It also means that if I choose to change technology or use multiple destinations the events will stay meaningful. For instance, the AdminStatusEvent will be used by our monitoring system to send a notification if the system crashes. I’ll probably want that as an SMS, maybe even a phone call as well as a twit (so the AdminStatusEvent will have a severity to designate how it should be handled)
   1: using System;
   2: using System.Collections.Generic;
   3: using System.Linq;
   4: using System.Text;
   5: using Dimebrain.TweetSharp.Fluent;
   6:  
   7: namespace xsights.Apps.PostOffice.Server.Twitter
   8: {
   9:     class TwitterSender
  10:     {
  11:         private string account;
  12:         private string password;
  13:         private string admin;
  14:  
  15:         public TwitterSender(string tweetAccount, string twitterPassword,string adminAccount)
  16:         {
  17:             account = tweetAccount;
  18:             password = twitterPassword;
  19:             admin = adminAccount;
  20:         }
  21:         public void Update(string msg)
  22:         {
  23:              foreach (var tweet in BreakToTwitts(msg))
  24:             {
  25:                 var update =
  26:                     FluentTwitter.CreateRequest().AuthenticateAs(account, password).Statuses().Update(tweet).AsJson();
  27:  
  28:                 update.Request();
  29:             }
  30:         }
  31:  
  32:         public void SendAdminMessage(string msg)
  33:         {
  34:             foreach (var twit  in BreakToTwitts(msg))
  35:             {
  36:                 var dm =
  37:                 FluentTwitter.CreateRequest().AuthenticateAs(account, password).DirectMessages().Send(admin, twit).AsJson();
  38:  
  39:                 Retry(2,dm.Request,false);
  40:             }   
  41:             
  42:         }
  43:  
  44:         private IList<string> BreakToTwitts(string originalString)
  45:         {
  46:             var list = new List<string>();
  47:             for (int i = 0; i < originalString.Length; i += 140)
  48:             {
  49:                 var len = 140;
  50:                 if (originalString.Length - i < 140) len = originalString.Length - i;
  51:                 list.Add(originalString.Substring(i, len));
  52:             }
  53:             return list;
  54:         }
  55:  
  56:         private void Retry(int retries, Func<string> call,bool shouldThrow)
  57:         {
  58:            
  59:             try
  60:             {
  61:                 call();
  62:             }
  63:             catch (Exception ex)
  64:             {
  65:  
  66:                 if (retries > 0)
  67:                     Retry(--retries, call,shouldThrow);
  68:                 else
  69:                 {
  70:                     if (shouldThrow)
  71:                         throw;
  72:                 }
  73:             }
  74:           }
  75:           
  76:         }
  77:     }
  78:  
  79: }

 
Tags: .NET | OO | SOA | SOA Patterns

Earlier today I read a post by Michael Feathers Called "10 Papers Every Developer Should Read (At  Least Twice). I knew some of the articles mentioned there and learnt about few interesting ones.I liked it so much,  I thought I'd compile a similar list for software architects - based on stuff I read over the years.

1. The Byzantine Generals Problem (1982) by Leslie Lamport, Robert Shostak and Marshall Pease - The problem with distributed consensus
2. Go To statements considered harmfull (1968) - by Edsger W. Dijkstra - Didn't you always want to know why ? :)
3. A Note on Distributed Computing (1994) - by Samuel C. Kendall, Jim Waldo, Ann Wollrath and Geoff Wyant - Also on Michael's list but it is one of the foundation papers on distributed computing
4. Big Ball of Mud (1999) - Brian Foote and Joseph Yoder - patterns or anti-patterns?
5. No Silver Bullet Essence and Accidents of Software Engineering (1987) - Frederick P. Brooks - On the limitations of Technology and Technological innovations.
6. The Open Closed Principle (1996) - Robert C. Martin (Uncle Bob) - The first in a series of articles on Object Oriented Principles (you remember the debate on SOLID...)
7. IEEE1471-2000 A recommended practice for architectural description of software intensive systems (2000) various- It is a standard and not a paper but it is the best foundation for describing a software architecture I know.
8. Harvest, Yield, and Scalable Tolerant Systems (1999) Armando Fox, Eric A. Brewer - That's where the CAP theorem was first defined
9. An Introduction to Software Architecture (1993) - David Garlan and Mary Shaw - one of the foundation articles of software architecture field (although based on earlier work by the two)
10. Who Needs an Architect? (2003) Martin Fowler - Do we or don't we?

I could come up with quite a few more articles not to mention books that aren't in this list. However these are definitely some of the most influential papers I read.




 
Tags: data | Design | OO | Software Architecture

February 16, 2009
@ 10:29 PM
Returning from a short blogging vacation (both personal and work related) I found my RSS reader is getting filled with posts relating to "Uncle Bob vs. Joal & Jeff or the case of the SOLID principles (OO principles by various people including Uncle Bob himself - you can also read the "OO Primer" paper (pdf/ppt) I worte on these and few related principles).

Codeclimber has the complete timeline, but in essence, Unclue bob talked about the SOLID principles in a Hanselminutes podcast; Joel Spolsky commented that "quality doesn't matter that much" on his and Jeff's StackOverflow podcast, which was followed by a few blog posts on both side like Uncle Bob's "Open letter to Joel Spolsky and Jeff Atwood" and Jeff Atwood's "The Ferengi Programmer".

I don't know Uncle Bob personally (I've seen him talk a couple of times and I read his blog/papers), so I might be off here. However (to me) Uncle bob usually comes off  as  being too religious about following practices to the letter. e.g. see the debate  between Uncle Bob and Jim Coplien on TDD where Bob says "nowadays it is irresponsible for a developer to ship a line of code he has not executed in a unit test."

Nevertheless, I am fully with Unclue Bob on this one. I especially like the following qoute from his "Getting a SOLID start"
"The SOLID principles are not rules. They are not laws. They are not perfect truths. The are statements on the order of “An apple a day keeps the doctor away.” This is a good principle, it is good advice, but it’s not a pure truth, nor is it a rule.

The principles are mental cubby-holes. They give a name to a concept so that you can talk and reason about that concept. They provide a place to hang the feelings we have about good and bad code. They attempt to categorize those feelings into concrete advice. In that sense, the principles are a kind of anodyne. Given some code or design that you feel bad about, you may be able to find a principle that explains that bad feeling and advises you about how to feel better."


You see, the point about these OO principles (and patterns and even so called "best practices") are an attempt to distill experience. Assuming these principles are worth anything (and the SOLID ones do) - You can either  ignore that knowledge and learn these lessons the hard way, i.e. through your mistakes or internalize them and apply them.

How do you apply principles - by applying critical thinking. I wrote here before that I don't like the term "best practices" since it implies you don't have to think. Sorry, being a developer means you have to think, deal with it. There's no "deus ex machina" - it is all up to you.
Any good idea can be abused, e.g. let's look at the Single Responsibility Principle. SRP helps bring cohesion into classes by  lowering the "reasons to change. It is very easy to apply it ad absurdum and get a single method classes that really only do one thing but that's not the way to go.
The thinking part means that when we implement some class, and we happen to know about SRP, we can say to ourselves, hey this looks like an SRP violation. Why am i doing this? Do I have a good reason for that etc. knowledge of principles and patterns promote making your design decisions explicit and thus having you control the design (and architecture) rather than the other way around.

One claim I read is that principles such as SOLID preach to the choir. The so called "20% of developers" who already read blogs, enrich themselves and generally know what they are talking about. Well, I don't know, but the way I see it, not all those developers are black belt gurus who feel good design by the direction of the wind or get perfect code by mumbling a few incantations. I know I am not one. That means I am always learning. Ok, so I already know about SOLID and a couple of other nifty patterns so I need to learn other stuff like good principles for functional programming or whatnot . If you are starting out on OOP though, I suggest you build yourself on a SOLID foundation and go form there


 
Tags: Design | OO | Trends

 A couple of days ago, one of our team members saw in the log that we are getting index out of bound exceptions in some of the code of the a piece of code I wrote (part of the EventBroker). Taking a look he saw the following:
       private void EventSender(object state)
{
var id = (Guid) state;
try
{
sagas[id].Dispatch();
}
catch (Exception e)
{

            Logger.DebugFormat("EventSender: Exeption Details={0}",e);
rwl.TryRLock();
var isClosed = true;
if (sagas.ContainsKey(id))
isClosed = sagas[id].IsClosed;
rwl.ExitReadLock();

if (!isClosed)
HandleSagaFault(id);

}
}

It is well known that one of the guidelines for exception throwing is "Do not use exceptions for normal or expected errors, or for normal flow of control".So,naturally, he raised an eyebrow and asked me why don't I check that the id exist before I try to call the Dispatch method.
e.g. something like:
            try
{

rwl.TryRLock();
var isClosed = true;
if (sagas.ContainsKey(id))

sagas[id].Dispatch();
rwl.ExitReadLock();
}

Well, as it happens the EventSender method runs on a thread pool thread (something you could have guessed from the object State parameter). It's role is to get the saga to dissipate the event. i.e. the Dispatch method on the Saga object handles actually sending   events to event's subscribers.And while it does its work in parallel it also waits for all the events to arrive before returning (collecting any communications exceptions etc.) What this means is that the Dispatch method takes awhile to return.

Ahh, there's the rub - Contrary to the guideline mentioned above, it is actually more worthwhile to pay the performance penalty of throwing an exception rather than pay the more severe penalty of holding a lock for a long time. The lock affects all the threads running whereas the exception only affects the isolated thread.

Indeed - on another MSDN page there a better version of the guideline "Do not use exceptions for normal flow of control, if possible. Except for system failures and operations with potential race conditions"

The lesson here is (again) is that we need to think before blindly following guidelines.Guidelines are, well er, guidelines not commandments

PS
if you release the lock before calling Dispatch you don't gain anything, since it is multi-threaded code and the id can still be collected between the check and the call itself.

PPS
in case you are wondering TryRLock() is an extension method which try to obtain a lock with a given (short) timeout and throw (instead of deadlock) if the timeout is reached.


 
Tags: .NET | Design | OO

October 18, 2008
@ 10:58 PM
As I mentioned in a couple of previous posts (like "Using REST along with other architectural ), I've been spending the last few weeks writing an Event system over WCF (probably also explains posts on  WCF gotchas like this;) ). Being a communication infrastructure it is still a long way from being completed, but it seems to be stabilizing and I think it turned out nicely so I thought I'd share a few details.

Let's start with the simple part - the usage.
The eventing is built on the idea of a bus (i.e. no centralized components) and the resources/services that want to use eventing have to use a library which I call EventBroker.  There are two modes for using the EventBroker. one is "regular" events which are contexless. This means that consecutive events can reach different services, and there is no context that flows from event to event:

bool raisedEvent = eb.RaiseEvent<SampleEvent>(new SampleEvent());
The second type of events are Sagas, which represent long running interactions. Sagas does have a "best effort" guarantee to reach the same recipients over consecutive calls. Also you can also End sagas (sucessful termination), Force End Saga (successful termination by a service that didn't initiate the saga) and Abot Saga (unsuccessful termination): Here is how you raise a saga event.
var evnt = new SampleEvent { data = somevalue};
var SagaId = Guid.NewGuid();
eb.RaiseSagaEvent<SampleEvent>(SagaId, evnt);
if you use the same Saga Id, the events are handled as part of the same saga, if you use a Saga Id that wasn't previously defined it will initialize a new saga.
The eventbroker translates events to the relevant contract and dispatches the events over to the different subscribers. Which brings us to to the next part which I  guess,   is also a little more interesting. How subscriptions are defined.

The first thing to do is to define the event itself.
    public class SampleClassEvent :ImEvent
{
public string DataMember1 {set;get;}
public int DataMember2 { set; get;}
}
There aren't any real constraints on the event, except that it has to "implement" the ImEvent interface. Which is really an empty interface but it marks the event as one for the event broker.
Then you have to define an interface for handling the event. The event broker, builds on the idea of convention rather than configuration (an idea popularized by the rails framework) so it is easier to generate the interface (something I do with a resharper template)
    [ServiceContract]
public interface IHandleSampleClass
{
[OperationContract]
int SampleClass(SampleClassEvent eventOccured);

}
The convention is that the interface will have a IHandle prefix followed by the name of the event. It will hold a single operation named like the event (without the Event suffix) and will recieve a single parameter which is the event data. Currently  events do return a value (int) but I am thinking about changing it to void and have everything marked as OneWay for added performance

Now, when we create a service which needs to handle events it will do that by specifing which events it handles. E.g.
    [ServiceContract]
public interface ImSampelResource : ImContract, IHandleSampleClass, IHandleSomeOtherThing
 {
}
So each contract declares all its subscriptions (by a list of IHandleXXX). It should also include the ImContract interface which holds all the service operation used by the eventbroker (e.g. ending sagas etc.).
Services that want to raise events should inherit from a ControlEdge class (base class Edge component that delegates control events to the event broker)

There's still the question of how does the event broker knows where to find other services. There are several ways this can be done (e.g. a service repository) but since we have  blogjecting watchdogs in place anyway, we use them to propagate liveliness (and location ) of services.

This sums up this post. It is basically just a little context for several planned posts where I hope to talk about some of the challenges, alternatives and design decisions that led me to the current design. Meanwhile, I'd also be happy to hear any comments, ideas or reactions you may have
 
Tags: .NET | Design | OO | SOA | SOA Patterns | Software Architecture | WCF | xsights

July 2, 2008
@ 01:07 PM
As I was going through my RSS list (which I publish as a linkblog) I noticed Reginald Braithwaite's post on the metaobject protocol. Reginald says

The bottom line: Languages like Java give you an object protocol: There is one way to do things with objects and classes and interfaces, period. Anything else, like adding generics or annotations, must be done outside of the language, it must be done by the creators of the language.

Languages like Common Lisp and Smalltalk give you a meta object protocol: You can decide for yourself how to do things with objects, classes, interfaces, generic functions, whatever you want. You don’t need to wait for a committee to try something different.

What's nice is that a couple of posts later I noticed a link by Harry Pierson to an attempt by Jeff Moser to port OMeta to the .NET world and to enable the very "meta object protocol" Reginald mentios. Here are the opening bits from Jeff's post (go read the rest):

What if programming language implementations were object-oriented? What if you wanted to change one itty-bitty part of your favorite language? Say you wanted to add an exponentiation operator (^^). How hard would it be?

Wouldn't it be nice if you could "subclass" a language like C# and then add 5 lines of code to make it work and then use it? What if you could add Ruby-like method_missing support in 20 lines?

What if you could conceive of a new language and start experimenting with it in production code in an hour? What if it leveraged your knowledge of existing frameworks like .NET?

Also check out the code Jeff posted so far (on CodePlex)


 
Tags: .NET | Design | OO | Trends

Dare Obasanjo complains about Resharper 4.0's recommendation to use implicitly typed locals (i.e. var someVariable = SomeMethod(); rather than SomeType someVariable = SomeMethod();)
It is a small issue and I would probably wouldn't comment about it except I hear some of the same complaints from members in our team. The main grievance Dare has is that using var impedes the readability of the code. He also says that that people using var will be more inclined to use long "hungarian style" variable names.
Dare also mentions MS recomendation on the use of var which recomends

As for me, I am just happy with anything that get me nearer to true duck typing and the extreme-late-binding it offers.
Dr. Alan Kay (inventor of Smalltalk and one of the fathers of OOP) even says that that extreme late-binding is one of the essential attributes of an Object oriented language:
"(I'm not against types, but I don't know of any type systems that aren't a complete pain, so I still like dynamic typing.)
OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things. It can be done in Smalltalk and in LISP. There are possibly other systems in which this is possible, but I'm not aware of them."
Yes, the var keyword is still a far cry from duck typing as it doesn't provide the true late binding to the interface needed you get in languages like Python or Ruby. It does however takes one worry from you and helps reduce the overall accidental complexity. var, extension methods and lambda expressions all help make C# more "dynamic" and easier to work with.

I think Ola Bini summed the issue best:
"A statically typed language with type inference will give you some of the same benefits as a good dynamic language, but definitely not all of them. In particular, you get different benefits and a larger degree of flexibility from a dynamic language that can't be achieved in a static language. Neal Ford and others have been talking about the distinction between dynamic and static typing as being incorrect. The real question is between essence and ceremony. Java is a ceremonious language because it needs you to do several dances to the rain gods to declare even the simplest form of method. In an essential language you will say what you need to say, but nothing else. This is one of the reasons dynamic languages and type-inferenced static languages sometimes look quite alike - it's the absence of ceremony that people react to."
Edit As for the code readability claim. I prefer to focus on stronger methods like keeping methods short, meaningful method and variable names and supporting tests (which can actually help you understand how the code behaves...). Not to mention that, if you really really need that, resharper will tell you the type if you put the mouse over the "var" keyword ;) 
 
Tags: .NET | OO

I saw a post (via Dzone) by Keith Elder called  "How to not screw up your application object model - Don't go all OOP on me!"
Its a long read, so I'll give you the skinny. Keith tries to make his point by an example. He designs an elaborate initial object model for a veterinary clinic (IPet, pet, cat, dog, spaniel, owner, address, work address ..), then takes the search screen from a real vet. app, maps the model to the information needed for that screen and explain that the initial model sucks and that that particular screen needs two UI objects (search and searchresults) that would better handle the problem.
He sums the post with
"The big takeaway from this litany of verbal spillage is when you design
applications you have to know how the end-user is going to use the
application and what the business requirements are.  Sitting down and
building an elaborate object model, while really fun and educational,
it is a big waste of time.  Don't do it.  Sit down and ask questions so
the behavior and business requirements can be accounted for.  Whether
you are using CSLA.Net or not the same thoughts must go into building
the application.  I hope this helps cements these concepts because I
know there is a large amount of disconnect with this out there among
developers who think everything is a perfect object graph like they
learned it in school. "

There's nothing really wrong in the conclusion on its own but in the light of the post itself there are few things to notice
  1. The "elaborate object model" Keith talks about has nothing to do with OOP.
  2. Keith just drafts an "elaborate object model" but fails to note that it is a "Problem Domain Object Model" (PDOM) and a bad one at that (since it wasn't built with a domain expert involvement). The role of a PDOM is to form an initial working vocabulary with domain experts. It is usually a big mistake to use it as a foundation for development
  3. Even as a PDOM and even considering it was built for the sake of argument, it still doesn't make sense to create a class for each type of dog. When you create a sub-class you need to have a new behavior not a different value in a property.
  4. The example of the client side objects which do not match business side objects doesn't prove anything. client object (esp. search dialogs) tend to cross or aggregate domain boundaries. You can still have a meaningful OO domain model running in a backend.
  5. The fact that the search screen aggregates data from multiple domain object doesn't mean that the other processes and screen don't use that but from the application type at hand I would bet that most of the screens only deal with CRUD which brings me to another point:
  6. Another consideration Keith fails to mention is the type of application at hand. For the example given. It is probably a viable option to develop the whole thing as an Access application with embedded SQL in the controls. no OO and and no nothing.
  7. In any event. It seems to me Keith doesn't provide enough requirements to justify either of the solutions
In summary, yes it is completely wasteful to draw an off-hand solution that doesn't support any concrete requirements but that's an absolute truth whether you are using OO or no.

Edit 5/5: After looking at Evan Hoff's post on this. I understand that the point is  better delivered if you can also see the object model in question. So here it is:


Figure 1: Poor OO model - don't use

 
Tags: Design | OO


Among the reactions I got for my previous post on the Singleton pattern in .NET were a couple that talked about the design rationale behind the solution I posted:

Adi Avnit posted on the risk of using a "generic singleton":
"However, there is one possible pitfall to this approach, as it makes this code possible:
Singleton obj = Singleton.Instance;
MyClass obj2 = new MyClass();

While I personally like the idea of having the freedom to use the same class in two different ways throughout the application, I know some people like their Singletons - well, single.
On the other hand, if you write a class from the start as a Singleton this is not an issue.

There is an inherit risk in decoupling a class from it's expected behavior, so take this into consideration before using this pattern."

And Cade Roux wrote in a comment:
"don't believe discoverability was a motivation for Singleton. The purpose was to ensure only one instance existed (a print spooler, a file system). You can get discoverability through the technique you explain but the primary purpose of singleton was a pattern of prohibition - to stop a programmer doing something they shouldn't do by accident by enforcing the primary goals of single point of access.

It is more than a global, and the OP, while comparing them to globals, does not acknowledge that this solves many of the problems which made "global" a 4-letter word. Knee-jerk reaction against globals is not healthy once you've mitigated their drawbacks.

It is true that usage of singleton can be misguided and cause coupling - which is why you need to ensure that the usage of the pattern matches the motivation in the first place.

This is a key point of design patterns which Alexander should have made clear in http://en.wikipedia.org/wiki/N...is_of_Form - the pattern is a result of a resolution of system of forces. It only satisfies that system - moving it to another different system will result in poor fitness. "
I guess that's a good opportunity  to talk about design issues regarding Design Patterns in general and the GoF ones in particular.

Aristotle Pagaltzis
noted (in a comment on Cedric's blog) that
"design patterns are a sign of a deficiency of a language for the purpose that the design pattern addresses. In other words, the Visitor pattern used in Java points to the fact that Java is deficient in terms of list processing: the `map` and `filter` constructs need to be emulated with lengthy OO incantations."
In other words Concrete patterns* (like GoF's) are tied to implementation which means that the implementation language is  part of their "context". Thus when you come to apply a pattern in a different language you need to consider the language in use and make sure that
  1. The pattern is needed
  2. The solution in the pattern is the best one for the language.
Let's look at a few examples.
If we take the Visitor pattern mentioned above - The intent of the visitor pattern is to "represent an operation to be performed on the elements of an object structure without changing the classes of the elements". In .NET 3.5 you can do that with the use of LINQ (to find relevant items) and extension methods (to add external functionality) - same intent, completely different implementation.

In the case of the Singleton pattern (mentioned in the previous post). The intent is to "Ensure a class only has one instance, and provide a global point of access to it". An implementation using templates (or generics) is superior since it provides the same intent but also adds better compliance with the Single Responsibility Principle. While it is possible to create the class as a non-singleton on the one hand it solves the multi-threading issue in one place preventing both duplication of code and mistakes (again this can be especially important in non-forgiving environments like C++). It also lets you (not in the implementation I provided) add flexibility and e.g. let singletons die and (if needed) resurrect themselves etc. Note that even if you are not convinced that using generics is better, there's no doubt that using .NET's thread-safe static readonly variable (public static readonly MySingleton = new MySingleton();)  is a much simpler solution than the GoF one and again, answers the original intent with a different solution.

Another example would be the Template method pattern. The idea behind the template method is to support the Open closed principle (classes should be open for extension but closed for modification) or as the GoF defines the intent:
"Define the skeleton of an algorithm in an operation, deferring some steps to subclasses. Template method lets subclasses redefine certain steps of an algorithm without changing the algorithm strucuture"
.NET 3.5 supports the notion of Generic Delegates so you can define functions as parameters and pass them along. This is a better implementation than the template method as now you don't have to subclass when you want to extend an algorithm you can just pass a function that matches the signature
e.g.
    class Program
    {
        static void Main(string[] args)
        {
            var alg = new Algorithm();
            alg.DoSomething(Funky);
        }

        static bool Funky(string value)
        {
            return value == "somevalue";
        }
    }


    class Algorithm
    {
        public void DoSomething(Func<string,bool>CanGoForward)
        {
            string someVariable = "somevalue";

            if (null!=CanGoForward && CanGoForward(someVariable))
            {
                //do one step
            }
            // whatever
        }
    }


You should note that this "refactoring to fit the language" phenomena isn't limited to GoF patterns (or .NET :)) , for instance you can consider the use (or lack thereof) of Dependency Injection in languages such as Ruby which I wrote about a few months ago.

To sum up - when you talk about design patterns you need to consider the implementation language, design pattern deal with deficiencies in the language and different languages have different deficiencies and may have better solutions to the problems solved by the patterns



* As opposed to architectural patterns which are more abstract and thus more reusable (e.g. Hohpe & Wolf Enterprise integration patterns, or Martin fowler Enterprise Architecture patterns)


 
Tags: .NET | Design | OO

December 6, 2007
@ 11:43 PM
Microsoft uses the "live labs" to release all sorts of test balloons. Sometimes we get really nifty stuff like Photosynth or SeaDragon. Unfortunately, sometimes we get stupid not so bright ideas like Volta.

Ok, so what is Volta? Here's what the project's homepage has to say (emphasis mine):
" The Volta technology preview is a developer toolset that enables you to build multi-tier web applications by applying familiar techniques and patterns. First, design and build your application as a .NET client application, then assign the portions of the application to run on the server and the client tiers late in the development process. The compiler creates cross-browser JavaScript for the client tier, web services for the server tier, and communication, serialization, synchronization, security, and other boilerplate code to tie the tiers together.

Developers can target either web browsers or the CLR as clients and Volta handles the complexities of tier-splitting for you.  Volta comprises tools such as end-to-end profiling to make architectural refactoring and optimization simple and quick. In effect, Volta offers a best-effort experience in multiple environments without any changes to the application."

The idea sounds very compelling - I kid you not. So what's the problem?

The first issue is that, as a platform/framework (MS would say factory), Volta tries to accomplish too much. On the one hand Volta is another go at the web/desktop convergence trend. On the other hand it is supposed to be a solution for "painless" tier-splitting. Both of these tasks are very heavy. My opinion is that the Single Responsibility Principle (while originally defined for objects) applies here. And Volta should choose one thing and try to excel in that.

What's more disturbing to me, is the automatic handling of the "complexities of tier-splitting". Here's another excerpt from the Volta site which further explains the "tier-splitting" concept:
Objective

We have an application that runs in a monolithic environment, say the browser. We want parts of this application to run in other environments, such as servers. We don’t want to litter the application with plumbing code.

Rationale

The standard techniques for distributed applications infuse our code everywhere with information about what parts run where. This makes the code hard to change. Typically, once we make these decisions we can’t change them because it is too expensive. However, environments, requirements, and performance profiles change and we’re stuck with applications that can’t adapt to new realities. We need to separate the concerns about what the application does from the concerns about where parts of the application run.

Without Volta, we are forced to decide where code runs before we know everything it is going to do, in particular before we know the communication frequencies and delays. Development methodologies force us to make irreversible decisions too early in the application lifecycle. Volta gives us the means to delay decisions until we have adequate information to base them on.

Recipe

Volta tier splitting automates the creation of the communication plumbing code, serialization, and remoting. Simply mark classes or methods with a custom attribute that tells the Volta compiler where they should run. Unmarked classes and methods continue to run on the client.

We may base our decisions about tier assignment on any criteria we like, such as performance or location of critical assets and capabilities. Because Volta automates boilerplate code and processes for dispersing code, it is easy for us to experiment with and change assignments of classes and methods to tiers.

Wow, Agile development at its best, allowing us to postpone architectural decisions,  that just sound too good to be true. Well, the problem is that it is too good to be true. Abstracting the network out, and providing location transparency without thinking about the  implications of distribution is the reason "distributed objects" failed. e.g. Here is what Harry Pierson (DevHawk) had to say about distributed objects:
"...back in 2003, mainstream platforms typically used a distributed object approach to building distributed apps. Distributed objects were widely implemented and fairly well understood. You created an object like normal, but the underlying platform would create the actual object on a remote machine. You'd call functions on your local proxy and the platform would marshal the call across the network to the real object. The network hop would still be there, but the platform abstracted away the mechanics of making it. Examples of distributed object platforms include CORBA via IOR, Java RMI, COM via DCOM and .NET Remoting.

The (now well documented and understood) problem with this approach is that distributed objects can't be designed like other objects. For performance reasons, distributed objects have to have what Martin Fowler called a "coarse-grained interface", a design which sacrifices flexibility and extensibility in return for minimizing the number of cross-network calls. Because the network overhead can't be abstracted away, distributed objects are a very leaky abstraction.


So here comes Volta and tells us just put a [RunAtOrigin] attribute on the code you want on another tier and if you don't like that you can change it to another place in your application and what not. Note that the notion that you can automate some or maybe even all of the distribution "boilerplate" code may be viable. The problem is in the premise that you can seamlessly move that boundary around. There's a fundamental  difference between tiers and layers. Tiers should be treated as a boundary .Volta designers do talk about Security but they seem to forget a few of the other fallacies of distributed computing...



 
Tags: .NET | Agile | OO | Software Architecture | Trends

A couple of thoughts on functional programming in general and on F# and Erlang in particular

Erlang - is it worth it?
I read Armstrong's Erlang thesis and other Erlang related material with great interest. After all Erlang claims some very compelling architectural benefits in the areas of scalability and availability.  However one thing that prevents me from digging too deep into it is the utter ugliness of the language vs. for example, ruby which is very compelling
i also tend to agree with Otaku that at least today you can achieve this tauted resilience with other languages as well.

On the other hand we start to see some really interesting things built on Erlang like RabbitMQ which is an AMQP implementation (AMQP - The Advanced Message Queue Protocol), MNesia persistence engine . Yaws web server etc. (you can see here one guy that even likes Yaws + MNesia better than Ruby + Rails)

Well, I am still on the fence  for this one, but I guess I will take a deeper look at Erlang eventually..

By the way, while other functional languages also have irritating syntax (e.g. Scheme ) - some look more intereseting (like F# or OCaml)

 which brings me to the next tidbit...

Functional Languages moving mainstream.

Functional programming is an old paradigm (see for example this paper from 1984 "Why Functional Programming Matters" explaining the advantages of functional programming over structured programming..).

For a decade or even more and even today the object oriented paradigm has ruled the software development world. In a way, and as part of the end of the "one size fits all" paradigm (I also mentioned this trend here) we also see more pluralism for languages so we get more dynamic languages vs. static typed one and we find a place for functional languages and not just object oriented ones. (I'll expand more on that in another post).

Anyway, Yesterday S. Somasegar (MS VP of DevDiv) announced on his blog that F# will be "productized" :
"We will be partnering with Don Syme and others in Microsoft Research to fully integrate the F# language into Visual Studio and continue innovating and evolving F#. In my mind, F# is another first-class programming language on the CLR. "
Microsoft is of course not the center of the universe, but when a company like Microsoft chooses to bring something closer to main stream it is a significant move which, in my opinion, shows that functional programming is getting more traction.




 
Tags: .NET | Everything | Functional Languages | OO

A side effect of my decision not to become an independent consultant at this time means that I have to shelve some of the projects I was considering. One of these projects was to create a training program for software architects which I was discussing with a couple of training centers here in Israel.
Since It seems I am not going to promote it, I thought I'd share what I think a training program for .NET/Java architects should look like in the hope that someone would find it useful and do promote it (or parts of it)

Soft Skills
The way I see it architects needs a bunch of soft skills to be able to perform their roles.
Here is the list I identified in the past (by the way, I began a series of posts on each of these skills and never got to finish it - maybe it is time that I will :) )
  • Leadership. Influencing others to accomplish tasks and following your guidance
  • System thinking. Understand decisions and constrains in the
    wide scope pertaining to whole of the solution at hand. This includes
    the ability to abstract problems.
  • Strategic thinking. Understanding decisions and constrains and their alighments to the overall business of the company.
  • Organizational politics. Understand the environment you operate in and how it influences you.
  • Communications. Making sure you get your point across.
  • Human relations. Understand the "people" aspects or human
    factors and dynamics. This includes things like negotiation, pragmatism
    etc
I am not sure if you can teach all of them, but few courses that can help (in my opinion) include:

  • Presentation Skills - While getting the architecture and technology right is what matters, if you can't explain it to the different stakeholders you're toast.
  • Strategic Planning - This has to do with the vision thing I expect architects to manifest. Note that having a vision should not be confused with future-proofing a solution. future-proof means excess work not needed. Having a vision is knowing where you want to end - it can still be perfectly valid to completely re-write your applications along the way


Project Management
While the architect is not the project manager (mostly anyway), I think understanding the constraints coming from the project management point of view is very important. Since most environments call for a mix of agile and formal disciplines (hey, you've got to be pragmatic). I would train architect both in SCRUM and RUP (or some other formal methodology)

Also while not all environments needs this I would give an 2 days overview of important standards. The first would be IEEE 1471, which defines a standard for documenting software architectures. I would also teach ISO 90003 and CMMI.

It should be noted that the ISO 90003 is much better than the previous incarnation (ISO 9003) as it basically lets you define what you want to do to cover the different areas. The standard just helps you make sure you think about the various parts of project management (requirements, environment etc.). For instance I demonstrated how key areas of 90003 can be mapped to SCRUM to get it approved on my last project.


Languages, Design  and Patterns

I would want the .Net/Java architect training program to include at least 2 of the following languages:
Ruby, Scala, Erlang, F#, Python , Groovy ,OCaml
The reason for this is that these languages have different design goals than .NET and Java so learning them gives you additional perspectives and broaden your horizons for other ways of thinking (even if you don't use them in your project directly). You might have noticed that there's no .NET or Java training here. The reason for that is that's a prerequisite as far as I am concerned. You should master at least one

Object Oriented principles - hopefully aspiring architects already know this. However, I often see people who discovered some of the principles by themselves but haven't heard about all of them.
I am talking about principles such as Liskov Substitution  Principle , Open Closed Principle, Single Responsibility Principle , IoC containers, Don't Repeat Yourself  and YAGNI (I summarized my opinion on most of them in this paper)

The next step is to cover some design issues like Domain Driven Design, UI Design, Database modeling, Database alternatives l(after all the database is dead, right? )

Advanced design patterns - When most people hear the term design patterns they think about the GoF patterns. There are however literally hundreds of design patterns. Some of them are even worth learning :) . For instance there are patterns for concurrent and parallel systems like Proactor, Reactor Half-sync/Half-Async etc; Workflow patterns like Cancelling partial Join, Recovery Action etc; SOA patterns (ok, so I am still working on that :) )



Architecture Workshops
Another important part of the training, in my opinion, is to do some workshops and actually try to apply some of the material covered.

  • Architecture Evaluation - workshop 2-3 days - It is probably worthwhile to delve a little on scenario based evaluation techniques such as LAAAM and ATAM. While I prefer evaluating architecture in code, the scenario based thinking is very valuable for eliciting architectural requirements
  • Distributed Systems Architectures workshop - I'll expand on this in the next post


Lastly, there are also a few miscellaneous subjects like architect 101, the SPAMMED architecture framework , Agile architecture, Behavior Driven Design , common frameworks (though hopefully this would  not be needed ) like Spring/Spring.Net, Hibernate/NHibernate, iBatis  etc.


PS

Note that there are a few architect training programs available out there
One is offered by the Software Engineering Institute (SEI) and includes a 6 courses. SEI program seems to be focuses on formal sides of architecture as it includes courses on documenting software architecture and ATAM (You can see an old presentation I have on ATAM here)
Dana Bredemeyer  also offer architect training. Dana offers several workshops that cover the software architecture profession.
TOGAF (which is more of an enterprise architecture framework) offers both a certification and courses
Lastly, IASA is considering creating a software architect program and has a few courses in development
If you know any others I'd be happy to hear about them


 
Tags: .NET | Agile | Design | Everything | Java | OO | Software Architecture

September 14, 2007
@ 11:41 PM

Having just read Doron's Yaacoby's post on some of the insights he got  from reading Tom & Lister's Peopleware, I'd thought I'd repost a couple of posts I wrote about a year ago in my Dr. Dobb's Journal blog (with a few edits):

I just finished reading Software Conflict 2.0: The Art and Science of Software Engineering, by Robert L. Glass.

This is a reprint (with few new retrospective additions) of his 1990 book. While the technology mentioned in the book is outdated, many of the author's ideas and views are still valid. The book is a collection of short articles on various subjects, and one of the more interesting articles is about the cognitive side of design.

Glass explains that research done showed that design includes the (obvious) steps of understanding the problem, decomposing it into goals and objects etc. The essence of design, however, are the mental steps taken by designers:

  1. They construct a mental model of proposed solution
  2. They mentally execute the model (i.e. simulate the model to see if it solves the problem)
  3. If the model isn't good enough (e.g., too simple) replay the simulation to find what wrong and remodel
  4. Repeat 1-3 until the model seems to solve the problem

Glass also said that people tend to start with a model that worked on a similar problem and that good designers perform this process extremely fast. As I was thinking how one can train himself to get better at performing this task, it occurred to me that I heard something similar  somewhere... While not the exact process this is very reminiscent of TDD - only TDD makes the process explicit. In a way we can say that TDD will not only help make your design better it would also  trains you to design better altogether.

Robert's book has several other views still relevant today. While it can seem odd that a 16-years old book contains relevant thoughts looking at my bookshelf I see there are quite a few other books (some of which are event older) which contain essential information and brilliant ideas that are still very relevant today. Here are just a few examples:

Getting back to TDD or at least the idea of test first. It seems (via Rob Keefer pomiet blog)  Gerald Weinberg talked about it more than 35 years ago in  Psychology of Computer Programming :

"By pursuing this test example to the point where he understands the problem, he will not only learn the one thing he did not know, but perhaps will learn others as well, for test programs such as this are often better learning instruments than are production programs.

As a matter of good practice, the test program should be constructed before the 'fix' is made to the production program. In the first place, there will be an all too human tendency to forget about the problem once the production program is working correctly, so we must impose a little discipline on ourselves. Possibly more important, however, is the chance that by the mere act of constructing the test case we shall discover the problem."

Anyway, with so much good advice lying around for years (11-40+) and the fact that only about 30% of the projects are successful (on-time;on budget; on scope) I think one question we should all ask ourselves is -- don't we ever learn?


 
Tags: Everything | General | OO | TDD

From time to time I read about the magic that is RESTful services and how they solve everything and anything like scalability, idempotency, simplicity etc. for instance in "RESTful Web Services" by Sam Ruby and Leonard Richardson they say
 "PUT and DELETE operations are idempotent. if you DELETE a resource, it's gone. If you DELETE it again, it's still gone..." (p.103)
or
"the safe methods, GET and HEAD, are automatically idempotent as well" (p.219)

Another example comes from Anne Thomas Manes who said

"The REST architectural style defines a number of basic rules (constraints), and if you adhere to these rules, your applications will exhibit a number of desirable characteristics, such as simplicity, scalability, performance, evolvability, visibility, portability, and reliability.

The basic rules are:
  • Everything that's interesting is named via a URI and becomes an addressable resource
  • Every resource exposes a uniform interface (e.g., GET, PUT, POST, DELETE)
  • You interact with the resource by exchanging representations of the resource's state using the standard methods in the uniform interface
"

I think such claims  are plainly wrong and misleading.
 
Don't get me wrong, I like the REST approach, since it encourages better service design - e.g. document oriented message exchange vs. the RPC like message exchange which the so called "WS-death-*s" (or actually the tools that support them) encourages.

It also encourages the above mentioned traits - however that's exactly the  point - REST encourages this thinking not solves scalability or other problems out of the box- you still need to design your services properly.

For instance if you follow Anne's rules you can still end up with a service which is stateful, that performs heavy distributed transactions against multiple databases and systems - i.e. a service that is neither simple, scalable or perfromant

DELETE will only be idempotent if the resource is idempotent (e.g. a specific version of a resource)  or the message is idempotent (e.g. requesting a deletion of a specific version) if you are deleting the "recent version" then it might have been recreated between your calls you are now deleting something completely different. heck, even a GET (read) message with a single reader can be made to be non-idempotent  if you decide to code something that alters the state of a resource significantly whenever it is read. When you have multiple readers and writers GET will not be idempotent "automatically" as two consecutive reads can give you two different representations as the resource might have changed (again unless the resources are idempotent)

REST is not different from other styles in this respect - for instance you can do Object orientation in C but working in an OO language encourages object orientation (the opposite is also true - using an Object Oriented language does not guarantee that you get an Object Oriented design)

At the end of the day, architects should still think about the design if they want to ensure the results matches the quality attributes they want to achieve - some environments/styles/tools will make some quality attributes more easy to achieve but nothing will solve the problems for you.



 
Tags: Everything | OO | scalability | SOA | Software Architecture | REST

A few weeks ago I wrote about Dependency Injection and how it doesn't really matter in Ruby. On one of the comments for this post Yoni said (in regard to extending classed - both in .Net 3.5 and Ruby):

"Regarding the issue of extending classes (and partial classes) I am very sorry it was ever invented. Developers should be encouraged to divide large classes by decomposing them into decoupled sub-components using design patterns and not by simply splitting them between files. Even though a feature is called "extending" a class it is in violation of the Open Closed Principle..."

that sounds like a serious charge :) So here's another round of  "Ruby does it better" the time featuring Bertrand Meyer's "Open Closed Principle" or OCP for short.
OCP was defined in 1988 in Bertrand's book "Object Oriented Software Construction"  as follows:
Modules should be both open (for extension and adaptation) and closed (to avoid modification that affect clients)
When we work with a language like C# or Java we have several ways to do that - here's what I had to say about it :


It is easy to see the benefit of having a class that answers this principle: When you need to add a requirement, instead of breaking dependent code (and tests) you just extend it somehow and everything is nice and dandy. Furthermore, violating OCP can result in Rigidity,Fragility, and Immobility.

But how do you do that? The obvious (and naïve) answer is inheritance. Every time something needs to change just add a sub-class. The parent class is not changes and voilà. However, if you add sub-classes all the time you'd get "lazy classes" or freeloaders--sub-classes without a real reason for existence not to mention a maintenance nightmare.

Thus, sub-classing is an option but we need to consider carefully where to apply it. Other (more practical ) OCP preserving steps include:

The way I see it Ruby (and other dynamic languages for that matter) just allows us to extend this repertoire by adding a few other options such as:

Singleton methods (that's not a very good name - but the more appropriate name "instance methods" was already taken by another poor naming choice...) which are methods added to a specific instance of a class:

class Foo
  def bar
    puts "foo.bar"
  end
end
   
obj = Foo.new
obj2 = Foo.new
def obj.bar  # redefining bar just for obj
  puts "foo.newbar"
end
obj.bar   # prints foo.newbar
obj2.bar # prints foo.bar


Closures - I already mentioned closures for C# but it existed in dynamic lanaguages for a long time now. closures are sort of like injecting a procedure. Here's a quick sample:

class Foo
  def bar(myProc)   # accepts a callable object
    foobar = "foo.bar"
    myProc.call(foobar) # call the object with a parameter
  end
end
   
obj=Foo.new
printer=Proc.new {|msg| puts msg}

obj.bar(printer)

Meta programming - Ruby has a lot of ways to add and change classes. methods and whatnot. Again. here is a simple example:
class Foo
end
   
obj=Foo.new

# obj.bar - error
Foo.class_eval do           # this simplistic example can also use regular def
    define_method :bar do   # but class_eval can also evaluate strings to create
      puts "foo.bar"        # dynamic methods like setters etc.
    end
end

obj.bar
And there are a few other similar ways.

All of them  are not violating OCP!

OCP is kept since we do not alter the original class. Anyone using the original class not in our context (i.e. not in the modified context) will not be affected by the change. The interface of the class, in the sense that was originally defined is not changed either, and remains stable. Any client that uses the modified or the original class will not have to change it syntax because of the changes we've maid. This is also in-line with the "protected variation" way of looking at OCP :
"Identify points of predicted variation and create a stable interface around them."
While in C# your would maybe want to add a template method or a specialized interface in Ruby you can apply YAGNI and only change the behavior if the need arise

Lastly, we can also extend classes - but OCP lets us do that so again, we are OK.

Nevertheless, we should be careful not to violate Liskov Substitution Principle when we do all that, which I guess is relatively tempting to do if you using Ruby - but that's for another post ...


 
Tags: .NET | Design | Everything | OO | ruby