I’m writing a short series of posts for MS Israel MCS blog (in Hebrew) and I’d thought I’d translate them to English, as it seems to me they are interesting enough.

In this series I am going to talk about Evolutionary Architecture or , some of the aspect of dealing with software architecture in agile projects. The topic is interesting since architecture and agile seems to have some conflicting forces at work to better understand that let’s start by defining software architecture

There are many definitions for software architecture, with the simplest one (attributed, I think, to Kent Beck) that software architecture is what software architects does. Leaving the fact that (unfortunately) sometimes software architects are very far from building software architectures, the definition doesn’t tell us much. There are many definitions around some are good and some are bad. My current definition is

“Software architecture is the collection of decisions affecting the system’s quality attributes; which have global effects and are hardest to change. Software architecture provides the frame within which the design (code) is built.”

Let’s review the components of this definition

• “affecting the system’s quality attributes” – I’ve written a lot about quality attributes in the past. In a nut shell, quality attributes (often going by the name “non-functional reqs.”) includes aspects of the system like scalability, security, availability etc. Architectural decisions have a direct effect on the system’s ability to meet these types of goal;
• “Global effects” – Design decisions effect the module or the class where they happen. Decisions with macro effect (e.g. choosing a technology, scaling approach) can completely alter a solution
• “Hardest to change” – The most interesting part of the definition, at least in regard to evolutionary architecture.The definition mentions that the code is built within the frame and rules set by the architectural decisions. Change in these decisions can have significant consequences. As a (over simplified) example - You can’t take a standalone system developed in Access and move it to a service oriented, Hadoop based solution with out major changes in the code, data flows and what not.

The definition of Software Architecture above, seems to prescribe that for best results we need to do a lot of up front design to get the architecture right. If that’s true than we have a severe mismatch with agile and/or lean where handling requirement and design up-front is a big no-no (YAGNI- you ain’t gonna need it, comes to mind) – Is there any way to make them work together.

I think yes, and as you’ve probably guessed, the answer is evolving the architecture over time. While this may sound simple it isn’t – I’ll try to give a few strategies to make that work later in the series. Before that, the next part, we’ll examine why design can be emergent which architectures need to evolve

Moving to architectures like SOA that increase the number of overall “moving parts” or components in the system means that reliability is going down. It is simple math really – if you have 10 components each with a 0.99 reliability then the total reliability is 0.99^10 or 0.904 and that’s before we take into account messages traveling over the wire and the network’s reliability (or lack thereof). What this does is leave us trying to build reliable systems from (a growing) bunch of unreliable components. I know, I know, there’s nothing new here. We’ve been using techniques like redundancy, statelessness etc. to help mitigate this since the beginning of times. With these techniques we decrease the “Mean Time Between Failure” (MTBF) but increase  the “Mean Time Between Critical Failure” (MTBCF) or the system’s overall MTBF.

Another aspect of reliability (and reliability calculations) is MTTR or “Mean Time To Repair” which in software mainly has to do with how much time does it take before we know something is wrong. The usual approach to that is monitoring which I’ve written about in the past (e.g. the blogjecting watchdog pattern). In this post I want to expand a little on another approach , which while not common in IT systems, can be useful at times.

Enter the BIT – which is short of “Built In Tests”. BIT is a technique I picked up when I worked on multi-disciplinary systems that also included embedded systems. Each and everyone of the embedded systems we developed (or integrated into the solution) supported BIT . Actually they usually supported several types of BIT at least PBIT, CBIT and IBIT

• PBIT – Power-On Built In Test – usually a short test the system runs to make sure all of its components are ready to go. You actually saw this one a lot of times since this is what motherboards do as you turn them on (all the blips and lights etc.)
• CBIT – Continuous Built In Test – Make sure the system is functioning, even when it isn’t really busy so we’ll know about problems before we actually try to use the system
• IBIT – Initiated Built In Test – provides a way to find out exactly what’s wrong when one of the other test types failed

BIT is very understandable for embedded systems, after all these are closed boxes with limited access to their innards and inner workings. but isn’t that also true for SOAs? After all we are building a bunch of blackboxes that interact to provide some business benefit, how can we be sure that everything is working fine esp. when we don’t control fully control some of the parts?

As mentioned above, a system, especially a distributed one, is built from relatively unreliable components. A continuous test helps us make sure things are working as expected. What we are doing is taking some of the code we wrote to run integration and acceptance tests (which runs a scenario end-to-end) deploy it as a service into the system which we call “liveliness check” and have it run periodically. Every time the liveliness  runs it sends a notification (twitter message) so we know the test itself works. If it fails it sends more notifications (twitter, Email, SMS etc.) to an administrator.

This liveliness or CBIT serves as an early warning system. Since the end result is known in advance we can have a pretty good idea if something went wrong. E.g. we know how much time it should take for a test Id, we know what the result of that image is etc. The fact that it works even when the system is in low utilization means we can find out about problems and deal with them before they happen to end-users. That’s a big plus for us.

The advantage over regular monitoring solutions (this is not an either/or – monitoring is also needed) is that you know the specific business scenarios are properly working, which is a higher confidence that things are ok from knowing a specific server or service is running.

On the flip side, or the downside of adding a periodic liveliness is adding complexity into the system. In our case, we have to add a process to clean the traffic data added by the test messages. Also, while we try to make the system behave as usual as much as possible,  certain parts of the system will have to know about the test messages and handle them differently. Again, in our case the reporting has to know to disregard test messages and not count them. This is even more problematic in other types of systems, for instance if you simulate an order, you don’t want the purchase order to actually go out to a supplier.

To sum this up, adding a liveliness check as part of the system to create a continuous built-in-test can increase your confidence that things are working as they should. It can also help you identify problems earlier. Like everything in life, it doesn’t come without tradeoffs and you should weight your benefits vs. costs before utilizing it in your systems.

You’ve probably read/heard that yesterday Google held a press event on making the ChromeOS the’ve been working on an open source project. Actually they’ve announced ChromeOS back in July, but now that the source is available it is making more waves. This is a very logical move for Google, and even though I think that they will need to enable some local/offline capabilities before it would be a viable option.

As usual* for Google, they are not the first in this space (If you want to get a webbook today you can try out litl, which looks like a very nice,  or the GOS cloud on the Gigabyte M912)  - but Google’s size and position in the market makes this interesting for everybody.

Anyway, I am almost thinking about a career change to analyst as I’ve been says that’s what they’ve up to since they released Chrome in Sept. 2008. when I published Google Chrome -The browser is the new Desktop. Here’s an excerpt:

Now,in my opinion, Google makes a bold move to change the rules and re-define the playground - if webapss need to run on the desktop, let's make the browser the new desktop.
What makes me say that? because it is focused on application (see the comics),because the browser runs each tab in its own process, because it has a process monitor, because it is a link on the google home page...

From the chrome "OS" point of view we can look at javascript,HTML etc. as the IL (bytecode in java speak) on which the application run. This makes cross-compilers like GWT and the good side of MS Volta (vs. the bad side) the next abstraction layer. I expect these will be more significant in the future

On the same note, it is probably good time to provide a link to a presentation on the future of PCs I’ve created in 2007, which highlights some of the trends that are more apparent today in moving to the cloud

* the most obvious example is of course search (altavista, hotbot etc. have all been there before..), a more recent example is the free GPS turn-by-turn where Waze (an Israeli company by they way) is already there but Google’s announcement is considered disruptive

Jesse Ezell  left the following comment on my previous WCF rant (Windows Trick-or-treat Foundation)

You wouldn't expect WCF to take care every TCP/IP registry setting as well would you? At some point, the things WCF exposes have to stop so transport specific settings come into play. What would really suck is if WCF completely abstracted every detail of every transport and came up with new names for things like cookies and specific http request and response headers in the name of creating a "unified" experience across every transport.
The point of WCF is to give you a unified communication API. You don't have to change your code to switch from HTTP, to MSMQ, to TCP, etc. However, the point of WCF is not to hide all the specifics of each transport from you. There are a lot of transport specific options in WCF that you won't find in a WSDL file. However... none of these settings actually require you to change any code to take advantage of them. IMO, WCF does a really good job of providing a uniform communication API and limiting transport specific details to configuration settings. Even in this case, you were able to resolve the issue by configuration settings rather than code changes.
If you had used http request manually, this is one of very few settings you would actually have control over via configuration. What about other popular competing APIs? Look at something like NServiceBus. NServiceBus has a transport model that abstracts communication to some degree... but what happens when you want to change the format of the messages on the wire? For example, maybe you want to switch from raw XML messages to messages with SOAP envelopes on an endpoint or limit the depth of XML node hierarchies to protect against XML attacks. Maybe you want to switch to a completely new transport that was provided by a vendor that has never seen your product. Maybe you want to add compression, or chunking, or certificate based security. Can you do that all via configuration files without ever touching code in any other .NET API? Maybe this is my own ignorance... but I don't know of any .NET communication API that offers even half of the flexibility of WCF.

Well, I would actually expect WCF to do one of two things either provide a complete API of all the communications need (next version of .NET communication , unified communication model and all that..) and retire the other .NET communication libraries or on the other hand provide a thin layer of abstraction that will make it clear you need to move to the specific underlying protocols.

What we’ve got now is something that isn’t quite there on the first and way away from the second – which means that when you try to do serious stuff with WCF you hit these unexpected (ok now they are) snags where you don’t know where to go – until you realize that this is a specific thing regarding TCP this or HTTP that which is not readily apparent and is not well documented or even worse you need to set it outside of WCF altogether.

In my experience , you can’t in fact, “just change the binding” and expect everything to work unless you are doing very simple stuff. For instance when  if you move from HTTP binding to TCP you’d find that the channels are suddenly getting closed after periods of inactivity and you need to “keep them alive” or if you move in the other direction from TCP to HTTP you’d find that the size of messages gets larger by an order of magnitude  etc.

Not to mention the  “training wheels” approach to setting defaults (at least some of it is fixed for .NET 4) which I talked about a few times in the past. Also there cryptic error messages that make you scratch your head looking for the configuration item you need to set. for instance if your send a large message (>8K) you won’t see any problem in the sending side but you’d get the following

“An exception of type 'System.ServiceModel.Dispatcher.NetDispatcherFaultException' occurred in mscorlib.dll but was not handled in user code

Additional information: The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:ClientPrintResult. The InnerException message was 'There was an error deserializing the object of type System.String. The maximum string content length quota (8192) has been exceeded while reading XML data. This quota may be increased by changing the MaxStringContentLength property on the XmlDictionaryReaderQuotas object used when creating the XML reader. Line 2, position 40523.'.  Please see InnerException for more details.”

I am sure all of you immediately understood you need to set  ReaderQuotas, MaximumReceiveMessageSize and MaxBuferSize on the binding e.g.:

var binding = new WebHttpBinding()
        {
            ReaderQuotas = { MaxArrayLength = 20 * 8192},
            MaxReceivedMessageSize = 20 * 8192,
            MaxBufferSize = 20 * 8192

        };

Don’t get me wrong, WCF isn’t all bad or anything like that but it does get annoying like hell at times.

As for the reference to other frameworks - I can’t speak for NServiceBus  because I didn’t write it (though I think the answer would be the same) - but if I consider the communication framework I did write for xsights (which builds on WCF by the way) it is not as presumptuous as WCF. It does not pretend to be an all-encompassing communication layer for .NET and  It is build to provide a specific architectural approach – which is why it shouldn’t be judged by the same standards.

There’s no Architecture in Business Service Orientation ! There, I’ve said it, there are no two types of SOA.  I am not trying to say that business-level service orientation doesn’t exist or isn’t valuable. However I am trying to say that labeling that SOA harms both Service Orientation at the business level and SOA (a.k.a. “technical SOA”)

For the record here are my definitions for both Business Service Orientation and Service Oriented Architecture (SOA)

Business Service Orientation is an IT  paradigm at the enterprise-level  that aims to componentize and partition the business’s software and to get composability and flexibility (and thus achieve better business and IT alignment etc.). Service Orientation can be implemented using various enterprise architecture practices (around governance, portfolio management etc)  as well as various software architectures including (but not limited to )  SOA, EDA, BPM, REST and combinations of them.

Service Oriented Architecture as an architectural style for building systems based on interacting coarse grained autonomous components called services. Each service expose processes and behavior through contracts, which are composed of messages at discoverable addresses called endpoints. Services’ behavior is governed by policies which are set externally to the service itself. SOA is derived of four predating architectural styles, namely Client/Server, Layered System, Pipes and Filters and Distributed Agents.

To reiterate -  calling Business Service Orientation SOA serves only to muddy the water and make both terms nebulous. If you just have to have a TLA just call them  BSO & SOA

PS

I know that in “What is SOA anyway” I also refer to two SOAs. I didn’t have enough confidence to say it bluntly when I wrote that paper, but I did emphasized Service Orientation for BSO and Architecture for SOA. Also regarding a more formal definition of SOA, that shows how it is derived from the four other styles – I started explaining that quite some time ago (Intro, Client-Server, Layered, Pipes and Filters) – I still need to explain Distributed Agents and summarize (about time I’ll do that)

Yes, this is another WCF rant…

We were getting ready to launch an open-for-all version of our service, we were also adding more cores to the system, to make sure our computation engine will be able to handle higher request loads (We’re basically implementing the Gridable Service pattern -  if it is interesting, I can expand on that in another post). We tried a few load tests, which on first look, seemed OK. However we then noticed that we get WCF timeouts on some of the calls.

WCF timeouts!? what gives? we already took care of all the annoying throttling defaults. After busting my head (and Google) for a few hours, I found that apparently, when using http bindings in WCF you only get 2 outgoing channels for each server (IP) you are connecting to. yep WCF thinks your services are novice users using a  browser. So if you have a service that tries to access a remote server with more than two requests simultaneously (say, under load…) you may find, like us, that you’d get occasional timeouts.

Yes, there’s also a solution, which also took time to find – it is called ConnectionManagment element in the network configuration (I am yet to find a programmable way to do this). So now the services app.config (see below) sets the value to 16 instead of 2 and everything is well again, at least until the next default hits us..

<system.net>
   <connectionManagement>
     <add address = "*" maxconnection = "16" />
   </connectionManagement>
 </system.net>

From trying to work with WCF in the last few years, it seems its abstractions are very leaky.They are almost leaky  to the point  of rendering it useless as a “unified” framework. So, in the spirit of the times, and as the title suggest – maybe they should just rename WCF to Windows Trick-or-treat Foundation – Alas, we are getting more and more of the “trick’ part rather that the “treat”, but at least the acronym fits well.

When we added our iPhone application xsights-light to the appstore about a month ago we intended it as a demo application we can use to showcase potential clients what we can do (with the aim of providing a white-label service). We activated a few images (3 on the site and a few others) and forgot about it.

We were very surprised to find out that even though we didn’t publicized it anywhere or anything like that, hundreds of people found it interesting, downloaded the app and tried it on just about anything. Here are just a few examples:

As I mentioned above, we only had a few images activated, so naturally, people weren’t very happy and we got some “rave” reviews like “would be nice if it worked as advertized”, “only identifies 3 images” and  others that used less refined language :)

Anyway, we’re listening – and we decided to make xsights-light a little more useful. For one we are adding content, we started with movie posters of movies playing in the US and we intend to add more in the future. What’s more interesting, is that we’re opening up the platform for general use i.e. you can upload (almost) any image you like (such as the images above), add a URL and activate anything you like be that a Pepsi can, an wedding invitation or whatnot (if you’d take a look at @xsightspulse on twitter, you can see what others are activating)

Apparently, we’re not the only ones who think this is interesting, as we also got a nice mention on Tech-Crunch. We agreed to let Tech-Crunch readers be the first users of our service so head over to the article for an invite, we hope to open it for general use in a week or so.