About me and how I see the role of an architect

 

There is a point where you start spending more time thinking on how you will build something than actually building it. You start thinking on what can you take out of your software instead of what can you put in. You realize that the only way to succeed today is to aim for the perfect simplicity. Although simplicity may sound easy, elegant simplicity is the most challenging goal in the software industry. If you are there you’ve passed the point of no return where you become an architect.

Although I have been on the role for a couple of years, and I had the basic skills associated with architecture only recently I have gone across that point and things look a lot more clear today. It’s a jungle out there.

Some believe that building software has become easier because technology has evolved, they could not be farther from the truth, building software has never been so hard, the timelines have never been so short and the basic requirements have never been so complex.

Shared identity, devices as in BYOD, form factors, usability, multi-tenancy, scalability, resilience are just a few of the attributes that users have been thought to take as normal, granted, part of any software that goes in the browser. Traditional software, at least the trend, had one, two, certainly not all. So how do you achieve it and still meet that weekly cycle?

Well the good news is that it doesn’t take a lot of code, only the right code and building on top of the right tools; it’s about how deeply you understand the problems, the protocols and the frameworks and how you put that knowledge in place to make it as simple as humanly possible and still make it great.

One example, I recently had to build an OAuth 2.0 server followed by an ASP.NET WebApi AuthorizationFilter to check the authorization token issued by that server. One print page of lines of code, weeks of reading to learn the protocol, to understand how claims can be used to simplicity the identity problems, how the token is digitally signed and how do you validate that signature, what can you do with OAuth 2.0, what does it enable, how can you extend it to go beyond what common OAuth providers do, and so one. It’s quite simple now because I have gone to the process of making it simple, a very hard process. If had just designed a custom solution, the best designed custom solution, it would have taken longer and it would be much less useful. An architect must embrace the pain of being ignorant about most of the things, and deal with it.

To finalize another example, also recently I have been working on applying different system metaphors (the list of parts you need to code to build a feature) to web development using .NET, trying to find one that isn’t just an hack that no one can evolve in the long run, but also that keeps the rate one screen one day (my goal for now). I really know how to build it with all the layers, make it SOLID, make it unit testable, as well designed as you can dream of but it breaks the one day goal, and no design is good if it doesn’t fit on the budget. I have finally made it, and guess what, it scales really well, I just have to tell my coworker what is the best example to follow and he can just add a new view in html with the bindings in place, a new viewmodel with those Knockout observables in place, a resource model, an api controller, a domain model, the repository, and a new property on the DbContext and it just works. If more coworkers are available they can repeat the steps and we can move really fast, it will have a balanced architecture, not to complex, not just an hack and you can make it move a little bit faster by adding more hands to the job. If I had a part to the metaphor I lose the productivity and if I take one out than it gets harder to test and to meet all the non-functional requirements, it’s a balance. In the end this is more about architecture then just the design or the protocols.

Multi-Targeting made easy

This is just a very short post to point to a CTP of a toolset that will be very important for those that like me have to write code that runs in multiple CLR hosts. The Portable Library Tools adds a new project template to visual studio that helps you through the process of writing code that runs on all selected platforms. Once the project goes RTM there is the promise to support executing the produced assembly in all the hosts. To do this today I have to maintain separate project files even if the code is exactly the same. You can read more about it here.

Model Based Programming

I am glad to announce the I decided to start blogging about what I have been working on on the past years - Model Based Software Development. If you want to learn about the new exciting age of generation full products based only on models stay tuned at Model Based Programming blog.

Some LINQ to Object Internals – How does the C# compiler mix Expression Trees with the LINQ operators extension methods?

I’m build a lot of code on top of LINQ that uses Queryables and Expression Trees and I have to admit it gave me a hard time. This post is about sharing some of the stuff I have learned by looking at the LINQ source code and I hope that some of you will also share some of your understanding on this topic here.

For the sake of simplicity lets stick with LINQ to Objects and group all extension methods and LINQ providers that deal with IEnumerable as Enumerables and the ones that deal with IQueryable as Queryables. If you look at their source code you will see that the actual code that runs behind the scenes is quite simple and most of the magic is actually triggered by the signatures.

Queryables receive Expression<Delegate> and Enumerables delegates. By reading through the documentation the explanation for this is quite simple. An Enumerable is designed to work locally while a Queryable is designed in a way that you can serialize the query and execute it somewhere else.

The first example here will start with a very simple query:

var root = new int[] { }.AsQueryable();
var query = from r in root
    where r > 10
    orderby r descending
    select r;

In the brackgroung the compiler is emitting something equivalent to the following snippet (I have changed the code a bit to make it easier to read):

var parameter = Expression.Parameter(typeof(int), "item");
var q = new int[0].AsQueryable<int>().Where<int>(
     Expression.Lambda<Func<int, bool>>(
         Expression.GreaterThan(parameter, Expression.Constant(10, typeof(int)))
         , new ParameterExpression[] { parameter })
 )
 .OrderByDescending<int, int>(
     Expression.Lambda<Func<int, int>>(
         parameter = Expression.Parameter(typeof(int), "r"), 
 new ParameterExpression[] { parameter }));

When the AsQueryable extension method is called it receives an array of items with 0 elements. The where clause in C# will translate into a call to the Where extension method (in this case the one that applies to Queryable). Because that extension method receives Expression<Delegate> and not the Delegate the compiler translates the “r > 10” into the equivalent expression tree. That expression tree can then be serialized and sent to be executed elsewhere or it can be translated to code by calling Lamdba.Compile.

I was trying to understand why taking an Expression from an IQueryable and calling provider CreateQuery on another queryable over the same data type did not work. By looking to the code generated by the compiler the awnser seams obvious. Although a part of the Expression is constructed by using Expression Trees the most important part (the Where) is still a call to the generic Where on top of an array that contains zero elements. The only way to change this is to create a visitor that emits an equivalent call chain on another data source. If I was using a LINQ to Blah Blah Blah provider that serialized the query it might work because it would depend on the interpretation that the provider gave to the root of the query, that is int[0], maybe a customers[] would be interpreted as the table of customers :).

Have fun,

Beatifull Code Part 2 – Combining RIA Services with Reactive Extensions to build a push based search

A common scenario in most applications is looking up values based on the user input in a auto-complete fashion. If you want to do this with a RIA service you have to (let’s ignore MVVM for the sake of simplicity) subscribe to the TextChanged event, grab the Text value, get the query from the DomainContext, apply a Where clause using the given text, let’s say StartsWith, create a LoadOperation, wait for the completed event and on the completed event clear the contents of the previous search and write the new search result. Its really messy!

It is complex because you are converting a data source that is push based (the textbox pushes text to your application) in to a pull based logic and you have to handle all those async problems yourself. But imagine that your text box is pushing a projection of the data filtered by the typed value into the search result control. You can think of a “eih C# just put the xxxs that StartWith ‘text’ into the data grid!” How do we write this?

I use a very dummy user interface:

<sdk:DataGrid AutoGenerateColumns="True" Height="296" HorizontalAlignment="Left" Margin="26,178,0,0" Name="dataGrid1" VerticalAlignment="Top" Width="598" />
<TextBox Height="23" HorizontalAlignment="Left" Margin="26,139,0,0" Name="textBox1" VerticalAlignment="Top" Width="278" />
<sdk:Label Height="28" HorizontalAlignment="Left" Margin="26,105,0,0" Name="label1" VerticalAlignment="Top" Width="120" Content="Name" />

And this is the code that pushes the TextChanged event into the service and back to the datagrid:

  var observableText = Observable.FromEvent<TextChangedEventArgs>(this.textBox1, "TextChanged");

  var text = (from change in observableText
             select ((TextBox)change.Sender).Text)
             .DistinctUntilChanged()
             .Throttle(TimeSpan.FromSeconds(1));

  var queryStream = from input in text
                    select context.GetCustomersQuery().Match(i => i.FirstName.StartsWith(input));

  var loadStream = from query in queryStream
                   select CreateLoad(query);

  var resultEventStream = from load in loadStream
                          select
                               Observable.FromEvent<EventArgs>(load, "Completed");

  var resultStream = from ev in resultEventStream
                     select ev
                         .ObserveOnDispatcher()
                         .Subscribe(current =>
                         {
                             var completedOperation = (LoadOperation<IndividualCustomer>)current.Sender;
                             this.customers.Clear();
                             foreach (var c in completedOperation.Entities)
                             {
                                 this.customers.Add(c);
                             }
                         });


  this.topmost = resultStream.Subscribe(); 

  this.dataGrid1.ItemsSource = this.customers;

The CreateLoad is just a helper function to allow customizing the query:

private LoadOperation<IndividualCustomer> CreateLoad(EntityQuery<IndividualCustomer> q)
{
    return this.context.Load(q);
}

The algorithm is quite simple, I take the event from the TextBox and convert it to an Observable of Text inputs. Then I take that Observable and create an Observable of Queries, them into an Observable of queries loading and  finally to an Observable of the results of loading those queries. I subscribe to this final one and conceptually I have the text that the user typed pushing loaded data back to me :).

Have fun,

Pedro

Beatifull Code Part 1 – Implement the repository pattern asynchronously with RIA Services

 

I’ve bought the book “Beautiful Code” published by O’Reilly and edited by Andy Oram and Greg Wilson and it has inspired my work in the last days. It doesn’t actually teach writing beautiful code or it would just be another GoF or Fowler book, it teaches you how the best think and it puts you thinking like the best. Hopefully thinking like the best will make you one of the best :).

Ten years ago when I left school if one would ask me where the best programmers spend their time I guess that I would answer writting very complex very fast. Yet today I spent hours thinking on an interface and how would people use it. I did that because I realized (through the book) that the best delete code instead of writing it. Code is bad! Code is ugly! Code is money spent! The less the better! What I really want is to reuse the same code over and over to solve equivalent problems by hiding the details.

What I was trying to do is to bridge the gap between the way you change collections of objects and the way you use RIA services to change tables. I looked at IEnumerable<T>, at ICollection<T>, at IList<T>, at EntitySet<T>, all the change tracking interfaces and the paged collection interfaces and at the end the only phrase on my brain was what a mess!

It is amazing how a mess can be created from a very simple mistake, the idea that operations that act on sets are synchronous. Have you ever asked yourself why would Add return void? Why would it return “immediately”? Imagine you are adding one element to a collection that as 10^9 elements and must always be sorted, it can take quite some time until add returns control to the caller. Until that happens the caller is blocked, wasting resources doing nothing. But if you say Add cannot be synchronous what would it look like? Well if I was on the Desktop market then I would say Task<T> but since I am on the Silverlight I must come up with a different name to the same idea because I don’t want to clash with that name in the future. So I came up with IResult:

 

/// <summary>
/// Represents the result of a method that cannot be completed synchronously.
/// </summary>
/// <remarks>
/// For void methods we consider that if no Exception was thrown the method executed successfully.
/// </remarks>
public interface IResult
{
    /// <summary>
    /// Occurs when all the activities initiated by the method that return this instance are completed.
    /// </summary>
    event EventHandler<EventArgs> Completed;

    /// <summary>
    /// Gets the error that ocurred during the execution of the activities triggered by the method that 
    /// returned this instance or null if no error occurred.
    /// </summary>
    Exception Error
    {
        get;
    }

    /// <summary>
    /// Gets a value indicating whether the method that returned this instance terminated in error.
    /// </summary>
    bool HasError
    {
        get;
    }

    /// <summary>
    /// Gets a value indicating whether the method that returned this instance finished its execution.
    /// </summary>
    bool IsComplete
    {
        get;
    }

    /// <summary>
    /// Gets a tag that identifies the method that returned this instance.
    /// </summary>
    string Action
    {
        get;
    }

    /// <summary>
    /// Gets a value representing the input of the method that returned this instance.
    /// </summary>
    IEnumerable Request
    {
        get;
    }

    /// <summary>
    /// Blocks the current thread until the underlaying activities complete.
    /// </summary>
    void Wait();

    /// <summary>
    /// Blocks the current thread until the underlaying activities complete but setting a timeout.
    /// </summary>
    /// <param name="miliseconds">The amount of time in miliseconds that the current thread will wait for a result.</param>
    void Wait(int miliseconds);
}

IResult is basically a pull based mechanism for someone to pull asynchronous method call state. We will get to a push base mechanism on a future article. IResult is inspired in OperationBase from RIA but it attempts to hide the semantics linked with web requests.

Ok, now that we have IResult how would a Repository contract look like?

/// <summary>
/// A set of elements that cannot be changed synchronously and that interfaces with a persistent storage
/// using a collection like interface.
/// </summary>
/// <typeparam name="T">The Type of elements in the repository.</typeparam>
public interface IRepository<in T>
{
    /// <summary>
    /// Gets a value indicating whether the repository can be modified.
    /// </summary>
    bool IsReadOnly
    {
        get;
    }

    /// <summary>
    /// Gets a value indicating whether the repository was modified.
    /// </summary>
    bool IsChanged
    {
        get;
    }

    /// <summary>
    /// Gets a value indicating whether the repository allows adding elements.
    /// </summary>
    bool CanAdd
    {
        get;
    }

    /// <summary>
    /// Gets a value indicating whether the repository allows removing elements.
    /// </summary>
    bool CanRemove
    {
        get;
    }

    /// <summary>
    /// Gets the number of elements in the set asynchronously.
    /// </summary>
    IResult<int> Count
    {
        get;
    }

    /// <summary>
    /// Adds the given element to the set asynchronously.
    /// </summary>
    /// <param name="item">The element to add to the set.</param>
    /// <returns></returns>
    IResult Add(T item);

    /// <summary>
    /// Determines asynchronously whether the given element is in the set or not.
    /// </summary>
    /// <param name="item">The element to check whether it is on the set or not.</param>
    /// <returns>A representation of the current state of the activities initiated by the method call.</returns>
    IResult<bool> Contains(T item);

    /// <summary>
    /// Removes asynchronously the element from the set.
    /// </summary>
    /// <param name="item">The element to add to the set.</param>
    /// <returns>A representation of the current state of the activities initiated by the method call.</returns>
    IResult Remove(T item);

    /// <summary>
    /// Accepts the changes made to the set making them permanent.
    /// </summary>
    /// <returns>A representation of the current state of the activities initiated by the method call.</returns>
    IResult AcceptChanges();

    /// <summary>
    /// Discards the changes made to the set reverting to the values that where previously on the persistent media.
    /// </summary>
    /// <returns>A representation of the current state of the activities initiated by the method call.</returns>
    IResult RejectChanges();
}

I would recommend you to spend a couple of minutes thinking on Count. In a standard collection Count would return the number of elements that are in the data structure holding the data, but when we are counting records on a table through a service call that can take quite some time. It makes sense to return an IResult. When the Count operation is completed then the Completed event will be raised and the program can use that value. Until that happens it can do other stuff. In the next article we will see how we can tweak this interface to take advantage of pull base mechanisms and reactive extensions. I’m building a framework based on these ideas and I will release the source code if someone is interested in it.

MEF TIP – How to initialize CompositionHost with all the assemblies contained in the application main XAP?

It’s a very a simple operation although not obvious (at least to me):

- Create an instance of the DeploymentCatalog class using the empty constructor. This gets the list of parts included in the manifest of the main XAP.

- Call CompositionHost.Initialize(deploymentCatalog, …)

If you are loading XAPs outside of MEF you must keep track of the assemblies you load and save that information, then create one AssemblyCatalog for each of then and merge them in a AggregateCatalog.

Have fun,

Pedro