Always remember about IQueryable when using Expressions in LINQ!

Today I’ve faced unexpected behavior during querying data and JSON deserialization while playing with Marten library which uses PostgreSQL Database as storage. So let’s imagine we have a very dummy 2 classes, one for saving data (in JSON format) and one class for the projection (as a response of some Web API for instance):

Origin class for data storing:

public class User
{
    private User() { }

    public User(int id, string name)
    {
        Id = id;
        Name = name;
        Status = "Pending";
    }

    public int Id { get; private set; }
    public string Name { get; private set; }
    public string Status { get; private set; }
}

Projection class

public class ReadOnlyUser
{
    public string Name { get; set; }
    public string Status { get; set; }
}

So as you see, the User class has private setters and due to MSDN Deserialization behavior, JSON Serializer ignores read-only properties. This means that this code will always create a User instance with Status Pending

var stringData = "{\"Id\": \"1\", \"Status\": \"Active\",\"Name\": \"John\"}";

var stream = new MemoryStream(Encoding.ASCII.GetBytes(stringData));
var textReader = new JsonTextReader(new StreamReader(stream));

var user = new JsonSerializer().Deserialize<User>(textReader);
Result: 

Output object:
User Id: 1 
User Name: John 
User Status: Pending

So if we follow the example above, making a projection from such class, will produce the ReadOnlyUser which also will have a Status Pending:

 var stringArrayData = "[{\"Id\": \"1\", \"Status\": \"Active\",\"Name\": \"John\"}]";
 JsonSerializer serializer = new JsonSerializer();

 var stream = new MemoryStream(Encoding.ASCII.GetBytes(stringArrayData));
 var reader = new JsonTextReader(new StreamReader(stream));

 var user = serializer
   .Deserialize<IEnumerable<User>>(reader)
   .Where(usr => usr.Id == "1")
   .Select(usr => new ReadOnlyUser {Name=usr.Name, Status=usr.Status})
   .FirstOrDefault();

 Console.WriteLine($"\r\nOutput object:\r\nUser Name: {user.Name} \r\nUser Status: {user.Status}");
Result:

Output object:
User Name: John 
User Status: Pending

But what if we are not querying from a string variable, but rather from the database? Most probably instead of IEnumerable, IQueryable interface will be used.

In a nutshell, IQueryable brings 1 huge benefit – it knows how to execute expressions on special data sources. (For instance Where statement will be implemented in the database, not on the memory side)

So, coming back to the previous example, this code will also return User with Status Pending, won’t it?

return await _session.Query<User>() //Returns IQueryable<User> from db
   .Where(usr => usr.Id == "1")
   .Select(usr => new ReadOnlyUser {Name=usr.Name, Status=usr.Status})
   .FirstOrDefaultAsync();
Result:

Output object:
User Name: John 
User Status: Active
Me at that moment

So how did it happen, why Status is Active if the User has a private setter on that field?

In the first example, as we used IEnumerable, all of the operations are processed in our App memory. The order of commands was like that:

  1. Deserialize the JSON to IEnumerable list of Users (by using the only 1 public constructor which set Status to Pending without possibility to set it to Active because of private setter)
  2. Use Where statement to filter the Users with Id = “1”
  3. Use the Select projection for creating a new ReadOnlyUser

In the example with an IQueryable interface, steps 1 and 2 were “outsourced” to the database. So commands look so:

  1. Find by Object Type a proper table from which data should be fetched and create a query like “Select * from User”
  2. Add Where the statement “Where Id = ‘1’ ” to the query from step 1
  3. Execute a query in the database and return the result as a JSON object
  4. Use JsonSerializer for deserialization from JSON to ReadOnlyUser

So as you see, in the second scenario, many steps executed on the database side, but in that particular case, we gain even more profit because we could omit the limitation of Newtonsoft.Json of ignoring read-only fields during deserialization. In fact, we are not creating a User object but just go directly to the creation of ReadOnlyUser projection.

Please be aware, that every implementation of an IQueryable interface has its own rules of converting command and using expression tree. So before relying on “outsourcing processing”, just spend a few minutes to check what exactly will be executed.

DDD & CQRS & Event Sourcing. Part 2: Extending domain model by events

Continuing the topic started in the previous post, today’s subject will be related to the domain model with events and why it is important.

So first of all, let’s try to understand what is the “event”?

Domain event describes something, which is happened with the Domain Model and it is important for the domain experts. So let’s revise our domain model:

public class User: Aggregate
{
    public Guid Id { get; private set; }
    public string FirstName { get; private set; }
    public string LastName { get; private set; }
    public string Email { get; private set; }
    
    public User(Guid id, string firstName, string lastName, string email)
    {
        if(id == Guid.Empty)
            throw new ArgumentException($"User id could not be empty")

        Id = id;
        
        ChangeName(firstName, lastName);
        ChangeEmail(email);
    }

    public void ChangeName(string firstName, string lastName)
    {
        CheckNullOrEmpty(firstName, nameof(frstName));
        CheckMaxLength(100, firstName, nameof(firstName));
        FirstName = firstName.Trim();
        
        CheckNullOrEmpty(lastName, nameof(lastName));
        CheckMaxLength(100, lastName, nameof(lastName));
        LastName = lastName.Trim();
    }
    
    public void ChangeEmail(string email)
    {
        CheckNullOrEmpty(email, nameof(email));
        CheckMaxLength(50, email, nameof(email));
        var trimmedEmail = email.Trim();
        
        if(!Regex.IsMatch(trimmedEmail, @"\A(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*@(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)\Z", RegexOptions.IgnoreCase))
            throw new ArgumentException($"Email {trimmedEmail} is invalid");

        Email = trimmedEmail;
    }
}

So for now, the User domain model has 3 changes (events) which are important to business experts (and developers as well, but let’s talk about that later):

  • User Created – event which signalizes that user has been created
  • User Name Changed – the event will be raised every time when first and/or last name of the user will be changed
  • User Email Changed – the event will be raised every time when using email will be changed

First step is that the events should be created:

public static class Events
{
    public static class V1
    {
        public class UserCreated
        {
            public UserCreated(Guid userId, string firstName, string lastName, string email)
            {
                UserId = userId;
                FirstName = firstName;
                LastName = lastName;
                Email = email;
            }

            public Guid UserId { get; }
            public string FirstName { get; }
            public string LastName { get; }
            public string Email { get; }
        }

        public class UserNameChanged
        {
            public UserNameChanged(Guid userId, string firstName, string lastName)
            {
                UserId = userId;
                FirstName = firstName;
                LastName = lastName;
            }

            public Guid UserId { get; }
            public string FirstName { get; }
            public string LastName { get; }
        }

        public class UserEmailChanged
        {
            public UserEmailChanged(Guid userId, string email)
            {
                UserId = userId;
                Email = email;
            }

            public Guid UserId { get; }
            public string Email { get;}
        }
    }
}

From the code above, there are 2 things worth to mention:

  • Names of the events use past tense (for ex. UserCreated). Such style underline, that something already happened in the past
  • Properties among with the whole object are read-only. Partly it relates to the previous point. If the event is smth which has happened in the past, so it is impossible to change the state of the event. (for instance, we could not change the value of the name in the UserNameChanged event, because unfortunately, we could not change the things from the past)

So probably now you think it should look like this:

public object ChangeName(string firstName, string lastName)
{
    CheckNullOrEmpty(firstName, nameof(frstName));
    CheckMaxLength(100, firstName, nameof(firstName));
    FirstName = firstName.Trim();
    
    CheckNullOrEmpty(lastName, nameof(lastName));
    CheckMaxLength(100, lastName, nameof(lastName));
    LastName = lastName.Trim();

    return new Events.V1.UserNameChanged(userId: Id, firstName: firstName, lastName: lastName)
}

Well, in fact, we could that, but there are a couple of problems in such an approach – from the business point of few, method ChangeName should not return anything. We don’t ask here smth like “GetFullName”, so there is nothing to return after the method was executed.

So let’s fix this concern by introducing private _events property and methods for access to them:

public class User
{
    private readonly IList<object> _events = new List<object>();

    public ICollection<object> DequeueEvents()
    {
        var events = _events.ToList();
        _events.Clear();
        return events;
    }

    private void Enqueue(object @event)
    {
        _events.Add(@event);
    }

rest of the code ......

and the method will stay with the void type :

public void ChangeName(string firstName, string lastName)
{
    CheckNullOrEmpty(firstName, nameof(firstName));
    CheckMaxLength(100, firstName, nameof(firstName));
    FirstName = firstName.Trim();
    
    CheckNullOrEmpty(lastName, nameof(lastName));
    CheckMaxLength(100, lastName, nameof(lastName));
    LastName = lastName.Trim();

    Enqueue(new Events.V1.UserNameChanged(userId: Id, firstName: firstName, lastName: lastName))
}

*Small change has to be done in the constructor now, as soon as domain methods enqueue events, we could not reuse them in other methods. So we have to move the validation logic to the common part or create a Value Object (which will not be probably covered in that post series, but I highly recommend getting familiar with this concept in Domain-Driven Design).

So example of usage of the User domain now looks like that:

var user = new Domain.User(Guid.NewGuid(), "testFirstName", "TestLastName", "test@email.com");
var events = user.DequeueEvents();

or 

var user = userRepository.Get(id);
user.ChangeEmail("newTest@email.com");
var events = user.DequeueEvents();
  • UserRepository allows getting users from DB. Later in the course, we will use the Marten Framework and Event Sourcing approach for implementing the User Repository.
  • Received events could be used to notify other parts of the application about the changes in the User module (We will cover this in the future post related to Microservice Architecture) or for saving our domain model using Event Sourcing

Introduction to Event Sourcing

So now imagine, that Users in our system could be edited by multiple users, and it is crucial to have a guarantee that we have the full history of changes made in the User domain model. So probably you think that we could just have a history table and catch all events, which User generates, so we could have all the changes regarding the particular user (we will come back to that approach when will be touching the Projections in CQRS topic).

The problem with such an approach is that someone could always change accidentally the data in the database (for instance changing email from test@email.com to newTest@email.com). In that case, there will be a difference between history and actual data.

So if we are publishing domain events after each business change, why we could not use these events as a source of truth for building the domain model state?

So after the User has been created, we raise the UserCreatedEvent which contains all information about the created User. Next, we want to change the user name? No problem, event UserNameChanged has all info about this change. So in fact, we don’t need to save the User domain model state itself, it is only enough to save the current version of this User and a list of the events, which will be used for rebuilding the state.

Let’s take a look on a User model, which could be rebuild from events (without versioning for now):

public class User
{
    private readonly IList<object> _events = new List<object>();

    public string FirstName { get; private set; }
    public string LastName { get; private set; }
    public string Email { get; private set; }
    
    public User(Guid id, string firstName, string lastName, string email)
    {
        Process(new Events.V1.UserCreated(userId:id, firstName: firstName, lastName: lastName, email: email ));
    }

    public void ChangeName(string firstName, string lastName)
    {
        Process(new Events.V1.UserNameChanged(userId: Id, firstName: firstName, lastName: lastName));
    }
    
    public void ChangeEmail(string userEmail)
    {
        Process(new Events.V1.UserEmailChanged(userId: Id, email: userEmail));
    }

    #region Applies
    private void Apply(Events.V1.UserCreated @event)
    {
        Id = @event.UserId;
        Apply(new Events.V1.UserNameChanged(userId: this.Id, firstName:  @event.FirstName, lastName: @event.LastName));
        Apply(new Events.V1.UserEmailChanged(userId: this.Id, email: @event.Email));
        Status = UserStatus.Pending;
    }
        
    private void Apply(Events.V1.UserNameChanged @event)
    {
        CheckNullOrEmpty(@event.FirstName, nameof(@event.FirstName));
        CheckMaxLength(100, @event.FirstName, nameof(@event.FirstName));
        FirstName = @event.FirstName.Trim();
        
        CheckNullOrEmpty(@event.LastName, nameof(@event.LastName));
        CheckMaxLength(100, @event.LastName, nameof(@event.LastName));
        LastName = @event.LastName.Trim();
    }

    private void Apply(Events.V1.UserEmailChanged @event)
    {
        CheckNullOrEmpty(@event.Email, nameof(@event.Email));
        CheckMaxLength(50, @event.Email, nameof(@event.Email));
        var trimmedEmail = @event.Email.Trim();
        
        if(!Regex.IsMatch(trimmedEmail, @"\A(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*@(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)\Z", RegexOptions.IgnoreCase))
            throw new ArgumentException($"Email {trimmedEmail} is invalid");

        Email = trimmedEmail;
    }

    public ICollection<object> DequeueEvents()
    {
        var events = _events.ToList();
        _events.Clear();
        return events;
    }

    private void Enqueue(object @event)
    {
        _events.Add(@event);
    }

    private void Process(object @event)
    {
        Type thisType = GetType();
        if (thisType == null) throw new NotSupportedException($"Current this type is null!");
        
        MethodInfo methodInfo = thisType.GetMethod("Apply",  BindingFlags.NonPublic | BindingFlags.Instance, Type.DefaultBinder, new [] { @event.GetType()}, null );
        if (methodInfo == null) throw new NotSupportedException($"Missing handler for event {@event.GetType().Name}");
        
        try
        {
            methodInfo.Invoke(this, new[] { @event } );
        }
        catch(TargetInvocationException ex)
        {
            ExceptionDispatchInfo.Capture(ex.InnerException).Throw();
        }

        Enqueue(@event);
    }
}

So there are a few changes in the User class above:

  • All state changes have been moved to the Applys methods.
  • The Process method has been introduced. This method use reflection to call correct Apply method based on event type and enqueue events.
  • All public domain methods use Process method, to introduce a changes in the model.

So in the previous implementation, the first User changed the state and later raised the event. Now, the User first raises an event, and based on that event, makes changes into its own state.

The next post will describe how to test the Domain Model which is using events.

The latest code base version of the project could be found on Github Org Page.

DDD & CQRS & Event Sourcing. Part 1: Creating a basic domain model

This is the very first post in a series about using DDD (Domain Driven Design), CQRS (Command Query Responsibility Segregation), and Event Sourcing. I’m going to create a small project, which will use the above-mentioned approaches for solving some real-life scenarios. Before we start, I want to underline, that this posts series among the code project will be created for presentational purposes only. That means, that the business challenges which will be handled here could be solved without any above described practices. At this point, I’m not sure how far and deep this will go, so I will start with very simple and naive examples. It is always better to go in direction of complication, rather than simplification.

I’ve not decided so far what problem this project will solve, but I’m sure that we will need registered users for it. So let’s start with a User Service.

First of all, we have to create a User Entity class with some basic properties

Entity – an object that is not defined by its attributes, but rather by a thread of continuity and its identity.

    public class User
    {
        public string FirstName { get; private set; }
        public string LastName { get; private set; }
        public string Email { get; private set; }
        public string Status { get; private set; }
        
        public User(Guid id, string firstName, string lastName, string email)
        {
            FirstName = firstName;
            Id = id;
            LastName = lastName;
            Email = email;
            Status = UserStatus.Awaiting;
        }

        public static class UserStatus
        {
            public const string Awaiting = "AWAITING";
            public const string Confirmed = "CONFIRMED";
        }
    }

As you could see above, all properties have a private setter. In such a way, we could guarantee, that entity will be created with all necessary information and will not be in a non-valid state (for example User without FirstName).

We could check if all necessary parameters are valid in the constructor.

   public class User
    {
        public string FirstName { get; private set; }
        public string LastName { get; private set; }
        public string Email { get; private set; }
        public string Status { get; set; }
        
        public User(Guid id, string firstName, string lastName, string email)
        {
            CheckNullOrEmpty(firstName, nameof(firstName));
            FirstName = firstName;

            CheckNullOrEmpty(lastName, nameof(lastName));
            LastName = lastName;

            CheckNullOrEmpty(id, nameof(id));
            Id = id;

            CheckNullOrEmpty(email, nameof(email));
            Email = email;

            Status = UserStatus.Awaiting;
        }

        private void CheckNullOrEmpty(string paramValue, string paramName)
        {
            if (string.IsNullOrWhiteSpace(paramValue))
                throw new ArgumentException($"{paramName} could not be null or empty");
        }

...

I believe that now you have a question “So if properties have private setters, how could change their value?”. Very reasonable one, the answer is that we are going to create methods, which will describe the business perspective of that operations. For example, we are going to create a method, which will change the first and last names. This method will check if it is not empty and also that it does not break the business requirement of max length 100 characters:

...

public void ChangeName(string firstName, string lastName)
{
    CheckNullOrEmpty(firstName, nameof(firstName));
    CheckMaxLength(100, firstName, nameof(firstName))
    FirstName = firstName;

    CheckNullOrEmpty(lastName, nameof(lastName));
    CheckMaxLength(100, lastName, nameof(lastName))
    LastName = lastName;
}

private void CheckMaxLength(int maxLength, string paramValue, string paramName)
{
    if (paramValue.Length > maxLength)
        throw new ArgumentException($"{paramName} could not be longer the {maxLength} characters");
}

...

After that, we could reuse this method in a constructor, to be sure, that our Entity has a valid state during the creation and change of the name properties:

...

public User(Guid id, string firstName, string lastName, string email)
{
    ChangeName(firstName, lastName);

    CheckNullOrEmpty(id, nameof(id));
    Id = id;

    CheckNullOrEmpty(email, nameof(email));
    Email = email;

    Status = UserStatus.Awaiting;
}

...

The ChangeEmail method also should be implemented (with some email regex etc), but I’m going to omit that for simplicity for now. The important thing is that the User Entity will not have a ChangeId method, because it would break the business requirement and the User just could not change its own identification number. So in a such way, we guarantee that Entity will always be in a valid state.

Below you could find a potential example of usage of such Entity:

...

public void ExampleOfUsage()
{
    var user = new User(Guid.NewGuid(), "John", "Doe", "john.doe@email.com");
    
    //We also could change user name or email if we want
    //But there is not way to change the Id of the user, 
    //because Id has private setter and entity doesn't have ChangeId method
    user.ChangeName("Barbra", "Streisand");
    user.ChangeEmail("barbra.streisand@email.com");

    // Also we could check the user status, which has been set to Awaiting,
    // as the Entity has been created in a valid s
    Console.WriteLine(user.Status)
}

...

So now we have a basic Domain Entity, which ensures a valid state and encapsulates business logic inside.

The last but not least, if you an interested to go deeper into the topic of DDD, which we touched on today (in a very very superficial way) I would like to recommend a few resources, which could be useful for a deeper understanding of key concepts like Value Objects, Bounded Context, Aggregates, etc. :

The latest code base version of the project could be found on Github Org Page.

In the next post, we will get familiar with the Events, and how we could use them in Entities.