by Carles

Decoupling architecture and domain with repositories


DISCLAIMER: There are a bunch of things that can be improved in the examples shown in this post. If you’re an experienced developer you’ll surely have noticed. In this post we wanted to bring the architecture isolation to the front. We will look forward to publish more posts about how we go further to improve the design and architecture of our applications.

Microservices have became one of the biggests challenges in software development since they started to be widely used in complex applications. In contrast to framework-based solutions that easily turned into Big Balls of Mud, microservices are aimed to increase the maintainability by separating pieces of functionality into independent procedures wired, usually, via HTTP. The tradeoff with microservices is the need for additional instruments to allow the communication between subsystems, wich means another layer of complexity to take care of.

If not handled properly, microservices can make us pay for the tradeoffs without experimenting their real benefits. In this post we will talk about one of the problems that can make microservices painful: self-awareness.

What is self-awareness?

A microservice usually contains both the business logic and the infrastructure needed to receive/send messages. In a typical HTTP-based application, a router is in charge of driving a request to the proper controller that will call the domain services to do the real work.

Basic microservice
Basic microservice

Self-aware applications are those in which the business logic knows its position in the overall system, including the other parts of the system it is communicating with. Have a look at the following example.

Self aware application
Self aware application

The figure to the right represents a remote application in an e-commerce platform. From it we extracted a small service responsible for generating and sending the invoices when a payment is received. The communication between them is performed via a small HTTP client that performs requests and parses JSON responses. Following is the source code of the payments processor.

module Invoicing
  class PaymentProcessor
    REMOTE_APPLICATION = 'http://remote.application.com:3001'

    def process(payment)
      customer = customer_by_id(payment.customer_id)
      items = items_by_order_reference(payment.order_reference)

      invoice = InvoiceGenerator.for(customer).with(items).generate
      invoice.pay(payment.amount)
      invoice.save

      invoice.send_to(customer.email)
    end

    def customer_by_id(customer_id)
      response = HttpClient.get(REMOTE_APPLICATION, "/customers/#{client_id}")
      Customer.new(response.parse_json)
    end

    def items_by_order_reference(order_reference)
      response = HttpClient.get(REMOTE_APPLICATION, "/orders/#{order_reference}/items")
      Items.new(response.parse_json)
    end
  end
end

The service receives a payment object, which contains the amount paid, the customer id and the order reference. First, it retrieves an entity representing a customer by sending an HTTP request to a remote application. The response will be parsed and provided to the constructor of the Customer class. Next it brings the ordered items by doing another HTTP request to the remote application.

With all the information gathered, it is able to create an instance of an invoice. For the sake of simplicity, we will assume that the InvoiceGenerator is in charge of generating the proper invoice number and filling in the required fields. After saving the invoice in a local storage, it is sent to the customer via email.

The problem with self-awareness is that the physical layer leaks into the logical one, polluting it with the architecture decisions taken. In a tiny example like the previous one, it doesn’t seem a big deal, but think of a more complex application with some more classes making requests. Any redistribution in the way the apps are organized in the architecture will have a significative impact in the domain classes. If this happens in 10 or 20 different applications, scaling the system architecture can be a pain. There will be little benefit, if any, compared to a traditional monolith. Plus, the addition of a physical layer and the communication mechanisms will make it even harder to test and to maintain.

Introducing repositories

Instead of using the HTTP libraries directly, we are going to introduce an abstraction between the logical and infrastructure layers that will make our domain architecture-agnostic. That abstraction will be an interface acting as a repository to mediate between the domain and the persistence strategy.

module Invoicing
  class PaymentProcessor
    def initialize(repositories)
      @repositories = repositories
    end

    def process(payment)
      customer = customer_by_id(payment.customer_id)
      items = items_by_order_reference(payment.order_reference)

      invoice = InvoiceGenerator.for(customer).with(items).generate
      invoice.pay(payment.amount)
      @repositories[:invoices].save(invoice)

      invoice.send_to(customer.email)
    end

    def customer_by_id(customer_id)
      @repositories[:customers].find_by_id(customer_id)
    end

    def items_by_order_reference(order_reference)
      @repositories[:orders].find_by_reference(order_reference)
    end
  end
end

Repositories will be responsible for doing the http requests, accessing to a local database or delegating to ActiveRecord. The domain logic will be completely agnostic about how things are persisted.

module Invoicing
  class ActiveRecordInvoiceRepository
    def save(invoice)
      invoice.save
    end
  end

  class RemoteCustomerRepository
    def find_by_id(customer_id)
      response = HttpClient.get(@remote_location, "/customers/#{client_id}")
      Customer.new(response.parse_json)
    end
  end

  class RemoteOrderRepository
    def find_by_reference(order_reference)
      response = HttpClient.get(@remote_location, "/orders/#{order_reference}/items")
      Items.new(response.parse_json)
    end
  end
end

Finally, we just need to inject the repositories into our application service. The following snippet is just an example, not intended to be a reference on how to pass the repositories in.

repositories = {
  invoices: Invoicing::ActiveRecordInvoiceRepository.new,
  customers: Invoicing::RemoteCustomerRepository.new('http://remote.service.com:4001'),
  orders: Invoicing::RemoteOrderRepository.new('http://remote.service.com:4001')
}
processor = Invoicing::PaymentProcessor.new(repositories)

Once we have decoupled infrastructure and business logic, we can move our microservices around, just by replacing the repositories injected to our services with newer versions. For instance, we could group microservices in a single application and use local storages due to performance needs. Or we could split remote services in two to improve the separation of concerns. Any change we do, our domain logic will be protected thanks to the repository abstraction.

Microservices architecture
Microservices architecture

Testing strategies

In order to test an application making remote requests with repositories, we’ll need to define the boundaries and test them in isolation by writing contract tests. Each boundary has two ends. One of them will be exercised by the test, while the other will be stubbed/mocked. It is highly recommended to watch the talk from J.B. Rainsberger: Integrated tests are a scam.

So, we could start with testing the boundary that comprehends the execution flow from the public service to the repository.

describe Invoicing::PaymentProcessor
  let(:repositories) { customers: double, orders: double, invoices: double }
  let(:the_customer) { Customer.new(...) }

  it 'saves the invoice generated' do
    payment = Payment.new(customer_id: 33, order_reference: 'the_order_reference', amount: 330)
    allow(repositories[:customers]).to receive(:find_by_id).
      with(33).
      and_return(the_customer)
    allow(repositories[:orders]).to receive(:find_by_reference).
      with('the_order_reference').
      and_return(the_orders)

    expect(repositories[:invoices]).to receive(:save) do |invoice|
      expect(invoice.amount).to eq(330)
      expect(invoice.customer_name).to eq(the_customer.name)
    end

    PaymentProcessor.new(repositories).process(payment)
  end
end

In the example above, we stub the retrievals and set expectations for the repository saving the invoice.

Boundary from the service to the repository
Boundary from the service to the repository

Next, each one of the repositories should be covered. The stubs we used previously in the arrange of the test will turn into the act in this side of the contract. The component doing HTTP Request will be mocked.

describe Invoicing::RemoteCustomerRepository do
  let (:repository) { RemoteCustomerRepository.new('http://remote.service.com:4001') }

  context 'bringing a customer by id' do
    it 'calls the remote endpoint' do
      expect(HttpClient).to receive(:get).
        with('http://remote.service.com:4001/customers/33')

      repository.find_by_id(33)
    end

    it 'it builds the Customer with the returned data' do
      allow(HttpClient).to receive(:get).
        and_return(code: 200, body: { id: 33, name: 'Sarah', surname: 'Connor' })

      customer = repository.find_by_id(33)

      expect(customer.full_name).to eq('Sarah Connor')
  end
end
Boundary from the repository to the HTTP client
Boundary from the repository to the HTTP client

Finally, the third part of the contracts contains the communication between the HttpClient and the remote service. Assuming the HttpClient is a third-party library, its tests are out of our business. We can assume it is working properly. If it was a home made component, we’d need to cover it in integration. Anyway, having covered each part of the contract separately, it is time to write an end-to-end test with real actors. How to do this end-to-end test is out of the scope of this test. Those upper level tests are fragile and hard to maintain, since we’ll probably need to start servers, prepare some fixtures, etc… Due to this complexity, it’s better to minimise the amount of end-to-end tests.

End to end test
End to end test

Carles Climent Granell.
Developer at peerTransfer.