by Miguel Angel

How we use github/scientist


Only a few weeks ago, the amazing folks at Github released the first version of Scientist (more about the gem in their blog) Scientist helps developers to refactor big chunks of code. More than refactor, it helps with rewriting parts of code, because sometimes a local refactor is not enough.

Scientist equips a legacy codebase with the ability of executing different branches of code (branches that perform the same actions) and comparing the current implementation with the new implementation. The behaviour will not change because the current branch is being executed and performing its tasks as usual, while the new rewritten part is executed with all the benefits of a production environment.

Lately, as we explained in past posts, we are fighting with our legacy code using different techniques, and Scientist quickly joined our toolbox. We have been using more rudimentary approaches for executing different branches of code without impacting the application so we started using Scientist straight away and enjoying all the benefits. Let’s introduce a couple of cases where it has been very useful to us.

Case #1: Changing the search engine.

At one point last year our search engine was not able to cope with the volume of data and our mates using the internal applications were experiencing turbulences when the system was under heavy load. This is a typical scenario where this technique specially shines since we can not refactor a search engine, we have to replace it.

The gem provides an abstraction for managing the experiment actions such enabling the experiment, publishing results after the execution, rescuing potential exceptions, etc. Here we have created an experiment to test the new search engine.

class ElasticExperiment
  include Scientist::Experiment

  # ...
  def publish(result)
    log_mismatch(result) if result.mismatched?
  end

  def log_mismatch(result)
    current_result = result.control.value.map(&:id)
    elastic_results = result.candidates.first.value.map(&:id)
    criteria = result.context[:criteria]
    elastic_query = TransferSearch::Elasticsearch::QueryBuilder.new(
      criteria
    ).query

    logger.error "DIFF: #{current_result - elastic_result}"
    logger.info "INPUT CRITERIA: #{criteria.to_json}"
    logger.info "QUERY: #{elastic_query.to_json}"
    logger.info "CURRENT IDS: #{current_result}"
    logger.info "ELASTIC IDS: #{elastic_result}"
  end

  # ...
end

And it is inevitable to change the production code, although the changes are small.

# ...
science 'transfer-search' do |experiment|
  experiment.context criteria: criteria
    experiment.use { TransferSearch::Sphinxsearch.search(criteria) }
    experiment.try { TransferSearch::Elasticsearch.search(criteria) }
    experiment.compare do |control, candidate|
      (control.map(&:id) - candidate.map(&:id)).none?
    end
  end
end
# ...

The use block contains the current code, the code that works and what the science block returns. The try block is the candidate (it could run more than one candidate). No matter what happens inside the experiment, the main execution will not be impacted.

Case #2: Replacing the user management system.

Also because of scaling issues, we have some challenges segregating users of different domains. Having all users in the same Users table creates an important issue when the volume increases only for some sets of users.

The implementation is very similar to the search engine one. There is an alternative execution branch that makes a remote call to a new service called Identity, where we are organizing the users now. The experiment logs the exception into a file when a given user is not authorized in the new service although it is authorized in the current system. That way we can browse all the exceptions later and fix them.

class AuthExperiment
  include Scientist::Experiment
  # ...
  def publish(result)
    logger.info "current auth duration: #{result.control.duration} | new auth duration: #{result.candidates.first.duration}"
    if result.candidates.first.exception && result.control.exception.nil?
      logger.info "Mismatch authenticating: #{result.control.value.email} #{result.candidates.first.exception.class.name}"
    end
  end
  # ...
end
science 'auth' do |experiment|
  experiment.use do
    user = Core.user_authenticate(email: email, password: password)
    create_valid_user(user)
  end
  experiment.try do
    authorized = IdentityClient.authenticate(username: email, password: password)
    user = { first_name: authorized.first_name,
      last_name: authorized.last_name,
      email: authorized.email,
      authentication_token: authorized.token }
    User.new(user)
  end
end

We are still finishing these systems migrations and with the help of this gem and some other techniques, like the mikado method, it is much easier to work with a legacy codebase. We have noticed that as we continue learning techniques to deal with legacy code, it becomes easier, safer and faster to modify old code, and the line between legacy and greenfield becomes blurry.

Miguel Angel Fernández. Developer at Flywire.