In the previous architecture spotlight entry, we discussed Event Sourcing and illustrated the concept with a simple banking account example. We laid out many of its pros and cons to help readers decide if the pattern would be useful to them.
In this post, we will be expanding the example and showing a working code implementation using a popular event streaming technology, Apache Kafka.
Kafka provides persistent messaging infrastructure and key/value data storage, supporting both event sourcing and derived state storage within a single package. These are the two primary pillars that make up a real-world event sourcing application. Kafka’s central construct of a messaging topic is used to support both concepts — its storage APIs act as a higher-level construct on top of its topic APIs.
However, event sourcing is not Kafka’s only use-case. It was originally designed to support event streaming — high-volume messaging, processing and analytics — so also works well as the general messaging fabric within your microservice system. It is easily scalable, and through its scalability, provides partitioned, replicated storage for the messages you send through it.
Scaffolding the Basic Example
We opt for a couple of abstractions to reduce boilerplate and allow us to focus on the event sourcing implementation. We will be using Spring Cloud Stream to provide the basis of an integration-enabled microservice app, together with its Kafka and Kafka Streams binders to allow us to operate with Kafka. Input, output and processing functions within the sample application will be designed as Spring Cloud Functions, both simplifying the processing code as well as allowing for easier deployment to serverless infrastructure on a cloud provider, if desired. We also use Lombok to reduce domain object boilerplate and to provide us with readily-available logging, as well as Jackson to handle marshaling data to and from JSON.
The project described in this article will be built using Spring Cloud, so the first step is to download a project scaffold from Spring Initializr. Once downloaded, edit the project’s pom.xml and add the Spring Cloud Stream Kafka binders to the project’s dependencies list:
You can now build and run the project from the command line, as follows (substitute mvnw for mvnw.cmd on a Windows platform):
Running this will show typical maven build output (including test executions), then log output from the running app. To start with, the log output will consist of the spring boot logo together with a few log messages indicating initial application startup, then shutdown, as right now the application contains no runtime processing logic.
We will also need a running Kafka instance that our application can interact with to send and receive messages, as well as to access our persistent derived-state storage.
The easiest way to get started locally is by using docker-compose. A suitable docker-compose.yml file is available (along with several other Spring Cloud Stream Kafka examples for you to take a look at). Once you have downloaded this docker-compose.yml file to your project, simply run:
Running this will start a zookeeper instance (which Kafka requires to orchestrate its clustering), as well as Kafka itself.
Adding the Domain Model
Before we can add some processing logic to our example application, we will need a few domain objects to model our Commands, Events and our Bank Account Entity. These objects will represent the data we pass as messages over Kafka, as well as general state containers, and will be what our processing logic operates on.
The basic data entity we will deal with is a bank account, described by the BankAccount class. Note we do include an account ID here, as Kafka needs a way to correlate messages by some form of key. All messages with the same key are assumed to operate on the same data object. Newer messages could also represent the complete updated state for the specified object, depending on how your system is designed. Kafka also stores all messages with the same key together on a given node, to increase efficiency when processing the complete message history for that specific data object.
We opt for a single event object together with an enum to differentiate between multiple event types, however, this is by no means the only way to model such a domain.
The event type enum:
In addition to credit and debit events, we also include
ACCOUNT_OPENED. This is used as a marker to 'open' an account, meaning that an initial account object (with zero starting balance) is created within the system. Credit and debit events can only operate on an already-existing account.
Account events are represented by an
Notice the event contains a representation of the latest bank account state within it. This is only done for point-in-time processing convenience and is by no means required. With event sourcing, it’s key to remember that the sequence of all events ultimately represents the master state record. Any other state representation is purely for convenience.
If you need help architecting your next enterprise application or improving your current architecture, please contact us to discuss how we can help!
Originally published at https://www.sitepen.com on August 6, 2020.