Introducing searcheable projections
We recently released a new feature that allows you to create searcheable projections. This page explains how to use it.
From launch, Serialized has supported projections. Projections are a way to create a read model of your data that is optimized for querying. The structure of a projection is defined declaratively using a Projection definition that is stored as configuration in your Serialized project.
Each projection definition can produce a number of Projection instances. Each instance is a separate read model that your application can query. The projection instances are created by processing the matching events from your aggregates using the handlers that are defined in the projection definition. The result is a queryable data structure that is a better fit for your query use case than the raw events.
For example, to create a projection that contains all reserved bookings in a hotel management system, you could create a
projection definition that will process all ReservationConfirmed events.
Using the idField property in the projection definition, makes the projections queryable by an ID of your choice that
is a field present in the projection data. This is useful if you want to be able to query your projections by a field
other than the aggregate ID. In this case we use the reservationId as the ID of the projection:
Using IDs is not enough
Querying projections by id is great, but it is not always enough. In many cases, you want to be able to query your data by different filters or search terms. For example, you might want to find all reservations for a specific room or all reservations for a specific customer.
Most of these scenarios can be solved by creating a projection that utilizes references. References offer a way to create a secondary index for your projections that can be used in listing queries.
Introducing full-text search
References and ID-based queries have been proven to work for many use cases. More than we thought when we first released the feature. However, we have also received feedback from our users that they would like to be able to search their projections using full-text search. We have heard from a number of customers that they like the projection support but also implemented their own full-text search solution with their own database as a search engine. After listening to our customers and hearing this feedback we have now added this feature to Serialized.
Why is this important?
Running and maintaining a search engine is a lot of work. It may often require separate infrastructure and maintenance, which makes it a potentially expensive addition. With this feature, you can now use Serialized as your search engine and avoid the need for a separate search engine/database.
As an example, the absolute cheapest option for AWS OpenSearch will set your team back €60 / month. A medium-sized cluster for production use will likely cost you at least €300 / month. This is a lot of money to spend on just search support for a small/medium team. For more heavy-load installations, the pure infrastructure lands above €1500 / month.
In addition to the running costs, you also need to maintain the search engine and implement the search functionality. Implementation of the processing of your events from your aggregates and index them in the search engine is also an implementation task that requires time and effort.
How does projection search work?
To create a searcheable projection, you add a indexedFields field to your projection definition. The
fields included will be indexed and made available for full-text search, automatically.
As an example, let's create a projection that contains all reservations for a hotel management system. We will make
the customerName and phoneNumber fields available for full-text search.
The projection definition above will make the projection instances searchable by the customerName and phoneNumber. A
search will apply to all fields provided as indexed fields using a search parameter in the request:
The search string can use any of the operators described in the docs.
For more information about how to create searchable projections or making search requests using our SDKs, see the documentation.
We're looking forward to your feedback on this new feature. If you have any questions or feedback, please reach out to us.