Designing Token APIs for Architecting Flow in Elixir
In previous articles, we discussed the Token approach and examined when to use a Plug-like Token and when to look at other options.
A Token is basically a struct which contains all information relevant to a certain use-case. If we were to create a time tracking app, we might have a Token for the time a user has entered:
defmodule MyTimeTracker.ManualEntry do
defstruct session_id: nil,
time: nil,
billable: nil,
# probably lots of other fields ...
end
Creating a Token API means creating functions to interact with our Token. In case of our imaginary time tracking app, this could lead to something like this:
defmodule MyTimeTracker.ManualEntry do
defstruct session_id: nil,
time: nil,
billable: nil,
# probably lots of other fields ...
@doc "Creates a time entry for the given `user`."
def build(user)
@doc "Reads the entered time for the given `entry`."
def time(entry)
@doc "Tries to parse the given `time_string` and add it to the given `entry`."
def time(entry, time_string)
@doc "Return `true` if the given `entry` is billable."
def billable?(entry)
end
Let’s see why this is useful.
The need for Token APIs
The usefulness might not be apparent at first. You might ask yourself:
Why would we need APIs for dealing with a Token? Can’t we just modify the damn thing?
Let’s take a look at why it is preferable to have a standardized API:
- Validation - using APIs instead of writing values directly into a struct gives us the chance to validate inputs in a central place
- Normalization - we can also ensure that certain pieces of information are always structured the same way (Plug does this by lowercasing all HTTP Header names it receives)
- Persistence - we can also defer the decision of how and where to store the information (which might be important for larger blobs of data, which we can then store in ETS tables behind the scenes)
By using a Token API to access information, we are also independent in all aspect of our data management, i.e. how we store, look up and index information inside our Token (which can, again, be important when dealing with larger amounts of data).
Other aspects you can deal with through a Token API are access management, caching, implementing domain-specific time-to-live concepts and basically anything you can think of.
Let’s look at a practical example.
Example: An Online Book Store
Let’s say we are building an online store for Elixir programming books. In this store, customers can register, login, curate a wishlist, fill and checkout a shopping cart.
We will use the shopping cart as our example for a Token: the cart is being used during the shopping process, which is the primary use-case of our online store.
Creating a Shopping Cart Token
So what do we “need” from our ShoppingCart Token?
- It has to belong to some kind of shopping session.
- It has to know when it was created, so we are able to identify abandoned carts.
- It has to contain the items the user wants to buy.
Here’s a good starting point to model this Token:
defmodule MyStore.ShoppingCart do
defstruct session_id: nil,
created_at: nil,
items: nil
def build(session_id) do
# we're using the `__MODULE__` special form here, so we don't have to refer
# to `MyStore.ShoppingCart` every time
%__MODULE__{
session_id: session_id,
items: [],
created_at: DateTime.utc_now
}
end
end
Creating a Shopping Cart API
Most of the time, I try to structure my Token so that I can read most, if not all, values directly from the Token. For writing values, I regularly defer that task to a function.
This pattern can also be found in Plug.Conn, where you can read assigns directly and use assign/3 to write new assigns to the struct:
iex> conn.assigns[:foo]
nil
iex> conn = assign(conn, :foo, :bar)
iex> conn.assigns[:foo]
:bar
This is much more readable than:
iex> conn = %Plug.Conn{conn | assigns: Map.put(conn.assigns, :foo, :bar)}
We can adapt this pattern to our shopping cart.
A put_item/2 function could help us to validate and/or cast items before putting them in the cart:
iex> cart = MyStore.ShoppingCart.build(user)
iex> cart.items
[]
iex> cart = put_item(cart, 43212332)
iex> cart.items
[%Item{...}]
iex> cart = put_item(cart, 87632785)
iex> cart.items
[%Item{...}, %Item{...}]
This is not only much nicer to read than
iex> cart = %MyStore.ShoppingCart{cart | items: [cart.items | item]}
but we can also have put_item/2 look up the item with the given id or see if the user doing the shopping is allowed to purchase the given item (e.g. due to his country’s age restrictions, although there are probably no such age restrictions in our imaginary Elixir programming book store 😉).
Lastly, we can ensure that, if given the id of an item instead of the Item itself, we look it up in order to have a consistent list of Item structs in our ShoppingCart.
With the help of put_item/2, we can validate and normalize inputs more easily.
Combining Tokens and APIs
At some point, our users want to checkout their ShoppingCart.
In that final step, we will have to let them choose a method of payment and provide further details.
As you might have guessed, we model the PaymentInformation as another Token:
defmodule MyStore.PaymentInformation do
defstruct session_id: nil,
created_at: nil,
provider: nil,
card_number: nil,
# lots of other payment info ...
def build(session_id) do
%__MODULE__{
session_id: session_id,
created_at: DateTime.utc_now
}
end
# lots of functions for setting payment providers, card numbers, etc.
end
We will now have to bring those two distinct pieces of data together: what a user wants to buy and how they plan to pay for it.
In other words: We need an API which takes both a ShoppingCart and a PaymentInformation Token.
defmodule MyStore.Checkout do
def perform_checkout(shopping_cart, payment_information) do
# suppose `CheckoutService.call/2` is our checkout API, which is
# implemented and maintained by another team of engineers
result = CheckoutService.call(shopping_cart, payment_information)
case result do
# `:ok` and `true` both mean that the checkout has gone through
success when success in [:ok, true] ->
# TODO: talk to checkout team why we need to check for two values
:ok
# one of the engineers from the checkout team mentioned that in case of
# an error the result is a Map, containing the reason for the failure
%{} = map ->
{:error, map["reason"]}
# if we get neither `:ok`, `true` nor a Map, let's raise to account
# for the unexpected input
value ->
raise "Unexpected return from CheckoutService. " <>
"Expected :ok or Map, got: #{inspect(value)}"
end
end
end
This example demonstrates how we can wrap an inconsistent external API and build a clean interface around it.
As in the example above, our own API might just return :ok or {:error, reason}.
It does not necessarily have to have its own Token, although it very easily could:
defmodule MyStore.Checkout do
defstruct shopping_cart: nil,
payment_information: nil,
success?: nil,
errors: nil
def perform_checkout(shopping_cart, payment_information) do
# `build/2` will ensure that our inputs are valid and create a Token
checkout = build(shopping_cart, payment_information)
# we are still calling the same checkout API, this time using
# the validated inputs from the Token
result =
CheckoutService.call(checkout.shopping_cart, checkout.payment_information)
case result do
success when success in [:ok, true] ->
# we're using internal Token APIs to modify our Token ...
add_success(checkout)
%{} = map ->
# ... this way, we have a central place that defines what
# "success" or "failure" means ...
add_failure(checkout, map["reason"])
value ->
# ... and we still raise in case of inputs we can't handle!
raise "Unexpected return from CheckoutService. " <>
"Expected :ok or Map, got: #{inspect(value)}"
end
end
# NOTE: this time, our `build` function is private, because we do not
# initialize a `Checkout` Token from outside this module
defp build(shopping_cart, payment_information) do
# we validate our input data here to ensure that, once we have a token,
# we can rely on the Token's contents being valid
valid? = shopping_cart.session_id == payment_information.session_id
if valid? do
%__MODULE__{
session_id: shopping_cart.session_id,
shopping_cart: shopping_cart,
payment_information: payment_information
}
else
raise "session_id not matching!"
end
end
# these functions define what "success" ...
defp add_success(checkout) do
%__MODULE__{checkout | success?: true, errors: []}
end
# ... and "failure" mean.
defp add_failure(checkout, errors) do
%__MODULE__{checkout | success?: false, errors: List.wrap(errors)}
end
end
Wrapping the checkout process in a Token like this has several advantages:
- We can define that the
errorsfield is always a list (although we only get a singlereasonfrom theCheckoutService.call/2function). - We can define that there is a
success?field, which tells a consumer if the checkout has gone through. - This is much clearer than telling the consuming programmer “Just check if the
errorsfield isnilor empty.”
One of my favourite quotes from the Elixir documentation states:
Remember that explicit is better than implicit. Clear code is better than concise code.
This is especially important since APIs have to be stable and provide a clear boundary people can rely on.
In conclusion
Tokens are a great way to provide a common interface to a shared domain.
And while APIs have to be stable, APIs are also constantly evolving, especially during the early stages of development.
Token APIs have to accomodate both: provide a well defined interface and a stable “language” to talk about shopping carts, payment information and checkouts.
Once we start implementing Token APIs that combine multiple Tokens, we have to tread lightly, since they have to rely on the interfaces of the Tokens they combine and provide a stable interface of their own.
This is why we need guiding principles for developing Token APIs.
To paraphrase one of my favourite guiding principles:
Be liberal in what you accept as input and be conservate in what you send as output.
The example above shows how to do just that by wrapping an inconsistent external interface into a cohesive and explicit interface using an internal Token.
