Shared Principles for Scalable Services on the Web
For the past several months, I've been working on a book for O'Reilly Media. The current title is "RESTful Web Microservices Cookbook" and it contains a collection of "recipes" for creating stable, scalable, reliable, and resilient services that can be pieced together across the public Web. The recipes work well for local enterprise work, too.
For the upcoming book, I decided to call out some base-level shared principles that act as a guide when selecting and defining the recipes. For this collection, I am using a single guiding principle. I am including service designs and patterns that:
Leverage global reach to solve problems you haven't thought of for people you have never met.
And we can break this principle down a bit further into its three constituent parts.
Leverage Global Reach ...
There are lots of creative people in the world and millions of them have access to the Internet. When we're working to build a service, define a problem space, or implement a solution, there is a wealth of intelligence and creativity within reach through the web. However, too often our service models and implementation tooling limits our reach. It can be very difficult to find what we're looking for and, even in cases where we do find a creative solution to our problem by someone else, it can be far too costly and complicated to incorporate that invention into our own work.
For the recipes in the book, I tried to select and describe them in ways that increase the likelihood that others can find your solution and lower the barrier of entry for using your solution in other projects. That means the design and implementation details emphasize the notions of context-specific vocabularies applied to standardized messages and protocols that are relatively easy to access and implement.
Good recipes increase our global reach. Both the ability to share our solutions and to find and use the solutions of others.
... To Solve Problems You Haven't Thought of ...
Another important part of our guideline is the idea that we're trying to create services that can be used to build solutions to problems we, ourselves, have not yet thought about. That doesn't mean we're trying to create some kind of "generic service" that others can use (e.g. data storage as a service or access control engines). Yes, these are needed, too, but that's not what I'm thinking about here.
To quote Donald Norman (from his 1994 video) :
"The value of a well-designed object is when it has such a rich set of affordances that the people who use it can do things with it that the designer never imagined."
I see these recipes as tools in a craft-person's workshop. Whatever work you are doing, it often goes better when you have "just the right tool for the job." For the book, I tried to select recipes that can add depth and a bit of satisfaction to your toolkit.
Good recipes make well-designed services available for others to use in ways we hadn't thought of yet.
... For People You Have Never Met
Finally, since we're aiming for services that work on the web -- a place with global reach -- we need to acknowledge that it is possible that we'll never get to meet the people who will be using our services. For this reason, it is important to carefully and explicitly define our service interfaces with coherent and consistent vocabularies. We need to apply Eric Evan's ubiquitous language across services. We need to make it easy for people to understand the intent of the service without having to hear it explained by ourselves. Our implementations need to be, to borrow Fielding's phrase "state-less" -- they need to carry with them all the context needed to understand and successfully use the service.
Good recipes make it possible for strangers (services and/or people) to safely and successfully interact with each other in order to solve a problem.
Dealing with Timescales
Another consideration we need to keep in mind is that systems have a life of their own and they operate on their own time scales. The Internet has been around since the early 1970s. While it's essential underlying features have not changed, the Internet itself has evolved over time in ways few could have predicted. This is a great illustration of Norman's "well-designed object..." quote from above.
Large scale systems not only evolve slowly, even the features that are rarely used persist for quite a long time. There are features of the HTML language (e.g. <applet>) that have been deprecated yet you can still find applets online today. It turns out it is hard to get rid of something once it gets out onto the Internet. Things we do today may have long term effects for years to come.
We can take advantage of long time timescales in our designs and implementations. Fielding, for example, has said that "REST is software design on the scale of decades: every detail is intended to promote software longevity and independent evolution."
Good recipes promote longevity and independent evolution on the scale of decades.
This Will All Change
Finally, it is worth saying that, no matter what we do, no matter how much we plot and plan, this will all change. The Internet evolved over the decades in unexpected ways. This is the role of the HTTP protocol and the original HTML message format. Software that we might have thought would be around forever is no longer available and applications that were once thought disposable are still in use today.
Whatever we build is likely to be used in unexpected ways, by unknown people, to solve as yet unheard of problems. For those committed to creating network-level software, this is our lot in life. To be surprised (pleasantly or not) by the fate of our efforts.
I've worked on projects that have taken more than ten years to become noticed and useful. And I've thrown together short term fixes that have been running for more than two decades. For me, this is one of the joys of my work. I am constantly surprised, always amazed, and rarely disappointed. And even when things don't go as planned, I can take heart that eventually, all this will change.
Good recipes recognize that nothing is permanent and things will always change over time.