Public Side Architectures
Next: Construction - Modular Building Concept (new topic)
This is the last post on an overly long series of writings on the public side services.
Let’s focus on a few options for building a architecture for the public side services. There are on very high level – sketches really.
Recommender Based Society
One can think of the recommender engine becoming the core of the society matching people with services.
There are really a few different types of recommenders out there.
Collaborative systems are either based on similar users or similar items. User based collaborative filtering looks for folks that are similar to me based on past use and looks what they have been doing and based on that draws up a list of potentially meaningful services for me. This might be useful in areas like education or health. “People who have taken courses on topic x have completed this set of trainings on that topic and gotten employed at salaries in the range ..”
Item based collaborative filtering in turn looks similarities between services based on some criteria – for example based on user feedback like how effective the treatment has been or how the user has rated some advice. Then it looks for people who are most likely to benefit from this type of service.
Context based recommendations look for example to location and time and based on that recommend actions. For example, when someone’s child comes of age to start school, a recommender system can automatically enroll them to a nearby school and ask parents if they like this option or would they like to change.
Demographic recommender looks for things like gender, age, education and use those as features for their model
Social recommender looks for what people in my social group (people I am closely connected to), like or find trustworthy. For example, the Onion Rater is strongly based on social network to filter out untrustworthy content and fraud
Most systems in use tend to be based on a combination of these generic strategies.
Such a recommender system could automatically act throughout major life events allocating to a recommended kindergarten, school at early age, calculating pension for you and showing what it means to retire at different times, making unemployment and other benefit decisions automatically and allowing citizens to change or opt out if they so wish.
As many needs of people span across public and private spheres, one can think that why not take a holistic approach and think of a society where one or more recommender systems could draw from a multitude of sources for suggestions. The image below tries to depict something like that:
Rather than a single recommendation engine, it would make sense for several to exists, even bringing on your own. Having multiple recommendation engines naturally creates the need to have a comparison service between them and a recommender for recommenders…
Going from information overload to recommendation overload must count as progress?
One problem with recommendation engines is that they tend to narrow down the choices and very small initial differences in prices or quality tend to accumulate when engines have no variability. A good recommendation engine really needs to fulfil three basic needs: understand what I need, present a good set of choices and ideally even surprise aka enlarge my understanding of options.
As a downside, recommender engines cannot always explain well why they make certain proposals due to the statistical nature of how recommendations are made from a large body of data.
This can be to a degree helped with an augmentation engine. Augmentation engines visualize things. Let’s take two examples how visualizations help. An augmentation engine on life habits could show what you look like with current drinking and smoking habits in 30 years versus if you slow down them and pick a few healthy ones on the side. Another might visualize natural catastrophes. As example migratory locusts can cause major agricultural catastrophes when they form swarms of trillions of locusts. Trillion as a number is hard to understand but an augmentation engine can visualize this by showing the locust swarm on top of your city or region to make it concrete, what would such an event mean in your surroundings.
The difference between recommenders and augmenters is in the diagram below:
Serverless State
Before discussing serverless state, a word on serverless computing. In the first wave of cloud computing the services were modularised so that the users’ need to orchestrate a whole circus of technical components – application servers, container environments, virtual machines, databases, load balancer and other whatnots to run their application.
Serverless computing is the next step where the service provider does all of that automatically in the boiler room. The user aka application developer only needs to focus on the logic of the application. Whenever there is an event/request, the logic supplied by user gets called. The provider does all needed background work for this to happen. Only useful work gets charged, if there are no request, there are no charges.
Serverless state is a way of using similar approach to automate public side services.
Whenever something significant happens in your life - a baby is born, you enlist to a school or graduate, want to start a company, get sick, divorce, get fired or retire etc, an event gets sent and the logic as defined by policies gets automatically executed. This can mean that an advisory service tells you what your options are, what similar users are doing, benefits get allocated automatically and so on.
This processing can be divided into two parts. Some actions are mandated by law and performed by the public side, other useful life event related services come from the private side. Automations apply for both.
Let’s take a few examples.
When starting a new business, there is a bunch of public services like registering and reserving the name of the company, enlisting to tax authorities etc. In addition, a company needs private services like a bank account and insurances. All of these can be organised into a smooth flow mixing public and private services.
Or consider a case where your holiday flight is late. There are regulations as to the compensation passengers are entitled in EU. You open an app, grant it access to you family’s flight data, select your option or speak to the app and explain that you want to complain . The app makes the complaint and follows the process to the end informing you at every step what is happening. At the same time (with airline’s approval) the app uses air line systems to re-route you to another flight if the original one is cancelled making sure you all stay together.
The architecture for the serverless state is as below:
Image. Serverless state gets started on major events and runs composite services tying public and private services to help you
Events are created either from major a person’s life goals like starting a school, starting a business or other life events like diseases, laboratory results or being retired. Some activities are started by sensor data as example a wearable informing of an accident (elderly person falling, car crash) or IoT devices. Some are just queries from users.
Events get published to the’ event bus’ and via it find their way to the event router. This first line handler consults the citizens personal preferences and what the laws say about such an event and further route the even to actual handlers (or send an additional event that triggers the right processing). Handlers take the event and perform the automated processing. Handlers may also communicate with the user, present you with results and what next step options are available or where to find more information (what online training courses or other material or user groups exists to gain a better understanding). These recommendations may be rule based on based on a recommendation engine.
The handlers perform their actions by calling various APIs of public and private providers and create and update records in various official repositories (for example centralised health, energy and ownership record hubs).
The handlers may generate additional events that then get fed into the same processing pipeline. For example, if you renew your passport and a new picture is needed, the handlers might consult commercial service providers and show set of places for taking passport pictures near your home or place of work.
At the end of each step or at least in the end, feedback is gathered from users. How satisfied they are in general, what ideas come to their mind as customer on how to improve etc.?
(As example, the Finnish program AuroraAI has tried to build something similar with AI, matching people with services. There isn’t much public info of practicalities except normal marketing materials online)
Some triggering events are more data related such as municipalities measurements from the environment (high pollution level today…), personal health measurements, product withdrawals (toxics detected etc.) It’s a. little bit of a personal preference whether one wants to see a difference between functional or data driven events as one or two slightly different categories. In the latter the storing of data causes an inspection function to be called that in turn fires an event.
In both cases the processing generates quite a lot of data.
Providers have their partners and internal structures that often useful to capture in records as well– for example to remember, who actually provided the service, not only who sold and charged for it. Providers here can be the public organisation and its employees itself, a partner company, device for example doing automatic analysis or an API on the cloud.
The results of these interactions are logged to underlying logging services and repositories of the nation and from those a set of statistics are drawn to understand better how the services are evolving. If someone wants to follow that, a set of database publishing tools can make automated reports and send notifications to interested parties.
After each service use end-users are asked whether they are happy or not and make improvement suggestions.
.And that’s a wrap for public services. Next week we’ll start looking into various verticals and see what future for them might look like. Feel free to agree or disagree.