To explain why, we need to go through a future scenario of how we build a business in order to explore the future potential of AWS step functions.
Building a business from a great idea, some future Monday
It’s Monday, it’s the year 2025 and I’ve woken up with a great new idea. [Actually, it’s a pretty lousy idea but hey, I just spent a few minutes on the scenario so let’s just assume it’s great]
I’m going to create a recommendation engine for stock picking based upon the mood of the internet. I quickly scribble down the user needs “make profitable trades” and “know what to buy” and write a basic map whilst grabbing breakfast. I have my map, I know the basic components that I need — the recommendation engine, trade feed etc. I start work at 8.30 am.
I know Amazon provides one of the components as a service (the lambda platform) and several others can be found as AWS lambda services in the marketplace. The company I work for also provides a stock portfolio service. I mark up what I think we can use (where we are the consumer) and what we need to build (where we are the provider) e.g. the recommendation engine and the mood system.
It’s 9.20 am, I send the map of to our spend control group. They act like an intelligence gathering organisation, collecting all the maps of everyone, comparing them and giving some feedback. They build profile diagram by finding common elements between the maps. I get a reply by 10 am with some details. They send me the profile diagram below. It seems some other team in the company has built a recommendation product. I’m the only person thinking about a mood system. In general, I’m roughly where everyone else.
However, 16 different teams are using trade feeds and everyone else works with a well developed lambda service. Apparently lots of groups are using some utility service for a trading engine. I click on the profile for the details, it’s Amazon.
They’ve also sent my map back, slightly modified. Ok, well at least this is not the like the bad old days of 2015 where my company had thousands of duplicated systems and endless rebuilding of stuff that already existed.
It’s 10.15 am. I start thinking about some metrics. The trade feed system is going to be providing trades to the recommendation system. Each one will need a call to the mood system, the risk system and so on. I start marking out where all the metrics are.
It’s 10.45am. I flip a switch and the map is converted into a business model. The same components, the same links, the same metrics. I start running a few scenarios, checking that I’ve made a truly variable cost business model.
It’s 11.30 am, I send the map and the model to finance. They come back with some helpful comments [in this case, it would be “how do we make money?” but then again the scenario took me a few minutes, so I’m taking liberties].
It’s 12.00 am. I send the maps and the improved model to the executive group. Thirty minutes later I get the go ahead and a small budget of $20k. I know from the spend control profile that some other cells are already building this stuff. I give them a call, tell them what I’m upto. They already know, spend control told them.
I know from the spend control profile that there is a group building a recommendation engine. I send them the map and model and outline my idea of adding a mood system to recommendation. We have a quick call and they’re up for it. We agree a metric of value for charging — everyone uses worth based development these days. Most of the stuff I need (i.e. 80%) is already built and provided as services. I just need a cell of pioneers to build the mood system, whatever that will be.
I update my map with the organisation structure and load it with the build map and financial model to the company’s job portal. I wait.
Our company operates in cells, using pioneers, settlers and town planners. We live in a constantly changing environment. Watch the movie! I love it.
YouTube video on pioneers, settlers and town planners
Because of this, we always have pools of people training and looking for their next cell to join. It’s 1pm and no-one has responded. I’m getting worried. I’m looking at the other exciting projects on the jobs board. Out of the blue, by 1.30 pm I’ve nabbed two pioneers willing to give this a go. They sign up.
We’re off to the races! Of course, HR is constantly monitoring the flow of components through the maps, the cells being formed, whether we need more pioneers, settlers and town planners. This goes on in the background. They’re checking what we build vs what we buy and whether we have the right balance of attitudes. Long gone are those old days where dullards would try to convince us that a company could have one culture. Long gone are those days we were weren’t looking for the right skills (aptitudes) and the right attitudes. HR is on a bit of recruitment drive at the moment, we’ve been lacking enough settlers especially in finance and engineering.
The pioneers start cracking away with the project. They’ve organised the cell the way they like it, recruit a few others and build the mood system. They understand the map and what they need to create. They have autonomy (their cell), mastery (their attitude e.g. pioneering) and purpose (the mood system and where it fits in with the wider scheme from the map).
It only took a few hours for them to get the mood system up and running. Most of it just used existing lambda functions. They’ve added interfaces for the recommendation engine to call and we’ve all started monitoring whether consumers use it. I start monitoring flow in the system, where’s the money going, are there bottlenecks, how are we doing on those metrics? Those metrics act as the fitness functions for the cells. It’s important we keep track of them.
Of course, we’re not the only ones monitoring. Part of the spend control group looks after strategy and they are already looking at the maps for new opportunities. Our stock portfolio system and recommendation engine are provided to other companies through public APIs. Spend control measure consumption of the services to identify future trends. But they’re also watching how the mood system is going, maybe we should provide it as a service to others and not just to our own recommendation engine?
Spend control notice the mood system is picking up and they’ve heard rumours that other companies are looking into this space. Differentials don’t last long in this world, you measure them in weeks. Spend control decide we should push the evolution of this towards more of a utility. It’ll need development and in this case, they recommend an open approach is worthwhile.
In the next few days, the project is soaring and I’ve noticed a new project on the company job board to turn the mood system into an open sourced project. Several settlers have joined our mood cell.
By the way, if it wasn’t obvious, I’m a pioneering manager. I’m not actually part of the mood system cell, I belong to a hierarchy of managers that monitor cells, help initiate them, define the fitness functions, keep track on operational flow, build the business case and basically remove any obstacles I find. I specialise in the genesis of new projects and concepts. On rare occasions I have good ideas, most of the times I get them from others or from searching the ecosystems around our company. When I’m not looking after cells, I work in spend control. It’s not so much that I manage the cell, they are autonomous, it’s more like tending a garden and helping set the right conditions.
I specialise in management and pioneering. I’m no different from those specialising in pioneering and engineering or finance or design except in aptitude (skillset). We all have the same attitude. We’re no different to settlers or town planners except in attitude. We’re all important. As pioneers we tend to discover the swamp and die off from malaria. We fail a lot, take huge risks. The settlers make the swamp liveable and the town planners build Rome. I’m always amazed by what they do.
It’s the end of a week. I’m in a good mood. The mood cell is up and running, it’s even growing into an open source effort. The changes to the recommendation engine are working. The metrics look good. It’s turning into a nice little business. I have a relaxing weekend and get a good sleep.
It’s Monday, it’s the year 2025 and I’ve woken up with a great new idea. Kitten internet where your kitten can order its own food! I check my messages. One of the pioneers in the mood cell has had a much better idea. I’m off to help her map it and see if we can’t get another cell up and running.
— —
The scenario was put together very quickly and is only an illustration designed to explain one thing. If you use a map then there is no reason why operations, build, strategy, finance, HR, strategy and other groups can’t happily work together without miscommunication, misalignment, duplication and bias. All of the above systems I’ve used in one form or another across multiple groups in a business over the last decade.
Unfortunately there is currently is no integrated tool for doing this even though it promises a future where development, operation, HR and financial can be combined together through the use of mapping of some form. Before you go “we have maps” — the chances are you don’t. Maps at a minimum need to be visual, context specific, have position relative to an anchor (e.g. value chain relative to user or physical position relative to compass) and finally they need to visualise movement (e.g. the evolution axis).
Where will such a tool come from?
The practices and systems above already exist to varying degrees within a number of organisations. As for codification of these practices into some formal tool then we’ve already started down that path at the level of code with AWS Step Functions. It’s well worth watching the video. Despite the appearance of being technical, the future of business development is being born here.
YouTube video on an Introduction to AWS Step Functions
There’s quite a long way to go but eventually it’ll move up the stack and as systems become more complicated then it’ll have to cope with evolution itself. As useful as micro services are, you will eventually find that you have custom built components on top of layers of custom built components. Fragility starts to bite. You’ll find yourself having to deal with the problem that evolution causes. However, as soon as you start to visualise this then you’re on the path to creating a map. The future scenario above becomes possible and this is where AWS step functions will head, in my humble opinion. Almost every past practice in business will go boom! This is marvellous. Thank you Amazon.
An alternative path comes from tools such as Atlas which have started with user needs and are making the journey down the value chain. Eventually these two approaches will overlap.
One final note
I often take things for granted and forget to spell things out enough. If it’s not clear the maps in the scenario above and the related systems (e.g. spend control) simply represent the codification of basic doctrine. If that word doesn’t mean much to you then can I suggest reading some basics of strategy (also, if you don’t know what a map is — start with chapter 1)
I’ve highlighted below (in orange), the basic doctrine that the maps and systems cover in the scenario. If this is unfamiliar ground for you then spend sometime going through the scenario and checking them off. You’ll find them all hidden in there.
When you think about AWS Lambda, AWS Step Functions et al then you need to view this through the lens of automating basic doctrine i.e. not just saying it and codifying in maps and related systems but embedding it everywhere. At scale and at the speed of competition that I expect us to reach then this is going to be essential.
For some further background reading on all the above try
Amazon is eating the software (which is eating the world)
Why all the fuss about serverless?
— —
This post is provided as Creative commons Attribution-ShareAlike 4.0 International by the original author, Simon Wardley, a researcher for the Leading Edge Forum.
Catch me on twitter as @swardley
Originally published at blog.gardeviance.org.