Platform teams are everywhere right now. I have worked with various shapes and sizes over the last few years. But first, let’s head to an authoritative source, to give us a vision of what a good platform might look like, rather than my messy reality.
Evan Bottcher gives an excellent definition in this article:
“A digital platform is a foundation of self-service APIs, tools, services, knowledge and support which are arranged as a compelling internal product. Autonomous delivery teams can make use of the platform to deliver product features at a higher pace, with reduced co-ordination.”
This definition sounds amazing and something worth working towards. However, the world is often not shaped like the definitions we create. I say this because as a tester, it’s good to know the terms under which you are working.
Anti-Platform Patterns
If I’ve learned one thing over the years it’s that you should test from where you are and work towards where you need to be. Rather than trying to test in a perfect vision of a platform team that doesn’t exist. This is especially true with concepts as ambiguous as a platform team. These are some patterns from my recent past:
- Project Driven Platform Teams – we will build you a platform, but the project must pay. A very strange beast which is looking to build for the long term, but the short term always wins. From a testing point of view, this approach prioritises adhoc requests over stability. Being generous, this is where the project funding model survives but there is a desire to have a common offering to enable project teams. If there is a one model that survives in virtually all conditions, it’s the project funding one. Recognisable by have no discernable product strategy or product people in close proximity.
- Hidden Within Delivery Teams Platform Teams – on one client, we had product people stalking the corridors. Looking for developers to build features. Not even asking teams but individuals. This was not the ideal place to build a platform team. So we did what any grown up would do, hid the platform engineers in the delivery teams. This was good as it spread the word widely but caused coherence problems with what the platform would consist of. With regard to testing, lots of co-ordination and curation of disparate information work.
- System Build and Platform Build Teams – strangely enough this was my first and best exposure to the platform team approach. The System Build team helped product teams with tools and advice to build their applications, Platform Build with what they would build it upon. With a nice set of interactions between the two Build teams too. Strange how life in software is like that, earlier experiences of a pattern were actually excellent but you didn’t know it.
- Individual as a Platform Team – taking the concept of Thinnest Viable Platform too far. I won’t say too much about this pattern. If your company has one person building and maintaining their whole platform, then no one is taking anything seriously. Apart from cruelty towards their people.
Being a tester on a platform team is a massive opportunity to influence the foundations of each product team. This make it an excellent spot to be able to influence testability, for both your benefit and that of the teams that consume the platform. Also, it has its problems. Many of the newly created platform engineers on this team may not be used to having testers around. From prior experience, the infrastructure world can be a more closed place.
Pro-Platform Testing Patterns
Aside from recognising what type of platform team you are on, starting from where you are and helping the platform to continuously improve, what should I do as a tester? My shortlist is below:
- Use consistency heuristics – first and foremost the platform is supposed to be aiming at becoming a coherent internal product. To help this journey, deploy your trusty consistency heuristics. As a general approach it ticks many of the platform team boxes. Take ‘History’ for example, the platform team provides a one to many relationship therefore breaking changes should be extremely rare. ‘User’s Desires’ speaks to taking the ‘Developer Experience’ into account. I use this heuristic as a tool during story kick off sessions for example.
- Adopt other teams tests – Use the tests that your project/product/stream aligned teams build, especially the load tests. Think about it, you will be building the ability for teams to provision what they need. While testing the platform you can’t become a deep expert in all the application that depend upon your platform. You will come across new environments provisioned for teams that just plain don’t work, especially early on. They have all the components, ports open, load balancing rules but the actual applications don’t run on them. Being able to understand and run the tests created by the team concerned will tease out configuration issues. The environment isn’t ready until the applications can be tested. Meaning it must not only be alive, it must be ready, You will make good, long lasting friendships if you recognise this. Being able to run load tests for applications is the ultimate version of this, giving insight beyond a few selected health checks to see if it stirs.
- Learn cloud provider capabilities – if you are on a platform team they will likely be using a cloud provider. It is extremely likely that any organisation who adopts a platform team approach is likely already down the cloud adoption path. Cloud computing has an obvious impact on testing. When combined with infrastructure as code being able to provision environments flexibly is a benefit. Don’t forget though, it’s integration into your systems still needs to be tested. For example, in a previous role we had a system that needed to scale rapidly for a specific time period at a specific time of day. After some fairly basic load tests it became obvious that although the compute power we needed to could be autoscaled. Scaling happened nowhere near fast enough. Thus we had to adopt the services provided by the platform to cope. Autoscaling but also timed fixed scaling to prepare for times of high load in short bursts. Most cloud providers offer loads of free training which provides the high level of your cloud providers capabilities, I use the AWS Architect path often to understand the bigger picture.
- You can’t test this – Watch out for “you can’t test this” comments, which are common for infrastructure changes. It’s not personal, it’s just that how it might be tested might not be obvious to the team. Or even to you. I start with pairing here, ask to be shown the change that has been made. You might decide not to test it, you might come up with some interesting ideas that were previously not considered. You might even decide to reach out to the teams that consume that bit of the platform to see what the impacts could be. The “you can’t test this” situation is usually an invitation to think outwards more than inwards in my experience. Start with the demo though. Its like any other relationship, build a rapport then move on to feedback.
- Do some platform engineering – need to add a new environment to the secrets management system? Volunteer. Need to change the instance type in your Terraform? Volunteer. Even write some Molecule tests for an Ansible playbook. You guessed it, have a go. I think this is really important for testing as part of a platform team, beyond that of on a project/product/stream aligned team. Infrastructure engineers are often not used to having testers around so contributing to the day to day work of the team is a good way to get involved. This then opens the door to more searching questions and involvement in story kick off and wider product sessions. The truly golden aim here is to get involved in discussions with the teams that consume the platform. Thus being able to improve the world for all teams trying to generate value. A good way to do that, is to contribute to the wider platform building effort. Useful team members get invited to things.
- Become the metrics collection and monitoring expert – testing is the information business. Gathering meaningful metrics and then choosing how to monitor and alert against them is a good match for a tester. Find out how your application has been instrumented (Micrometer for example), where those metrics are going (Cloudwatch perhaps) and where your information will be displayed and alerts configured (something like Grafana). The benefit of getting involved in all this is that it’s yet another touchpoint with the teams consuming the platform. Information like this can help to model performance and load scenarios. Aim testing at the areas that provide most value/show under-tested but regularly used areas. All in all, very useful indeed.
I will caveat the above with one warning. If you are less technically minded (argue amongst yourselves), it may be a place which is very hard for you to make your mark. For example, if the types of tasks in the ‘Do some platform engineering’ section make you feel very uncomfortable then make sure the commitment to support you exists before moving on to a platform team.
There is a lot of rhetoric around shifting testing to where it might add greater value. Being a tester on a platform team is one of the highest expressions of the shifting testing approach. Shifting testing anywhere has its roots in continuous improvement, as does building a platform. If you get the opportunity and its right for your career, give it a try. You might be enabling better testing for every team, not only your own.
Diagram Industries 