My approach to agile* adoptions is based quite a bit around Tai Chi and Taoist philosophy which I won’t go in to as you can search about that on the net. There are a few principles worth noting from these:
To the mind that is still, the whole universe surrenders ~ Lao Tzu – 6th century, BCE
Empty your mind. Be formless, shapeless. Like water. You put water into a bottle and it becomes the bottle. You put in a teapot, it becomes the teapot. Water can flow, or it can crash. Be water, my friend! ~ Bruce Lee, 1971**
which are a good philosophical summary of my approach to agile adoptions (well, actually life in general ;)
This series of posts is not directly about philosophy, it’s about the day-to-day progress of an agile adoption: The Good, The Bad and The Interesting.
So stay tuned, check back and drop in if you’re interested as this is just the beginning…
* In this series I’ll use the term agile with a lower-case ‘a’ to indicate “real ‘good’ agile” which I’ll explain as we go, compared to “Industrial Agile”, “Consulting Agile” and “Corrupted Agile” which start with an upper-case ‘A’
After a finishing up my current contract, holidays and a bit of Bloggers Block, a chat with John Wenger about humanity and process a while ago has inspired me to do this post that I’ve had brewing for a while… Your first response may be a glib / tongue-in-cheek response “What, humanity and PMs, that’s an Oxymoron”. And therein lies the problem – I feel I almost need to point out that PMs are people too :-) Most of them are trying to do a good job in a probably difficult situation. They have families, lovers, pets: and they’re just at work like you – doing their best…
To tease this apart more, the first thing we need to do is separate the person from the role. With that, we can acknowledge that people are generally the same, so there should be no core difference between a PM and others in an org. Then we have the role – this is unfortunately where distortions can come in. Not only from the role, but the organisation and it’s view of that role which often puts them forward as “Managers”. I mostly see them as “Administrators”, though and I’m not saying this in a mean way – as an Architect (or any other core-IT role) I see them there to help me to build things for an organisation. Sure, they have their plans, but any good PM knows how to work the plan with the people and not the other way around.
In most of my engagements in large organisations, as a facilitator I’ve gotten along well with the PMs personally and professionally and they’ve always helped the project (well, there’s always the exception, but one can say that about almost anything). Strangely though, this seems to be the opposite of most of the “agile crowd” who seem to constantly complain about and want to get rid of PMs. What’s up? Have I just been lucky the last 15-20 years? I think not…
How would you feel if I came in to your work and told you I was a BoingBoing Master and that part of the BoingBoing Approach (tm :) I’d be replacing you, or if you were lucky would turn you in to a BoingBoing Apprentice where you’d have to start from the bottom again and work your way up?
Well that’s the attitude most agile people do! And then they wonder why PMs are so “difficult”…
As I touched on above, I acknowledge that while PM is not an essential role in agile, it can be accommodated. Part of this probably comes from my earlier work with (Iterative) RUP where PM is an acknowledged role. When I progressed to Scrum, it was using RUP Iterations, so again, the PMs were there – helping with Iteration and Project planning, while we ran the Scrums and handed most problems over to them to sort out.
This brings me to my current approach: As I mentioned, I view PMs as “skilled administrators” and “organisational facilitators” who are necessary in in large company. On that basis I help them understand how I can work with them as I’ll help the team focus on their work and maximising it, while they can help communicate this to the wider organisation and help remove obstacles that the team will be surfacing with them. This way, we both benefit, generally doing what we all enjoy.
Perfect? No, but it’s sure better than going in to a situation where you have exist with a powerful and combative “opponent” every day…
This approach also allows for what I’d call a “classic” Scrum/Kanban/… Facilitator, where this is not a full time job (which I can remember in the early days of agile and scrum). The rest of the time, I get to perform a team role (usually Architect, but it could really be any – e.g. (Business) Analyst, Developer, Tester, Infrastructure) which enables me to be deeply involved in the project and therefore not need “explanation sessions” where the team explains what they’re doing to someone who doesn’t really understand.
So is that it? Am I advocating we just sit back and not cause “too much trouble” for the entrenched system? NO! It’s quite the opposite as I’m suggest a campaign for Hearts and Minds conducted with Love and Compassion rather than the “Shock and Awe” campaigns that are often conducted by the Agile Extremists…
It’s amazing how time flies as it seemed like yesterday, but it’s now been 8 years. Must be that old adage as I’ve certainly been having a lot of fun here. When I’d finished my last post on this, Agile Baby Steps – Iteration 3 (2 & 1), I’d finished up in Australia and moved to the UK. By coincidence it was where my Agile journey got a bit of a Turbo-Boost
After a few months of being a tourist and “kinda looking for work” I finally ended up in Andover which is a nice “little” market town just over an hour out from London working with Lloyds TSB. The role was initially as a Service Designer, but due to some enlightened management there I was able to introduce Scrum. Remember that in Aus I’d only just used it with a Scrum Master to integrate with Rational Unified Process, so I’d only just started down this route. To do a proper implementation we ran training with about 15 people and a Scrum Alliance trainer. This was the first time I’d been exposed to “pure Scrum” and it absolutely blew my mind! During the time there we rolled Scrum out across the various projects in the Business Process Re-engineering programme, even scaling it. Probably the key reason things went so smoothly was that the programme was Agile in it’s BPR activities and there had been a Lean effort in the past. Also, the programme had the concept of “Daily Prayers” that were like a Scrum, only unstructured so Scrum was really just a refinement (as the DP’s weren’t scaling) and extension which made sense as the program was just starting.
Next along was TNT and I had no intent originally for any process work as I was just there to do some performance tuning and architectural refactoring. After a few weeks though, it emerged that the team they were building up was having problems keeping track of things. One of the lessons I learnt from the previous engagement was not to be too prescriptive, so I said I knew a simple technique that could help. I told them the basic rules of Scrum and had Backlog, Doing, Done but did not tell them what to put on the stick notes! In the beginning it was just the task but after the first retrospective someone suggested adding the person responsible – “great idea” I said ;-) After the next retrospective, they suggested adding time! They’d invented Scrum Sticky Notes! I think this is a great example of what happens when you trust people and let them own “the process solution”.
Even better, was the fact that this was integrated with their existing Project Management process as at the end of each sprint I gathered up all the sticky notes and gave them to my enlightened PM who then did whatever she did with Gantt charts, reports etc… As the Scrum Master, I worked hand-in-hand with my PM and it only took less than 1h per day for me as the PM handled all the issues. Oh, but it didn’t stop there – we had an offshore team also. Again, thanks to enlightened management, we had a number of devs from offshore being rotated in and the timing was perfect. After a few weeks some of them went back to India and because they knew the process (it was theirs after all) Scrums were quickly up and running over there. We’d then have a teleconference Scrum to sync everyone up which worked a treat.
The interesting thing about the TNT implementation was that the “pure Scrum” people would be frothing at the mouth as we’d probably broken a number of “rules” – e.g. estimating in time, rather than story points, bananas, etc… Yet, I’d regard this as on of my best Scrum implementation so far as we’d cracked two of the classic problems that people have:
- Working in a more conventional Project Management process and
- Working with offshore teams
Most of this was not to do with me, it was obviously the management chain running through 3 levels who all supported this as what they’d been doing obviously didn’t work as well – they were ripe for change.
After a “pure SOA Architect” role I then ended up in Agile Land, this time at Yell (RIP ;) which was an Agile environment – and I really mean this! They were running a whole BPR programme using Agile with some of the best coaches I’d worked with. Needless to say, I learnt a lot about the subtleties of Agile and was even able to contribute at the end in helping develop a more scaled version of agile based around Dean Leffingwell’s Scaling Software Agility for the entire enterprise. One of the key lessons though was that you can be “too agile” – at one stage, as an Agile SOA Architect, I was continually (every few weeks) altering service designs for the Implementation teams. In the end, they (nicely) screamed “Enough! We can’t absorb this much change!”. As a result of a retrospective workshop we did on this issue, it was agreed that there should be a bit more up-front work and broader investigation in order to minimise the change to the designs they were implementing.
Coming off the pure agile role, I landed in what was at one stage one of the largest Agile projects in Europe at British Gas. Due to it’s scale (we’re talking thousands of people) there were some waterfall elements to it, but the base rule was that wherever Agile made sense, we’d use it. Again, more amazing scum masters and even some of the ones from Yell. For me, this role followed on nicely from Yell to a place that had implemented Agile at scale. There has been a lot of debate as to how Agile their approach was, but from my perspective, I was at a transition point from Waterfall into Agile and it was working, but it wasn’t easy. There were some great features though, especially where they’d applied an “Agile Mindset” to a “Waterfall Mechanism” which resulted in a more people-centric and communicative process.
For a while though, I’d been reading and learning about this new “Kanban Thingy”. It certainly had an appeal and I realised that I’d actually been using some of it’s practices in Scrum, so I decided to start injecting some Kanban tools in to Scrum (this was before I’d heard of Scrumban). I then managed to use this approach successfully in two Enterprise Architecture assessments for global organisations topped off by an Agile implementation of Agile Governance for SOA a few years ago.
As I went through this beginning of my Scrumban phase (which I’m still technically in) I had a dawning realisation: there is quite a disconnect between the work I do in Solutions, SOA and Enterprise Architecture (where I’ve used Agile, Scrum, Lean and Kanban most of the time over the last decade) and what I was reading and hearing about in “Cool Agile Land” (that’s where all the cool hipsters are xDD’ing ;).
It was then that I realised that there was just as much of a need in these areas – the problem was that this was not where the majority of “activity” was happening and that in general the mindset was more waterfall. There were however, some mavericks (who both backed and worked with me) like my self who believed that there was a middle path – we could use Agile techniques in Architecture in Big Organisations without conflict and hopefully start a revolution…
A scramjet (supersonic combusting ramjet) is a variant of a ramjet airbreathing jet engine in which combustion takes place in supersonic airflow.
A Skramjet (ScRuM, KAnban, lEan, Just in Time) is a process in which work takes place in a SuperPersonic flow state environment.
A personal exploration of the use of Kanban, Scrum and Lean Principles to create a Service Oriented Architecture within a large organisation.
Within the last few years, for reasons that are only partially known to me, I accepted a contract at a “Self Confessed pUre Waterfall Organisation” – let’s call them SCUMO for short ;-) I know why I worked there originally – the two people who interviewed me were great – one was the head of architecture, very outgoing and inspiring – the kind of guy you wanted to work with; and the other (my boss to be) was a rather quiet spoken yet meticulous fellow – as Douglas Adams would say – “Mostly Harmless”. I mean hey, just because the organisation was waterfall, with those people and a great team (which it was) – what could go wrong?
It didn’t all happen at once though – the first project I was on was a “dream project” – great PM, good BA, tech team who knew what they were doing – I was seriously wondering what everyone else was moaning about – my world was GREAT! It was after this though, that things started to change – I was drafted in to doing a quick piece of Solution Design for a really cool sounding and disruptive project within the organisation, which of course went nowhere. Then, I was put on a project with a PM who resigned after less than a month (I later found out that this sort of thing was “normal” within SCUMO, but they regarded that as people “just not being up to the task” or whatever self-serving platitude (and there were many) that came to peoples minds) which was a disaster. We had A Business Process Primadonna (let’s call him BAPP :) who was a bully (I was very close to reporting him as such to the organsation) who had the full support of the management – “Oh, that’s just J, but he has decades of experience and we need him, so you just have to put up with iT” (which is why I didn’t report him – that would be an Express Career Limiting Move). Finally though, there was a small light at the end of the tunnel – another “really cool and transformational project” but that was rapidly pulled in to a small clique (who had all worked at another SCUMO type organisation) who were grabbing all the interesting and high profile projects for themselves – that was the last straw, the writing on the wall, so I left.
Why did I stay at all then? A question I asked myself a few times… I came to the conclusion that the negatives were temporarily damped down by the fact I had such a great team that I worked with on a daily basis. Our “boss” was fairly Laissez-faire and we had quite a number of highly competent individuals. Still, it’s hard to really enjoy things when whatever you’re working on is either totally ignored or stripped down to a degree where it’s not your original design, and what’s worse – is of lower quality and costs more!!! Again, and again, and again…
Back to the Waterfall Process – everything above is a consequence of having a totally Command and cOntrol, Waterfall-based Structure (or COWS for short). Maybe it works in the military, or in a Cotton Mill in the 18th century, but please tell me how it actually applies to modern information technology solutions..?
- Waterfall based systems have no concept of flow!
Because of all the silos and steps, they’re based on the naive assumption that things actually work this way (and perfectly). The only problem of course is that they don’t. As you look further up the “organisational stack”, you’d see people / senior managers who were inundated with requests for their time (often because people had to cover their asses or were dis-empowered), yet they had no real way to manage it (Personal Kanban anyone?). It really just depended on what was the most politically critical issue was for the day… The one time I did attempt to introduce flow based thinking, I received the rather patronising response from the CEO of “baby steps” – if they go at his rate, it will be next century before they embrace contemporary approaches… which is probably about par for the course for that organisation ;-)
- Waterfall based systems have minimal feedback
I was absolutely amazed at how little capacity for feedback was present in the system. When it was, you’d generally have to go through a committee – do a presentation and documentation for them and if you were lucky – 6 months to 6 years later (depending on how much politicking you’ve done), your suggestion would actually be implemented…
- Waterfall based systems have no compassion
Time and time again, I was reminded by “the system” about how “people” were really “resources” or “interchangeable units”. The ultimate irony was that Waterfall Systems actually focus their efforts on individuals such as BAPP and their like to create “central points of failure”. The only people who were shown “compassion” (in a weirdly inverted way) by the organisation were people like BAPP. Everyone else was on their own and could always leave the organisation as they’re not “up to the task”
- Waterfall systems struggle with early risk mitigation
Ironic isn’t it? The nature of the system dictates that Risks are left until the latest point in the development cycle as “it just might fix itself” or we could make it “someone else’s problem” or even a “problem that we know will occur later but will be outside of the current contract warrantee period” – the list goes on… At the end of the day, I was astounded with the amazing capacity of Waterfall Systems to act in the worst interests of the organisation and their clients, which brings me on to:
- Are Waterfall Systems (more) Hypocritical?
I’m phrasing this as a question as there seems to be a fair dose of Hypocrisy in the air these days no matter what the organisational and process type, but I felt an extremely strong presence at SCUMO who supposedly focused on and excelled in:
- The Customer
but they managed to implement exactly the opposite of all of those! Systems Thinking anyone? If ever there was an argument against ruthless command and control systems, this must be it. No matter how hard they tried to shove, threaten, reward or cajole people about these values, nothing worked!
- Waterfalls help and encourage Dunning-Krueger
Every organisation has their share of people who think more of themselves beyond their ability, but this organisation seemed to excel in it. BAPP was only one of many mediocre / “excellent” people there. It was not unusual to have a meeting with over 50% D-K people, in which case the best thing was to just sit back and let them pontificate amongst themselves… So we had a situation where the blind were not only leading the blind… They were reporting to a committee in a dark room!
- Waterfalls have no concept of a system!
Sounds crazy doesn’t it? Yes, they have Change Control Boards and a whole bunch of other crap, but at the end of the day, there is no acknowledgement that they are a system that is part of a wider system. Because of that, they don’t stand a snowflakes chance in hell of understanding their context or of being able to effect any change beyond minor tweaks such as changing the format of a table in a document which would often take months
- What a Waste!
Coming from fairly agile environments that had at least some element of the lean concept of minimising waste, I was astounded at the amount of waste that was produced during a project – even sometimes even in things I was working on! As a rough estimate, easily 80% of what was being done over a whole project was waste, like documents that would be read once or never by the “supposed target” audience. What sort of impact would this have on a project basis? There was one project to change the background colour on a set of web pages which took months!
- What a Waste of People :-(
Most of the people that I worked with on a daily basis were really trying to do the best for “the customer” (although they were quite often punished by BAPP’s and the organisation for doing so). Eventually, there seemed to be only 3 “attractors” which would allow people to exist in this environment:
- Give in and just play the “jobsworth game”
- Get involved in the politics and back stabbing – become a “power player”, BAPP COW or the like
- Try and change the system – this was only really a temporary state as I never saw anyone that could truly maintain this as it sets up a cognitive dissonance
None of the above result in a person permanently acting to anywhere near their full potential within the organisation. If they’re lucky, they get to do a bit when they attempt change, but as mentioned, this can never be a permanent situation.
The sad fact is therefore that most or almost all of the people I worked with weren’t happy. Were the managers happy, in their controlling positions? For the most part, no. Some people had material happiness through incentives, but most of them seemed quite miserable too. Oh, there was one person who was happy – the security guy who sat at the front desk…
In summary, I found the whole environment extremely primitive (as in Cro-magnon) and insensitive (1950’s management). Not that I’m saying that Agile is nirvana, but more that in a company that uses some form of Lean / Agile / Kanban etc…, you’re more likely to get or be able to influence people to adopt a more effective, humane and respectful approach to software development.
From a personal perspective, this just showed me how much I have learnt and grown over the past decade as the sad fact is that SCUMO are not the only company like this. What this experience gave me was the realisation that I took the red pill ages ago and that I really can’t turn back – it’s just too painful.
You take the blue pill, the story ends, you wake up in your bed and believe whatever you want to believe.
You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes.
If you haven’t already, you should probably read Enterprise Kanban Part 1, which brings us up to the end of the first week where we had enough to get started and had used a “rough Kanban” for a week. At the end of that week we ran a retrospective:
Remember, these guys had never done Scrum or Kanban before. They were self-confessed “standard process heads” who came in to this with open minds, probably as they knew that a standard waterfall approach would be meaningless in a 1 1/2 month context and probably fail miserably.
The insight shown here is great and they’re contributing well, which you can see by the fact that I happened to use a green pen and my teammates used black. Interestingly, two of the 3 improvements voted for are really reflections from them wanting to use the process more effectively. The thing this revealed to me about retrospectives, is they can give people an objective way to ask for help. Rather than say “I need to use the system better, can you please help me?”, they are able to say “We need to improve the system so it is used better”. This takes the ego out of the equation and introduces objectivity, probably reduces the violence of the dialogue and encourages working as a team rather than a set of isolated individuals who need to “buck up” and “get with the program”. We already had a “Daily Scrum” in the morning which included our PM and others (such as our CEO) if they wanted to attend as Chickens but the other improvement, “End of day catch up (internal)”, was a great piece of insight given the pace we were working at (fast!) and the fact that we couldn’t let mistakes (which would occur) stay around for long. It also gave us a chance to quietly reflect at the end of the day amongst only ourselves and set us up nicely for the next days Scrum.
Note: I didn’t tell them beforehand that we’d select the top 3 improvements, it’s just a coincidence that there were only 3.
Next, it was time to set up a new board based on what we’d learnt the previous week – ah… nothing like a nice shiny new board, ready to be populated :-)
You can see that the Project (with sticky notes) plan now has an official place, and we’ve distinguished between Waiting (as a lot of our work was information gathering, and in a large company, that means a lot of waiting) and Stalled, which is the conventional concept of something that can’t be done for a blocking reason (other than waiting). Finally, we have the Deadlines area as the others wanted a way to track these. I personally didn’t think this was necessary but knew better than to impose my view – people need trust and the freedom to learn and who knows? They could be right!
I was obviously really happy as we’d got through the first week, my other team members thought this was useful and even better, were immersed in the process and contributing deeply to it. This should not really be a surprise though, as they were both smart people with about the same amount of experience as me and were open enough to embrace something that was a bit different but would help them work more effectively.