He's not wrong. Having been involved in the Agile movement since before the term Agile was coined, I think of Scrum as the least interesting of Agile processes, but also the most successful in terms of adoption.
I used to think that was a contradiction. Now I think it's almost inevitable. I wrote more about it elsewhere [1], but the basic deal is that most companies have other priorities than being effective, so the processes that dominate will reflect those priorities.
On average, the first priority of managers and execs is maintaining the power structures that make them a big deal. But true Agile processes are about empowering teams to self-organize around serving users. Ergo, Sturgeon's Law [2] applies here too: 90% of company software process is crap.
I think reason why Scrum in specific won out is that it combined a process framework unthreatening to executives with the multilevel marketing scheme of Scrum certification. Executives got to keep doing what they were doing, the workers got a "didn't stop breathing" certificate, and coaches got paid. Win, win, win!
Scrum is badly misunderstood and maybe that's a failing in and of itself but it's really a victim of the developers who sold it as a magical process. Scrum simply can't be implemented as a purely developer process. The backlog exists as a rolling contract between dev and product owners and sponsors.
The most common failure I see is when project leadership agrees to a fixed scope and timeline then tries to execute in an agile way. Agile is the contract. But it's also a tough sell. It requires a high degree of trust to say "we'll pay you $XM in exchange for X sprints of whatever we prioritize with no fixed end state" but that's what you need to be agile. If you layer sprints on top of a fixed timeline, you're just adding needless complexity since your end state is predetermined and how you get there is irrelevant.
I have almost never seen Scrum sold as magical by developers. Executives? Yes. Consultants? Yes. But developers?
I agree that fixed-scope contacts cause a lot of problems for Agile approaches. However, they also cause a lot of problems for non-Agile approaches. If I have to deal with supposedly fixed-scope situation, I'm going with and Agile approach.
There are two basic cases. One is that scope is truly fixed (which is rare). In that case, having a new releasable version every week with highest-priority features first is excellent risk management. When the date comes, you'll have something to ship. You also get to continually validate and improve internal processes, so you're more likely to be using the time available effectively.
The other is that scope was fixed in the contract but is in practice variable. So every week you deliver something to the customer. Every week you build trust. And every week you encourage them to do tests, deploy early, anything so they start getting value. At that point they get user and business feedback, and come to you with changes. That's the point where you start shifting from fixed-bid to time-and-materials. Maybe you keep to the same budget, but now they're getting more for their money. Maybe they look at the backlog and say, "Wow, there's still lots more value to deliver, let's keep going."
Seconded. I've also seen it promoted by developers who are in what one could call a "honeymoon period" of their careers. First or second job, probably learned programming at university so every task is an interesting challenge, SCRUM is their first agile methodology, they don't have enough broad knowledge about programming and the industry to become disillusioned and cynical. I've had such people evangelize SCRUM to me, with pride in their eyes, like it was the best thing since sliced bread.
I’ve noticed this, too. For some reason a junior developer on my team 2 years our of school was made into scrum master, and if we so much as sneeze there’s no JIRA ticket for it he starts whining.
it's really a victim of the developers who sold it as a magical process
Developers may advocate small-a agile, but I have never heard of an actual developer pushing for Scrum, SAFe, or any of the name-brand "agile" methodologies, that come with expensive consultants and certifications and "coaches" and conferences and a whole ecosystem around them. These are things for project managers for the benefit of project managers who see project management the main deliverable l in and of itself. The same with JIRA actually, it's not a software development tool, it's a project management tool, sold to the same people who previously bought MS Project. These things are all overheads in the software development process, not enablers, as another poster says, the goal of most organisations is not to be effective at their stated mission, but to maintain their internal power structures.
Have you seen any methodology work consistently when dealing with fixed deadlines? The reality when dealing with large contracts, as you mention, is that there are almost always timetables with expectations. This poses an inherant problem due to the unreliability of estimates, so either quality or features must be sacrificed if the timeline is in jeopardy. My only experience in such an environment was using some hybrid waterfall/sprint methodology that mostly worked fine, but i suspect that was more to with the general culture and people involved than anything else.
I think no industry producing anything new ever found a working process to predict and keep the timeline. (If omniscience really existed, it would have more lucrative applications.)
Developing new aircraft, building a new ship, building a custom-designed bridge (most of them are) are processes that often run out of time and / or budget.
If you want predictability, you want repeatability. But in software all reliably repeatable parts become automated away.
Absolutely. And I'd add that predictability is dependent upon not learning anything over the course of the project. To be predictable either you know everything that matters up front (which is only true for trivial projects with no competitors) or you refuse to learn anything along the way (with, e.g., a big-bang release at the end).
But I think good projects release early and often precisely so that they can learn as they go. At which point predictability goes out the window.
? All the specified above endavours - usually run into what one could call the end of the map. Here be science! If you run into e.g. new material science because your goal stretched beyond the scope of what has been done previously, you can blame the engineer for not telling you that you will leave the boundary of the knowledge of a field- you can not blame him for the estimates beeing wrong.
I have seen more success in cases where teams estimate for 90% percent confidence instead of 50% confidence. By that I mean "it should almost never take longer than that" vs. "it will probably take that long." Unfortunately, to do that, you need enlightened business management that appreciates the difference between an estimate and a promise, as opposed to business management that pays lip service to the distinction.
I haven't seen a good way to turn the concept of "it should almost never take longer than that" into a concrete process. I've always seen it as a gut check, often implemented as "take your initial estimate and double/triple it." What I actually see is that a lot of teams implement that but walk back the doubling when the business/PM delegation persistently asks for more in less time. Only in really engineer driven cultures have I seen engineering teams successfully push back.
Exactly. Probably 90% of the burden on a successful agile project is in setting expectations with stakeholders for an iterative delivery without a fixed scope. It can be a tough sell but it shouldn't be. If you've ever done a top line estimate on a fixed scope project, you know it's just absolutely not possible. Why set a deadline you know with 100% certainty you're not going to hit? Just bake it into the contract and upfront expectations.
Yes, absolutely. Many times. I've done a bunch of projects that were tied to immovable event dates. The only way to deal with it is having very flexible scope. Typically, these have been more "creative" projects and not tied to a lot of critical functionality so we just make sure we keep our MVP quite small and treat everything else as iterative enhancements so we can cut off whenever we're out of time and still have something presentable.
This is true, but fixing the deadline and not committing to the scope requires clear communication by the project management / sales teams with the clients about it, and this is rare.
> Scrum is badly misunderstood and maybe that's a failing in and of itself but it's really a victim of the developers who sold it as a magical process.
Even if so, that would be in itself a failure of Scrum. A process that works only sporadically and only with exceptionally well organised groups ain't really all that much.
> It requires a high degree of trust to say "we'll pay you $XM in exchange for X sprints of whatever we prioritize with no fixed end state" but that's what you need to be agile.
...no? It requires a well-defined end-state, and that includes not defining the irrelevant. It's not really "give me $XM and we'll deliver anything from accounting software to a really nice puppet."
There are in fact two kinds of Scrum (or rather, Scrum implementations), both of them compatible with the Scrum guide: ‘Left to Right’ Scrum (backlog-driven, implementation-focussed) and ‘Right to Left’ Scrum (goal-oriented, iterative).
Unfortunately Scrum is too often explained and implemented that first way, leading to the anti-Agile feedback we see on HN with some regularity. The second way is much more compatible with complementary tools such as Lean Startup and Kanban and I wonder if this (to me very welcome) non-exclusivity explains why it is less talked about.
"Right-to-left scrum" seems like a hollow phrase made up by that consulting firm you linked.
From what I can tell, it has all the meaninglessness of an empty buzzword used primarily as a method of trying to engage people and give an opening on selling their services by stating "oh if you don't understand the difference sit down with us and let us show you how different and better it is."
It looks like if scrum doesn't work they call it "falling left to right scrum", and if it works it's their "brilliant right to left scrum".
Even by their own definition they both have a backlog that gets prioritized and selected each 2 weeks into a sprint backlog, and executed during the sprint.
The rest is just hand waving. "One is goal focused vs the other is backlog focused. Oh yeah but we do put our goals in the backlog." So they're both backlog focused then? The only thing they're really sayings is "When prioritizing tasks, make sure they accrue to something and aren't just random work." Which is both obvious, and of course too simple to write and sell a book about, so instead this whole other terminology is made for it.
If I ever find a developer who can sell, selling is not a developer's USP in general, there are exceptions and I would hire those exceptions instantly!
Waterfall was coined by a paper discussing why waterfall wasn't a good way to do software projects.
You aren't wrong, either. I found Scrum to be a series of waterfalls that weren't well thought out. "Iterative Design" essentially meant, "We aren't sure what the button should do exactly, but we know we need it there and it kinda has to do this and we'll figure the rest out for the next iteration."
That caused so many problems with tech debt.
Stories began to take longer due to the increasingly large and discombobulated code base, but the expectation was that we continue to deliver the same number of story points each sprint.
Then we got blasted for losing pace. "Why are you under-performing? We are just adding little features. You've already implemented this button on another form before? Why does it take longer the second time? You already know how to do it!"
It became absurd and no one could hear or understand what was happening was easily predictable and in fact -- was predicted by several members of the team many months prior.
When the project management was informed of these predictions now having come true, the development team was accused of intentionally causing the delays.
> "How could it possibly take that long? How do you not know how long it will take?"
Well, if they really said that, you can just dismiss the comment as incompetent. A witty retort may be in order if a non-technical manager is in earshot, e.g.:
"How long would it take for you to earn a green belt in karate?"
"Hmm, what does a green belt entail?"
"Exactly, you don't even know what you don't know, yet."
"We don't know because we've never done this before, and we can only compare to the most similar work we've done, and any existing data."
Before I could say more, they I interrupted: "You have to know, it's your job, or you don't know what your doing," or something along those lines. It was the most heated I've felt. I know what I'd reply with today, but I was younger and more naive then.
In these situations I shift the conversation to what is the PM really after. I explain that I can tell them any estimate they want (or they can make it up on their own - no need to even discuss), but does that really do them any good when it is missed? There is usually some pushback at first, but when items are delivered within estimate a few times in a row they realize it's better to take whatever hit up front than it is to miss and adjust.
Also, someone asking how could something take so long is not a belittling question. I don't see any problem with either question.
Personally I believe this is one of the major gaps in Scrum, and there is no prescripted way to solve this problem. XP's focus on technical aspects is much better in this regard, but the possibility of runaway technical debt still exists.
There are so many places where this can sneak in and become an issue, too:
- The development team can groupthink, underestimate and/or fail to account for technical debt payback in planning poker. They may estimate the cost for the new feature, but not include any time for refactoring to keep the codebase healthy
- The Scrum Master may not shield the development team from pressure to deliver coming from the product owner, which may cause the team to cut the above corners
- Codebase health may be split out from vertical stories into its own tasks that get deprioritised by the product owner
- Even if the development team identifies the need for codebase health, this may not be communicated back to the product owner in a way that makes business sense ("but we can still get this feature out the door quickly and fix the mess later, right?" - yes, once or twice, but the downside is a slowing of velocity)
- Even if the team started well with ground rules the Scrum Master is supposed to enforce surrounding technical quality, the Scrum Master may not be effective in enforcing the process - either because they lack authority or because they lack the ability or knowledge to do so
Among several others.
In your situation, the big red flag is that the team predicted the issue many months prior. The process has failed to extract that information from the team members and deliver a consensus between them and the product owner.
Though ultimately, as with any project, if the product owner has more organisational authority than the development team and chooses to exert this authority, no process will be able to save you.
The tricky bit is having a solution and implementing it.
Having something well thought out is hard work. Is hard work rewarded? Is initiative to even try rewarded? I'd say the answer is no.
The answer becomes yes in small teams that are given time, freedom and resources, or in times of crisis (same thing as small team really, minus the time bit), as far as I can tell. Otherwise, most people's natural tendency (including me) is to phone it in. We'd be wise to let people in software development and many other professions that are not manual labor, to switch careers after 30-35 if they haven't done anything worthwhile by then.
Working with people who have to phone it in for the next 30 years and know it is modern day hell for anyone with an iota of ability.
It's permanent compromise and insincerity, or being an outcast that's temporarily tolerated for doing 5-10x the work of the person next to you.
Because Tech debt and I reading messy code base never happened in waterfall...
The problem is always : features and customer request over bug fixing or design fixes.
The system, project type does not matter. That's why some PO recommend 10 percent allocation to devops / ops, 10 percent to improving code base . And "no bug survives the Sprint" philosophy. And what is left,is you next feature or customer request.
For every example of scrum operating badly there are examples of scrum working well. The common denominator in all of those situations - good and bad - is the project and middle/upper management.
In business process analysis, when a process is dependent on individual people, you don't have a mature process but an ad hoc process.
In other words, Scrum methodology in itself is not mature as a methodology and depends on individual fiat, just like any project that doesn't have any process at all.
The only tangible benefit to Scrum might be paying lip service to development departments while firmly retaining the status quo.
Scrum is no more or less dependant on individual people as any other methodology.
The rest of your post is just warping your initial conjecture to arrive at conclusions you'd already decided upon. I neither agree with those conclusions nor the chain of twisted logic you used to arrive that.
I'm not going to stand on a soapbox and sing for the glory of scrums. People - like yourself it seems - can get very tribal when talking about such things. Which is as weird to read as it is pointless for you to argue. In my experience leading different teams using different methodologies, the thing that really makes the big noticeable impact is people and not methodologies. If you work in a blame culture or have colleagues in your team who don't follow process - then whatever process you put in place will be undermined at every opportunity regardless of it's methodology. However if you work in an environment where people respect one another and want to collaborate in getting work done, then you pick a methodology that works best for the day to day work (eg project work or support operations) and for the way people like their work organised. All the rest of the arguments are superfluous.
> good and bad - is the project and middle/upper management.
Yep. People do not realize that the whole company has to adopt agile/scrum for it to work. It's a shift that many companies can not or will not make. They are tied to fixed deadline, fixed scope for various reasons.
I mostly agree with you but I wouldn’t go so far as to say the whole company has to adopt it. However you do certainly need the support from the wider company. I’ve seen scrum work really well despite waterfall being the predominant methodology in that business - there people were happy to embrace the preferred working styles of whichever team they had to liaise with. I’ve also worked in places where scrum failed badly despite the CEO pushing for it. Those places usually suffered from a blame culture that was also the CEOs doing and that blame culture meant that everyone was spending their time working against the best interest of any of the other teams. By “teams” I mean project managers, sales guys, support or operations teams (if they differ from your developers and/or DevOps), finance staff, directors/upper management and even your own paying clients (if you’re a hired service) etc.
I’ve noticed comments for and against scrum are often so focused on the technical aspects of the methodology that they overlook the human aspect. Which matters more in my opinion.
Scrum is iterated waterfall. By iterating faster, inaccurate estimation is shown up sooner. On the other hand, developers are treated like cogs in a feature factory, munching through backlog items fed to them by product managers.
I think it works well enough, for a few years. I don't think it's sustainable - the blinkers of "sprints" encourage growth of tech debt because nobody has an eye on the future and Product won't prioritize refactorings, and sprints are too short for devs to sneak refactoring into the schedule.
No, it isn't. Iterated waterfall is at least as old as the first paper discussing waterfall, but while scrum mandates interations, it doesn't mandate much about how work is done in the iterations, and specifically does not mandate the process steps associated with waterfall; further, it emphatically rejects the role separations and handoffs associated with waterfall during the iterations.
> munching through backlog items fed to them by product managers.
That's...not actually Scrum, as it implies that either the role of Product Owner is taken by a PM outside of the Scrum Team or that that the Scrum Team is not self-organizing, either of which is a significant (even if common) deviation from Scrum.
> the blinkers of "sprints" encourage growth of tech debt because nobody has an eye on the future
Tech debt should manifest in reduced velocity which should be noticed, taken as a signal of a process defect, and addressed in the Scrub Team’s various inspection and process adjustment points.
OTOH, if the Scrum Team is properly cross-functional and self-organizing instead of having a non-team-member imposed as Product Owner, then including appropriate restarting as components of completion of relevant backlog items shouldn't be a problem.
You can scream "That's not actually Scrum" untill the day you die. It doesn't change the fact that that's how it is perceived and done in almost every place that says they do Scrum.
What percentage of Scrum teams do you believe are "properly cross-functional and self-organizing"? And could you point me to examples of people losing their Scrum certifications for not living up to that standard?
> What percentage of Scrum teams do you believe are "properly cross-functional and self-organizing"?
About the same percentage as that of “Agile” software development shops that put people and interactions above processes and tools.
OTOH, at any place that is considering implementing either, there are decision makers who can influence (or in the case of Scrum more than Agile, authoritatively direct) whether or not that's the case, so for them, at least, it's worth distinguishing between problems with Scrum as prescribed and problems which often occur because decision-makers decided to ignore key parts of Scrum-as-prescribed.
Most nominally "Agile" shops are in effect doing Scrum, so I'm not sure that answer helps your case. Either way, it sounds like we agree the "good Scrum" shop is at best rare.
Given that, I think it's worth considering that the problem is Scrum. Especially given that Scrum is not just a process, but an organization and an army of "certified" people that sell services.
When something generally doesn't work for its stated purpose but keeps making money, I think it's worth asking what its real purpose is. E.g., things like crystals and psychics. As Eric Hoffer wrote, “Every great cause begins as a movement, becomes a business, and eventually degenerates into a racket.”
> Most nominally "Agile" shops are in effect doing Scrum
No, most nominally Agile shops are also nominally doing Scrum, but actually doing not-Scrum in a not-Agile context, largely due to sabotage of both core principles of Agile and foundational elements of Scrum by management.
It doesn't matter what name and superficial ritual you put on the process if it's all window dressing over top-down disempowering command-and-control by persons who are neither doing the work nor experts on the work.
Which isn't to say that the Agile and Scrum bodies of work aren't part of the problem: neither really addresses as a key point how the team effectuates ownership of process and how interaction with management works, which means those gaps get filled in (or rendered moot, in the first case) in ways which compromise what those bodies of work do prescribe because.
Lean, which comes from basically the same perspective (while they don't cite exactly the same values as expressed in the Agile Manifesto, being Lean essentially implies being Agile and vice versa) and is a good body of knowledge to draw from alongside Agile, is better in this regard, and so places nominally drawing on Lean seen to be more likely to be doing what they say, because Lean doesn't leave as much of the core vital parts without good guidance.
Which is a really impressive con. Generally the point of certification is to make clear you're getting the official version. E.g., doctors and lawyers self-police because they know it's harmful to have quacks running loose. But somehow Scrum has been able to keep making money despite not bothering with that.
This. I've run programs that purport to be agile but the work input stream was waterfall and disconnected from the means of output so it was really timeboxed waterfall, unfortunately my management did not agree to re label. You know it is real Scrum if it is painful for the business team and they come to realise that now it's software all the way down.
> Tech debt should manifest in reduced velocity which should be noticed, taken as a signal of a process defect, and addressed in the Scrub Team’s various inspection and process adjustment points.
This process adjustment usually means "sorry about your vacation" and/or "you aren't doing enough overtime".
Yes, if external actors rather than the self-organizing teams own the process, that is probably the first response, but that won't actually stop the buildup of tech debt and the velocity impacts, though it may produce a one-time, non-repeatable improvement, so even in that case if someone (even the wrong actor) is monitoring velocity and doing process improvement, there will be an impetus for further change.
(Of course, if the wrong actor is monitoring velocity and controlling process, then the team has an incentive to mask the effect of tech debt but continuously adjusting estimates to maintain the illusion of constant velocity, a d avoid the first bad response, or, if that opportunity is missed, to avoid the subsequent external interventions.
"Fortunately," hardly anyone in Silicon Valley plans to be working on the same codebase in a few years. There's a good chance the problem space won't be relevant anymore by then, and on the off chance it's still funded, the new team will rewrite it whether or not it's good.
>sprints are too short for devs to sneak refactoring into the schedule.
As I've gotten more senior, I've just gotten more brazen about doing this less sneakily.
Is the Silicon Valley representative of software development though ?
Because on the other hand we have professors telling us that the average (surviving?) software lifetime is 20 years...
This is just intuition but I suspect that the lifetime for software is u shaped. Much of it is very short lived but software that lasts more than 1-2 years is very likely to live for a decade or more.
A lot of 6 month old code gets thrown away either because its been rewritten or because it didn't achieve its stated objective. Meanwhile a bunch of companies are relying on systems that were first created in the 90s because that software achieves its objectives and the cost justification for a rewrite isn't there.
There are many 20 years and older code bases currently. But if you start a new code base today it might get rewritten many times in the next few years because software users no longer have the power to stay on an old version due to the SaaS paradigm.
The biggest problem with Scrum is it lacks any sort of design phase. You do the minimum. Oh, it doesn't work quite right? We'll fix it in the next sprint...
The "spiral" model is closer to a true iterated waterfall. I've seen it used successfully in more mature companies.
It's like "build one to throw away", in that the goal is to rapidly explore the feature space and discover useful features, but what actually happens is that it's a trap and you won't throw away something that works no matter how poorly implemented.
absolutely. If you ever see a group afraid to produce a prototype, it's because they're afraid management will try to ship it as a final product.
The effective strategy is to build an oblique, fatally limited prototype, which can not possibly be mistaken for a shippable product. The bad prototype can only be used to test the hardest parts of an idea and must have major holes in it with no way of filling them. It should also have a largely fixed timeline to ensure it is put to bed before it gets "hamstered" into the shipping product.
absolutely. If you ever see a group afraid to produce a prototype, it's because they're afraid management will try to ship it as a final product.
I once had a manager who, on the subject of upper management wanting to ship the prototype, would say “the difference between dev and prod is one letter.”
You're confusing "investing in design" with "having a design phase". I agree many Agile teams underinvest in design. But so do many non-Agile teams. The problem isn't the lack of a formal phase. The problem is not taking it seriously.
I'm certainly not defending Scrum. But I think one of Scrum's big problems is relying on proximate measures rather than actual results. It values output much more than outcome. So I think the way to get good design isn't to try to measure design. It's to release early and often, making sure that products fulfill their purpose.
I think our whole industry is full of proxies for the things we actually care about, which is for instance why we rant so often about the hiring process.
For another example, look at what we talk about with Clean Code or SOLID principles. We have that old joke about the two hardest things are naming things, cache invalidation and off by one errors. I think the fourth hardest one is resolving merge conflicts.
You work carefully on code for weeks or months and then in one fell swoop it all gets broken when you aren't looking at it anymore. It's really bad for morale.
If you read through all of these Best Practices with the notion that merge conflicts are dangerous, then you start to see them as designed to keep conflicts from happening. Putting related code together isn't just about reading comprehension. It's also about keeping unrelated code apart. You and I can work on unrelated stories without ever doing a 3 way merge.
And if you look at merges as the problem, you start doing things like alphabetizing data structures that don't have a strong cohesiveness to them because then when you and I add a feature to the same data structure, we aren't both modifying the bottom of the file. I put something under T and you put it under G.
To put this another way, if in some perfect future someone invented a merge tool that never screws up, so that we only get merge conflicts when two people are accidentally working on the same feature/bug, we might start to question all of these practices as overwrought.
Experience. I've been developing software for 20+ years. In practice, due to the nature of agile / scrum, design is very minimal or thrown out the window entirely. It encourages short term thinking. Rarely is anything beyond the "current sprint" considered.
I was explicitly told not to write unit tests because they took too much time, which required me to spend entire days retesting almost 100 scenarios when the business logic changed.
Of course the business didn't know all the scenarios at the beginning of the feature development and didn't care because: iterative development means we'll figure it out later.
A lot of devs trap themselves by insisting they can write the tests after. Once managers know the code exists they want to use it, or move on to the next thing people are breathing down their necks for. And code written without tests is difficult to test so becomes a self fulfilling prophecy.
In fact it’s such a reliable mechanism for self sabotage that I look at carefully at people who bring this on themselves and try to figure out if it’s naïveté, learned helplessness, or malice.
As you've discovered, good automated testing is mandatory for iterative development. If execs tell you to do something unprofessional, it's ok to say no. Indeed, if we're going to think of ourselves as professionals, I think it's mandatory.
But one developer quitting rather than do bad work does change something. At the very least, it means that the developer can find another, better job. But it's also an opportunity for companies to learn. And for those who don't, good people refusing to prop up bad companies is a step forward.
I'd also add there's a a lot that can happen between refusing to do bad work and quitting/getting fired. Saying no is the beginning of a negotiation. A negotiation that won't happen if we just say yes all the time.
Then that's not an Agile process. That sounds more like mini-waterfall.
It's perfectly possible to deliver small units of work weekly. One still gets frameworks while doing that. It's just that the work of making the framework happens over time, not up front. Which is good, because good designs are based on good information, and the beginning of the project is when you have the least information.
I love these commitments. When my Scrum leaders asks me why we didn’t finish what we ‘committed’ to doing these two weeks I want to strangle him (nothing personal, I do otherwise like the guy :P).
The terminology is so incredibly developer hostile.
I agree. The last "waterfall" company I worked for was Novell. In my opinion, all the Novells of the world glommed onto Scrum because it was shiny and new and allowed them to call all their bloated and unnecessary project managers "Scrum Masters" and all their wasteful status meetings "Stand-ups." It was just the same old garbage with a different name, but hey, We're Scrumming!
Waterfall with little or no modelling. If you write the wrong system, you are left with nothing. With waterfall you could at least start over with your design artefacts.
Scrum wins out at companies that did not exist in the waterfall era, staffed by developers and engineering managers who have only ever worked in startups. Similarity to existing process may explain a large part of the adoption in places that had existing process, but a lot of Scrum shops didn't.
You're asked to deliver a feature that would take 4 weeks to implement without tech debt in two weeks. This requires sacrifices like ensuring network connections remain alive, handling exceptions, optimizing queries, abstracting functions for re-use, handling all the data input scenarios, etc.
This creates a confusing code-base with lots of IF statements, copy/pasted code, data modifications from several areas of the system, etc.
You'll fix a problem with data getting into the database incorrectly and realize a couple sprints later, fixing that bug caused other bugs elsewhere.
Automated testing doesn't catch it because automated testing only handled certain scenarios.
Because it's a sprint. You've got a finish line and you're racing towards it. Businesses don't function without prediction, and refactoring gets cut before roadmapped features. If feature development is a bottleneck in company growth, debt will grow, quickly.
Agreed. I regularly ask "Why are we running a marathon in a thousand sprints?".
Besides tech debt, a concern I have that I don't see brought up is burn out. With Scrum, every action you perform is micromanaged and with a push for "high velocity". There is no proverbial breathing room in this where the pressure lets up. At least with waterfall (for how we did it before Scrum), the windows of high pressure times were shorter. During the beginning of our 6 month waterfall, in parallel to spec work we'd be taking care of tech debt or implementing our pet feature and it was a time of mental recovery.
imho a problem with the implementation of SCRUM, but not necessarily restricted to SCRUM.
In a perfect SCRUM world you could not be forced to implement this in a time not of your own estimate. A feature that is so big that it takes more than a few days to implement has to be split...but well we all know how this plays our in reality most of the times...
My experience shows that proper testing and documentation is the first thing that management wants taken out of the story, often with the excuse "We can handle that in a later sprint." But since your life is a neverending series of sprints (note: that's actually an ultramarathon), and management gets to pick priorities, you may never return to the technical debt.
I have not worked in a scrum environment in a couple of years, but when we did it those items were not included in the story. They were part of a definition of done that the team did not really talk about publicly.
Refactoring stories were also rare, it was just sort implicit in the task, unspoken as part of doing it.
I'd imagine that is why they did not get cut.
But yes, management picked feature priorities, but left the implementation up to the team. So at any given time were were working on the "most important" thing from a business perspective.
The most successful (but still crappy) solution to this that I have seen is developers undertaking guerilla tech debt work. This can be done by bundling maintenance work into an existing task where the two are actually not that related or by simply carving out bits of time between official tasks.
While it "works" it is certainly not ideal that people have to go off the reservation to ensure that the project doesn't implode in the future due to the accumulation of buggy code, performance problems and likely security vulnerabilities.
> proper testing and documentation is the first thing that management wants taken out of the story
That's not breaking down a story into smaller stories though, that's simply not doing some of the tasks necessary for a story to be complete. "Story" is not a synonym for "task"; a story is user-centric not about internals. "Write the documentation for this feature" is not a story and "write the tests for this feature" is not a story, they should be part of your definition of "done".
You're not doing Scrum then. One of the few meetings that Scrum dictates is "backlog refinement" where the dev team works with the PO to get the stories into a workable state.
If you can't touch the stories, you can ask whoever does to break it down. If they don't then you shouldn't ever pick it up into your sprint. If they insist and it doesn't work out then you should call out the incident in your retrospective and hopefully get everyone to agree to do things differently in the future.
Sadly, it’s more like the dev team points out that we really cannot work with these stories then management telling us to deal with it while the BA’s suck their teeth.
Scrum won out IMO because it's an awesome word and conjures up subliminal images of brute force pushing obstacles out of the way, bringing success to the team. Execs can relate to it without even knowing what it is.
I actually think scrum won because of its multiple rituals that allowed most of middle managers and project managers to keep their job by recycling it.
When I hear the word scrum I am always reminded that my school stopped playing Rugby because a kid in the next town got his back broken, in a scrum. So from then on we played football (soccer) in Rugby kit, until eventually all the kids had grown out of it, or left. Similarly, if you sprint and sprint and sprint you will collapse from exhaustion.
I often wonder if they picked these words because they knew nothing at all about sports, of if there is a coded warning in them...
Scrum (and other methods that try in vain to make developers replaceable cogs) won because it enabled bad developers to hide among the good ones, and avoid personal responsibility.
I think it won out because the number one thing any kind of project management framework has to provide is ambiguous blame-avoiding fungibility to management. If a framework does not offer that, managers will politically subvert the policy-making process to rule out that framework.
It’s the same reason why demonstrably value-additive, cost-effective prediction markets are rejected by managers despite evidence in their favor: it removes their ability to create ambiguous Dutch books out of different project deliverables and resort to exploiting subjective psychological biases of superiors to win money & status increases. Anything that holds them accountable to the actual measured result status of a deliverable can’t be tolerated.
Scrum is just the by-product of corporate status-seeking evolutionary war games against in-house scientific thinking.
Agreed, it feeds the existing power structures. And it did hit at the right time when over-communication is not only appreciated but everything else is seen as intransparent and potentially suspicious.
"You don't even know what your team is doing?" - is the best way for middle-management to pick on each other. So in order to never get into that situation daily standups, slack, jira, basecamp all these "transparency tools" are welcome as they offer defense for this manager.
Does anyone know if there is an anti-movement to that over-communication trend? Something along the lines: "The only way to go fast is to go well",
* Plan
* Honor the flow
* Adjust
yes, I too was involved with XP before it was under umbrella of Agile which got consumed by the Scrum borg.
I feel we kind of lost something where creating code/designs using strong feedback loops was a "first class citizen with royal honors" of the software development process. Scrum kind of makes it a sideline issue, a peon of middle management
My dad started developing in the late 1960s. He did a lot of in-house software, mainly manufacturing and insurance systems. His approach was to go out and see something in use, talk to the people using it, go back to his office, write something, release it, and go talk to the people again. He'd do that loop in as little as a couple of days.
Today we'd call that obviously agile. But both back then and now, higher-ups wanted more "control", by which they meant feelings and appearance of control. Which caused longer feedback loops. Which made things more out of control, which required more process and even longer feedback loops.
I think we have the same struggle today, and I have no idea how to get out of it.
Agile has a problem with introducing Processes and Tools (by definition, as the blog post describes). However, Scrum offers a solution to this problem by creating an Agile space or bubble within those organizations that are still stuck with the traditional methodologies (Waterfall, etc.). So Scrum is a nice way to introduce organizations to Agile concepts but it is certainly not the final goal.
IMHO, the idea is that over time the organization will adopt new Agile practices and not depend on Scrum anymore. In practice, however, many organizations think they are Agile after they introduced Scrum, which is a simple fallacy. After all, becoming Agile is about a culture change and not about the introduction of a new process framework.
I don't find much constructive value in your comment. To me it seems like a politely worded screed about the incompetence of management and organizations.
To drive the point home I could easily transform "[o]n average, the first priority of managers and execs is maintaining the power structures that make them a big deal" into a derogatory comment about engineers, fad chasing, and resume padding by a simple word substitution. It would be neither accurate nor fair to many (perhaps most) engineers: just as your comment isn't really accurate or fair to managers or organizations.
I have been both a manager and an executive. As well as a consultant in many companies both small and large. I'm not saying that those people are necessarily incompetent. I'm saying that, per POSIWID, the purpose of most software organizations is not effectiveness in making software (or in serving users via software).
That's not constructive in the sense of providing ways to solve the problem. But that wasn't my goal. My goal was to support and confirm the article's thesis. I have also written plenty of constructive things on software development, but I think it's absurd to suggest that every HN comment must contain that kind of constructive feedback. (And if that were a reasonable standard, you aren't living up to it.)
> On average, the first priority of managers and execs is maintaining the power structures that make them a big deal. But true Agile processes are about empowering teams to self-organize around serving users.
But that doesn't explain the success of Scrum, because Scrum does away with the power structure of managers, and should empower the teams. "We're doing Scrum now" is the big stick I see scrum masters use to beat back Business managers to keep them from interfering with and micromanaging the team.
At least, that's what they should be doing when management interferes too much. And then address this higher up with whoever decided that we're doing scrum now, and explain to them what that means, if necessary.
There's no excuse to tolerate micromanaging managers in a Scrum process. They don't belong there. Kick them out.
At the company I work at, we have the following scrum anti-patterns. I wish I knew, whether we could "do scrum right" or just move onto something simpler (fta; priority queue)
* Daily standup, nobody wants to be at. We have multiple teams arrive, with roughly 20 people in a small room. Some people stand, some people sit. Sometimes the front-end team goes, sometimes the back-end team goes. Its limited to 15 minutes, so nobody says much of importance, and just parrots what is already on the Jira board.
* Scrum master driving process above scrum. The scrum master's role is to make sure scrum rules are adhered to. But at this company, any deviation from scrum itself poses a threat to this person's job security, so it doesn't happen. More high-level processes are not optimized because the scrum master defines everything.
* Technical debt. I see this time and time again. User Stories are supposed to be forecasts, not commitments. But the business doesn't like stories carried over, so they become commitments. At the end of each sprint, everyone rushes to get their stuff done, and hacks are implemented to meet an arbitrary deadline. Many times I want to begin my work by refactoring something to what it needs to be first, then do the actual user story. But its risky because the refactoring might take more than the allocated story points, and you get dinged. So I do the story first, and if there is time do the refactoring but it almost never happens.
* Poor product owners. During maintenance phase, we could work on cleaning up technical debt, but this value is not appreciated by business so they keep us busy with bikeshedding. One gripe I have about scrum, is there is nobody representing engineering, as the product owner represents the business.
These things combined have dragged down the happiness of the people I work with, but we all feel imprisoned by it. I have a stack of scrum books here I plan on reading, I figure this process isn't going away and I need to up my game with how to play it - but I wish I could use something else, perhaps kanban.
The way I decided to play the game was like this, "I won't be attending the Daily Standups anymore as I don't think they add value and do subtract value."
The project mgmt response was, "Attendance at standups is mandatory."
Regardless, I didn't go to anymore standups and when I got flack for that, I stopped going to the office all together. When I got flack for that, I stopped working all together.
Then I got fired. That solved all my scrum problems.
I got criticized for zoning out at scrum meetings. So I watched what the managers do. They show up for the first few minutes, look alert, then leave as if in a hurry. I started doing the same thing, and never had a problem again.
Because what do they all actually do? They spend a large portion of their time waiting around for numbers they add[0] to a spreadsheet and report those numbers to their manager.
I feel like you work at my last company. But so many companies f* this up that it could be any company.
I got fired from any last job after being one of the only engineers willing to say something to management about this. I figured I had a popular opinion - many other engineers often DM'ed me on slack encouraging me to continue speak out, including my own manager, so I somehow figured I should be safe speaking out, politely, as a respected majority representative.
In fact I was dead wrong. By saying that scrum was a problem, I ended up making product managers / CTO who loved scrum and used it as an hour long opportunity to lecture the team every day feel threatened.
Getting fired was one of the best things that ever happened to my career, because in retrospect it was a dead end company that was run by fear. Management was afraid of ideas or challenge to power to the point where innovative ideas and feedback were never well received and a culture of fear and not stepping out of line arose, even though it was sugar coated with fake company values of "openness" that nobody really believed in but that managers loved as a way to elevate themselves.
I suspect many companies are a my least a little bit like this. Unfortunately it is human nature and there are so many stories in history, usually of narcissistic dictators, that mirror this.
This was a very extreme example but I'm sure many other companies struggle with this when it comes to challenging scrum. Managers tend to love scrum because it's a chance, daily, to "manage" and to get status updates so that they feel comforted. It makes it so that trust is no longer necessary, and bad managers are often bad at trust.
I guess another way to view scrum is to realize that it is often a reflection on management's subtle fears and insecurities being projected onto the processes of the company. It is an instance where employees unfortunately must manage up.
> Daily standup, nobody wants to be at. We have multiple teams arrive, with roughly 20 people in a small room
It is central to the idea of the daily standup is that it is one team.
> Its limited to 15 minutes, so nobody says much of importance, and just parrots what is already on the Jira board.
If you have another mechanism for sharing what each person has done and is doing, then the standup should just be for sharing information about and resolving barriers.
> Stories are supposed to be forecasts, not commitments. But the business doesn't like stories carried over, so they become commitments.
If you organizationally can't get away from commitments, then you need to cut back the amount you plan to do so that there is some slack. As long as you are also doing backlog grooming and have items groomed beyond the current sprint commitments, you can take additional items opportunistically, if there is excess capacity after doing the committed items properly.
> As long as you are also doing backlog grooming and have items groomed beyond the current sprint commitments, you can take additional items opportunistically, if there is excess capacity after doing the committed items properly.
The powers that be at my work decided that “sprint predictability” is the most important metric here. This means that if we pull in stuff near the end of the sprint but don’t finish, predictability goes down. Thus, we are encouraged to not take on extra work if we don’t expect to finish. What this means in reality is that we start working on it without pulling it into the sprint and then get a head start for next sprint.
My (related) experience has led me to believe reporting to management should be strictly separate from any tools or processes used to track actual work, specifically to avoid dumb shit like this. Team task-tracking should be a team communication tool. If you want to report task-related stuff upstream it should be totally separate, with updates/sync done by a human (project manager would make sense).
I also think speculative future make-believe planning junk should be kept out of the real getting-shit-done tool. Put it in there when it actually matters. There should be no "on ice" or "phase 3" (when in "phase 1") crap in the tool the devs use. It may or may not make sense to have them involved in planning stuff that early, but keep the output of that planning away from the real-work trackers until it's closer to time to do it.
I think that's part of why Jira (and similar, heavy, report-focused tools) is so god-awful. It not only tries to have features for teams to get stuff done and for five other roles to dig around in tasks and reports for whatever reason, it enables and encourages that, and I'm fairly sure, at this point, that it's fundamentally a bad idea.
> My (related) experience has led me to believe reporting to management should be strictly separate from any tools or processes used to track actual work,
This probably increases the accuracy of the work trackers but increases the B.S. level of management reporting in organizations where there are fundamental problems between management and the working level.(Which is the only reason anyone would want to separate these two things.)
While this probably seems like an improvement from the working level, it's probably bad for the organization. The solution is not dis-integrating communication tools so that management's view is not connected to the actual work tracking, but dealing with the fundamental trust issues. Which is hard, of course, but things that are important often are.
I get why they want it, but I think it's a case of false improvements from adding more computers. If the reports to management are such BS that they don't usefully resemble reality they should be able to figure that out before long and sort things out, one way or another. If they want insight into the tools the bottom-of-the-ladder workers are using to coordinate then that coordination will suffer, greatly, and those tools themselves will be full of BS, for sure, gaining little aside from some pretty "real-time" BS graphs for management and worse communication for workers.
[EDIT] more to the point, I think if it worked we wouldn't still constantly see management surprised when things aren't delivered "on time", and yet, that still happens all the time. IMO the team needs someone "on their side" to report reality upwards, diplomatically, not their own work tools reporting up directly, or they'll lie to their tools, which leads to more surprises, not fewer.
There's certainly environments where that makes a lot of sense, particularly where the software at issue had immediate effects on external users who require advance notice of changes (I actually work in that kind of environment, and though we do have opportunistic items they generally are restricted to items that don't impact external users.)
What answer are you expecting? Your example indeed makes no sense, but it has nothing do with scrum and has everything to do with clueless people do stupid things. Do you expect this guy will suddenly do only sane logical things when using something other than scrum? I expect not.
If you're trying to do scrum properly, you shouldn't be consistently finishing work early. That implies you're overestimating stories.
When you do finish a bit early I've not normally found it a problem to find some small bits of tech debt, research, or admin work to fit in before the end of the sprint.
Starting work on stuff from the next sprint is not ideal because it'll throw your estimates off for the next sprint, plus you don't actually know for sure what is going into the next sprint.
I passionately hate being asked for "commitments". If it's stuff of any reasonable complexity or novelty I will have no idea how long it will take and therefore can't make any commitments. The only thing I can commit to is to make sure that people don't waste time and work towards the goal. The problem is that management has no problem wasting a lot of time with useless meetings or not committing to the final feature set but instead changing requirements all the time.
Even better is being told to commit by the Scrum Jesus (who does not even try to understand the business domain or engineering issues, since he's getting hired for being the Scrum Jesus) to stories with no one available for even defining or discussing them. Our story might have been just a word someone saw at a trade fair.
Then, two weeks later, being asked why I did not complete what I committed to.
In that situation I've gotten a lot of mileage (and quiet amusement) from pointing out that Scrum dropped the term "commitment" for "forecast" in 2011. Scrum Jesus has no defense against Scrum itself!
"Development Teams do not commit to completing the work planned during a Sprint Planning Meeting. The Development Team creates a forecast of work it believes will be done, but that forecast will change as more becomes known throughout the Sprint."
If you can say more than a minute, but less than a decade, you've already got SOME idea of the timeframe for a task. Granted that sort of estimate would get you called to HR for insubordination. But can you pull the bounds in from either direction at all?
I say this as someone just moving into project planning and management. From that perspective you start to see that some level of estimation s critical. My preference is to ask devs for just the tightest bounds to timeframe they're actually comfortable with, then estimate based on that and pass up the chain. I consider it to be on my shoulders if things take longer than I communicated upwards.
I understand that estimates are important and I give estimates. But recently in my company it’s fashionable to talk about “commitments” and I won’t give those. In my view this is just a way to get overtime and weekend work out of people once things fall behind so they meet their commitments. I commit to do my work as best as i can but I can’t commit to work overtime because of issues that are often out of my control.
One manager I’ve worked with had the brilliant notion of “x2+1” time of what the dev says.
Anecdotally this has worked out remarkably well throughout my career - from single dev to cto - even stuff that I _new_ was going to take for example 2 days, if done properly ended up in like 5.
Didn’t matter if I was doing the estimate or someone else. At some point I just gave up and started doing the “my gut says 3 hours, so it must be 7 hours of work then, and send that up the chain. That cheeky “+1” had saved my ass more times than I can count.
One of the points of Scrum is that you don't estimate hours at all. You estimate intentionally vague and abstract "story points". You don't need to know what that is in hours, just try to make sure that two stories of roughly equal complexity have roughly equal points. Run a few sprints, check what velocity you actually end up on, and you've got the averages you need to give a slightly more long term planning.
But without the feedback of actually having done the work, any estimate is bound to be wrong.
I also generally have had good success with guessing a number and then multiplying it by 5 to get the time when it’s completely done, tested and documented.
> One gripe I have about scrum, is there is nobody representing engineering, as the product owner represents the business.
That is definitively not be the feeling you should have. In "proper" scrum the Team is supposed to represent engineering. If you, for some reason, don't feel like you can represent these concerns then that is a huge issue. The PO shouldn't be your boss either. At least it isn't so in the company I work for. It sounds like the PO is the boss of that team, if he "keeps you busy". I am super grateful that our Scrum Master knew what he was doing and suggested creating a separate line on the Org chart for the POs so that they are explicitly not in charge of development. The team is in charge and POs are more advisory.
Sounds to me like there are some deeper problems in the company then just the scrum. There are so many red flags in your comment. I don't know if those books are going to help you "play" scrum better, because what you guys are doing does not sound like scrum at all.
Kanbans probably also not a solution. I really like Kanban, but it isn't going to remove the bad elements that seem to control that company.
Yeah, that's an odd complaint. You are engineering, are you not? The meeting is already about engineering. If you have more engineering as an outside stakeholder that the PO isn't able to represent, by all means bring someone in to represent them.
In scrum the dev teams maintains a technical backlog and fills the sprintbacklog as THEY see fit- from the product backlog AND the technical backlog.
the PO has no say in this! he can complain that he thinks not enough features from his product backlog are planned, but should trust that the team knows what they are doing.
First of all, most issues with process issues are more a reflection of the organization and what drives them than the process. I would say you should start talking about, 'Our process' instead of 'Scrum'.
>These things combined have dragged down the happiness of the people I work with, but we all feel imprisoned by it. I have a stack of scrum books here I plan on reading, I figure this process isn't going away and I need to up my game with how to play it - but I wish I could use something else, perhaps kanban.
Spoiler, your Scrum books aren't going to help you figure things out, unless your org actually wants to change. You have your own unique process, but that you have a unique process isn't an issue. The issue is that your management seems to not respect the input of engineering. No process will work the way you want as long as that is true.
Some thoughts,
* Most critical, your team is way to big. Should be half as big
* Scrum master should be a team member and rotated along the team. The process should be a result of constant tuning via the Retrospective. The scrum master here, should lose his job.
* Engnineering owns the implementation of the stories, product owners should not be a part of the conversation on how to do something, they only can say 'what'.
* If you can't decide your own points, then again, you have your own process. If you can, and you need to refactor, you need to add points for that.
If you can't do anything to change, then again reading scrum books isn't going to help. You can say, 'this isn't the way your supposed to do this' as much as you want. processes always reflect the organization. Unless the engineering team is an equal to the product team, it won't matter what process you have.
> Scrum master should be a team member and rotated along the team.
I don't agree. For us the scrum master is a fixed position who has as a responsibility to advise the teams on how to fix problems with their processes or to help build these processes in the first place. This works very well for us, because the scrum master is explicitly only an adviser.
Meaning I, as a team lead, do not have to do what the scrum master says, but I always have someone experienced with scrum who I can ask when problems arise or when the team wants to change the processes.
However, many companies face similar issues. For a moment I thought the OP worked for my company because we face the same issues (that's not possible however because in person stand ups are banned at our company due to the distributed nature).
1) While I agree that 20 people seems like too large a team in general, I have a fundamental problem with the idea that scrum can determine what size team is right.
What seems to be happening, it appears to me, is that we have a fair idea of how large a meeting can be before it becomes unwieldy, and scrum has a daily meeting as it's fundamental component, so basically decides that for it's fundamental component to be effective there shouldn't be more than 8 people so therefore teams shouldn't be larger than 8 people.
That seems backwards to me.
2) Almost every scrum training usually has a separate scrum master. I do think the idea of a team member being the scrum master makes a lot of sense and would probably alleviate a lot of the concerns.
3) Scrum story points are supposed to reflect end user benefits. A refacorin's benefit will only show up after several sprints. In the meanwhile you hurt your velocity significantly because your refactoring is likely to be 0 points. Now, this actually makes sense to me, but the issue is that in most places management is carefully monitoring velocity and is gonna hold the drop against the team.
To be fair though, I think this is another issue where the real problem is management converting a measurement metric into a target, which basically negates any use it may have as a metric.
My biggest issues with scrum are:
1) A daily standup is ridiculous. It's highly disruptive and gives me the feeling of someone constantly peeking over my shoulder. A better strategy is a simple email when someone has an update, or a blocker to the rest of the team. Very few of my projects are done in a single day, and it's psychologically stressful to join every meeting saying I worked on story A, will continue working on story A for 3-4 days in a row, while the managerial types rattle off a list of meetings they attended, emails they sent, trainings they completed, etc.
2) Sprints are far too inflexible. Each story should basically define a new Sprint for the people involved in that story, which may or may not be longer than originally expected. I don't see the point of having a fixed 2 week period that applies to everyone. Why not just have a weekly or biweekly meeting where you go over everyone's stuff and see where you stand. It's basically a more flexible version of what sprints try to achieve, without the psychological issues created by having to fit things into a Sprint or alternatively having to carry it over into a whole new Sprint. That same meeting can track how many points were earned since the last meeting and you have your velocity being tracked as well.
Retrospectives are great. But they should be included in the same 2 week catch up meeting.
I find scrum tries to create a lot of artificial deadlines, possibly to encourage people to break things into smaller pieces and make regular progress, but I find that's not how people usually work. In my experience people tend to work more in spurts, delivering a ton of features and bug fixes over a couple of days, and then going relatively quiet for the next few days. Not because they aren't working, but because thats just how things usually pan out since software development is a decidedly non linear process.
> Scrum story points are supposed to reflect end user benefits.
That’s not my understanding at all. In our org, that’s the role of backlog priorities. Story points are the cost to get there, and are explicitly a tool to estimate relative amounts of effort involved.
If tech debt is to be incurred, it is expected that engineers negotiate that, and that other estimates will change as a result (and/or other tasks be created to track that debt). But then, nearly 100% of the managers in our org are, themselves, engineers or SMEs in the field they are managing, so they have realistic goals and understanding of the sausage-making process. What you’re describing really does sound more to me like a management problem than a process problem.
Sometimes I find myself so stressed out by the standup, I basically doing nothing the rest of the day. Also I have a habit now to get up early and try to do yesterday's portion of work in an hour or two before standup just to report it. I completely stop thinking in time periods wider than sprint, and I thought before that was my strength.
Worst thing, the process forces an idea that my struggle with it is, basically, my fault.
I hate "scrum".
I think the scrum team size is determined in part because communication overhead is exponential, getting beyond 6-8 people means the whole team has to spend a lot of extra time coordinating. This is true regardless of whether you're dong scrum or not. It's true some teams may be correctly sized at 20, but I would suspect that breaking that into 2-3 other teams would be a benefit most times.
Story Points represent effort and uncertainty, not business/customer value - those are Business Value Points. Devs are only committing to X story points per sprint. If refactoring is needed, it's built into the story points, or added as it's own task. And as others have said if the Product Owners don't take engineering input on paying down tech debt/infrastructure/internal tooling, then you have a broken company no matter what process you're following.
> 3) Scrum story points are supposed to reflect end user benefits.
Scrum.org states[0] A Story Point is a relative unit of measure, decided upon and used by individual Scrum teams, to provide relative estimates of effort for completing requirements.
According to Jeff Sutherland himself[1], story points are based on team effort and not end-user benefit: Estimates are estimates for the team to get a story done.
Eh, the idea that a team should be no larger than the number of people that can meet together is not unreasonable, if also not incontrovertible. What makes a team a team if they can't meet together as a team effectively?
I'd say the most important characteristic is whether the team can work together effectively.
As an example, one could consider the various people working on an individual open source project as part of a team. And most successful O/S projects rarely require everyone to meet together at the same time.
I personally strongly prefer small teams (I find even 8 members too large), but arguably certain projects may need bigger teams, and I think they should still be able to practice Agile with the bigger team, which scrum doesn't allow for.
Thanks everyone for correcting my understanding of story points (which was based on scrum training provided by my current company).
Effort makes a lot more sense, and I particularly like that it tries to incorporate uncertainty as well (I guess that's why they chose story points, to make it vague, since using hours leads to people treating estimates as deadlines instead).
I have never worked at a company where this actually goes right and could not be replaced by [insert ticketing system here]. We use Basecamp to do daily check-ins on a very high priority issue, if we have one, which we usually don't.
Managers need to justify their jobs. Reading off a slack report every day seems too simple and not worthy the salary they're paid. Compare that to a large meeting and live status updates, everyone stands, etc. Now that is some "serious work" happening.
Think of it from the point of view that managers have mangers they report to. When shit goes wrong they have to cover their asses. "So it's late. But did you check on their progress? Wait just over slack, no actual meetings? Ok, that obviously needs improvement".
It's not like it's a requirement that "managers" be at a standup meeting. Some of the best teams I've worked on have done daily standups without managers, and I dare to say most of us enjoyed them and found them productive!
Managers are often looking for ways to fill their calendar. Meetings, calls that could be a two-minute Slack exchange, et c. Some will hop into meetings like that if they can. They love the recurring ones, especially, bonus if they can apologize any time they have to skip it for something that actually, maybe, matters ("uh, yeah, we don't care if you come, OK, thanks for apologizing I guess?")
I have never been in a team where managers were present at the standup meeting. In fact, sometimes I have no idea who my "manager" even is. We've got a team, we've got stakeholders, we do the work. If a manager wants to join, that's fine, but he's not in charge of that meeting.
All this talk of standup meetings being primarily for managers is really weird. It's not for managers, and they have no reason to be present. It's for the team, to improve cooperation within the team. If the team has a better way to accomplish that, then they should use that.
Why are your managers attending stand up? The Daily Scrum is the team's meeting. I swear Scrum is only fragile because no one reads the bloody book, or if they do they skim it or ignore it.
From the scrum guide... "The Daily Scrum is a 15-minute time-boxed event for the Development Team."
It is the team's event, if there are other people there it should be at their request. The guide goes on to say that.. "If others are present, the Scrum Master ensures that they do not disrupt the meeting."
If something is self-organized, there is no need for all this ceremony and bullshit that comes with capital-A Agile or Scrum. You just shoot the shit and talk about what you're doing.
Stand-ups and all the process attendant appears to be universally imposed from on high as a micromanagement technique, or because they were sold the idea as some kind of silver bullet by a talk or a snake-oil consultant.
I think Scrum provides a good baseline of working for new teams, but ultimately the team is free to shape it however it works best for them.
The main value of Scrum as an official process, is that it gives you a stick with which to chase toxic management out of the room. They're not sufficiently Scrum if they crash your meetings and demand to be in charge. Though in my experience, most companies that do Scrum have a management that keeps their distance unless invited.
We do it in person on MWF and slack TTh. I find slack updates to be both more effective and more efficient. It is obviously why it is more efficient; but it’s actually more effective because unfortunately I am one of those many people who simply drift off when in a meeting, and putting updates down in writing helps infinitely.
I like standup on my current team. We're in an open office already, and just (physically!) stand up and say "I'm doing X today". Whole thing never goes over 2-3 minutes, and it's a friendly way to keep everyone up to date with the bigger picture.
Generally if things can get unblocked as you go, that's better, and so this isn't particularly common. But sometimes you need your manager to go kick down a door for you and it's a handy time to let him know. Sometimes it's more benign and it's just easier to ask the group in person when everyone's together sipping their coffee together than it is to fire off a bunch of Slack messages or a group email or whatever.
And to be clear, when I say "blocked" I'm not talking like "I can't get any work done until X", I'm more talking like "this particular avenue needs X to happen before I can continue it, so I'm shelving it for now and doing something else until we can get it resolved." If you find yourself in the former case, something has already gone horribly wrong that should probably have been resolved days ago.
There are different ways something can be blocked. It's possible something is too complex to wrap your head around and you need an extra set of eyes and brains, or it might be blocked by outside impediments. It doesn't really matter, all obstacles need to be addressed and solved somehow.
Exactly. I've always understood Scrum to be moderately successful at getting usable work out of terrible developers. Companies that use it should be avoided. If your employer introduces it it's time to quit.
This is a very important point that's often forgotten. You shouldn't wait for the appropriate meeting, but address it right away. The next stand up meeting is more of a backstop. It's the time to admit: "I guess I really am stuck here. Can anyone else look at this?"
Smoothly running teams don't need this, but many teams don't run that smoothly, and then it's better to address it at the stand up than not at all.
From what I understand, that's the end goal, with daily standup being a way to break the ice and get people who like to silo themselves off to actually ask instead of making no progress.
I think this gets at a very common problem with managers in the hierarchy. They forget that iteration of processes is crucial to good agile process. Your goal isn't to adhere to "true scrum." Your goal is to tailor scrum to what your team needs.
If your daily stand-ups are just repeating the JIRA board, then just give people back their time. If your features are coming out badly designed, then stick a designer in the engineering pods, or call out the design phase in project lifecycle.
If your projects are consistently late, then... slow down the estimates!
Of course teams that do it right tend to know this, but the bad implementations of scrum I've seen consistently lack attentive iteration on the process itself.
Are you in a position to effect change? If not, leave if you can. If not and you can't leave, suck it up.
Otherwise, here's a couple of notes freely given with no claim to excellence except that I've worked with successful scrum teams and unsuccessful ones:
* The 'business' is everything. All code is liability. Code that can't do things the business wants to do and hinders those things is a particular cost to the business. Product Owners who don't understand that "refactoring code to make it extensible before extending it" is the cost of doing business just have to learn that lesson. If they won't, let them go. Bad ones are a dime a dozen so you can always hire another clueless one if you want. It's like having a loan that you're paying lots of interest on. Saying "We can't spend time refinancing that. We have to go earn money." doesn't make sense as some absolute truth.
* Engineering is not in contrast to the business side. There is no need to represent engineering because the product owner is supposed to be able to understand all inputs into the program. If engineering is slowed down by the lack of dealing with some technical debt that's not some engineering problem divorced from the business. It's the business. This requires reaching across from the engineering side, being clear when things aren't going well, not starting on ridiculously long "refactoring" projects that are just lateral changes, and being clear what the expected outcome of any tech debt relief is. Avoid "feels cleaner", "is more elegant", "is more extensible". Talk in terms of outcomes: faster response to outages, the ability to add things like X (requires you to understand the product some, which is something an engineer should do), less time spent debugging issues. If trivially measurable, measure and demonstrate improvement. Builds trust.
* I'm not too convinced "Scrum Master" needs to be a solo job for anyone. The incentives are skewed. If you have a team of ideal people and they all pick up on the principles and you correct the bad patterns they have, they won't need you. The job inherently obsoletes itself which means the median person taking the role will entrench themselves in process to keep themselves relevant.
* 20 people at daily standup makes no sense. It's 10 minutes or 15 or whatever so you have 30 s to describe something. It isn't so much the time, though, but the fact that there are too many people here together in the room. It makes no sense. Find smaller units of organization. Conway's law presents but the alternative is much worse.
It sounds like you're missing probably the most important part of agile which is "at regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly".
In Scrum, it's retrospectives.
Continuous improvement is a theme that runs across agile, scrum, lean. Without giving feedback, how are you going to improve?
> nobody says much of importance, and just parrots what is already on the Jira board
Worse, the moment anything of interest gets being discussed, SM stops it, saying "let's take this offline" (in German-speaking projects at least, where there's a tendency to use English words to really say nothing at all). Which brings us to
> any deviation from scrum itself poses a threat to this person's job security, so it doesn't happen
MBAs have identified and captured the role of a Scum master (sic!) to comfortably represent a project towards management, without the responsibilties of a project manager (I'm just moderating the project), yet with an instrument to turn time-based freelancing and employed work into a fixed-price deal through commitments/forecasts.
Standups with multiple teams, or simply very large teams, are a terrible idea. I'm currently in a team that has very quick, fun and productive daily standups (despite not even having a good standup space for it), but we have only 3 devs and a handful or other people involved.
But I've also been in projects where standups drag on and on and on, and they're hard to get under control. Usually, those projects had much larger teams.
I suggest smaller teams, and if you have this kind of multi-team standup, discuss with your team to have your own independent standup instead and send one representative to the big one.
> Create tickets for debt, put them on the backlog.
That will risk making a huge technical debt section in the backlog that will find it hard to get priority. It's better to condition new features/bugfixes on the cleanup that caused them. "I can add this feature in 1 day but will need 4 days to clean it up. So my estimate is a week." I constantly nag developers to do this when they find that they are pressed for time and want to make a hack.
-"I'm nearly done but I can't find a nice way of doing this because of an existing code smell so I'm considering adding another smell"
-"Did you ever find that this was NOT the case?"
-"No..."
-"So do you always double or triple your estimates so you know you can comfortably refactor"
-"No..."
I think this is a disease of Scrum. Too short estimates exist because developers pat themselves on the back for finishing things quickly.
That will just lead to stories to change something simple the color of a button taking 3 days because you hit a tech debt minefield refactoring. You don’t want someone looking at JIRA and seeing it took you 3 days to change a simple button color.
Keep technical debt in separate tasks to make the tech debt visible and to keep management accountable for not making efforts to get rid of it.
The best place I've worked at as a developer essentially had no process. It's also the place I really saw the benefits of having good managers (or a project manager, but here the manager took this role).
Essentially the "process" was:
- Someone gets an idea to do something, e.g. PMs wants to add a feature.
- The manager (maybe with the help of a dev) figures out which teams need to be involved (e.g. dependencies).
- Get a very rough estimate from developer. Are we talking a few days, a few weeks or a few months?
- Manager, PM and other teams get together to figure out priorities, schedules and who's likely to be working on it.
- Based on some discussions, areas of expertise etc, some number of devs gets assigned to work on this feature.
From here the devs know what they're trying to accomplish, what the constraints are and who to go to for questions (PM, design etc). They also know who else from other teams they're collaborating with and they just figure out amongst themselves how to get things done. They'll keep their managers and the PM updated on progress and any blockers.
Every once in a while manager etc have to step in. For example if some big that was previous unknown came up, there is a big risk, priorities need to be adjusted etc. But for the most part things just worked really smoothly.
Sometimes we need to give estimates and really hit it (legal or security issues, big marketing launch etc), but for the most part we were trusted to be doing things as quickly and efficiently as possible. So none of the commits, sprints and burn down charts BS. It's not hard to gauge develop's productivity based on output anyways. If there was something slowing the team down, we communicated the need to the manager and PM, and we worked on fixing it.
So for me, if you have good competent people that communicate well, you don't really need much process.
100% this. For a team of good developers this is by far the best model to get things done. SCRUM only serves to prevent a very bad team from doing even worse then they would do otherwise. However for a team of competent developers SCRUM is painfully stifling.
We are better off building over-all engineers that know what they are doing, than trying to catch up with the latest shiny thing (Scrum, Agile, Waterfall, Kanban, ...Jesus Christ)
I think the challenge with having no process is this: You have to rely on great chemistry and raw talent.
Look at Spotifys videos on their process. Every team is self-driving, because everyone knows how to do stuff. If you work at a company where there is no process, and not every is able to complete the whole task from A-Z, you end up with deadends and people who get stuck. If the culture doesn't encourage knowledgesharing, people end up making crappy solutions.
You have direct access to the PM and work closely with them. If you weren’t clear on something you ask, if you don’t agree with something you debate it out.
Like anything else there is no silver bullet. Sometimes you got the feature right, other times it misses the mark.
Not saying that this is not a good way to go, but I rarely see this happening in large complex projects, especially those that are on their way to delivering v1.0. v2.0 and beyond have a better chance of these types of "process" to work where the product evolution is not so volatile.
In my very surface-level, western understanding of Chinese philosophy, Agile seems very much like Taoism and Scrum is like Confucianism, in several ways.
The Agile Manifesto describes a set of ideals but gives no true set of instructions to follow, to do so would not be the Agile way. More than anything, it prescribes an attitude around which you should generally approach things.
Scrum conversely gives rules for how things can be organized and executed in a functional organization. It gives little room for flexibility. Any problems you have with Scrum, the first reaction should be "how are we following the rules wrong?".
Similarly, Tao and Agile are ideals for the individuals (and small teams), Confucianism and Scrum are rules for getting things to work functionally within a society/company.
The comparison may be very apt. After all, a key tenet of Taoism is that the Tao that can be told is not the eternal Tao. It may very well be that an Agile development process that can be fully specified and documented is not truly Agile.
There's a set of practices here in Seattle that I've come to call Skragile, which evinces the veneer of of Scrum, Kanban, and Agile, but embraces very little of the fundamental philosophies. The most distinctive characteristics:
- daily standups (often one per day per team so multiple per person)
- sprint based development cycles, often with retrospectives
- fondness for the "as an X I want to Y" story
- story points defined in terms of developer hours or days
- Kanban style "pick a thing to work on", even in sprints
- burndown charts
Seems nearly inevitable that when anyone here says "we need more process" they're looking to have more of one or more of the above. I essentially never hear anyone in project management say anything like "we need to build more prototypes" or "we need to streamline our deployment process" or any of the other concepts that would actually characterize a (lower case a) agile development process.
This is pretty typical elsewhere too (midwest US as a data point).
Its popular because managers can implement it. Agile- the manifesto version- was intended as a way to re-introduce the notion of discipline that newer generations of programmers had lost (where previously they were engineers and scientists who wrote code, now people who code exclusively eithout other backgrounds).
It feels like we have come full circle to where the new generations of programmers have delegated (or lost) all discipline to managers' Scrum plannings.
Managers and "product owners" aren't engineers or programmers. They shouldn't be expected to "manage" a developer's day-to-day, and yet them presiding over sit-down-stand-up meetings and demanding points and determining deadlines and features is precisely what it has come to. We are right back where we were when the manifesto was drafted.
The irony is, when I first saw Agile/Scrum, I thought of it mostly as a tool to provide discipline further up the chain rather than down.
User stories? Oh, a neat way to keep requirements general and open-ended so we can properly address how we're actually going to solve a user's problem. Surely it'll prevent PMs from over-specifying requirements that lose sense of true objectives.
2 week Sprints? Cool, estimates are hard, and now I never have to estimate more than 10 business days worth of labor.
Retrospectives? Great idea, we can finally do proper post-mortems and knowledge-sharing!
Scrum Masters? Wonderful, there's someone whose dedicated to running the process and making sure we have everything we need!
What I wasn't anticipating:
User Stories? But what about critical requirements that need to be prioritized that don't fit into "As an X I need Y"
2 Week Sprints? I now have so much technical debt a repo man is confiscating my laptop.
Retrospectives? This is always going to be 100% about how we didn't estimate correctly and not about far more important matters like: how these features didn't help our users, technical knowledge-sharing, and ticking-out technical debt.
My favorite team experience was scrum done "right". Standups were run by a scrum master who was the most experienced engineer on the team- programming longer than I've been alive type of guy.
Probject managers and product owners were kept at arms length in the sense that they didn't dictate how, what or when we did what we did. They translated the business requirements and timelines into something we could understand and react to.
Somehow, all of that turned into two week sprints that felt sustainable... At least until the company bought out another company, and everything went downhill. That, however, is a story for another time.
The most salient truth about power is that it cannot be disciplined, except by greater power.
Here's the thing: I have been lucky enough to work with engineer-founders as my bosses for most of my career. You know what the downsides are? None. It's just pure awesome.
It's getting to the point that they're the only people I want to work for.
Managers, not understanding the difference between latency (how long each task takes) and throughput (how much work is getting done in total), always try to optimize for latency. The predictable result: throughput goes to hell, and then latency goes with it.
People who actually write software understand that you have to optimize for throughput first. Not to worry: latency won't be forgotten! But a primary focus on throughput will result in a clean codebase, that will maximize throughput and minimize latency.
The rationale is that if you minimize latency, throughput has to be maximum too, so optimizing latency is enough.
In practice latency is the goto target for optimizing actual processes. It's the most linked with all the risks. But software development is not an actual process.
> The rationale is that if you minimize latency, throughput has to be maximum too
Exactly. And it's quite wrong even without taking the growth of complexity into account — as every engineer knows, or should know. Getting every task done as quickly as possible requires a lot of context switching, which is murder on throughput. When you add in the effects of complexity growth (aka technical debt, though I think "complexity growth" is clearer) the disadvantages of optimizing for latency become that much more serious. And the worst part is, as the disease progresses and latency deteriorates, managers try to cure it by applying even larger doses of the poison.
This idea that managers optimize for latency, while I optimize for throughput, occurred to me only recently. But as I look back over the disagreements I've had with managers through the years (including disagreements over the usefulness of Scrum processes!), it's quite remarkable how many of them seem to come down to this.
There's pretty good theory behind the idea of minimizing latency to improve throughput. One of the better books on this is "Managing the Design Factory", and a followup called "The principles of product development flow".
These books are not about software development, but of product development in general. The first one actually predates agile, published in 1997.
Core idea is that minimizing the size of the tasks is the best way to improve productivity. Not getting it done as quickly as possible, but to decrease the size.
Remember that product development (new, innovative, uncertainties) vs product manufacting (repeatable), is not a new problem, and not unique to the software industry. There's a lot to learn from product development in other industries.
Another interesting read is "The Toyota Product Development System: Integrating People, Process And Technology" which talks about ways of making product development predictive, and less risky.
It's interesting to see how little software is used to improve the process of software development. Other industries use a lot of software (cad/cam, visual modelling, testing, impact analysis) to improve efficiency and quality of product development. Software for product development is a huge market.
It is basically correct, as the math does add up. As yourself pointed, if throughput isn't optimized, latency goes to hell. Thus minimized latency leads to optimized throughput.
The problem is that except on the bare minimum, latency is a bad proxy on development projects. So the idea is perfectly correct, yet it's useless.
I've worked on more than 10 different Scrum teams, and have seen it done well exactly once. When it was good, it was very good.
But we spent one entire workday (7 hours) on each sprint follow-up meeting, and then another entire workday planning the next sprint. That is what it took to write the stories, break them down into one-point pieces, prioritize with the PO, pass the stories out to the devs, etc.
Most places just don't plan for enough time to do things right, and quality is second priority, so no matter what methodology you use, you lose, because no cared about doing it right to begin with.
I'm on a team that started doing Scrum a few months ago, and we're still figuring it out.
To be clear: are you saying that the one team that did Scrum well did so because they spent more time on the process? Reading what you wrote, spending two full days every two weeks to plan sounds, well, terribly dragged out. Does it feel like the time was well spent, or was it a slog?
It did not seem dragged out, it was just thorough, and reasonably un-rushed. It was somewhat tiring, but definitely worth it.
By the end of the planning meeting, we had a clear idea of what we were going to accomplish, and a reasonable amount of confidence that we considered all the tasks that went into our plan.
Because when a story was, e.g. "add addresses to the clients page", it was broken down, discussed, thought out, and agreed upon. The moments when I discovered, oh shit, this story will actually take 10 hours longer than I had allocated, and now I have to stay until 10PM two days this week to meet my committment. Because our story points were approximately an hour each, and almost every story was broken down until there were no tasks more than 3 points each, 1 or 2 preferred.
It was all thanks to our scrum-master/project manager, who had actually spent a lot of time learning about scrum/agile/kanban/etc, read many books on it, and most of all, was committed to doing it right.
I think our sprints were a bit longer than 2 weeks.
I had a similar experience, where we would spend at a full day every sprint planning the sprint (team of 5).
During the project, it felt like a bit of a waste. We were spending a full 10% of our time on project management. However, looking back, I see that a) it seems to match up with other's experience, and b) it's pretty much the same amount of overhead for project management that the project would have had using any methodology.
At least with the 1-2 days every 2 weeks everyone sees it and it's something you can get better at. I'll take that over magical GANTT charts any day.
> are you saying that the one team that did Scrum well did so because they spent more time on the process?
Creating fine-grained, detailed user stories and making sure that everyone understand them and agrees on the prioritization is not time spent on "the process", it's requirements engineering.
I just read your comment again, and I want to add this to my other reply:
I would say the top, most useful, make-or-break practice that I would say is most essential to Scrum succeeding is a combination of:
* 1 point ≈ 1 hour of work
and
* no task above 3 points, try to only have 1-point tasks
Coincidentally, this process is what took the longest in our planning meetings, because the coders sat down and planned out the tasks, kind of the way you would in an algorithms course.
The payoff is that our estimates, after the first couple of sprints, were dead on, and there was very little "discovered work" mid-sprint. No midnight oil. And no corner-cutting.
I had the same experience. I worked in 2 different companies, in 4 different teams. And I can see the value of planning. In the first company we broke down our 2 weeks work into 3-5 hour long tasks. By doing that we had to know exactly what is the work needed and we discovered the work that we didn't think about at the very beginning. Plannings would take 1-2 days. It was exhausting but in my opinion worth it.
In my current company we create 5-8 tasks for whole sprint and when someone takes it then they work on it for 2-3 days and discover more work to be done. At the end we usually don't make it on time. Even if we do, we spend about 40% of our time on bugs/improvements that we didn't think about.
My personal view is lack of experienced scrum masters and developers who have never seen scrum working at least once xD Lack of management support who would push the process.
My favourite: ("what?! you want to spend 2 days on planning?! I can implement it in that time!").
Whenever I've used scrum, the developers set the estimate and task breakdowns. We even had the developer _doing_ the work provide the estimate. It's not really fair to have someone else estimate your work, it doesn't breed commitment either. We did sanity check things - sometimes an estimate would seem big and everyone else would ask why. The answer was either we had missed something (typically), or (rarely) the estimator misunderstood the task and thought it was something bigger.
At the end of each sprint/cycle we shared estimate vs actual effort, to improve our ability to make a good estimate and set an overall velocity.
On my first Scrum/Agile project, there were complaints of overwork the first sprint. Then I pointed out that we were the ones setting our own estimates, setting the pace and causing our own problems.
After that, the estimates got reasonable. We stopped playing "Name that Tune" with our estimates and the project settled down.
I am curious how much process research is being done as an industry. With billions at stake, it seems like I would run across more studies where teams were paid to produce the same software independently.
But virtually none of the methodologies I read are backed by much rigorous experimentation.
Maybe it exists and I am just not reading the correct articles.
You're 100% right that this is exactly the kind of thing that needs to be tested in various settings to figure out what an optimal setup looks like for different problems. Unfortunately, I had a hell of a time finding actual comparative data with real metrics [1][2]. Pretty sad that 90% of the "research" available is either snake oil or mostly useless comparisons of methodology.
There have been a lot of software development methodology but one thing that's been overlooked a lot is the competence of the people doing the execution of the project. I am talking not just about the developers / engineers who are building the product but everyone across the pipeline - product owners, business stakeholders, process specialists, business analysts, project managers and just about everyone else.
People attribute failure to the process because it is demeaning to put the blame on people when established best practices can receive the finger pointing.
Stop the notion of being nice to the lack of skills of people. Hiding behind being nice does not resolve the issue but rather propagates it. Fix the issue by providing the grounds for people to learn and become productive.
> Stop the notion of being nice to the lack of skills of people.
Projects don't fail because people are "too nice".
Projects staffed with fully qualified people with hardcore skills fail too. And projects staffed with utterly under-qualified people sometimes do just fine.
The thing is projects succeed or fail for many different reasons, usually multiple reasons operating in concert.
I think the best approach is not to be dogmatic about process and to recognize that "pointing fingers" rarely resolves anything, regardless of whether one is pointing at the process or the people.
Yes! One thing I don't see people mention is that Scrum requires that everyone on a team to be competent. It's for teams that are already great that want to be even greater. It's not for teams with inexperience and incompetence. My guess is many people are afraid of scrum because it may out them as incompetent.
I'm not saying you're right or wrong, but "many people are afraid of scrum because it may out them as incompetent" sounds like the cliche of "X cannot fail, it can only be failed"
Yes, it does sound like a cliche. However, the point of scrum is to find problems that get in the way. Through the process, it's clear who knows what they are doing and who doesn't.
The position may be right, but it's so badly argued. Arguments he uses in support:
- the dictionary definition of "agile"
- how many times the word "agile" appears in the Scrum Guide
- how "lengthy" the Scrum Guide "seems"
- that Scrum is a process, so does not "sound Agile".
I think because people are frustrated. My read of the conversation is that there's a lot of latent resentment towards modern Agile and Scrum practices.
A long time ago, ppl figured out that no one process was right for every organization, project, technology, etc. They created process frameworks (things used to create a process) and they saw that they were good.
Then ppl came along and said that they were "too complicated" we need something Agile! In actual fact, those ppl didn't understand the process frameworks and just created a specific instance of one and they wondered why it didn't work for all projects, organizations, technologies, etc. And then they said those processes are "too simple" - what can we do? "Maybe we should create a way of generating a process?", they then said. And so it went...
Yep, I've never been in a team where the process was not a regular topic of debate. Once it stops being a topic of debate, it actually just means people are no longer adapting. I prefer a good dose of pragmatism here. Changing the rules all the time is very draining on team morale. Similarly not adapting to obvious process issues is equally demoralizing.
Since everyone is doing, or claims to be doing, agile, the whole notion is completely meaningless. Unless they are actively advertising to be doing some form of waterfall, you can safely assume there is some notion of iterations involved.
What matters more these days is whether the team is mature enough to have continuous deployment without process bureaucracy. Deployment fear is a good sign the process sucks. If you have a need for human gatekeepers, something is wrong with the test automation. Deploying often and with confidence is a good sign you are dealing with a smoothly running team.
I don't really have that much industry experience but have been reading a lot about development processes and practices. It seems there just isn't one development process that will solve all your problems and allow you always create successful software. To me it seems you just need to find what works for your team and continuously work on improving on it. What might work today, probably won't work for your team tomorrow so you need to always look at how can you improve your process.
ACM recently had a series of webinars[0] by Ivar Jacobson on Essence[1]. Essence was kind of confusing at first, but it is essentially a language to describe practices from the different processes. The idea is that you can build up a library of the various practices in your company and allow teams to pick their own and evolve their process by swapping out practices that just don't work out. Essence seems like an interesting idea, especially if it allows teams to create a development process that fits them.
I have decades of industry experience and I mostly agree.
> find what works for your team
For the combination of team and project. Different software projects need to take very different tradeoffs in how they managed. Use scrum in aerospace for flight control, and people will die. Use scrum for system software, and you'll have hard time delivering quality and performance, system software often requires non-trivial amount of engineering. Implement NASA's best practices when working on videogame or web app, and you'll blow budget.
You may not have much industry experience, but you’ve managed to hit the core of the problem. There is no one process that will work for every team, or even every project.
Every team at my workplace runs a different process. We’ve got a team developing new features who are running an iterative agile process, allowing them to react to changes ok requirements or unexpected issues with as little disruption as possible. We’ve also got a team doing B2B projects, essentially rebranding our mobile apps and platform for different companies, and that’s pretty traditional waterfall - I wouldn’t in a million years use waterfall for new features, but when you’re running through a set of well understood tasks that need delivering on schedule to coincide with work by third parties it gets the job done.
From all the comments, I see that arguments are there both ways. I also understand that we should follow a process that bets suits the business. I would really like to know how we could set up an Agile process provided that is what is required.
Is there anywhere this would be documented to explain -
a. How an agile process in initiated?
b. How an agile process is maintained?
I have been part of a team with mis-applied agile processes -
1. Requirements change frequently. And not due to users asking for them. More like the Project Managers asking for more configurablity.
2. Requirements are not thought through. Even the simplest cases.
3. Changes don't get vetted by users. Instead, we go through another round of requirements!
4. Every once in a while requirements (and teams) get re-organized, re-planned which puts all the developers off.
I have also been part of a team which focussed on deliveries. This team has been able to write tests and iterate over requirements fast. The satisfaction level of all the developers in the team as well as the project owners were high.
1. Requirements were phased.
2. Whatever was required to be done in each phase was thought through as far as possible. Anything that wasn't clear or was more complex to think out was sent back to the project owners / users for clarity.
3. Code and tests get written. Phase delivered.
Although this process was extremely successful, I don't know if I would call this Agile.
I agree that Scrum is fragile but not that it isn't agile.
It's agile up to the point that it is broken, which happens often due to it's fragility. That it is easily broken is an unfortunate but unavoidable consequence of its complexity, and common misunderstandings/misuses.
Scrum is complex because large software projects (large as in number of people, budget and expected velocity) are never simple to manage. Scrum is complex because correct usage can not be read from a book nor gained from a qualification - it requires experienced judgement to pick the optimal usage and apply it to a team and project.
Nothing can beat the effectiveness and efficiency of a single strong developer working on a stream of well considered backlog items. That is engineer nirvana! However many projects simply require more velocity than one person can manage. From here, there is a whole spectrum of processes that should be chosen depending on resources and needs. From my experience, full scrum kicks in once you have four or five engineers on the same system.
Product management is a key point of failure in Scrum, true also with other processes. But it's felt so much more when an entire team grinds to a halt and an iteration fails.
If Scrum doesn't feel agile then you are doing something wrong. This is exactly the question the team should be asking itself at retrospectives (doesn't need to be three questions!). What's changing in your process sprint to sprint? Scrum masters are there to facilitate incremental process changes driven by team members, not a system that doesn't result in agile software delivery.
The author had really bad experiences. SCRUM is super agile when done right. Change can be introduced during sprint transitions (reviews, retrospectives and planning) - never in the middle of a sprint. Doing so will kill moral and break the rhythm of the dev team.
SCRUM is meant to be owned by the developers - not management. Shame on companies that hijack and impose such process.
SCRUM gives dev teams rhythm with sprints. Allows them to deliver mini releases of functional software in 2-week increments (could be 1 to 4 weeks too - lenght should be decided during planning). The goal of a sprint is to be decided by the developer and doesn't have to be about the software. Can be about learning, improving as a team or helping team members to grow. Here are a few examples I've seen:
- Measure 5 metrics in Kibana
- Marie-Pier leads sprint review demo
- Try self-merging PR's and measure the results
- No more loose ends
Of course, in parallel the team builds and delivers an increment of functional software.
The planning, review and retro rituals are paramount. SCRUM can only work well if those are well planned and executed. The retrospective is particularly important to allow the team to improve themselves. Its an opportunity to change their own process (i.e. change the definition of done).
Contrary to wide belief, story points are not part of SCRUM. Its a mechanism that can be used to gauge the size of work - it should be up to the dev team to decide whether to use them or not. Again dev teams should own that process - should not be imposed by management.
These lessons I've learned in the past two years thanks to an awesome SCRUM evangelist we've hired. I'm proud to say its changed our lives for the better.
In my current team, we post our daily status on a slack channel: what we do yesterday and we are going to do today. We try to keep to one or two points. We have remote people so that works out well. Standup is optional.
On Monday meetings, each person tell one thing that we are going to do. Sometimes it is two things but the point is to focus.
Meetings should only be between business / leads to get scope onto canvas.
If I'm blocked I'll seek out the person who can most help me. I can easily find 5-10 minutes to write out a summary, but on my terms when I'm not in coding mode.
Now, I'm freelancing, likely to be team lead for the company assuming they get financing - and I keep taking notes and reading ways to organize better. We have a ton of work to do, it's a financial mobile app that has 0 tests on the api (I'm single-handedly running the API development Laravel + vue (for some web views)). Money is tight so I'm the only dev on web side and we have 1 ios dev.
The hope is to be full-time at this by Christmas w/ a 6 figure salary (my first +6 figures).
Scrum truly does end up being "Waterfall" most of the time, in my experience. I think this is largely because teams or management inevitably get bored with projects and end up shifting priorities regardless.
From what I've seen, projects almost always start out in a requirements/analysis phase because management, stakeholders, etc., need the reassurance. I've yet to see any public-facing project of meaningful size start out releasing actual MVPs as opposed to "big bang" releases.
Design often happens first because engineers don't want to(and sometimes can't) begin coding away and then receive designs that are either counter to their existing work or are unworkable; management usually sides with design because, well, it looks cool.
Engineers eventually get coding, but non-automated testing doesn't really occur until later because it can be impractical to have people testing unfinished software. Then there's alpha testing, users break the software, engineers fix bugs, A/B testing of different versions of features, then beta testing, where users again break the software and the engineers fix more bugs.
The project continues to drag on because the first 80% of a project is always the easiest part, and management is always hesitant to release "unfinished" software.
Inevitably, there's a big bang release, by which point management has already gotten bored and has dreamt up other "big ideas" for the team to focus on, at which point the project is, for all intents and purposes, placed into maintenance limbo where bugs are fixed, junior developers place their awful code, and a new "feature" is added now and then to satisfy the marketing department.
Senior developers rationalize the waterfall-like nature of the project, so they opt for "continuous integration", but all that ends up translating to is maintenance limbo without versioning.
The actual next version of the software doesn't get built until after most if not all the original team left, and the original software became "legacy" enough that bugs keep popping up and the new engineers don't want to touch the old code. Management eventually gives in, for better or worse.
Am I wrong? This is what I've seen happen to the vast majority of software projects, all under companies that were either Scrum, Agile, or pretend "Agile". Maybe I've just had bad luck.
I wish more teams could pick and choose what methodologies to use for their purposes, rather than buy into MLM garbage.
Said it a million times before - I deal with way more process and paperwork using scrum than I ever did with waterfall. I'm not saying waterfall was an answer, but at the places I worked on it, it did a better job of staying out of my way.
The SCRUM guide is not long, it's 19 pages in a large font. If I remember correctly it does not even mention a lot of the processes which make up SCRUM for many teams.
However the article is not wrong, it's too easy to make SCRUM all about the process, without changing anything about interactions, developer empowerment and feedback cycles.
I think the SCRUM doesn't do a good job of putting limits on what is still agile SCRUM and what is not. Maybe because it's written in such a friendly language.
The aim of Scrum, and other methodologies tagged Agile, is -- near as I can tell -- to emulate a good programmer with a team of mediocre programmers. Not that all developers working on Agile teams are mediocre programmers, only that the business wants to achieve good-programmer results while still being able to leverage the wider talent pool and lower risk and cost offered by hiring mediocre programmers, so it tends to adopt formalized processes designed to work well with them in the hopes that following the process will make them achieve good results. The process is a formalization of the relationship a good programmer has with their client, which is known to produce good results; therefore, the process must also produce good results by definition (and any failure to produce good results can immediately be blamed on lack of adherence to the process).
Unfortunately there ain't no such thing as a free lunch, and when you emulate you necessarily incur overhead, which can be prohibitively large if you're emulating something powerful -- plus, if you have actual good programmers on your team, they will either quit or start emulating mediocre programmers to remain compatible with your mediocre-programmer process. Either way, you lose the benefits of having them.
The problem often is that scrum (or any iterative process) fails when it's not paired with good XP engineering practices. There's constant learning and change on the product side, which drives constant learning and change on the implementation side, the code needs to be constantly refactored to best handle the changes.
TDD/yagni/refactoring allows you to keep your code base nimble. I can't imagine doing an iterative process by slamming features in one after another without refactoring the code as you go to handle what "you now know", or not having good test coverage to support the changes you need to make.
Refactoring also shouldn't be something the PO is aware of, it's not a story, or a 2 month break from feature development. It's part of the job as each story is implemented. Yes, sometimes you don't notice a good way of doing something until some time passes and that area of the code becomes a bigger refactor than usual, just have to deal with it as soon as possible.
That's a bit of an easy critic of Scrum on its literal sense rather than a critic on what doesn't work or isn't agile or is fragile. I've seen Scrum badly and imperfectly implemented everywhere I've been, yet I think this is preferable as having a strict interpretation of its principles and processes. I don't care actually if Scrum is intrinsically not agile (or Agile): I think what matters and is actually what's happening is people adopting Scrum with agility.
It has good principles like backlog, sprint and daily meeting. If teams only pick one of those or anything else and that it help them, great! By the way holding solely Scrum accountable for failures is narrowing down the analysis to only one part of all things that can go wrong and have nothing to do with Scrum. That's a cognitive biases concentrate.
Reading the comments on this post are truly sad. The amount of "me too" juxtaposed with "actually, this isn't a true scotsman" are truly mind boggling.
I wish there was a prescriptive way to successfully make and deliver software. I have seen very little evidence that there is.
Although I have remained sceptical of Scrum, I do find it a bit strange that many of the negative comments about Scum here seems to be related to not doing Scrum (The old, you are not doing Scrum properly). All the talk about managers, and being told what to do in a sprint etc. None of that sounds like Scrum to me. Isn’t the whole idea of Scrum about self organising teams (?), independent of management etc. All you have is a Scrum Master to facilitate the process, and a Product Owner (part of the team) who prioritises the backlog because that role has an understanding of what is needed to solve the problem at hand (in terms of functionality needed).
Where this stuff really fails (IMO) is when execs throw together a half-baked structure around a project, and throw minimally qualified human resources and a few consultancy firms at it. Knowing that "shit rolls downhill", it's always the developers who end up sweeping up the mess working endless hours trying to make deadlines, ensuring everyone further up the hill looks good to their bosses, while praying that their project doesn't get terminated.
In my experience this seems to be an issue unique to enormous corporations. When I used to work at small / medium-sized companies this never happened.
I think that longer term strategic planning is missing from the Agile Scrum manifesto.
Agree with the Software development priority queue part being good thing.
Long term planning was in the waterfall process which does not work that great either since it tends to over focus on planning.
How do you find a good balance between Agile and Waterfall?
I think it can also be different what process works well if you are a startup and need to deliver a MVP minimum viable product as soon as possible. Compared a bit to if you are a long term enterprise company and need to maintain your code base over time. Ie dealing with technical debt from short term solutions vs long term stable.
Strategic planning doesn’t belong in a process designed to provide constant iteration on a two week basis. In successful organisations I’ve worked with strategic planning is one of the things that feeds into the backlog of features to be developed, and runs in parallel to development.
In my experience, I was lucky to apply Scrum and Agile correctly so now I understand them and can distinguish between them. In my current job, there are no chances to replicate those conditions and I do not expect Scrum to succeed (although we use a lot of Scrum terms to define what we do).
I think one should have experienced a successful scrum project before being able to criticise it.
Scrum is a tool: you may be Agile without using Scrum or you may find easier to be Agile using Scrum. Doing Scrum does not make you Agile.
Scrum is hard and implementing it takes different skills than those proven by individuals who can write code.
Honestly, the comments on this thread baffle me. Endless complaining about scrum and everyone who is not a developer is apparently clueless about software. Is this an adequate reflection of reality? This doesn't seem like a very mature attitude. Scrum is a simple framework with only three roles, the development team being one of them - so if the process fails you are (partly) to blame for it yourself as well. Take responsibility!
What's the better alternative? What have you been doing to make things better? The answers to these questions as much more interesting than endless complaining.
I searched the entirety of the Scrum Guide (https://www.scrumguides.org/scrum-guide.html) and could not find a single reference to "agile". And a quick Google search about the origins of Scrum reveals:
In the early 1990s, Ken Schwaber used what would become Scrum at his company, Advanced Development Methods; while Jeff Sutherland, John Scumniotales and Jeff McKenna developed a similar approach at Easel Corporation, referring to it using the single word Scrum.
> In other words, it is a very specific and well-defined process. This does not sound agile and it also does not sound Agile (remember: “Individuals and interactions over processes and tools”)
This is a many times debunked myth about the Agile manifesto. Even the authors themselves explained that "Individuals and interactions over processes and tools" does not mean and shouldn't be taken as "no processes".
Also, agile is not about moving quickly but about being able to easily change direction any time, therefore dealing better with changing environment.
Scrum is a communication contract between customers / stakeholders, managers and dev teams. It is a poor cure for social and managerial dysfunction, but it is a cure.
Scrum is a problem when it is mandated and imposed on functional organisations that do better and more fluid communication. This is often done as a way of flexing corporate power and does huge damage.
A key problem is that we do not have ways of auditing and measuring the performance of software teams no matter what methods they are using - until then this is all anecdote.
> A key problem is that we do not have ways of auditing and measuring the performance of software teams no matter what methods they are using - until then this is all anecdote.
Of course we have ways of doing that - one is the code quality, another one is historical performance on agility.
I wonder what happened to so many developers that turned them so jaded and cynical towards scrum and project management. So far I've dealt with the good and bad of scrum with a good pinch of horrendous PMs in the mix, but I still came out of it with a positive view of the framework. It is hard to run it somewhat smooth and takes lots of motivated people to accomplish[1], but the result had a positive impact on my day-to-day work.
1 - Some people said that this can be a failing of scrum and I 100% agree.
Scrum only works if you need sprints, and you only need sprints if you have non-engineering stakeholders whose mandatory approval can only be attained in one meeting per sprint length.
Some large orgs really work like that, where the approval of one ham-fisted iron-gripped executive is deemed more important than hundreds of thousands of dollars of engineering talent going to waste. I don't think this is very effective but alright then.
Everybody else should just be using Kanban boards with periodic review.
Reading all the comments out there, there is some atrocious stuff people call Scrum. I'm not even a huge Scrum fan, but it folks seem to criticize what people do in the name of Scrum instead of what is actually in the process.
The TLDR of Scrum is simple: (product) management gets to set the priority of things every 2 or 3 weeks. After that, we see what got done and check to see if the priorities are still the same.
If the priorities are wrong, don't blame Scrum, blame management. If tech debt is increasing, don't blame Scrum, blame management.
There's no magic bullet to determine what is important, and certainly Scrum Master training won't help an incompetent manager a competent one.
I don't think I have ever seen it done as badly as some of these comments. Scrum itself is a very light and quite flexible framework, half of the complaints here aren't even Scrum related. User stories aren't part of Scrum, neither are story points or burndown charts.
Of all the people complaining about Scrum it doesn't sound like any of them have read the Scrum Guide. Symptoms of Scrum are not managers pressuring people at stand ups, squeezing 4 weeks of work into 2 and having a "team" of 20 people. These are symptoms of poor leadership.
Agile works fine, if you ignore the Scrum hocus pocus. Stick to basic project management principles (work on one project at the time, wrap things up before moving on to the next project, set aside a bit of time for unrelated work) with the stakeholder put at the center of what you're up to.
The problem with Scrum is that it's usually (always?) implemented as serialized waterfall projects with completely arbitrary looking deadlines (i.e. the sprint's length). All too often things get sort of done but not done quite as well as everybody would be happy with. It's stressful for everyone involved and, insofar as I've seen it used, it makes accumulating technical debt even more likely than more traditional waterfall projects.
Waterfall was terrible in many ways but it seems to me for all the problems with that methodology that "agile" has fixed it has created at least one new one, and often more.
I’m quite amazed by the lack of substance in this article. It takes potshots at certain ideas and processes, without managing to explain these potshots, or even coherently explain the ideas and processes. It is a flimsy opinion piece with no substance, but a clickbait title so everyone ran here to comment.
“To me it seems that one very important part of high quality software development is to maintain a simple priority queue of tasks. The weight is a combination of the value a task provides for the customer / developers and the estimated effort to implement this task.”
Scrums point for me is to prepare your development time. Talk about the issues, prepare the issues, ask the client, make a mock-up and then implement in code.
Usually it's hack together something and see if it sticks, which is awful.
just to offset a bit about scrum being bad: I have used SCRUM over 15 years to run teams and it worked. Most of the problems I am hearing on here are culture problems and no process can fix that. Regarding planning, a day per scrum is normal - sometimes longer, you have to use your judgement to determine when to move from design to implementation. Also, heavy use of spiking stories to learn enough to create the right design is important
> "We are simply caught in a rat race and not able to make a short break in order to look at and learn from all the things that happened around us, maybe even before our time."
But isn't that exactly what Scrum wants you to do? After every sprint, you spend some time to reflect, look at how you're working, look at the bigger picture, etc.
Scrum is by no means perfect; it's a tool, not the infallible silver bullet the author wants it to be. If you misuse the tool, you're still going to get wrong results, but that's the same with every other method.
The real question is: is there a better way to do it that can easily and reliably be implemented by large corporations? I think the popularity of Scrum is probably due to it being more successful than what was used before.
It might not be truly Agile, but for large corporations, being truly Agile may be a bit much to ask. They need reliability and reproduceability, and that means they're always going to need some focus on processes and procedures, and can't always rely on people who might leave or have something happen to them. Not that Scrum is always a good fit for those processes and procedures, but it does help make it appealing for large companies, and at least it puts a good part of the process in the hands of the people who will be using it.
on the contrary: you have a PO in charge of communication and priorization of features and a dev team in charge of developing- no manager on top...at least in theory...
I think the key motivation of agile is the recognition that the plan made in advance is often not right.
If completing "the plan" is seen as success, then of course sticking to the plan is the best way to success. But if maximizing business/user value is the success, then "no plan survives first contact with the enemy." (I don't like "the enemy" concept, but this is a famous quote that gets across the concept).
And I think the reason why "agile" fails is because many actors in many organizations find they are effectively rewarded for "completing the plan" regardless of business/user value, indeed.
This is just not accurate. The types of changes thrown in after early requirements gathering are most often coming from product managers and executives, not from customers and not driven by genuine feedback or budget constraints, etc.
“Sticking to the plan” does not mean rigidly enforcing zero changes, but rather means keeping a commitment to the general scope and direction that was mapped out. Compromises should require extraordinary hard evidence before being accepted.
This is really why Agile as a general set of guidelines is so easy to subvert and ends up being a misused tool in most every situation where Agile is deployed (Scrum of otherwise).
Project management guidelines need to start out by specifying a way that quality is strictly disallowed from being subverted by competing interests that lobby for changing the plan.
If a set of software project management guidelines doesn’t start out with an unchallengeable quality-above-all-else mandate that creates policy barriers to the natural entropy of different interests trying to lobby for why their preferred change has to be made, then it’s doomed to just get politically subverted.
Doesn’t matter if it’s Agile, Waterfall, extreme programming, whatever.
“Sticking to the plan” in the sense of setting up preemptive, high-cost barriers to anti-quality modifications to what was agreed is _the_ thing.
The problem is that if the environment or requirements have changed and you don't take account of that then you may have a successful project delivering a product that is no longer relevant.
This is the overblown excuse people use to argue for their preferred changes, when really the environment or requirements usually don’t change, and when they really do then either it’s small and you should still stick to the plan or else it’s big and the need for change is so overwhelmingly self-evident that you don’t need things like Agile slogans to generate motivation or buy-in to change things.
Either way, harping on the idea of “but what if circumstances changed” is mostly just what corporate politicians do to subvert people demanding evidence for the requirement of changes.
To think of scrum as a process is wrong. It's actually an organizational pattern language. The idea is that social interactions are a complex system and there are patterns of interactions which, from experience, seem to work in certain contexts.
It's not even software specific. Which is actually a strength.
Example, one common misconception is the daily standup is a status meeting. In fact you are supposed to replan in standup, it's a tiny planner. Not sure how you can call the ability to pivot every day not agile.
EDIT: Unpopular option, software quality is poor and it has nothing to do with scrum.
I'm already able to pivot every day—it's part of the autonomy inherent to me being hired as a professional. I can make decisions about my work, and ask my colleagues to make decisions about our group work, as necessary to deliver on business needs. Making a process and a specific time about how to get approval to pivot is less agile.
You ability to pivot as an individual is already limited based on your team. In fact a unilateral pivot is destructive and can inflict pain on everyone else. I'm not sure how having implicit and ad-hoc patterns of interactions is agile. Why? because you can't improve what you can't see.
I guess part of my skill as a professional is knowing how to work with my team, communicate as needed, understand whay they need to deliver business results and get them to understand what I need to deliver business results, prioritize as needed, and make sure I'm not inflicting pain on everyone else.
If you don't have that level of trust in your team members, sure, formal process makes sense. And I'm all for it if that's what's needed to deliver business results successfully. But you're then valuing processes and tools over individuals and interactions, following a plan of when and how you may respond to change over letting individuals figure it out, etc. Again, it's perfectly okay if it's not agile if it's what the business needs. But my assertion is it's not agile.
He's not wrong. Having been involved in the Agile movement since before the term Agile was coined, I think of Scrum as the least interesting of Agile processes, but also the most successful in terms of adoption.
I used to think that was a contradiction. Now I think it's almost inevitable. I wrote more about it elsewhere [1], but the basic deal is that most companies have other priorities than being effective, so the processes that dominate will reflect those priorities.
On average, the first priority of managers and execs is maintaining the power structures that make them a big deal. But true Agile processes are about empowering teams to self-organize around serving users. Ergo, Sturgeon's Law [2] applies here too: 90% of company software process is crap.
I think reason why Scrum in specific won out is that it combined a process framework unthreatening to executives with the multilevel marketing scheme of Scrum certification. Executives got to keep doing what they were doing, the workers got a "didn't stop breathing" certificate, and coaches got paid. Win, win, win!
[1] http://williampietri.com/writing/2011/agiles-second-chasm-an...
[2] https://en.wikipedia.org/wiki/Sturgeon%27s_law
Scrum is badly misunderstood and maybe that's a failing in and of itself but it's really a victim of the developers who sold it as a magical process. Scrum simply can't be implemented as a purely developer process. The backlog exists as a rolling contract between dev and product owners and sponsors.
The most common failure I see is when project leadership agrees to a fixed scope and timeline then tries to execute in an agile way. Agile is the contract. But it's also a tough sell. It requires a high degree of trust to say "we'll pay you $XM in exchange for X sprints of whatever we prioritize with no fixed end state" but that's what you need to be agile. If you layer sprints on top of a fixed timeline, you're just adding needless complexity since your end state is predetermined and how you get there is irrelevant.
I have almost never seen Scrum sold as magical by developers. Executives? Yes. Consultants? Yes. But developers?
I agree that fixed-scope contacts cause a lot of problems for Agile approaches. However, they also cause a lot of problems for non-Agile approaches. If I have to deal with supposedly fixed-scope situation, I'm going with and Agile approach.
There are two basic cases. One is that scope is truly fixed (which is rare). In that case, having a new releasable version every week with highest-priority features first is excellent risk management. When the date comes, you'll have something to ship. You also get to continually validate and improve internal processes, so you're more likely to be using the time available effectively.
The other is that scope was fixed in the contract but is in practice variable. So every week you deliver something to the customer. Every week you build trust. And every week you encourage them to do tests, deploy early, anything so they start getting value. At that point they get user and business feedback, and come to you with changes. That's the point where you start shifting from fixed-bid to time-and-materials. Maybe you keep to the same budget, but now they're getting more for their money. Maybe they look at the backlog and say, "Wow, there's still lots more value to deliver, let's keep going."
>I have almost never seen Scrum sold as magical by developers. Executives? Yes. Consultants? Yes. But developers?
In many places, especially smaller ones, project managers are also developers (or lead developers etc), and they often drink the Scrum kool-aid.
Seconded. I've also seen it promoted by developers who are in what one could call a "honeymoon period" of their careers. First or second job, probably learned programming at university so every task is an interesting challenge, SCRUM is their first agile methodology, they don't have enough broad knowledge about programming and the industry to become disillusioned and cynical. I've had such people evangelize SCRUM to me, with pride in their eyes, like it was the best thing since sliced bread.
I’ve noticed this, too. For some reason a junior developer on my team 2 years our of school was made into scrum master, and if we so much as sneeze there’s no JIRA ticket for it he starts whining.
It was heavily hyped by developers about 10 years ago. Now they've all given up because they didn't learn the right lesson.
it's really a victim of the developers who sold it as a magical process
Developers may advocate small-a agile, but I have never heard of an actual developer pushing for Scrum, SAFe, or any of the name-brand "agile" methodologies, that come with expensive consultants and certifications and "coaches" and conferences and a whole ecosystem around them. These are things for project managers for the benefit of project managers who see project management the main deliverable l in and of itself. The same with JIRA actually, it's not a software development tool, it's a project management tool, sold to the same people who previously bought MS Project. These things are all overheads in the software development process, not enablers, as another poster says, the goal of most organisations is not to be effective at their stated mission, but to maintain their internal power structures.
Have you seen any methodology work consistently when dealing with fixed deadlines? The reality when dealing with large contracts, as you mention, is that there are almost always timetables with expectations. This poses an inherant problem due to the unreliability of estimates, so either quality or features must be sacrificed if the timeline is in jeopardy. My only experience in such an environment was using some hybrid waterfall/sprint methodology that mostly worked fine, but i suspect that was more to with the general culture and people involved than anything else.
I think no industry producing anything new ever found a working process to predict and keep the timeline. (If omniscience really existed, it would have more lucrative applications.)
Developing new aircraft, building a new ship, building a custom-designed bridge (most of them are) are processes that often run out of time and / or budget.
If you want predictability, you want repeatability. But in software all reliably repeatable parts become automated away.
Absolutely. And I'd add that predictability is dependent upon not learning anything over the course of the project. To be predictable either you know everything that matters up front (which is only true for trivial projects with no competitors) or you refuse to learn anything along the way (with, e.g., a big-bang release at the end).
But I think good projects release early and often precisely so that they can learn as they go. At which point predictability goes out the window.
? All the specified above endavours - usually run into what one could call the end of the map. Here be science! If you run into e.g. new material science because your goal stretched beyond the scope of what has been done previously, you can blame the engineer for not telling you that you will leave the boundary of the knowledge of a field- you can not blame him for the estimates beeing wrong.
Nope, never.
I have seen more success in cases where teams estimate for 90% percent confidence instead of 50% confidence. By that I mean "it should almost never take longer than that" vs. "it will probably take that long." Unfortunately, to do that, you need enlightened business management that appreciates the difference between an estimate and a promise, as opposed to business management that pays lip service to the distinction.
> "it should almost never take longer than that" vs. "it will probably take that long."
This sounds like a much better way to estimate. Do you have any links to content discussing this?
Here's an article that explains the phenomenon much better than I can. It made its rounds in HN a week or two ago: https://erikbern.com/2019/04/15/why-software-projects-take-l...
I haven't seen a good way to turn the concept of "it should almost never take longer than that" into a concrete process. I've always seen it as a gut check, often implemented as "take your initial estimate and double/triple it." What I actually see is that a lot of teams implement that but walk back the doubling when the business/PM delegation persistently asks for more in less time. Only in really engineer driven cultures have I seen engineering teams successfully push back.
On the actual use and calibration of confidence intervals - https://www.lesswrong.com/posts/ybYBCK9D7MZCcdArB/how-to-mea...
AFAIC, the only thing that works with a fixed deadline is a variable scope. Which is basically a tenet of Agile.
The alternative is some sort of sub-iteration hybrid where you attempt to fix deadlines for much smaller units of work and constantly revise.
This is OK for some industries. But 2/3rds of a car doesn't quite sell...
One thing about agile is that usually companies not only set fixed deadlines but also fixed feature set; which usually means lower quality.
That in turn means technical debt; which puts future deadlines and feature sets at risk.
Since in SCRUM as a team you try to not compromise on quality the only way out of this is to push back on deadlines or feature set (or both).
So instead of blindly executing orders; the development teams can push back on deadlines.
Most of the time these are actually negotiable. Even the "hard" deadlines....
It doesnt mean the dev teams always win however they are better equipped to inform "management" about the consequence of the deadline.
Exactly. Probably 90% of the burden on a successful agile project is in setting expectations with stakeholders for an iterative delivery without a fixed scope. It can be a tough sell but it shouldn't be. If you've ever done a top line estimate on a fixed scope project, you know it's just absolutely not possible. Why set a deadline you know with 100% certainty you're not going to hit? Just bake it into the contract and upfront expectations.
Yes, absolutely. Many times. I've done a bunch of projects that were tied to immovable event dates. The only way to deal with it is having very flexible scope. Typically, these have been more "creative" projects and not tied to a lot of critical functionality so we just make sure we keep our MVP quite small and treat everything else as iterative enhancements so we can cut off whenever we're out of time and still have something presentable.
DHH has a nice methodology that works with fixed deadlines: his basic rule is that features can be dropped/simplified, but deadlines must be met.
https://www.amazon.com/Doesnt-Have-Be-Crazy-Work/dp/00628747...
He's applying an old concept. The Iron Triangle has been in use since the 1950s.
https://en.wikipedia.org/wiki/Project_management_triangle
If time is fixed, then scope and/or cost must change.
This is true, but fixing the deadline and not committing to the scope requires clear communication by the project management / sales teams with the clients about it, and this is rare.
> Scrum is badly misunderstood and maybe that's a failing in and of itself but it's really a victim of the developers who sold it as a magical process.
Even if so, that would be in itself a failure of Scrum. A process that works only sporadically and only with exceptionally well organised groups ain't really all that much.
> It requires a high degree of trust to say "we'll pay you $XM in exchange for X sprints of whatever we prioritize with no fixed end state" but that's what you need to be agile.
...no? It requires a well-defined end-state, and that includes not defining the irrelevant. It's not really "give me $XM and we'll deliver anything from accounting software to a really nice puppet."
There are in fact two kinds of Scrum (or rather, Scrum implementations), both of them compatible with the Scrum guide: ‘Left to Right’ Scrum (backlog-driven, implementation-focussed) and ‘Right to Left’ Scrum (goal-oriented, iterative).
Unfortunately Scrum is too often explained and implemented that first way, leading to the anti-Agile feedback we see on HN with some regularity. The second way is much more compatible with complementary tools such as Lean Startup and Kanban and I wonder if this (to me very welcome) non-exclusivity explains why it is less talked about.
[1] https://blog.agendashift.com/2018/07/04/righttoleft-works-fo...
"Right-to-left scrum" seems like a hollow phrase made up by that consulting firm you linked.
From what I can tell, it has all the meaninglessness of an empty buzzword used primarily as a method of trying to engage people and give an opening on selling their services by stating "oh if you don't understand the difference sit down with us and let us show you how different and better it is."
It looks like if scrum doesn't work they call it "falling left to right scrum", and if it works it's their "brilliant right to left scrum".
Even by their own definition they both have a backlog that gets prioritized and selected each 2 weeks into a sprint backlog, and executed during the sprint.
The rest is just hand waving. "One is goal focused vs the other is backlog focused. Oh yeah but we do put our goals in the backlog." So they're both backlog focused then? The only thing they're really sayings is "When prioritizing tasks, make sure they accrue to something and aren't just random work." Which is both obvious, and of course too simple to write and sell a book about, so instead this whole other terminology is made for it.
Thanks for sharing the link though.
This logical fallacy is sometimes called No True Agile.
>> developers who sold it as a magical process
If I ever find a developer who can sell, selling is not a developer's USP in general, there are exceptions and I would hire those exceptions instantly!
>Scrum simply can't be implemented as a purely developer process.
aka, WaterAgileFall.
aka WaterFail
Scrum won out because it was waterfall in disguise. Commitments and sprints become terribly destructive over time.
Waterfall was coined by a paper discussing why waterfall wasn't a good way to do software projects.
You aren't wrong, either. I found Scrum to be a series of waterfalls that weren't well thought out. "Iterative Design" essentially meant, "We aren't sure what the button should do exactly, but we know we need it there and it kinda has to do this and we'll figure the rest out for the next iteration."
That caused so many problems with tech debt.
Stories began to take longer due to the increasingly large and discombobulated code base, but the expectation was that we continue to deliver the same number of story points each sprint.
Then we got blasted for losing pace. "Why are you under-performing? We are just adding little features. You've already implemented this button on another form before? Why does it take longer the second time? You already know how to do it!"
It became absurd and no one could hear or understand what was happening was easily predictable and in fact -- was predicted by several members of the team many months prior.
When the project management was informed of these predictions now having come true, the development team was accused of intentionally causing the delays.
This mirrors my experiences to a tee.
The most upsetting experience in my career thus far was being belittled by an incompetent project manager.
"How could it possibly take that long? How do you not know how long it will take?"
The happiest times of my career have been when we're lacking a PM -- coincidentally the most successful!
> "How could it possibly take that long? How do you not know how long it will take?"
Well, if they really said that, you can just dismiss the comment as incompetent. A witty retort may be in order if a non-technical manager is in earshot, e.g.:
"How long would it take for you to earn a green belt in karate?"
"Hmm, what does a green belt entail?"
"Exactly, you don't even know what you don't know, yet."
They did, and I replied
"We don't know because we've never done this before, and we can only compare to the most similar work we've done, and any existing data."
Before I could say more, they I interrupted: "You have to know, it's your job, or you don't know what your doing," or something along those lines. It was the most heated I've felt. I know what I'd reply with today, but I was younger and more naive then.
I commented above that the initial questioning was fine.
> "or you don't know what your doing,"
That though was unprofessional, and should be addressed as such.
What would you reply today?
"Yeah, well I had sex with your wife!"
Or just offer over-inflated estimates since they won't accept something realistic anyway.
Not OP, but I will answer with the second option and ask him what I should do?
> A witty retort may be in order
I don't think this will score you a lot of points.
Depends. Are we talking self-esteem/dignity points, or good doggie/loyalty points?
I'd guess career-progression points, or even keep-getting-a-paycheck points at worst.
This low-stakes exchange won't affect much, it's about putting Mr. Incomp in his place. Remember it's a PM we're talking about, not "the Boss."
Best PM I ever had dropped by every week or so to ask if we needed anything.
That's what the best ones do.
In these situations I shift the conversation to what is the PM really after. I explain that I can tell them any estimate they want (or they can make it up on their own - no need to even discuss), but does that really do them any good when it is missed? There is usually some pushback at first, but when items are delivered within estimate a few times in a row they realize it's better to take whatever hit up front than it is to miss and adjust.
Also, someone asking how could something take so long is not a belittling question. I don't see any problem with either question.
Personally I believe this is one of the major gaps in Scrum, and there is no prescripted way to solve this problem. XP's focus on technical aspects is much better in this regard, but the possibility of runaway technical debt still exists.
There are so many places where this can sneak in and become an issue, too:
- The development team can groupthink, underestimate and/or fail to account for technical debt payback in planning poker. They may estimate the cost for the new feature, but not include any time for refactoring to keep the codebase healthy
- The Scrum Master may not shield the development team from pressure to deliver coming from the product owner, which may cause the team to cut the above corners
- Codebase health may be split out from vertical stories into its own tasks that get deprioritised by the product owner
- Even if the development team identifies the need for codebase health, this may not be communicated back to the product owner in a way that makes business sense ("but we can still get this feature out the door quickly and fix the mess later, right?" - yes, once or twice, but the downside is a slowing of velocity)
- Even if the team started well with ground rules the Scrum Master is supposed to enforce surrounding technical quality, the Scrum Master may not be effective in enforcing the process - either because they lack authority or because they lack the ability or knowledge to do so
Among several others.
In your situation, the big red flag is that the team predicted the issue many months prior. The process has failed to extract that information from the team members and deliver a consensus between them and the product owner.
Though ultimately, as with any project, if the product owner has more organisational authority than the development team and chooses to exert this authority, no process will be able to save you.
It's easy to say something is shit.
The tricky bit is having a solution and implementing it.
Having something well thought out is hard work. Is hard work rewarded? Is initiative to even try rewarded? I'd say the answer is no.
The answer becomes yes in small teams that are given time, freedom and resources, or in times of crisis (same thing as small team really, minus the time bit), as far as I can tell. Otherwise, most people's natural tendency (including me) is to phone it in. We'd be wise to let people in software development and many other professions that are not manual labor, to switch careers after 30-35 if they haven't done anything worthwhile by then.
Working with people who have to phone it in for the next 30 years and know it is modern day hell for anyone with an iota of ability.
It's permanent compromise and insincerity, or being an outcast that's temporarily tolerated for doing 5-10x the work of the person next to you.
Because Tech debt and I reading messy code base never happened in waterfall...
The problem is always : features and customer request over bug fixing or design fixes.
The system, project type does not matter. That's why some PO recommend 10 percent allocation to devops / ops, 10 percent to improving code base . And "no bug survives the Sprint" philosophy. And what is left,is you next feature or customer request.
For every example of scrum operating badly there are examples of scrum working well. The common denominator in all of those situations - good and bad - is the project and middle/upper management.
In business process analysis, when a process is dependent on individual people, you don't have a mature process but an ad hoc process.
In other words, Scrum methodology in itself is not mature as a methodology and depends on individual fiat, just like any project that doesn't have any process at all.
The only tangible benefit to Scrum might be paying lip service to development departments while firmly retaining the status quo.
Scrum is no more or less dependant on individual people as any other methodology.
The rest of your post is just warping your initial conjecture to arrive at conclusions you'd already decided upon. I neither agree with those conclusions nor the chain of twisted logic you used to arrive that.
I'm not going to stand on a soapbox and sing for the glory of scrums. People - like yourself it seems - can get very tribal when talking about such things. Which is as weird to read as it is pointless for you to argue. In my experience leading different teams using different methodologies, the thing that really makes the big noticeable impact is people and not methodologies. If you work in a blame culture or have colleagues in your team who don't follow process - then whatever process you put in place will be undermined at every opportunity regardless of it's methodology. However if you work in an environment where people respect one another and want to collaborate in getting work done, then you pick a methodology that works best for the day to day work (eg project work or support operations) and for the way people like their work organised. All the rest of the arguments are superfluous.
> good and bad - is the project and middle/upper management.
Yep. People do not realize that the whole company has to adopt agile/scrum for it to work. It's a shift that many companies can not or will not make. They are tied to fixed deadline, fixed scope for various reasons.
I mostly agree with you but I wouldn’t go so far as to say the whole company has to adopt it. However you do certainly need the support from the wider company. I’ve seen scrum work really well despite waterfall being the predominant methodology in that business - there people were happy to embrace the preferred working styles of whichever team they had to liaise with. I’ve also worked in places where scrum failed badly despite the CEO pushing for it. Those places usually suffered from a blame culture that was also the CEOs doing and that blame culture meant that everyone was spending their time working against the best interest of any of the other teams. By “teams” I mean project managers, sales guys, support or operations teams (if they differ from your developers and/or DevOps), finance staff, directors/upper management and even your own paying clients (if you’re a hired service) etc.
I’ve noticed comments for and against scrum are often so focused on the technical aspects of the methodology that they overlook the human aspect. Which matters more in my opinion.
Scrum is iterated waterfall. By iterating faster, inaccurate estimation is shown up sooner. On the other hand, developers are treated like cogs in a feature factory, munching through backlog items fed to them by product managers.
I think it works well enough, for a few years. I don't think it's sustainable - the blinkers of "sprints" encourage growth of tech debt because nobody has an eye on the future and Product won't prioritize refactorings, and sprints are too short for devs to sneak refactoring into the schedule.
> Scrum is iterated waterfall.
No, it isn't. Iterated waterfall is at least as old as the first paper discussing waterfall, but while scrum mandates interations, it doesn't mandate much about how work is done in the iterations, and specifically does not mandate the process steps associated with waterfall; further, it emphatically rejects the role separations and handoffs associated with waterfall during the iterations.
> munching through backlog items fed to them by product managers.
That's...not actually Scrum, as it implies that either the role of Product Owner is taken by a PM outside of the Scrum Team or that that the Scrum Team is not self-organizing, either of which is a significant (even if common) deviation from Scrum.
> the blinkers of "sprints" encourage growth of tech debt because nobody has an eye on the future
Tech debt should manifest in reduced velocity which should be noticed, taken as a signal of a process defect, and addressed in the Scrub Team’s various inspection and process adjustment points.
OTOH, if the Scrum Team is properly cross-functional and self-organizing instead of having a non-team-member imposed as Product Owner, then including appropriate restarting as components of completion of relevant backlog items shouldn't be a problem.
You can scream "That's not actually Scrum" untill the day you die. It doesn't change the fact that that's how it is perceived and done in almost every place that says they do Scrum.
A sibling comment to yours made essentially the same point, and I had already addressed it, before you posted your comment: https://news.ycombinator.com/item?id=20020206
What percentage of Scrum teams do you believe are "properly cross-functional and self-organizing"? And could you point me to examples of people losing their Scrum certifications for not living up to that standard?
> What percentage of Scrum teams do you believe are "properly cross-functional and self-organizing"?
About the same percentage as that of “Agile” software development shops that put people and interactions above processes and tools.
OTOH, at any place that is considering implementing either, there are decision makers who can influence (or in the case of Scrum more than Agile, authoritatively direct) whether or not that's the case, so for them, at least, it's worth distinguishing between problems with Scrum as prescribed and problems which often occur because decision-makers decided to ignore key parts of Scrum-as-prescribed.
Most nominally "Agile" shops are in effect doing Scrum, so I'm not sure that answer helps your case. Either way, it sounds like we agree the "good Scrum" shop is at best rare.
Given that, I think it's worth considering that the problem is Scrum. Especially given that Scrum is not just a process, but an organization and an army of "certified" people that sell services.
When something generally doesn't work for its stated purpose but keeps making money, I think it's worth asking what its real purpose is. E.g., things like crystals and psychics. As Eric Hoffer wrote, “Every great cause begins as a movement, becomes a business, and eventually degenerates into a racket.”
> Most nominally "Agile" shops are in effect doing Scrum
No, most nominally Agile shops are also nominally doing Scrum, but actually doing not-Scrum in a not-Agile context, largely due to sabotage of both core principles of Agile and foundational elements of Scrum by management.
It doesn't matter what name and superficial ritual you put on the process if it's all window dressing over top-down disempowering command-and-control by persons who are neither doing the work nor experts on the work.
Which isn't to say that the Agile and Scrum bodies of work aren't part of the problem: neither really addresses as a key point how the team effectuates ownership of process and how interaction with management works, which means those gaps get filled in (or rendered moot, in the first case) in ways which compromise what those bodies of work do prescribe because.
Lean, which comes from basically the same perspective (while they don't cite exactly the same values as expressed in the Agile Manifesto, being Lean essentially implies being Agile and vice versa) and is a good body of knowledge to draw from alongside Agile, is better in this regard, and so places nominally drawing on Lean seen to be more likely to be doing what they say, because Lean doesn't leave as much of the core vital parts without good guidance.
This is the great con of scrum certification — you can always point to some team and say “well that’s not the official version of scrum”
Which is a really impressive con. Generally the point of certification is to make clear you're getting the official version. E.g., doctors and lawyers self-police because they know it's harmful to have quacks running loose. But somehow Scrum has been able to keep making money despite not bothering with that.
This. I've run programs that purport to be agile but the work input stream was waterfall and disconnected from the means of output so it was really timeboxed waterfall, unfortunately my management did not agree to re label. You know it is real Scrum if it is painful for the business team and they come to realise that now it's software all the way down.
> Tech debt should manifest in reduced velocity which should be noticed, taken as a signal of a process defect, and addressed in the Scrub Team’s various inspection and process adjustment points.
This process adjustment usually means "sorry about your vacation" and/or "you aren't doing enough overtime".
Yes, if external actors rather than the self-organizing teams own the process, that is probably the first response, but that won't actually stop the buildup of tech debt and the velocity impacts, though it may produce a one-time, non-repeatable improvement, so even in that case if someone (even the wrong actor) is monitoring velocity and doing process improvement, there will be an impetus for further change.
(Of course, if the wrong actor is monitoring velocity and controlling process, then the team has an incentive to mask the effect of tech debt but continuously adjusting estimates to maintain the illusion of constant velocity, a d avoid the first bad response, or, if that opportunity is missed, to avoid the subsequent external interventions.
>I think it works well enough, for a few years.
"Fortunately," hardly anyone in Silicon Valley plans to be working on the same codebase in a few years. There's a good chance the problem space won't be relevant anymore by then, and on the off chance it's still funded, the new team will rewrite it whether or not it's good.
>sprints are too short for devs to sneak refactoring into the schedule.
As I've gotten more senior, I've just gotten more brazen about doing this less sneakily.
Is the Silicon Valley representative of software development though ? Because on the other hand we have professors telling us that the average (surviving?) software lifetime is 20 years...
This is just intuition but I suspect that the lifetime for software is u shaped. Much of it is very short lived but software that lasts more than 1-2 years is very likely to live for a decade or more.
A lot of 6 month old code gets thrown away either because its been rewritten or because it didn't achieve its stated objective. Meanwhile a bunch of companies are relying on systems that were first created in the 90s because that software achieves its objectives and the cost justification for a rewrite isn't there.
There are many 20 years and older code bases currently. But if you start a new code base today it might get rewritten many times in the next few years because software users no longer have the power to stay on an old version due to the SaaS paradigm.
The biggest problem with Scrum is it lacks any sort of design phase. You do the minimum. Oh, it doesn't work quite right? We'll fix it in the next sprint...
The "spiral" model is closer to a true iterated waterfall. I've seen it used successfully in more mature companies.
It's like "build one to throw away", in that the goal is to rapidly explore the feature space and discover useful features, but what actually happens is that it's a trap and you won't throw away something that works no matter how poorly implemented.
absolutely. If you ever see a group afraid to produce a prototype, it's because they're afraid management will try to ship it as a final product.
The effective strategy is to build an oblique, fatally limited prototype, which can not possibly be mistaken for a shippable product. The bad prototype can only be used to test the hardest parts of an idea and must have major holes in it with no way of filling them. It should also have a largely fixed timeline to ensure it is put to bed before it gets "hamstered" into the shipping product.
absolutely. If you ever see a group afraid to produce a prototype, it's because they're afraid management will try to ship it as a final product.
I once had a manager who, on the subject of upper management wanting to ship the prototype, would say “the difference between dev and prod is one letter.”
You're confusing "investing in design" with "having a design phase". I agree many Agile teams underinvest in design. But so do many non-Agile teams. The problem isn't the lack of a formal phase. The problem is not taking it seriously.
You get what you measure and Scrum has no measure at all for design. I’m now curious to see some projects that Ken Schwaber worked on.
I'm certainly not defending Scrum. But I think one of Scrum's big problems is relying on proximate measures rather than actual results. It values output much more than outcome. So I think the way to get good design isn't to try to measure design. It's to release early and often, making sure that products fulfill their purpose.
I think our whole industry is full of proxies for the things we actually care about, which is for instance why we rant so often about the hiring process.
For another example, look at what we talk about with Clean Code or SOLID principles. We have that old joke about the two hardest things are naming things, cache invalidation and off by one errors. I think the fourth hardest one is resolving merge conflicts.
You work carefully on code for weeks or months and then in one fell swoop it all gets broken when you aren't looking at it anymore. It's really bad for morale.
If you read through all of these Best Practices with the notion that merge conflicts are dangerous, then you start to see them as designed to keep conflicts from happening. Putting related code together isn't just about reading comprehension. It's also about keeping unrelated code apart. You and I can work on unrelated stories without ever doing a 3 way merge.
And if you look at merges as the problem, you start doing things like alphabetizing data structures that don't have a strong cohesiveness to them because then when you and I add a feature to the same data structure, we aren't both modifying the bottom of the file. I put something under T and you put it under G.
To put this another way, if in some perfect future someone invented a merge tool that never screws up, so that we only get merge conflicts when two people are accidentally working on the same feature/bug, we might start to question all of these practices as overwrought.
Yes, you're correct... I didn't mean a "formal" phase, so much.
I completely agree here. Instead of understanding the volatility of the requirement(s) and designing for that, we refactor and react.
It doesn't lack a design phase. Where do you get this information from? It isn't explicitly prescribed, but no one prohibits you to make designs.
Experience. I've been developing software for 20+ years. In practice, due to the nature of agile / scrum, design is very minimal or thrown out the window entirely. It encourages short term thinking. Rarely is anything beyond the "current sprint" considered.
I was explicitly told not to write unit tests because they took too much time, which required me to spend entire days retesting almost 100 scenarios when the business logic changed.
Of course the business didn't know all the scenarios at the beginning of the feature development and didn't care because: iterative development means we'll figure it out later.
All straight out of the TDD playbook.
A lot of devs trap themselves by insisting they can write the tests after. Once managers know the code exists they want to use it, or move on to the next thing people are breathing down their necks for. And code written without tests is difficult to test so becomes a self fulfilling prophecy.
In fact it’s such a reliable mechanism for self sabotage that I look at carefully at people who bring this on themselves and try to figure out if it’s naïveté, learned helplessness, or malice.
As you've discovered, good automated testing is mandatory for iterative development. If execs tell you to do something unprofessional, it's ok to say no. Indeed, if we're going to think of ourselves as professionals, I think it's mandatory.
> if we're going to think of ourselves as professionals
We need to unify around these principles, possibly revoke memberships for unprofessional behavior. One dev getting himself fired changes nothing..
I'd love to see that kind of organizing happen.
But one developer quitting rather than do bad work does change something. At the very least, it means that the developer can find another, better job. But it's also an opportunity for companies to learn. And for those who don't, good people refusing to prop up bad companies is a step forward.
I'd also add there's a a lot that can happen between refusing to do bad work and quitting/getting fired. Saying no is the beginning of a negotiation. A negotiation that won't happen if we just say yes all the time.
Scrum is unsuited to actually building a decent framework for you application.
We tend towards 4 or 5 sprints in which nothing is completed, only to suddenly have a flood of completed things the next.
Then that's not an Agile process. That sounds more like mini-waterfall.
It's perfectly possible to deliver small units of work weekly. One still gets frameworks while doing that. It's just that the work of making the framework happens over time, not up front. Which is good, because good designs are based on good information, and the beginning of the project is when you have the least information.
I love these commitments. When my Scrum leaders asks me why we didn’t finish what we ‘committed’ to doing these two weeks I want to strangle him (nothing personal, I do otherwise like the guy :P).
The terminology is so incredibly developer hostile.
Which is why the scrum handbook went with the term forecast instead. Many years ago.
I agree. The last "waterfall" company I worked for was Novell. In my opinion, all the Novells of the world glommed onto Scrum because it was shiny and new and allowed them to call all their bloated and unnecessary project managers "Scrum Masters" and all their wasteful status meetings "Stand-ups." It was just the same old garbage with a different name, but hey, We're Scrumming!
Waterfall with little or no modelling. If you write the wrong system, you are left with nothing. With waterfall you could at least start over with your design artefacts.
Scrum wins out at companies that did not exist in the waterfall era, staffed by developers and engineering managers who have only ever worked in startups. Similarity to existing process may explain a large part of the adoption in places that had existing process, but a lot of Scrum shops didn't.
Scrumfall is, sadly, widespread
> Commitments and sprints become terribly destructive over time.
Could you elaborate?
You're asked to deliver a feature that would take 4 weeks to implement without tech debt in two weeks. This requires sacrifices like ensuring network connections remain alive, handling exceptions, optimizing queries, abstracting functions for re-use, handling all the data input scenarios, etc.
This creates a confusing code-base with lots of IF statements, copy/pasted code, data modifications from several areas of the system, etc.
You'll fix a problem with data getting into the database incorrectly and realize a couple sprints later, fixing that bug caused other bugs elsewhere.
Automated testing doesn't catch it because automated testing only handled certain scenarios.
It's a mess.
>You're asked to deliver a feature that would take 4 weeks to implement without tech debt in two weeks.
Well there's your problem. How is that inherent to scrum?
Because it's a sprint. You've got a finish line and you're racing towards it. Businesses don't function without prediction, and refactoring gets cut before roadmapped features. If feature development is a bottleneck in company growth, debt will grow, quickly.
“Sprints” are, pretty much by definition, a non-sustainable pace.
Agreed. I regularly ask "Why are we running a marathon in a thousand sprints?".
Besides tech debt, a concern I have that I don't see brought up is burn out. With Scrum, every action you perform is micromanaged and with a push for "high velocity". There is no proverbial breathing room in this where the pressure lets up. At least with waterfall (for how we did it before Scrum), the windows of high pressure times were shorter. During the beginning of our 6 month waterfall, in parallel to spec work we'd be taking care of tech debt or implementing our pet feature and it was a time of mental recovery.
imho a problem with the implementation of SCRUM, but not necessarily restricted to SCRUM.
In a perfect SCRUM world you could not be forced to implement this in a time not of your own estimate. A feature that is so big that it takes more than a few days to implement has to be split...but well we all know how this plays our in reality most of the times...
>You're asked to deliver a feature that would take 4 weeks to implement without tech debt in two weeks.
There is something odd with this, if you did the estimation. Shouldn't the story points/sizing/estimates preclude this?
Doesn't the team pick the items for the sprint that they think they can complete?
You can never take a 4 week story in a two week sprint. When would you ever be able to complete it?
By breaking it down into smaller stories?
My experience shows that proper testing and documentation is the first thing that management wants taken out of the story, often with the excuse "We can handle that in a later sprint." But since your life is a neverending series of sprints (note: that's actually an ultramarathon), and management gets to pick priorities, you may never return to the technical debt.
I have not worked in a scrum environment in a couple of years, but when we did it those items were not included in the story. They were part of a definition of done that the team did not really talk about publicly.
Refactoring stories were also rare, it was just sort implicit in the task, unspoken as part of doing it.
I'd imagine that is why they did not get cut.
But yes, management picked feature priorities, but left the implementation up to the team. So at any given time were were working on the "most important" thing from a business perspective.
The most successful (but still crappy) solution to this that I have seen is developers undertaking guerilla tech debt work. This can be done by bundling maintenance work into an existing task where the two are actually not that related or by simply carving out bits of time between official tasks.
While it "works" it is certainly not ideal that people have to go off the reservation to ensure that the project doesn't implode in the future due to the accumulation of buggy code, performance problems and likely security vulnerabilities.
> proper testing and documentation is the first thing that management wants taken out of the story
That's not breaking down a story into smaller stories though, that's simply not doing some of the tasks necessary for a story to be complete. "Story" is not a synonym for "task"; a story is user-centric not about internals. "Write the documentation for this feature" is not a story and "write the tests for this feature" is not a story, they should be part of your definition of "done".
That would be ideal, but we are devs, we don’t get to touch the stories other than to mark them as done...
You're not doing Scrum then. One of the few meetings that Scrum dictates is "backlog refinement" where the dev team works with the PO to get the stories into a workable state.
Yeah, we do waterfall and scrum with the worst of both.
If you can't touch the stories, you can ask whoever does to break it down. If they don't then you shouldn't ever pick it up into your sprint. If they insist and it doesn't work out then you should call out the incident in your retrospective and hopefully get everyone to agree to do things differently in the future.
Sadly, it’s more like the dev team points out that we really cannot work with these stories then management telling us to deal with it while the BA’s suck their teeth.
Then they ask us why there isn’t any progress.
Yep. :)
https://www.brightball.com/articles/reality-driven-developme...
Scrum won out IMO because it's an awesome word and conjures up subliminal images of brute force pushing obstacles out of the way, bringing success to the team. Execs can relate to it without even knowing what it is.
I actually think scrum won because of its multiple rituals that allowed most of middle managers and project managers to keep their job by recycling it.
When I hear the word scrum I am always reminded that my school stopped playing Rugby because a kid in the next town got his back broken, in a scrum. So from then on we played football (soccer) in Rugby kit, until eventually all the kids had grown out of it, or left. Similarly, if you sprint and sprint and sprint you will collapse from exhaustion.
I often wonder if they picked these words because they knew nothing at all about sports, of if there is a coded warning in them...
Scrum won out because it's basically feudalism. It's great for those in the upper echelon.
Scrum won because in reality not everyone is a wizz bang developer like your guys so make yourself out to be.
Scrum (and other methods that try in vain to make developers replaceable cogs) won because it enabled bad developers to hide among the good ones, and avoid personal responsibility.
I think it won out because the number one thing any kind of project management framework has to provide is ambiguous blame-avoiding fungibility to management. If a framework does not offer that, managers will politically subvert the policy-making process to rule out that framework.
It’s the same reason why demonstrably value-additive, cost-effective prediction markets are rejected by managers despite evidence in their favor: it removes their ability to create ambiguous Dutch books out of different project deliverables and resort to exploiting subjective psychological biases of superiors to win money & status increases. Anything that holds them accountable to the actual measured result status of a deliverable can’t be tolerated.
Scrum is just the by-product of corporate status-seeking evolutionary war games against in-house scientific thinking.
Agreed, it feeds the existing power structures. And it did hit at the right time when over-communication is not only appreciated but everything else is seen as intransparent and potentially suspicious. "You don't even know what your team is doing?" - is the best way for middle-management to pick on each other. So in order to never get into that situation daily standups, slack, jira, basecamp all these "transparency tools" are welcome as they offer defense for this manager.
Does anyone know if there is an anti-movement to that over-communication trend? Something along the lines: "The only way to go fast is to go well", * Plan * Honor the flow * Adjust
yes, I too was involved with XP before it was under umbrella of Agile which got consumed by the Scrum borg.
I feel we kind of lost something where creating code/designs using strong feedback loops was a "first class citizen with royal honors" of the software development process. Scrum kind of makes it a sideline issue, a peon of middle management
Totally. Although this has long been a problem.
My dad started developing in the late 1960s. He did a lot of in-house software, mainly manufacturing and insurance systems. His approach was to go out and see something in use, talk to the people using it, go back to his office, write something, release it, and go talk to the people again. He'd do that loop in as little as a couple of days.
Today we'd call that obviously agile. But both back then and now, higher-ups wanted more "control", by which they meant feelings and appearance of control. Which caused longer feedback loops. Which made things more out of control, which required more process and even longer feedback loops.
I think we have the same struggle today, and I have no idea how to get out of it.
Agile has a problem with introducing Processes and Tools (by definition, as the blog post describes). However, Scrum offers a solution to this problem by creating an Agile space or bubble within those organizations that are still stuck with the traditional methodologies (Waterfall, etc.). So Scrum is a nice way to introduce organizations to Agile concepts but it is certainly not the final goal.
IMHO, the idea is that over time the organization will adopt new Agile practices and not depend on Scrum anymore. In practice, however, many organizations think they are Agile after they introduced Scrum, which is a simple fallacy. After all, becoming Agile is about a culture change and not about the introduction of a new process framework.
I don't find much constructive value in your comment. To me it seems like a politely worded screed about the incompetence of management and organizations.
To drive the point home I could easily transform "[o]n average, the first priority of managers and execs is maintaining the power structures that make them a big deal" into a derogatory comment about engineers, fad chasing, and resume padding by a simple word substitution. It would be neither accurate nor fair to many (perhaps most) engineers: just as your comment isn't really accurate or fair to managers or organizations.
I have been both a manager and an executive. As well as a consultant in many companies both small and large. I'm not saying that those people are necessarily incompetent. I'm saying that, per POSIWID, the purpose of most software organizations is not effectiveness in making software (or in serving users via software).
That's not constructive in the sense of providing ways to solve the problem. But that wasn't my goal. My goal was to support and confirm the article's thesis. I have also written plenty of constructive things on software development, but I think it's absurd to suggest that every HN comment must contain that kind of constructive feedback. (And if that were a reasonable standard, you aren't living up to it.)
> On average, the first priority of managers and execs is maintaining the power structures that make them a big deal. But true Agile processes are about empowering teams to self-organize around serving users.
But that doesn't explain the success of Scrum, because Scrum does away with the power structure of managers, and should empower the teams. "We're doing Scrum now" is the big stick I see scrum masters use to beat back Business managers to keep them from interfering with and micromanaging the team.
At least, that's what they should be doing when management interferes too much. And then address this higher up with whoever decided that we're doing scrum now, and explain to them what that means, if necessary.
There's no excuse to tolerate micromanaging managers in a Scrum process. They don't belong there. Kick them out.
At the company I work at, we have the following scrum anti-patterns. I wish I knew, whether we could "do scrum right" or just move onto something simpler (fta; priority queue)
* Daily standup, nobody wants to be at. We have multiple teams arrive, with roughly 20 people in a small room. Some people stand, some people sit. Sometimes the front-end team goes, sometimes the back-end team goes. Its limited to 15 minutes, so nobody says much of importance, and just parrots what is already on the Jira board.
* Scrum master driving process above scrum. The scrum master's role is to make sure scrum rules are adhered to. But at this company, any deviation from scrum itself poses a threat to this person's job security, so it doesn't happen. More high-level processes are not optimized because the scrum master defines everything.
* Technical debt. I see this time and time again. User Stories are supposed to be forecasts, not commitments. But the business doesn't like stories carried over, so they become commitments. At the end of each sprint, everyone rushes to get their stuff done, and hacks are implemented to meet an arbitrary deadline. Many times I want to begin my work by refactoring something to what it needs to be first, then do the actual user story. But its risky because the refactoring might take more than the allocated story points, and you get dinged. So I do the story first, and if there is time do the refactoring but it almost never happens.
* Poor product owners. During maintenance phase, we could work on cleaning up technical debt, but this value is not appreciated by business so they keep us busy with bikeshedding. One gripe I have about scrum, is there is nobody representing engineering, as the product owner represents the business.
These things combined have dragged down the happiness of the people I work with, but we all feel imprisoned by it. I have a stack of scrum books here I plan on reading, I figure this process isn't going away and I need to up my game with how to play it - but I wish I could use something else, perhaps kanban.
The way I decided to play the game was like this, "I won't be attending the Daily Standups anymore as I don't think they add value and do subtract value."
The project mgmt response was, "Attendance at standups is mandatory."
Regardless, I didn't go to anymore standups and when I got flack for that, I stopped going to the office all together. When I got flack for that, I stopped working all together.
Then I got fired. That solved all my scrum problems.
I got criticized for zoning out at scrum meetings. So I watched what the managers do. They show up for the first few minutes, look alert, then leave as if in a hurry. I started doing the same thing, and never had a problem again.
If you do this often enough. People will assume you are busy if you don’t show up at all.
Don't forget to always roll up your sleeves and carry around some papers.
If you are in a paperless office, tapping a tablet with a stylus as you walk confers the same office space prestige.
That seems to be secret for managers anyway. Always appear eager and motivated and then run away.
Because what do they all actually do? They spend a large portion of their time waiting around for numbers they add[0] to a spreadsheet and report those numbers to their manager.
That's it. (Ok, once a year they do reviews.)
[0] If they don't bug a dev to do it.
Great story from start to finish. I kept waiting for a happy turnaround but that ending was perfect.
Yea, same here.
This pretty much sums up my experience with Software Dev at one of the Big 4.
Thank you for giving me a really good laugh.
Attrition!
Underrated post
I feel like you work at my last company. But so many companies f* this up that it could be any company.
I got fired from any last job after being one of the only engineers willing to say something to management about this. I figured I had a popular opinion - many other engineers often DM'ed me on slack encouraging me to continue speak out, including my own manager, so I somehow figured I should be safe speaking out, politely, as a respected majority representative.
In fact I was dead wrong. By saying that scrum was a problem, I ended up making product managers / CTO who loved scrum and used it as an hour long opportunity to lecture the team every day feel threatened.
Getting fired was one of the best things that ever happened to my career, because in retrospect it was a dead end company that was run by fear. Management was afraid of ideas or challenge to power to the point where innovative ideas and feedback were never well received and a culture of fear and not stepping out of line arose, even though it was sugar coated with fake company values of "openness" that nobody really believed in but that managers loved as a way to elevate themselves.
I suspect many companies are a my least a little bit like this. Unfortunately it is human nature and there are so many stories in history, usually of narcissistic dictators, that mirror this.
This was a very extreme example but I'm sure many other companies struggle with this when it comes to challenging scrum. Managers tend to love scrum because it's a chance, daily, to "manage" and to get status updates so that they feel comforted. It makes it so that trust is no longer necessary, and bad managers are often bad at trust.
I guess another way to view scrum is to realize that it is often a reflection on management's subtle fears and insecurities being projected onto the processes of the company. It is an instance where employees unfortunately must manage up.
> Daily standup, nobody wants to be at. We have multiple teams arrive, with roughly 20 people in a small room
It is central to the idea of the daily standup is that it is one team.
> Its limited to 15 minutes, so nobody says much of importance, and just parrots what is already on the Jira board.
If you have another mechanism for sharing what each person has done and is doing, then the standup should just be for sharing information about and resolving barriers.
> Stories are supposed to be forecasts, not commitments. But the business doesn't like stories carried over, so they become commitments.
If you organizationally can't get away from commitments, then you need to cut back the amount you plan to do so that there is some slack. As long as you are also doing backlog grooming and have items groomed beyond the current sprint commitments, you can take additional items opportunistically, if there is excess capacity after doing the committed items properly.
> As long as you are also doing backlog grooming and have items groomed beyond the current sprint commitments, you can take additional items opportunistically, if there is excess capacity after doing the committed items properly.
The powers that be at my work decided that “sprint predictability” is the most important metric here. This means that if we pull in stuff near the end of the sprint but don’t finish, predictability goes down. Thus, we are encouraged to not take on extra work if we don’t expect to finish. What this means in reality is that we start working on it without pulling it into the sprint and then get a head start for next sprint.
What’s your take on that?
My (related) experience has led me to believe reporting to management should be strictly separate from any tools or processes used to track actual work, specifically to avoid dumb shit like this. Team task-tracking should be a team communication tool. If you want to report task-related stuff upstream it should be totally separate, with updates/sync done by a human (project manager would make sense).
I also think speculative future make-believe planning junk should be kept out of the real getting-shit-done tool. Put it in there when it actually matters. There should be no "on ice" or "phase 3" (when in "phase 1") crap in the tool the devs use. It may or may not make sense to have them involved in planning stuff that early, but keep the output of that planning away from the real-work trackers until it's closer to time to do it.
I think that's part of why Jira (and similar, heavy, report-focused tools) is so god-awful. It not only tries to have features for teams to get stuff done and for five other roles to dig around in tasks and reports for whatever reason, it enables and encourages that, and I'm fairly sure, at this point, that it's fundamentally a bad idea.
> My (related) experience has led me to believe reporting to management should be strictly separate from any tools or processes used to track actual work,
This probably increases the accuracy of the work trackers but increases the B.S. level of management reporting in organizations where there are fundamental problems between management and the working level.(Which is the only reason anyone would want to separate these two things.)
While this probably seems like an improvement from the working level, it's probably bad for the organization. The solution is not dis-integrating communication tools so that management's view is not connected to the actual work tracking, but dealing with the fundamental trust issues. Which is hard, of course, but things that are important often are.
I get why they want it, but I think it's a case of false improvements from adding more computers. If the reports to management are such BS that they don't usefully resemble reality they should be able to figure that out before long and sort things out, one way or another. If they want insight into the tools the bottom-of-the-ladder workers are using to coordinate then that coordination will suffer, greatly, and those tools themselves will be full of BS, for sure, gaining little aside from some pretty "real-time" BS graphs for management and worse communication for workers.
[EDIT] more to the point, I think if it worked we wouldn't still constantly see management surprised when things aren't delivered "on time", and yet, that still happens all the time. IMO the team needs someone "on their side" to report reality upwards, diplomatically, not their own work tools reporting up directly, or they'll lie to their tools, which leads to more surprises, not fewer.
There's certainly environments where that makes a lot of sense, particularly where the software at issue had immediate effects on external users who require advance notice of changes (I actually work in that kind of environment, and though we do have opportunistic items they generally are restricted to items that don't impact external users.)
What answer are you expecting? Your example indeed makes no sense, but it has nothing do with scrum and has everything to do with clueless people do stupid things. Do you expect this guy will suddenly do only sane logical things when using something other than scrum? I expect not.
If you're trying to do scrum properly, you shouldn't be consistently finishing work early. That implies you're overestimating stories.
When you do finish a bit early I've not normally found it a problem to find some small bits of tech debt, research, or admin work to fit in before the end of the sprint.
Starting work on stuff from the next sprint is not ideal because it'll throw your estimates off for the next sprint, plus you don't actually know for sure what is going into the next sprint.
I passionately hate being asked for "commitments". If it's stuff of any reasonable complexity or novelty I will have no idea how long it will take and therefore can't make any commitments. The only thing I can commit to is to make sure that people don't waste time and work towards the goal. The problem is that management has no problem wasting a lot of time with useless meetings or not committing to the final feature set but instead changing requirements all the time.
Even better is being told to commit by the Scrum Jesus (who does not even try to understand the business domain or engineering issues, since he's getting hired for being the Scrum Jesus) to stories with no one available for even defining or discussing them. Our story might have been just a word someone saw at a trade fair.
Then, two weeks later, being asked why I did not complete what I committed to.
In that situation I've gotten a lot of mileage (and quiet amusement) from pointing out that Scrum dropped the term "commitment" for "forecast" in 2011. Scrum Jesus has no defense against Scrum itself!
"Development Teams do not commit to completing the work planned during a Sprint Planning Meeting. The Development Team creates a forecast of work it believes will be done, but that forecast will change as more becomes known throughout the Sprint."
https://scrumguides.org/revisions.html, "Changes between 2010 and 2011 Scrum Guides"
If you can say more than a minute, but less than a decade, you've already got SOME idea of the timeframe for a task. Granted that sort of estimate would get you called to HR for insubordination. But can you pull the bounds in from either direction at all?
I say this as someone just moving into project planning and management. From that perspective you start to see that some level of estimation s critical. My preference is to ask devs for just the tightest bounds to timeframe they're actually comfortable with, then estimate based on that and pass up the chain. I consider it to be on my shoulders if things take longer than I communicated upwards.
I understand that estimates are important and I give estimates. But recently in my company it’s fashionable to talk about “commitments” and I won’t give those. In my view this is just a way to get overtime and weekend work out of people once things fall behind so they meet their commitments. I commit to do my work as best as i can but I can’t commit to work overtime because of issues that are often out of my control.
That makes perfect sense. And I applaud your push against a practice that does seem to be pushing devs for unpaid overtime. You're right.
One manager I’ve worked with had the brilliant notion of “x2+1” time of what the dev says.
Anecdotally this has worked out remarkably well throughout my career - from single dev to cto - even stuff that I _new_ was going to take for example 2 days, if done properly ended up in like 5.
Didn’t matter if I was doing the estimate or someone else. At some point I just gave up and started doing the “my gut says 3 hours, so it must be 7 hours of work then, and send that up the chain. That cheeky “+1” had saved my ass more times than I can count.
One of the points of Scrum is that you don't estimate hours at all. You estimate intentionally vague and abstract "story points". You don't need to know what that is in hours, just try to make sure that two stories of roughly equal complexity have roughly equal points. Run a few sprints, check what velocity you actually end up on, and you've got the averages you need to give a slightly more long term planning.
But without the feedback of actually having done the work, any estimate is bound to be wrong.
I also generally have had good success with guessing a number and then multiplying it by 5 to get the time when it’s completely done, tested and documented.
I remember sitting in a meeting with the team discussing how we regularly failed to meet the our estimates and how this was bad.
I made exactly this suggestion and was laughed out of the room.
Granted this was for large and small estimates, but I still don't think it's the worst idea.
Changing requirements is fine, as long as they recognize that stuff like that impacts the schedule.
Generally they don’t though.
> One gripe I have about scrum, is there is nobody representing engineering, as the product owner represents the business.
That is definitively not be the feeling you should have. In "proper" scrum the Team is supposed to represent engineering. If you, for some reason, don't feel like you can represent these concerns then that is a huge issue. The PO shouldn't be your boss either. At least it isn't so in the company I work for. It sounds like the PO is the boss of that team, if he "keeps you busy". I am super grateful that our Scrum Master knew what he was doing and suggested creating a separate line on the Org chart for the POs so that they are explicitly not in charge of development. The team is in charge and POs are more advisory.
Sounds to me like there are some deeper problems in the company then just the scrum. There are so many red flags in your comment. I don't know if those books are going to help you "play" scrum better, because what you guys are doing does not sound like scrum at all.
Kanbans probably also not a solution. I really like Kanban, but it isn't going to remove the bad elements that seem to control that company.
Yeah, that's an odd complaint. You are engineering, are you not? The meeting is already about engineering. If you have more engineering as an outside stakeholder that the PO isn't able to represent, by all means bring someone in to represent them.
this.
In scrum the dev teams maintains a technical backlog and fills the sprintbacklog as THEY see fit- from the product backlog AND the technical backlog.
the PO has no say in this! he can complain that he thinks not enough features from his product backlog are planned, but should trust that the team knows what they are doing.
First of all, most issues with process issues are more a reflection of the organization and what drives them than the process. I would say you should start talking about, 'Our process' instead of 'Scrum'.
>These things combined have dragged down the happiness of the people I work with, but we all feel imprisoned by it. I have a stack of scrum books here I plan on reading, I figure this process isn't going away and I need to up my game with how to play it - but I wish I could use something else, perhaps kanban.
Spoiler, your Scrum books aren't going to help you figure things out, unless your org actually wants to change. You have your own unique process, but that you have a unique process isn't an issue. The issue is that your management seems to not respect the input of engineering. No process will work the way you want as long as that is true.
Some thoughts, * Most critical, your team is way to big. Should be half as big
* Scrum master should be a team member and rotated along the team. The process should be a result of constant tuning via the Retrospective. The scrum master here, should lose his job.
* Engnineering owns the implementation of the stories, product owners should not be a part of the conversation on how to do something, they only can say 'what'.
* If you can't decide your own points, then again, you have your own process. If you can, and you need to refactor, you need to add points for that.
If you can't do anything to change, then again reading scrum books isn't going to help. You can say, 'this isn't the way your supposed to do this' as much as you want. processes always reflect the organization. Unless the engineering team is an equal to the product team, it won't matter what process you have.
> Scrum master should be a team member and rotated along the team.
I don't agree. For us the scrum master is a fixed position who has as a responsibility to advise the teams on how to fix problems with their processes or to help build these processes in the first place. This works very well for us, because the scrum master is explicitly only an adviser.
Meaning I, as a team lead, do not have to do what the scrum master says, but I always have someone experienced with scrum who I can ask when problems arise or when the team wants to change the processes.
However, many companies face similar issues. For a moment I thought the OP worked for my company because we face the same issues (that's not possible however because in person stand ups are banned at our company due to the distributed nature).
1) While I agree that 20 people seems like too large a team in general, I have a fundamental problem with the idea that scrum can determine what size team is right.
What seems to be happening, it appears to me, is that we have a fair idea of how large a meeting can be before it becomes unwieldy, and scrum has a daily meeting as it's fundamental component, so basically decides that for it's fundamental component to be effective there shouldn't be more than 8 people so therefore teams shouldn't be larger than 8 people.
That seems backwards to me.
2) Almost every scrum training usually has a separate scrum master. I do think the idea of a team member being the scrum master makes a lot of sense and would probably alleviate a lot of the concerns.
3) Scrum story points are supposed to reflect end user benefits. A refacorin's benefit will only show up after several sprints. In the meanwhile you hurt your velocity significantly because your refactoring is likely to be 0 points. Now, this actually makes sense to me, but the issue is that in most places management is carefully monitoring velocity and is gonna hold the drop against the team.
To be fair though, I think this is another issue where the real problem is management converting a measurement metric into a target, which basically negates any use it may have as a metric.
My biggest issues with scrum are: 1) A daily standup is ridiculous. It's highly disruptive and gives me the feeling of someone constantly peeking over my shoulder. A better strategy is a simple email when someone has an update, or a blocker to the rest of the team. Very few of my projects are done in a single day, and it's psychologically stressful to join every meeting saying I worked on story A, will continue working on story A for 3-4 days in a row, while the managerial types rattle off a list of meetings they attended, emails they sent, trainings they completed, etc. 2) Sprints are far too inflexible. Each story should basically define a new Sprint for the people involved in that story, which may or may not be longer than originally expected. I don't see the point of having a fixed 2 week period that applies to everyone. Why not just have a weekly or biweekly meeting where you go over everyone's stuff and see where you stand. It's basically a more flexible version of what sprints try to achieve, without the psychological issues created by having to fit things into a Sprint or alternatively having to carry it over into a whole new Sprint. That same meeting can track how many points were earned since the last meeting and you have your velocity being tracked as well.
Retrospectives are great. But they should be included in the same 2 week catch up meeting.
I find scrum tries to create a lot of artificial deadlines, possibly to encourage people to break things into smaller pieces and make regular progress, but I find that's not how people usually work. In my experience people tend to work more in spurts, delivering a ton of features and bug fixes over a couple of days, and then going relatively quiet for the next few days. Not because they aren't working, but because thats just how things usually pan out since software development is a decidedly non linear process.
> Scrum story points are supposed to reflect end user benefits.
That’s not my understanding at all. In our org, that’s the role of backlog priorities. Story points are the cost to get there, and are explicitly a tool to estimate relative amounts of effort involved.
If tech debt is to be incurred, it is expected that engineers negotiate that, and that other estimates will change as a result (and/or other tasks be created to track that debt). But then, nearly 100% of the managers in our org are, themselves, engineers or SMEs in the field they are managing, so they have realistic goals and understanding of the sausage-making process. What you’re describing really does sound more to me like a management problem than a process problem.
Sometimes I find myself so stressed out by the standup, I basically doing nothing the rest of the day. Also I have a habit now to get up early and try to do yesterday's portion of work in an hour or two before standup just to report it. I completely stop thinking in time periods wider than sprint, and I thought before that was my strength. Worst thing, the process forces an idea that my struggle with it is, basically, my fault. I hate "scrum".
You are not alone. Meetings just suck all the energy out of me.
I think the scrum team size is determined in part because communication overhead is exponential, getting beyond 6-8 people means the whole team has to spend a lot of extra time coordinating. This is true regardless of whether you're dong scrum or not. It's true some teams may be correctly sized at 20, but I would suspect that breaking that into 2-3 other teams would be a benefit most times.
Story Points represent effort and uncertainty, not business/customer value - those are Business Value Points. Devs are only committing to X story points per sprint. If refactoring is needed, it's built into the story points, or added as it's own task. And as others have said if the Product Owners don't take engineering input on paying down tech debt/infrastructure/internal tooling, then you have a broken company no matter what process you're following.
> 3) Scrum story points are supposed to reflect end user benefits.
Scrum.org states[0] A Story Point is a relative unit of measure, decided upon and used by individual Scrum teams, to provide relative estimates of effort for completing requirements.
[0]: https://www.scrum.org/resources/blog/why-do-we-use-story-poi...
According to Jeff Sutherland himself[1], story points are based on team effort and not end-user benefit: Estimates are estimates for the team to get a story done.
[1]: https://www.scruminc.com/story-points-why-are-they-better-th...
Eh, the idea that a team should be no larger than the number of people that can meet together is not unreasonable, if also not incontrovertible. What makes a team a team if they can't meet together as a team effectively?
I'd say the most important characteristic is whether the team can work together effectively.
As an example, one could consider the various people working on an individual open source project as part of a team. And most successful O/S projects rarely require everyone to meet together at the same time.
I personally strongly prefer small teams (I find even 8 members too large), but arguably certain projects may need bigger teams, and I think they should still be able to practice Agile with the bigger team, which scrum doesn't allow for.
Thanks everyone for correcting my understanding of story points (which was based on scrum training provided by my current company).
Effort makes a lot more sense, and I particularly like that it tries to incorporate uncertainty as well (I guess that's why they chose story points, to make it vague, since using hours leads to people treating estimates as deadlines instead).
> Daily standup
I have never worked at a company where this actually goes right and could not be replaced by [insert ticketing system here]. We use Basecamp to do daily check-ins on a very high priority issue, if we have one, which we usually don't.
Managers are the only people that like daily standups, so far as I can tell.
I've never figured out why you couldn't just do it over slack or equivalent. It's massively inefficient to get everyone to meet up in the morning.
Managers need to justify their jobs. Reading off a slack report every day seems too simple and not worthy the salary they're paid. Compare that to a large meeting and live status updates, everyone stands, etc. Now that is some "serious work" happening.
Think of it from the point of view that managers have mangers they report to. When shit goes wrong they have to cover their asses. "So it's late. But did you check on their progress? Wait just over slack, no actual meetings? Ok, that obviously needs improvement".
It's not like it's a requirement that "managers" be at a standup meeting. Some of the best teams I've worked on have done daily standups without managers, and I dare to say most of us enjoyed them and found them productive!
Managers are often looking for ways to fill their calendar. Meetings, calls that could be a two-minute Slack exchange, et c. Some will hop into meetings like that if they can. They love the recurring ones, especially, bonus if they can apologize any time they have to skip it for something that actually, maybe, matters ("uh, yeah, we don't care if you come, OK, thanks for apologizing I guess?")
I have never been in a team where managers were present at the standup meeting. In fact, sometimes I have no idea who my "manager" even is. We've got a team, we've got stakeholders, we do the work. If a manager wants to join, that's fine, but he's not in charge of that meeting.
All this talk of standup meetings being primarily for managers is really weird. It's not for managers, and they have no reason to be present. It's for the team, to improve cooperation within the team. If the team has a better way to accomplish that, then they should use that.
Why are your managers attending stand up? The Daily Scrum is the team's meeting. I swear Scrum is only fragile because no one reads the bloody book, or if they do they skim it or ignore it.
From the scrum guide... "The Daily Scrum is a 15-minute time-boxed event for the Development Team."
It is the team's event, if there are other people there it should be at their request. The guide goes on to say that.. "If others are present, the Scrum Master ensures that they do not disrupt the meeting."
If something is self-organized, there is no need for all this ceremony and bullshit that comes with capital-A Agile or Scrum. You just shoot the shit and talk about what you're doing.
Stand-ups and all the process attendant appears to be universally imposed from on high as a micromanagement technique, or because they were sold the idea as some kind of silver bullet by a talk or a snake-oil consultant.
I agree. Scrum is training wheels. If your team is mature and knows when to talk to each other then awesome, let the team do what works for them.
I feel like Scrum is for teams that suffer Stockholm syndrome from poor management and need to be taught how to human beings again.
I think Scrum provides a good baseline of working for new teams, but ultimately the team is free to shape it however it works best for them.
The main value of Scrum as an official process, is that it gives you a stick with which to chase toxic management out of the room. They're not sufficiently Scrum if they crash your meetings and demand to be in charge. Though in my experience, most companies that do Scrum have a management that keeps their distance unless invited.
We do it in person on MWF and slack TTh. I find slack updates to be both more effective and more efficient. It is obviously why it is more efficient; but it’s actually more effective because unfortunately I am one of those many people who simply drift off when in a meeting, and putting updates down in writing helps infinitely.
My current place does it on slack and has a standup. Oh, and everything is also logged in jira tickets. Makes little sense, in my opinion.
Many places do it over Slack.
I like standup on my current team. We're in an open office already, and just (physically!) stand up and say "I'm doing X today". Whole thing never goes over 2-3 minutes, and it's a friendly way to keep everyone up to date with the bigger picture.
It's similar at my office. We also use it to let the rest of the team know on occasions when you're blocked on something.
Why wait until a meeting to air that you are blocked, just sort it out when you become blocked. Where is the agility?
Generally if things can get unblocked as you go, that's better, and so this isn't particularly common. But sometimes you need your manager to go kick down a door for you and it's a handy time to let him know. Sometimes it's more benign and it's just easier to ask the group in person when everyone's together sipping their coffee together than it is to fire off a bunch of Slack messages or a group email or whatever.
And to be clear, when I say "blocked" I'm not talking like "I can't get any work done until X", I'm more talking like "this particular avenue needs X to happen before I can continue it, so I'm shelving it for now and doing something else until we can get it resolved." If you find yourself in the former case, something has already gone horribly wrong that should probably have been resolved days ago.
There are different ways something can be blocked. It's possible something is too complex to wrap your head around and you need an extra set of eyes and brains, or it might be blocked by outside impediments. It doesn't really matter, all obstacles need to be addressed and solved somehow.
Exactly. I've always understood Scrum to be moderately successful at getting usable work out of terrible developers. Companies that use it should be avoided. If your employer introduces it it's time to quit.
Having dedicated communication time allows everyone to plan their dedicated focus or "deep work" time.
This is a very important point that's often forgotten. You shouldn't wait for the appropriate meeting, but address it right away. The next stand up meeting is more of a backstop. It's the time to admit: "I guess I really am stuck here. Can anyone else look at this?"
Smoothly running teams don't need this, but many teams don't run that smoothly, and then it's better to address it at the stand up than not at all.
From what I understand, that's the end goal, with daily standup being a way to break the ice and get people who like to silo themselves off to actually ask instead of making no progress.
I think this gets at a very common problem with managers in the hierarchy. They forget that iteration of processes is crucial to good agile process. Your goal isn't to adhere to "true scrum." Your goal is to tailor scrum to what your team needs.
If your daily stand-ups are just repeating the JIRA board, then just give people back their time. If your features are coming out badly designed, then stick a designer in the engineering pods, or call out the design phase in project lifecycle.
If your projects are consistently late, then... slow down the estimates!
Of course teams that do it right tend to know this, but the bad implementations of scrum I've seen consistently lack attentive iteration on the process itself.
Are you in a position to effect change? If not, leave if you can. If not and you can't leave, suck it up.
Otherwise, here's a couple of notes freely given with no claim to excellence except that I've worked with successful scrum teams and unsuccessful ones:
* The 'business' is everything. All code is liability. Code that can't do things the business wants to do and hinders those things is a particular cost to the business. Product Owners who don't understand that "refactoring code to make it extensible before extending it" is the cost of doing business just have to learn that lesson. If they won't, let them go. Bad ones are a dime a dozen so you can always hire another clueless one if you want. It's like having a loan that you're paying lots of interest on. Saying "We can't spend time refinancing that. We have to go earn money." doesn't make sense as some absolute truth.
* Engineering is not in contrast to the business side. There is no need to represent engineering because the product owner is supposed to be able to understand all inputs into the program. If engineering is slowed down by the lack of dealing with some technical debt that's not some engineering problem divorced from the business. It's the business. This requires reaching across from the engineering side, being clear when things aren't going well, not starting on ridiculously long "refactoring" projects that are just lateral changes, and being clear what the expected outcome of any tech debt relief is. Avoid "feels cleaner", "is more elegant", "is more extensible". Talk in terms of outcomes: faster response to outages, the ability to add things like X (requires you to understand the product some, which is something an engineer should do), less time spent debugging issues. If trivially measurable, measure and demonstrate improvement. Builds trust.
* I'm not too convinced "Scrum Master" needs to be a solo job for anyone. The incentives are skewed. If you have a team of ideal people and they all pick up on the principles and you correct the bad patterns they have, they won't need you. The job inherently obsoletes itself which means the median person taking the role will entrench themselves in process to keep themselves relevant.
* 20 people at daily standup makes no sense. It's 10 minutes or 15 or whatever so you have 30 s to describe something. It isn't so much the time, though, but the fact that there are too many people here together in the room. It makes no sense. Find smaller units of organization. Conway's law presents but the alternative is much worse.
Standups can have their use but I can't imagine a 20 person standup being useful. Why do you bring so many teams together for it?
> User Stories are supposed to be forecasts, not commitments.
Indeed the commitment is supposed to be with the sprint goals. I've seen this vital aspect of scrum getting overlooked too often.
It sounds like you're missing probably the most important part of agile which is "at regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly".
In Scrum, it's retrospectives.
Continuous improvement is a theme that runs across agile, scrum, lean. Without giving feedback, how are you going to improve?
> nobody says much of importance, and just parrots what is already on the Jira board
Worse, the moment anything of interest gets being discussed, SM stops it, saying "let's take this offline" (in German-speaking projects at least, where there's a tendency to use English words to really say nothing at all). Which brings us to
> any deviation from scrum itself poses a threat to this person's job security, so it doesn't happen
MBAs have identified and captured the role of a Scum master (sic!) to comfortably represent a project towards management, without the responsibilties of a project manager (I'm just moderating the project), yet with an instrument to turn time-based freelancing and employed work into a fixed-price deal through commitments/forecasts.
Standups with multiple teams, or simply very large teams, are a terrible idea. I'm currently in a team that has very quick, fun and productive daily standups (despite not even having a good standup space for it), but we have only 3 devs and a handful or other people involved.
But I've also been in projects where standups drag on and on and on, and they're hard to get under control. Usually, those projects had much larger teams.
I suggest smaller teams, and if you have this kind of multi-team standup, discuss with your team to have your own independent standup instead and send one representative to the big one.
> Daily standup
Change them so that people talk about what is blocked then.
> Scrum master
Tricky, like anyone who doesn't want to adapt.
> Technical debt
Create tickets for debt, put them on the backlog.
> Poor product owners
Tricky.
> Create tickets for debt, put them on the backlog.
That will risk making a huge technical debt section in the backlog that will find it hard to get priority. It's better to condition new features/bugfixes on the cleanup that caused them. "I can add this feature in 1 day but will need 4 days to clean it up. So my estimate is a week." I constantly nag developers to do this when they find that they are pressed for time and want to make a hack.
-"I'm nearly done but I can't find a nice way of doing this because of an existing code smell so I'm considering adding another smell"
-"Did you ever find that this was NOT the case?"
-"No..."
-"So do you always double or triple your estimates so you know you can comfortably refactor"
-"No..."
I think this is a disease of Scrum. Too short estimates exist because developers pat themselves on the back for finishing things quickly.
That will just lead to stories to change something simple the color of a button taking 3 days because you hit a tech debt minefield refactoring. You don’t want someone looking at JIRA and seeing it took you 3 days to change a simple button color.
Keep technical debt in separate tasks to make the tech debt visible and to keep management accountable for not making efforts to get rid of it.
This is terrible but has nothing to do with scrum. Poor management won't magically disappear after adopting scrum.
The best place I've worked at as a developer essentially had no process. It's also the place I really saw the benefits of having good managers (or a project manager, but here the manager took this role).
Essentially the "process" was:
- Someone gets an idea to do something, e.g. PMs wants to add a feature.
- The manager (maybe with the help of a dev) figures out which teams need to be involved (e.g. dependencies).
- Get a very rough estimate from developer. Are we talking a few days, a few weeks or a few months?
- Manager, PM and other teams get together to figure out priorities, schedules and who's likely to be working on it.
- Based on some discussions, areas of expertise etc, some number of devs gets assigned to work on this feature.
From here the devs know what they're trying to accomplish, what the constraints are and who to go to for questions (PM, design etc). They also know who else from other teams they're collaborating with and they just figure out amongst themselves how to get things done. They'll keep their managers and the PM updated on progress and any blockers.
Every once in a while manager etc have to step in. For example if some big that was previous unknown came up, there is a big risk, priorities need to be adjusted etc. But for the most part things just worked really smoothly.
Sometimes we need to give estimates and really hit it (legal or security issues, big marketing launch etc), but for the most part we were trusted to be doing things as quickly and efficiently as possible. So none of the commits, sprints and burn down charts BS. It's not hard to gauge develop's productivity based on output anyways. If there was something slowing the team down, we communicated the need to the manager and PM, and we worked on fixing it.
So for me, if you have good competent people that communicate well, you don't really need much process.
100% this. For a team of good developers this is by far the best model to get things done. SCRUM only serves to prevent a very bad team from doing even worse then they would do otherwise. However for a team of competent developers SCRUM is painfully stifling.
I think the above can simply be summarised as "real managing and communication".
As opposed to trying to create a couple of metrics because you don't really understand what people around you are doing.
Absolutely. I have nearly identical experience.
We are better off building over-all engineers that know what they are doing, than trying to catch up with the latest shiny thing (Scrum, Agile, Waterfall, Kanban, ...Jesus Christ)
I think the challenge with having no process is this: You have to rely on great chemistry and raw talent.
Look at Spotifys videos on their process. Every team is self-driving, because everyone knows how to do stuff. If you work at a company where there is no process, and not every is able to complete the whole task from A-Z, you end up with deadends and people who get stuck. If the culture doesn't encourage knowledgesharing, people end up making crappy solutions.
But how did you ensure that you udnerstood the requirements if you were potentially just left to it for a few months?
At the end was there ever an instance of the client not using what you produced because it wasnt what they wanted?
You have direct access to the PM and work closely with them. If you weren’t clear on something you ask, if you don’t agree with something you debate it out.
Like anything else there is no silver bullet. Sometimes you got the feature right, other times it misses the mark.
Not saying that this is not a good way to go, but I rarely see this happening in large complex projects, especially those that are on their way to delivering v1.0. v2.0 and beyond have a better chance of these types of "process" to work where the product evolution is not so volatile.
In my very surface-level, western understanding of Chinese philosophy, Agile seems very much like Taoism and Scrum is like Confucianism, in several ways.
The Agile Manifesto describes a set of ideals but gives no true set of instructions to follow, to do so would not be the Agile way. More than anything, it prescribes an attitude around which you should generally approach things.
Scrum conversely gives rules for how things can be organized and executed in a functional organization. It gives little room for flexibility. Any problems you have with Scrum, the first reaction should be "how are we following the rules wrong?".
Similarly, Tao and Agile are ideals for the individuals (and small teams), Confucianism and Scrum are rules for getting things to work functionally within a society/company.
The comparison may be very apt. After all, a key tenet of Taoism is that the Tao that can be told is not the eternal Tao. It may very well be that an Agile development process that can be fully specified and documented is not truly Agile.
Somehow I'm reminded of "we demand rigidly defined areas of doubt and uncertainty!" but I'm not sure which side it supports.
Sounds like Theodore Sturgeon's description of the difference between morality and ethics in his classic "More than Human":
Morality is society's rules for individual survival.
Ethics is the individual's rules for society's survival.
There's a set of practices here in Seattle that I've come to call Skragile, which evinces the veneer of of Scrum, Kanban, and Agile, but embraces very little of the fundamental philosophies. The most distinctive characteristics:
- daily standups (often one per day per team so multiple per person)
- sprint based development cycles, often with retrospectives
- fondness for the "as an X I want to Y" story
- story points defined in terms of developer hours or days
- Kanban style "pick a thing to work on", even in sprints
- burndown charts
Seems nearly inevitable that when anyone here says "we need more process" they're looking to have more of one or more of the above. I essentially never hear anyone in project management say anything like "we need to build more prototypes" or "we need to streamline our deployment process" or any of the other concepts that would actually characterize a (lower case a) agile development process.
(edited for formatting)
This is pretty typical elsewhere too (midwest US as a data point).
Its popular because managers can implement it. Agile- the manifesto version- was intended as a way to re-introduce the notion of discipline that newer generations of programmers had lost (where previously they were engineers and scientists who wrote code, now people who code exclusively eithout other backgrounds).
It feels like we have come full circle to where the new generations of programmers have delegated (or lost) all discipline to managers' Scrum plannings.
Managers and "product owners" aren't engineers or programmers. They shouldn't be expected to "manage" a developer's day-to-day, and yet them presiding over sit-down-stand-up meetings and demanding points and determining deadlines and features is precisely what it has come to. We are right back where we were when the manifesto was drafted.
The irony is, when I first saw Agile/Scrum, I thought of it mostly as a tool to provide discipline further up the chain rather than down.
User stories? Oh, a neat way to keep requirements general and open-ended so we can properly address how we're actually going to solve a user's problem. Surely it'll prevent PMs from over-specifying requirements that lose sense of true objectives.
2 week Sprints? Cool, estimates are hard, and now I never have to estimate more than 10 business days worth of labor.
Retrospectives? Great idea, we can finally do proper post-mortems and knowledge-sharing!
Scrum Masters? Wonderful, there's someone whose dedicated to running the process and making sure we have everything we need!
What I wasn't anticipating:
User Stories? But what about critical requirements that need to be prioritized that don't fit into "As an X I need Y"
2 Week Sprints? I now have so much technical debt a repo man is confiscating my laptop.
Retrospectives? This is always going to be 100% about how we didn't estimate correctly and not about far more important matters like: how these features didn't help our users, technical knowledge-sharing, and ticking-out technical debt.
Scrum Masters? Oh, you mean Project Managers?
My favorite team experience was scrum done "right". Standups were run by a scrum master who was the most experienced engineer on the team- programming longer than I've been alive type of guy.
Probject managers and product owners were kept at arms length in the sense that they didn't dictate how, what or when we did what we did. They translated the business requirements and timelines into something we could understand and react to.
Somehow, all of that turned into two week sprints that felt sustainable... At least until the company bought out another company, and everything went downhill. That, however, is a story for another time.
The most salient truth about power is that it cannot be disciplined, except by greater power.
Here's the thing: I have been lucky enough to work with engineer-founders as my bosses for most of my career. You know what the downsides are? None. It's just pure awesome.
It's getting to the point that they're the only people I want to work for.
you just described perfectly how scrum works at my company in Germany.
Managers, not understanding the difference between latency (how long each task takes) and throughput (how much work is getting done in total), always try to optimize for latency. The predictable result: throughput goes to hell, and then latency goes with it.
People who actually write software understand that you have to optimize for throughput first. Not to worry: latency won't be forgotten! But a primary focus on throughput will result in a clean codebase, that will maximize throughput and minimize latency.
The rationale is that if you minimize latency, throughput has to be maximum too, so optimizing latency is enough.
In practice latency is the goto target for optimizing actual processes. It's the most linked with all the risks. But software development is not an actual process.
> The rationale is that if you minimize latency, throughput has to be maximum too
Exactly. And it's quite wrong even without taking the growth of complexity into account — as every engineer knows, or should know. Getting every task done as quickly as possible requires a lot of context switching, which is murder on throughput. When you add in the effects of complexity growth (aka technical debt, though I think "complexity growth" is clearer) the disadvantages of optimizing for latency become that much more serious. And the worst part is, as the disease progresses and latency deteriorates, managers try to cure it by applying even larger doses of the poison.
This idea that managers optimize for latency, while I optimize for throughput, occurred to me only recently. But as I look back over the disagreements I've had with managers through the years (including disagreements over the usefulness of Scrum processes!), it's quite remarkable how many of them seem to come down to this.
There's pretty good theory behind the idea of minimizing latency to improve throughput. One of the better books on this is "Managing the Design Factory", and a followup called "The principles of product development flow".
These books are not about software development, but of product development in general. The first one actually predates agile, published in 1997.
Core idea is that minimizing the size of the tasks is the best way to improve productivity. Not getting it done as quickly as possible, but to decrease the size.
Remember that product development (new, innovative, uncertainties) vs product manufacting (repeatable), is not a new problem, and not unique to the software industry. There's a lot to learn from product development in other industries.
Another interesting read is "The Toyota Product Development System: Integrating People, Process And Technology" which talks about ways of making product development predictive, and less risky.
It's interesting to see how little software is used to improve the process of software development. Other industries use a lot of software (cad/cam, visual modelling, testing, impact analysis) to improve efficiency and quality of product development. Software for product development is a huge market.
It is basically correct, as the math does add up. As yourself pointed, if throughput isn't optimized, latency goes to hell. Thus minimized latency leads to optimized throughput.
The problem is that except on the bare minimum, latency is a bad proxy on development projects. So the idea is perfectly correct, yet it's useless.
There is no manager role in scrum, so not sure if this has anything to do with scrum.
I've worked on more than 10 different Scrum teams, and have seen it done well exactly once. When it was good, it was very good.
But we spent one entire workday (7 hours) on each sprint follow-up meeting, and then another entire workday planning the next sprint. That is what it took to write the stories, break them down into one-point pieces, prioritize with the PO, pass the stories out to the devs, etc.
Most places just don't plan for enough time to do things right, and quality is second priority, so no matter what methodology you use, you lose, because no cared about doing it right to begin with.
I'm on a team that started doing Scrum a few months ago, and we're still figuring it out.
To be clear: are you saying that the one team that did Scrum well did so because they spent more time on the process? Reading what you wrote, spending two full days every two weeks to plan sounds, well, terribly dragged out. Does it feel like the time was well spent, or was it a slog?
It did not seem dragged out, it was just thorough, and reasonably un-rushed. It was somewhat tiring, but definitely worth it.
By the end of the planning meeting, we had a clear idea of what we were going to accomplish, and a reasonable amount of confidence that we considered all the tasks that went into our plan.
Because when a story was, e.g. "add addresses to the clients page", it was broken down, discussed, thought out, and agreed upon. The moments when I discovered, oh shit, this story will actually take 10 hours longer than I had allocated, and now I have to stay until 10PM two days this week to meet my committment. Because our story points were approximately an hour each, and almost every story was broken down until there were no tasks more than 3 points each, 1 or 2 preferred.
It was all thanks to our scrum-master/project manager, who had actually spent a lot of time learning about scrum/agile/kanban/etc, read many books on it, and most of all, was committed to doing it right.
I think our sprints were a bit longer than 2 weeks.
I had a similar experience, where we would spend at a full day every sprint planning the sprint (team of 5).
During the project, it felt like a bit of a waste. We were spending a full 10% of our time on project management. However, looking back, I see that a) it seems to match up with other's experience, and b) it's pretty much the same amount of overhead for project management that the project would have had using any methodology.
At least with the 1-2 days every 2 weeks everyone sees it and it's something you can get better at. I'll take that over magical GANTT charts any day.
> are you saying that the one team that did Scrum well did so because they spent more time on the process?
Creating fine-grained, detailed user stories and making sure that everyone understand them and agrees on the prioritization is not time spent on "the process", it's requirements engineering.
I just read your comment again, and I want to add this to my other reply:
I would say the top, most useful, make-or-break practice that I would say is most essential to Scrum succeeding is a combination of:
* 1 point ≈ 1 hour of work
and
* no task above 3 points, try to only have 1-point tasks
Coincidentally, this process is what took the longest in our planning meetings, because the coders sat down and planned out the tasks, kind of the way you would in an algorithms course.
The payoff is that our estimates, after the first couple of sprints, were dead on, and there was very little "discovered work" mid-sprint. No midnight oil. And no corner-cutting.
I had the same experience. I worked in 2 different companies, in 4 different teams. And I can see the value of planning. In the first company we broke down our 2 weeks work into 3-5 hour long tasks. By doing that we had to know exactly what is the work needed and we discovered the work that we didn't think about at the very beginning. Plannings would take 1-2 days. It was exhausting but in my opinion worth it. In my current company we create 5-8 tasks for whole sprint and when someone takes it then they work on it for 2-3 days and discover more work to be done. At the end we usually don't make it on time. Even if we do, we spend about 40% of our time on bugs/improvements that we didn't think about. My personal view is lack of experienced scrum masters and developers who have never seen scrum working at least once xD Lack of management support who would push the process. My favourite: ("what?! you want to spend 2 days on planning?! I can implement it in that time!").
7 hours of meetings? did people show up the next few days for work?
Yeah. It was a bit tiring, but also fun. I think it helped that we all liked each other.
Whenever I've used scrum, the developers set the estimate and task breakdowns. We even had the developer _doing_ the work provide the estimate. It's not really fair to have someone else estimate your work, it doesn't breed commitment either. We did sanity check things - sometimes an estimate would seem big and everyone else would ask why. The answer was either we had missed something (typically), or (rarely) the estimator misunderstood the task and thought it was something bigger.
At the end of each sprint/cycle we shared estimate vs actual effort, to improve our ability to make a good estimate and set an overall velocity.
On my first Scrum/Agile project, there were complaints of overwork the first sprint. Then I pointed out that we were the ones setting our own estimates, setting the pace and causing our own problems.
After that, the estimates got reasonable. We stopped playing "Name that Tune" with our estimates and the project settled down.
I am curious how much process research is being done as an industry. With billions at stake, it seems like I would run across more studies where teams were paid to produce the same software independently. But virtually none of the methodologies I read are backed by much rigorous experimentation.
Maybe it exists and I am just not reading the correct articles.
You're 100% right that this is exactly the kind of thing that needs to be tested in various settings to figure out what an optimal setup looks like for different problems. Unfortunately, I had a hell of a time finding actual comparative data with real metrics [1][2]. Pretty sad that 90% of the "research" available is either snake oil or mostly useless comparisons of methodology.
[1] https://pdfs.semanticscholar.org/85d0/00404206914501d26e0bb4...
[2] https://www.researchgate.net/publication/261047173_Agile_and...
There have been a lot of software development methodology but one thing that's been overlooked a lot is the competence of the people doing the execution of the project. I am talking not just about the developers / engineers who are building the product but everyone across the pipeline - product owners, business stakeholders, process specialists, business analysts, project managers and just about everyone else.
People attribute failure to the process because it is demeaning to put the blame on people when established best practices can receive the finger pointing.
Stop the notion of being nice to the lack of skills of people. Hiding behind being nice does not resolve the issue but rather propagates it. Fix the issue by providing the grounds for people to learn and become productive.
Projects staffed with fully qualified people with hardcore skills fail too. And projects staffed with utterly under-qualified people sometimes do just fine.
The thing is projects succeed or fail for many different reasons, usually multiple reasons operating in concert.
I think the best approach is not to be dogmatic about process and to recognize that "pointing fingers" rarely resolves anything, regardless of whether one is pointing at the process or the people.
Yes! One thing I don't see people mention is that Scrum requires that everyone on a team to be competent. It's for teams that are already great that want to be even greater. It's not for teams with inexperience and incompetence. My guess is many people are afraid of scrum because it may out them as incompetent.
I'm not saying you're right or wrong, but "many people are afraid of scrum because it may out them as incompetent" sounds like the cliche of "X cannot fail, it can only be failed"
Yes, it does sound like a cliche. However, the point of scrum is to find problems that get in the way. Through the process, it's clear who knows what they are doing and who doesn't.
The position may be right, but it's so badly argued. Arguments he uses in support:
- the dictionary definition of "agile" - how many times the word "agile" appears in the Scrum Guide - how "lengthy" the Scrum Guide "seems" - that Scrum is a process, so does not "sound Agile".
Ugh. How did this reach #1 on HN?
I think because people are frustrated. My read of the conversation is that there's a lot of latent resentment towards modern Agile and Scrum practices.
At least the discussion in the comments is better argued.
A long time ago, ppl figured out that no one process was right for every organization, project, technology, etc. They created process frameworks (things used to create a process) and they saw that they were good.
Then ppl came along and said that they were "too complicated" we need something Agile! In actual fact, those ppl didn't understand the process frameworks and just created a specific instance of one and they wondered why it didn't work for all projects, organizations, technologies, etc. And then they said those processes are "too simple" - what can we do? "Maybe we should create a way of generating a process?", they then said. And so it went...
https://en.wikipedia.org/wiki/Rational_Unified_Process
https://en.wikipedia.org/wiki/Microsoft_Solutions_Framework
Are we still talking about these dated concepts?
Agile is a mindset.
Understand the mindset, absorb it, make it yours, and build the process that's working for you based on that mindset.
Stop complaining about Scrum, it's just a tool, and tools deprecate.
Yep, I've never been in a team where the process was not a regular topic of debate. Once it stops being a topic of debate, it actually just means people are no longer adapting. I prefer a good dose of pragmatism here. Changing the rules all the time is very draining on team morale. Similarly not adapting to obvious process issues is equally demoralizing.
Since everyone is doing, or claims to be doing, agile, the whole notion is completely meaningless. Unless they are actively advertising to be doing some form of waterfall, you can safely assume there is some notion of iterations involved.
What matters more these days is whether the team is mature enough to have continuous deployment without process bureaucracy. Deployment fear is a good sign the process sucks. If you have a need for human gatekeepers, something is wrong with the test automation. Deploying often and with confidence is a good sign you are dealing with a smoothly running team.
I don't really have that much industry experience but have been reading a lot about development processes and practices. It seems there just isn't one development process that will solve all your problems and allow you always create successful software. To me it seems you just need to find what works for your team and continuously work on improving on it. What might work today, probably won't work for your team tomorrow so you need to always look at how can you improve your process.
ACM recently had a series of webinars[0] by Ivar Jacobson on Essence[1]. Essence was kind of confusing at first, but it is essentially a language to describe practices from the different processes. The idea is that you can build up a library of the various practices in your company and allow teams to pick their own and evolve their process by swapping out practices that just don't work out. Essence seems like an interesting idea, especially if it allows teams to create a development process that fits them.
[0]: https://www.sigsoft.org/resources/webinars.html
[1]: https://www.ivarjacobson.com/services/what-essence
I have decades of industry experience and I mostly agree.
> find what works for your team
For the combination of team and project. Different software projects need to take very different tradeoffs in how they managed. Use scrum in aerospace for flight control, and people will die. Use scrum for system software, and you'll have hard time delivering quality and performance, system software often requires non-trivial amount of engineering. Implement NASA's best practices when working on videogame or web app, and you'll blow budget.
This article is from 2002 but still good: https://www.joelonsoftware.com/2002/05/06/five-worlds/
You may not have much industry experience, but you’ve managed to hit the core of the problem. There is no one process that will work for every team, or even every project.
Every team at my workplace runs a different process. We’ve got a team developing new features who are running an iterative agile process, allowing them to react to changes ok requirements or unexpected issues with as little disruption as possible. We’ve also got a team doing B2B projects, essentially rebranding our mobile apps and platform for different companies, and that’s pretty traditional waterfall - I wouldn’t in a million years use waterfall for new features, but when you’re running through a set of well understood tasks that need delivering on schedule to coincide with work by third parties it gets the job done.
I'm doing Kanban right now at my job and, I have to tell you, it's so liberating.
I hate artificial deadlines (i.e., sprints). Kanban let's you focus on the work without all the process distractions.
From all the comments, I see that arguments are there both ways. I also understand that we should follow a process that bets suits the business. I would really like to know how we could set up an Agile process provided that is what is required.
Is there anywhere this would be documented to explain - a. How an agile process in initiated? b. How an agile process is maintained?
I have been part of a team with mis-applied agile processes -
1. Requirements change frequently. And not due to users asking for them. More like the Project Managers asking for more configurablity.
2. Requirements are not thought through. Even the simplest cases.
3. Changes don't get vetted by users. Instead, we go through another round of requirements!
4. Every once in a while requirements (and teams) get re-organized, re-planned which puts all the developers off.
I have also been part of a team which focussed on deliveries. This team has been able to write tests and iterate over requirements fast. The satisfaction level of all the developers in the team as well as the project owners were high.
1. Requirements were phased.
2. Whatever was required to be done in each phase was thought through as far as possible. Anything that wasn't clear or was more complex to think out was sent back to the project owners / users for clarity.
3. Code and tests get written. Phase delivered.
Although this process was extremely successful, I don't know if I would call this Agile.
I agree that Scrum is fragile but not that it isn't agile.
It's agile up to the point that it is broken, which happens often due to it's fragility. That it is easily broken is an unfortunate but unavoidable consequence of its complexity, and common misunderstandings/misuses.
Scrum is complex because large software projects (large as in number of people, budget and expected velocity) are never simple to manage. Scrum is complex because correct usage can not be read from a book nor gained from a qualification - it requires experienced judgement to pick the optimal usage and apply it to a team and project.
Nothing can beat the effectiveness and efficiency of a single strong developer working on a stream of well considered backlog items. That is engineer nirvana! However many projects simply require more velocity than one person can manage. From here, there is a whole spectrum of processes that should be chosen depending on resources and needs. From my experience, full scrum kicks in once you have four or five engineers on the same system.
Product management is a key point of failure in Scrum, true also with other processes. But it's felt so much more when an entire team grinds to a halt and an iteration fails.
If Scrum doesn't feel agile then you are doing something wrong. This is exactly the question the team should be asking itself at retrospectives (doesn't need to be three questions!). What's changing in your process sprint to sprint? Scrum masters are there to facilitate incremental process changes driven by team members, not a system that doesn't result in agile software delivery.
The author had really bad experiences. SCRUM is super agile when done right. Change can be introduced during sprint transitions (reviews, retrospectives and planning) - never in the middle of a sprint. Doing so will kill moral and break the rhythm of the dev team.
SCRUM is meant to be owned by the developers - not management. Shame on companies that hijack and impose such process.
SCRUM gives dev teams rhythm with sprints. Allows them to deliver mini releases of functional software in 2-week increments (could be 1 to 4 weeks too - lenght should be decided during planning). The goal of a sprint is to be decided by the developer and doesn't have to be about the software. Can be about learning, improving as a team or helping team members to grow. Here are a few examples I've seen: - Measure 5 metrics in Kibana - Marie-Pier leads sprint review demo - Try self-merging PR's and measure the results - No more loose ends Of course, in parallel the team builds and delivers an increment of functional software.
The planning, review and retro rituals are paramount. SCRUM can only work well if those are well planned and executed. The retrospective is particularly important to allow the team to improve themselves. Its an opportunity to change their own process (i.e. change the definition of done).
Contrary to wide belief, story points are not part of SCRUM. Its a mechanism that can be used to gauge the size of work - it should be up to the dev team to decide whether to use them or not. Again dev teams should own that process - should not be imposed by management.
These lessons I've learned in the past two years thanks to an awesome SCRUM evangelist we've hired. I'm proud to say its changed our lives for the better.
Relevant: https://pragdave.me/blog/2014/03/04/time-to-kill-agile.html
In my current team, we post our daily status on a slack channel: what we do yesterday and we are going to do today. We try to keep to one or two points. We have remote people so that works out well. Standup is optional. On Monday meetings, each person tell one thing that we are going to do. Sometimes it is two things but the point is to focus.
This is the way it should be.
Meetings should only be between business / leads to get scope onto canvas.
If I'm blocked I'll seek out the person who can most help me. I can easily find 5-10 minutes to write out a summary, but on my terms when I'm not in coding mode.
Now, I'm freelancing, likely to be team lead for the company assuming they get financing - and I keep taking notes and reading ways to organize better. We have a ton of work to do, it's a financial mobile app that has 0 tests on the api (I'm single-handedly running the API development Laravel + vue (for some web views)). Money is tight so I'm the only dev on web side and we have 1 ios dev.
The hope is to be full-time at this by Christmas w/ a 6 figure salary (my first +6 figures).
Scrum truly does end up being "Waterfall" most of the time, in my experience. I think this is largely because teams or management inevitably get bored with projects and end up shifting priorities regardless.
From what I've seen, projects almost always start out in a requirements/analysis phase because management, stakeholders, etc., need the reassurance. I've yet to see any public-facing project of meaningful size start out releasing actual MVPs as opposed to "big bang" releases.
Design often happens first because engineers don't want to(and sometimes can't) begin coding away and then receive designs that are either counter to their existing work or are unworkable; management usually sides with design because, well, it looks cool.
Engineers eventually get coding, but non-automated testing doesn't really occur until later because it can be impractical to have people testing unfinished software. Then there's alpha testing, users break the software, engineers fix bugs, A/B testing of different versions of features, then beta testing, where users again break the software and the engineers fix more bugs.
The project continues to drag on because the first 80% of a project is always the easiest part, and management is always hesitant to release "unfinished" software.
Inevitably, there's a big bang release, by which point management has already gotten bored and has dreamt up other "big ideas" for the team to focus on, at which point the project is, for all intents and purposes, placed into maintenance limbo where bugs are fixed, junior developers place their awful code, and a new "feature" is added now and then to satisfy the marketing department.
Senior developers rationalize the waterfall-like nature of the project, so they opt for "continuous integration", but all that ends up translating to is maintenance limbo without versioning.
The actual next version of the software doesn't get built until after most if not all the original team left, and the original software became "legacy" enough that bugs keep popping up and the new engineers don't want to touch the old code. Management eventually gives in, for better or worse.
Am I wrong? This is what I've seen happen to the vast majority of software projects, all under companies that were either Scrum, Agile, or pretend "Agile". Maybe I've just had bad luck.
I wish more teams could pick and choose what methodologies to use for their purposes, rather than buy into MLM garbage.
Said it a million times before - I deal with way more process and paperwork using scrum than I ever did with waterfall. I'm not saying waterfall was an answer, but at the places I worked on it, it did a better job of staying out of my way.
The SCRUM guide is not long, it's 19 pages in a large font. If I remember correctly it does not even mention a lot of the processes which make up SCRUM for many teams.
However the article is not wrong, it's too easy to make SCRUM all about the process, without changing anything about interactions, developer empowerment and feedback cycles.
I think the SCRUM doesn't do a good job of putting limits on what is still agile SCRUM and what is not. Maybe because it's written in such a friendly language.
The aim of Scrum, and other methodologies tagged Agile, is -- near as I can tell -- to emulate a good programmer with a team of mediocre programmers. Not that all developers working on Agile teams are mediocre programmers, only that the business wants to achieve good-programmer results while still being able to leverage the wider talent pool and lower risk and cost offered by hiring mediocre programmers, so it tends to adopt formalized processes designed to work well with them in the hopes that following the process will make them achieve good results. The process is a formalization of the relationship a good programmer has with their client, which is known to produce good results; therefore, the process must also produce good results by definition (and any failure to produce good results can immediately be blamed on lack of adherence to the process).
Unfortunately there ain't no such thing as a free lunch, and when you emulate you necessarily incur overhead, which can be prohibitively large if you're emulating something powerful -- plus, if you have actual good programmers on your team, they will either quit or start emulating mediocre programmers to remain compatible with your mediocre-programmer process. Either way, you lose the benefits of having them.
The problem often is that scrum (or any iterative process) fails when it's not paired with good XP engineering practices. There's constant learning and change on the product side, which drives constant learning and change on the implementation side, the code needs to be constantly refactored to best handle the changes.
TDD/yagni/refactoring allows you to keep your code base nimble. I can't imagine doing an iterative process by slamming features in one after another without refactoring the code as you go to handle what "you now know", or not having good test coverage to support the changes you need to make.
Refactoring also shouldn't be something the PO is aware of, it's not a story, or a 2 month break from feature development. It's part of the job as each story is implemented. Yes, sometimes you don't notice a good way of doing something until some time passes and that area of the code becomes a bigger refactor than usual, just have to deal with it as soon as possible.
That's a bit of an easy critic of Scrum on its literal sense rather than a critic on what doesn't work or isn't agile or is fragile. I've seen Scrum badly and imperfectly implemented everywhere I've been, yet I think this is preferable as having a strict interpretation of its principles and processes. I don't care actually if Scrum is intrinsically not agile (or Agile): I think what matters and is actually what's happening is people adopting Scrum with agility.
It has good principles like backlog, sprint and daily meeting. If teams only pick one of those or anything else and that it help them, great! By the way holding solely Scrum accountable for failures is narrowing down the analysis to only one part of all things that can go wrong and have nothing to do with Scrum. That's a cognitive biases concentrate.
Reading the comments on this post are truly sad. The amount of "me too" juxtaposed with "actually, this isn't a true scotsman" are truly mind boggling.
I wish there was a prescriptive way to successfully make and deliver software. I have seen very little evidence that there is.
Although I have remained sceptical of Scrum, I do find it a bit strange that many of the negative comments about Scum here seems to be related to not doing Scrum (The old, you are not doing Scrum properly). All the talk about managers, and being told what to do in a sprint etc. None of that sounds like Scrum to me. Isn’t the whole idea of Scrum about self organising teams (?), independent of management etc. All you have is a Scrum Master to facilitate the process, and a Product Owner (part of the team) who prioritises the backlog because that role has an understanding of what is needed to solve the problem at hand (in terms of functionality needed).
Where this stuff really fails (IMO) is when execs throw together a half-baked structure around a project, and throw minimally qualified human resources and a few consultancy firms at it. Knowing that "shit rolls downhill", it's always the developers who end up sweeping up the mess working endless hours trying to make deadlines, ensuring everyone further up the hill looks good to their bosses, while praying that their project doesn't get terminated.
In my experience this seems to be an issue unique to enormous corporations. When I used to work at small / medium-sized companies this never happened.
I think that longer term strategic planning is missing from the Agile Scrum manifesto.
Agree with the Software development priority queue part being good thing.
Long term planning was in the waterfall process which does not work that great either since it tends to over focus on planning.
How do you find a good balance between Agile and Waterfall?
I think it can also be different what process works well if you are a startup and need to deliver a MVP minimum viable product as soon as possible. Compared a bit to if you are a long term enterprise company and need to maintain your code base over time. Ie dealing with technical debt from short term solutions vs long term stable.
Strategic planning doesn’t belong in a process designed to provide constant iteration on a two week basis. In successful organisations I’ve worked with strategic planning is one of the things that feeds into the backlog of features to be developed, and runs in parallel to development.
In my experience, I was lucky to apply Scrum and Agile correctly so now I understand them and can distinguish between them. In my current job, there are no chances to replicate those conditions and I do not expect Scrum to succeed (although we use a lot of Scrum terms to define what we do).
I think one should have experienced a successful scrum project before being able to criticise it.
Scrum is a tool: you may be Agile without using Scrum or you may find easier to be Agile using Scrum. Doing Scrum does not make you Agile. Scrum is hard and implementing it takes different skills than those proven by individuals who can write code.
Honestly, the comments on this thread baffle me. Endless complaining about scrum and everyone who is not a developer is apparently clueless about software. Is this an adequate reflection of reality? This doesn't seem like a very mature attitude. Scrum is a simple framework with only three roles, the development team being one of them - so if the process fails you are (partly) to blame for it yourself as well. Take responsibility!
What's the better alternative? What have you been doing to make things better? The answers to these questions as much more interesting than endless complaining.
I searched the entirety of the Scrum Guide (https://www.scrumguides.org/scrum-guide.html) and could not find a single reference to "agile". And a quick Google search about the origins of Scrum reveals:
In the early 1990s, Ken Schwaber used what would become Scrum at his company, Advanced Development Methods; while Jeff Sutherland, John Scumniotales and Jeff McKenna developed a similar approach at Easel Corporation, referring to it using the single word Scrum.
> In other words, it is a very specific and well-defined process. This does not sound agile and it also does not sound Agile (remember: “Individuals and interactions over processes and tools”)
This is a many times debunked myth about the Agile manifesto. Even the authors themselves explained that "Individuals and interactions over processes and tools" does not mean and shouldn't be taken as "no processes".
Also, agile is not about moving quickly but about being able to easily change direction any time, therefore dealing better with changing environment.
Scrum is a communication contract between customers / stakeholders, managers and dev teams. It is a poor cure for social and managerial dysfunction, but it is a cure.
Scrum is a problem when it is mandated and imposed on functional organisations that do better and more fluid communication. This is often done as a way of flexing corporate power and does huge damage.
A key problem is that we do not have ways of auditing and measuring the performance of software teams no matter what methods they are using - until then this is all anecdote.
> A key problem is that we do not have ways of auditing and measuring the performance of software teams no matter what methods they are using - until then this is all anecdote.
Of course we have ways of doing that - one is the code quality, another one is historical performance on agility.
I wonder what happened to so many developers that turned them so jaded and cynical towards scrum and project management. So far I've dealt with the good and bad of scrum with a good pinch of horrendous PMs in the mix, but I still came out of it with a positive view of the framework. It is hard to run it somewhat smooth and takes lots of motivated people to accomplish[1], but the result had a positive impact on my day-to-day work.
1 - Some people said that this can be a failing of scrum and I 100% agree.
Scrum only works if you need sprints, and you only need sprints if you have non-engineering stakeholders whose mandatory approval can only be attained in one meeting per sprint length.
Some large orgs really work like that, where the approval of one ham-fisted iron-gripped executive is deemed more important than hundreds of thousands of dollars of engineering talent going to waste. I don't think this is very effective but alright then.
Everybody else should just be using Kanban boards with periodic review.
Reading all the comments out there, there is some atrocious stuff people call Scrum. I'm not even a huge Scrum fan, but it folks seem to criticize what people do in the name of Scrum instead of what is actually in the process.
The TLDR of Scrum is simple: (product) management gets to set the priority of things every 2 or 3 weeks. After that, we see what got done and check to see if the priorities are still the same.
If the priorities are wrong, don't blame Scrum, blame management. If tech debt is increasing, don't blame Scrum, blame management.
There's no magic bullet to determine what is important, and certainly Scrum Master training won't help an incompetent manager a competent one.
I don't think I have ever seen it done as badly as some of these comments. Scrum itself is a very light and quite flexible framework, half of the complaints here aren't even Scrum related. User stories aren't part of Scrum, neither are story points or burndown charts.
Of all the people complaining about Scrum it doesn't sound like any of them have read the Scrum Guide. Symptoms of Scrum are not managers pressuring people at stand ups, squeezing 4 weeks of work into 2 and having a "team" of 20 people. These are symptoms of poor leadership.
Has anyone ever tested whether Scrum or Agile actually work better than "Waterfall" for software development?
Agile works fine, if you ignore the Scrum hocus pocus. Stick to basic project management principles (work on one project at the time, wrap things up before moving on to the next project, set aside a bit of time for unrelated work) with the stakeholder put at the center of what you're up to.
The problem with Scrum is that it's usually (always?) implemented as serialized waterfall projects with completely arbitrary looking deadlines (i.e. the sprint's length). All too often things get sort of done but not done quite as well as everybody would be happy with. It's stressful for everyone involved and, insofar as I've seen it used, it makes accumulating technical debt even more likely than more traditional waterfall projects.
I dunno, by now I would actually be really happy with a waterfall requirements document.
Waterfall was terrible in many ways but it seems to me for all the problems with that methodology that "agile" has fixed it has created at least one new one, and often more.
I’m quite amazed by the lack of substance in this article. It takes potshots at certain ideas and processes, without managing to explain these potshots, or even coherently explain the ideas and processes. It is a flimsy opinion piece with no substance, but a clickbait title so everyone ran here to comment.
What a virtuous cycle
“To me it seems that one very important part of high quality software development is to maintain a simple priority queue of tasks. The weight is a combination of the value a task provides for the customer / developers and the estimated effort to implement this task.”
A.k.a. common sense :)
Agile – the only word in the English language that inverses its meaning when capitalised.
Scrums point for me is to prepare your development time. Talk about the issues, prepare the issues, ask the client, make a mock-up and then implement in code.
Usually it's hack together something and see if it sticks, which is awful.
just to offset a bit about scrum being bad: I have used SCRUM over 15 years to run teams and it worked. Most of the problems I am hearing on here are culture problems and no process can fix that. Regarding planning, a day per scrum is normal - sometimes longer, you have to use your judgement to determine when to move from design to implementation. Also, heavy use of spiking stories to learn enough to create the right design is important
Can you explain a little bit more about your "spiking stories" and how you use them?
The best of agile can be summed up in one piece of advice: "Create short feedback loops". Nail that and you can't go too far wrong.
> "We are simply caught in a rat race and not able to make a short break in order to look at and learn from all the things that happened around us, maybe even before our time."
But isn't that exactly what Scrum wants you to do? After every sprint, you spend some time to reflect, look at how you're working, look at the bigger picture, etc.
Scrum is by no means perfect; it's a tool, not the infallible silver bullet the author wants it to be. If you misuse the tool, you're still going to get wrong results, but that's the same with every other method.
The real question is: is there a better way to do it that can easily and reliably be implemented by large corporations? I think the popularity of Scrum is probably due to it being more successful than what was used before.
It might not be truly Agile, but for large corporations, being truly Agile may be a bit much to ask. They need reliability and reproduceability, and that means they're always going to need some focus on processes and procedures, and can't always rely on people who might leave or have something happen to them. Not that Scrum is always a good fit for those processes and procedures, but it does help make it appealing for large companies, and at least it puts a good part of the process in the hands of the people who will be using it.
tl;dr: Scrum is not perfect, but what is?
uh oh, is this the one article that will make people go "A-HA, told you so?" #confirmation-bias
and, I never saw so much interpretation of what something should be as with Agile.
not everyone is entitled to say or mean something, that includes myself and this post.
"ahh-gee-lay! must be italian!"
Scrum is micromanagement. I think it’s as simple as that.
no this is completely and entirely wrong.
on the contrary: you have a PO in charge of communication and priorization of features and a dev team in charge of developing- no manager on top...at least in theory...
This is very light on specifics? What, exactly, is fragile about Scrum? Where does it break down, and what can I do to prevent it from breaking down?
Without that info, this just feels like buzzword bingo, a la "What should I write about to stir the pot today?"
Reminds me of this goto conference talk from one of the original Agile developers "pragmatic" Dave Thomas.
Speaks a lot bout what agile truly is and how scrum is a very large mismatch.
https://youtu.be/a-BOSpxYJ9M
You may find this very much relevant: https://www.youtube.com/watch?v=OOAMNOso46g
Agile itself is pretty bad too. “Responding to change over sticking to the plan” is pretty much the single biggest reason why I see projects fail.
I think the key motivation of agile is the recognition that the plan made in advance is often not right.
If completing "the plan" is seen as success, then of course sticking to the plan is the best way to success. But if maximizing business/user value is the success, then "no plan survives first contact with the enemy." (I don't like "the enemy" concept, but this is a famous quote that gets across the concept).
And I think the reason why "agile" fails is because many actors in many organizations find they are effectively rewarded for "completing the plan" regardless of business/user value, indeed.
This is just not accurate. The types of changes thrown in after early requirements gathering are most often coming from product managers and executives, not from customers and not driven by genuine feedback or budget constraints, etc.
“Sticking to the plan” does not mean rigidly enforcing zero changes, but rather means keeping a commitment to the general scope and direction that was mapped out. Compromises should require extraordinary hard evidence before being accepted.
This is really why Agile as a general set of guidelines is so easy to subvert and ends up being a misused tool in most every situation where Agile is deployed (Scrum of otherwise).
Project management guidelines need to start out by specifying a way that quality is strictly disallowed from being subverted by competing interests that lobby for changing the plan.
If a set of software project management guidelines doesn’t start out with an unchallengeable quality-above-all-else mandate that creates policy barriers to the natural entropy of different interests trying to lobby for why their preferred change has to be made, then it’s doomed to just get politically subverted.
Doesn’t matter if it’s Agile, Waterfall, extreme programming, whatever.
“Sticking to the plan” in the sense of setting up preemptive, high-cost barriers to anti-quality modifications to what was agreed is _the_ thing.
The problem is that if the environment or requirements have changed and you don't take account of that then you may have a successful project delivering a product that is no longer relevant.
This is the overblown excuse people use to argue for their preferred changes, when really the environment or requirements usually don’t change, and when they really do then either it’s small and you should still stick to the plan or else it’s big and the need for change is so overwhelmingly self-evident that you don’t need things like Agile slogans to generate motivation or buy-in to change things.
Either way, harping on the idea of “but what if circumstances changed” is mostly just what corporate politicians do to subvert people demanding evidence for the requirement of changes.
Nothing can replace Will and Wisdom
To think of scrum as a process is wrong. It's actually an organizational pattern language. The idea is that social interactions are a complex system and there are patterns of interactions which, from experience, seem to work in certain contexts.
http://www.scrumabok.org/
It's not even software specific. Which is actually a strength.
Example, one common misconception is the daily standup is a status meeting. In fact you are supposed to replan in standup, it's a tiny planner. Not sure how you can call the ability to pivot every day not agile.
EDIT: Unpopular option, software quality is poor and it has nothing to do with scrum.
I'm already able to pivot every day—it's part of the autonomy inherent to me being hired as a professional. I can make decisions about my work, and ask my colleagues to make decisions about our group work, as necessary to deliver on business needs. Making a process and a specific time about how to get approval to pivot is less agile.
You ability to pivot as an individual is already limited based on your team. In fact a unilateral pivot is destructive and can inflict pain on everyone else. I'm not sure how having implicit and ad-hoc patterns of interactions is agile. Why? because you can't improve what you can't see.
I guess part of my skill as a professional is knowing how to work with my team, communicate as needed, understand whay they need to deliver business results and get them to understand what I need to deliver business results, prioritize as needed, and make sure I'm not inflicting pain on everyone else.
If you don't have that level of trust in your team members, sure, formal process makes sense. And I'm all for it if that's what's needed to deliver business results successfully. But you're then valuing processes and tools over individuals and interactions, following a plan of when and how you may respond to change over letting individuals figure it out, etc. Again, it's perfectly okay if it's not agile if it's what the business needs. But my assertion is it's not agile.
No.
Agile is a subset of TPS (toyota production system)/Lean applied to software.
Actually, lean is lean applied to software. Agile is Henry Ford's interpretation of lean applied to his factories, applied to software.
And scrum? TPS is explicitly an influence on scrum.
Scrum is just an instance of the agile manifesto. TPS/Lean > Agile > Scrum.
Kanban for example is another instance of the agile manifesto.
Scrum existed before the agile manifesto. It's probability more accurate to say Agile is a mix of those things.