Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Design ‘debt’ and quality process

10 August 2009 by Clark 8 Comments

A tweet from Joshua Kerievsky (@JoshuaKerievsky) led me to the concept of design debt in programming.   The idea is (quoting from Ward Cunningham):

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite…. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.

I started wondering what the equivalent in learning design would be. Obviously, software design isn’t the same as learning design, though learning design could stand to benefit from what software engineers know about process and quality.   For example, the Personal Software Process‘ focus on quality review and data-driven improvement could do wonders for improving individual and team learning design.

Similarly, refactoring to remove typical bad practices in programming could easily analogize to the reliable patterns we see in Broken ID.   There are mistakes reliably made, and yet we don’t identify them nor have processes to systematically remedy them.

What are the consequences of these mistakes?   It’s clear we often take shortcuts in our learning design, and let’s be honest, we seldom go back. For big projects, we might create iterative representations (outlines, finished storyboards), and ideally we tune them once developed, but seldom do we launch, and then reengineer based upon feedback, unless it’s heinous.   Heck, we scandalously seldom even measure the outcomes with more than smile sheets!

For software engineering, the debt accrues as you continue to patch the bad code, rather than fixing it properly (paying off the principal).   In learning design, the cost is in continuing to use the bad learning design.   You’ve minimized the effectiveness, and consequently wasted the money it cost and the time of the learners.   Another way we accrue debt is transfer learning designed for one mode, e.g. F2F delivery, and then re-implement it as elearning, synchronous or asynchronous.

In software engineering, you’re supposed to design your code in small, functional units with testable inputs and outputs, and there might be different ways of accomplishing it inside, but the important component are the testable results.   Our learning equivalent would be how we address learning objectives, and of course first we have to get the objectives right, and how they build to achieve the necessary outcome, but then it shifts to getting the proper approach to meeting objectives. If we focus on the latter, it’s clear we can think about refactoring to improve the design of each component.

Frankly, our focus on process is still too much on a waterfall model that’s been debunked as an approach elsewhere.   We don’t have quality controls in a meaningful way, and we don’t check to see what reliable mistakes we’re making.   Maybe we need a quality process for design. I see standards, but I don’t see review.   We have better and better processes (e.g. Merrill’s Ripple in a Pond), but still not seeing how we bake review and quality process into it.   Seems to me we’ve still a ways to go.

Comments

  1. Dave Ferguson says

    11 August 2009 at 3:36 AM

    I hadn’t heard “design debt” before, but I have heard (and spoken about) processes that are front-end loaded.

    Which is just a complicated way of saying “pay me now, or pay me later.”

    Your comments about quality remind me of the initial resistance to quality in the latter part of the 20th century–remember that Deming had his initial success in Japan, rather than in the U.S.

    Quality seemed like extra work, the way you’ll hear critiques about “paralysis by analysis.”

    Even today, I think, a fair number of organizations claim that they value quality. It’s like the cartoon I saw once of two Romans at the Colliseum, looking down at people being thrown to the lions. “You know,” says one, “I’m a Christian, too, but I’m not a fanatic about it.”

  2. Clark says

    11 August 2009 at 9:50 AM

    Dave, ah yes, “pay me now or pay me later”. M’lady has two Fram oil filter cups (they look like the filters!) with that legend.

    Good point about the overhead. What Watts Humphries found with PSP was that initially it slowed people down, but then it actually made them more productive, as their estimates were more accurate, and they made fewer mistakes requiring rework. Let alone the quality of the output. Reckon some peer review and systematic analysis of ways to improve would work wonders. Thanks for the feedback.

  3. Steve Flowers says

    12 August 2009 at 5:53 PM

    While debateable whether I was actually programming, I have ‘some’ experience designing and building software. I see many opportunities for improvement in the ISD vertical space using well established software engineering and management practices. Here are the biggies:

    1. Patterns. I’m not sure what it is in the world of the ISD. Whether we think our world of work is too complex to document problem / solution pairs or we are protective of our ideas. Either way, there’s nothing but good that can come from an establishment of a common patterns library, common associated language, and open capture and extensions of these common patterns. Did some presentations and wrote some position papers with Ian Douglas at FSU. He’s a big proponent of patterns. Sadly, the same people we need to convince can’t see the promise of ‘another best practices compilation’. Sigh:)

    2. Unit testing. Here’s another one that baffles me. I’ve spent a significant amount of time on the development side and in my experience the design eats up the first 75% of the resource / time budget. This is before metal is bent, before one wrench is turned, before anything is actually developed. The last 5% are for (a) QA and / or (b) Scramble to unf*#k the output – usually unsuccessfully. Not sure that this is common everywhere, but it is in government contracting and it’s complete lunacy (I railed against this mindset unsuccessfully for years). Modular abstraction and unit testing is among the many great practices in software engineering that allows large teams to accomplish seemingly impossible feats. Why we aren’t moving evaluation up front is beyond me – formative evaluation creates data points, data points correct course, course correction helps avoid abysmal failure.

    3. UxD. This one is relatively new. But this field (some say horizontal intersection with design verticals) closely reflects ISD goals. The difference being that if you are a bad UxD you aren’t employed for long – in stark contrast to my experience with bad ISD. The field accession is also different. There is no DAT (Design Aptitude Test) for entry into an ISD program. If you are flashing the green, you’re in the machine… Principles can be taught, but sadly talent is non-transferrable. UxD’s move through the ranks by demonstration of aptitude. Those who make hiring decisions for ISD’s typically either don’t understand the field well enough to make a good judgment or poor performance is masked by a good interview and a team effort that doesn’t isolate what the ISD actually did. If it walks like a rant and quacks like a rant it’s probably a rant:)

    We did try a few experiments to make our ID and Development processes a bit more like efficient software project management processes. How did it come out? Depends on how you define success. Too many factors to isolate, but it didn’t seem to hold and I split the scene to work as a hermit. ISD folks couldn’t understand what was wrong with the take-your-time-waterfall, plow horse in blinders approach. The technical folks didn’t understand the unique process elements and considerations within the performance engineering realm (warm) that differed from the software engineering realm (which feels cold to me).

    As an inclusive process element definition, there’s nothing wrong with ADDIE. I’m not sure how else you would distill the neck up activities that contribute to success. The problem is the way it’s leveraged. Big A, NO I… or Big D, little D, tiny I and a sprinkle of E if you’re lucky at the very end. I say stratify A on multiple tiers and move the E up front… Reach cogent hypothesis faster, sprint, test, correct.

    There is plenty that we can learn from the software engineering crowd. There’s also plenty of practices that simply DO NOT fit. In each case, the biggest problem we have is personnel selection / preparation. Without changing how we pick and prepare our ISD crop, we’re in the vat of sadness for the long haul – the minority of superstars and visionaries can’t stem the tide of baddies:(

  4. Clark says

    13 August 2009 at 4:49 PM

    Steve, as I taught interface design while researching learning technology, I’ve often tried to cross-fertilize, but I often found UxD to be ahead of ID (which I think you’re also saying). I’d be curious what you think *doesn’t* fit. My one response would be that creating learning experience, is different than creating task ability.

    Thanks for the feedback!

  5. Steve F says

    13 August 2009 at 7:37 PM

    One example: OOP, the way software engineers view OOP, doesn’t fit learning design. Too much cold rigidity and might need it in the future refactoring. This was part of the driving concept behind SCORM. SCORM didn’t come out as intended in my view.

  6. Dave Ferguson says

    14 August 2009 at 7:51 AM

    I love “might-need-it-in-the-future refactoring.” It’s got me imagining the engineering or design equivalent of the person whose apartment is hip-deep in stacks of old newspapers, bread-bag ties, every receipt ever issued. And a large number of cats, many of them still alive.

  7. John Schulz says

    20 August 2009 at 8:57 PM

    Clark,

    Once again, thanks for putting a name to concerns I’ve had for years. I can’t even begin to tell you about the debt I was lugging around at some organizations. Somehow, validating root cause and documenting projects always seemed to get cut from project plans in the rush to meet some deadline. Always enjoyable to unravel spaghetti code after its creator has left the organization.

    Steve – I always find your comments enlightening. While I agree with all of your points, I especially connect on your comments about talent selection. I had actually included this very idea in a presentation I recently did regarding mistakes organizations make with regard to eLearning. Before tapping in to all of the wonderful brain power I am finding on Twitter, I thought that maybe ‘we’ were just getting lazy. That IDs had given in to the corporate demand for ‘NOW’ at the expense of producing great products.

    But after thinking about Cynefin (thanks, Clark!), Dryfus, and a little light reading around cognitive science, I came to your very conclusion (I even drafted a causal loop!). Our courseware design (and the very fact that we can’t think of anything BUT COURSEware) is being driven by the fact that we have few experts in our field. David Merrill suggested that 95% of ID is done by people without ID education (or ‘by assignment’ as he called it). Cammy Bean’s informal survey indicated that 60% of us don’t have formal ID degrees. Without this education (through formal University programs, or professional development), and without a personal drive to become an expert, most of us only know learning theory as a set of bullet points from a conference presentation. Let’s not even get in to our lack of skills required for software development, UxD, assessment, visual design, etc. (I still don’t understand why most people don’t treat elearning as a application development project. Ohh, I know … because our tools pretend that no coding is required!)

    Dryfus suggested that without experts there’s no one who CAN be visionary, no one to push the envelope – or as Kathy Sierra says, create the revolutionary change that gets us over that ‘big freakin wall’. Because experts have acquired a broad-based, conceptual understanding of the domain, they are able to innovate. The rest do what we’ve always done. What we’re comfortable with. And, someday soon, someone in an industry outside of L&D is going to come along and take our work – because they’ll be able to produce product that creates meaningful results – and without any of that debt stuff.

    (Didn’t want to leave you out Dave – love your stuff too!)

  8. Clark says

    21 August 2009 at 8:14 AM

    Interesting thoughts, John. My short response is how do we create a framework where we can collect those expert reflections. I see them distributed across some of the top reflective practitioners, but I don’t see a systematic process whereby they can interact and refine their understanding, yet think that would accelerate the field. Am I missing something?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Clark Quinn

The Company

Search

Feedblitz (email) signup

Never miss a post
Your email address:*
Please wait...
Please enter all required fields Click to hide
Correct invalid entries Click to hide

Pages

  • About Learnlets and Quinnovation

The Serious eLearning Manifesto

Manifesto badge

Categories

  • design
  • games
  • meta-learning
  • mindmap
  • mobile
  • social
  • strategy
  • technology
  • Uncategorized
  • virtual worlds

Blogroll

  • Charles Jennings
  • Christy Tucker
  • Connie Malamed
  • Dave's Whiteboard
  • Donald Clark's Plan B
  • Donald Taylor
  • Harold Jarche
  • Julie Dirksen
  • Kevin Thorn
  • Mark Britz
  • Mirjam Neelen & Paul Kirschner
  • Stephen Downes' Half an Hour

License

Previous Posts

  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021
  • May 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • September 2009
  • August 2009
  • July 2009
  • June 2009
  • May 2009
  • April 2009
  • March 2009
  • February 2009
  • January 2009
  • December 2008
  • November 2008
  • October 2008
  • September 2008
  • August 2008
  • July 2008
  • June 2008
  • May 2008
  • April 2008
  • March 2008
  • February 2008
  • January 2008
  • December 2007
  • November 2007
  • October 2007
  • September 2007
  • August 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007
  • March 2007
  • February 2007
  • January 2007
  • December 2006
  • November 2006
  • October 2006
  • September 2006
  • August 2006
  • July 2006
  • June 2006
  • May 2006
  • April 2006
  • March 2006
  • February 2006
  • January 2006

Amazon Affiliate

Required to announce that, as an Amazon Associate, I earn from qualifying purchases. Mostly book links. Full disclosure.