Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Designing a conference

by Leave a Comment

When I agreed to join as co-director of the Learning & Development Accelerator, I’d already attended their first two conferences. Those had been designed to reflect the circumstances at the time, e.g. the pandemic. In addition, there was a desire on the part of Matt Richter & Will Thalheimer (the original directors) to reflect certain values. Matt and I are running the event again, but times have changed. That means we have to rethink what’s being done. So here’s my thinking about designing a conference.

First, the values Matt and Will started with included being as global as possible, and being virtual. The former was reflected in having presentations given twice, once early in the US day, and then again later. That supported everything from Europe, Africa, and the Mideast to Asia and Australia. The virtual was, at least partly, a reaction to the lack of desire to travel and meet face to face, but also to provide options for those who might struggle.

We’re definitely still focusing on being virtual. Folks who would find it challenging to arrange travel for whatever reason can attend this event. There’s also the environmental considerations. Yes, technology requires resources, but not as much as collective travel. While there’s also a desire to meet different time needs, we’ve found less demand for multiple times. However, we will be recording sessions that are synchronous, so they can be viewed at convenient times. We also are spreading it over six weeks, so that there’s time to consume as much as you want. Further, faculty can choose when they’re offering ;).

The original design was focused on evidence-based L&D (which remains a key guiding principle for the LDA). Matt & Will solicited their presenters based upon their representation, but the agenda was largely what those folks wanted to present. Which, in many ways, reflects what other conferences do. In this new era, we wondered what would make a compelling proposition when you can travel to F2F events. We decided that we wanted to step away from ‘what we get’, and focus on ‘what the audience needs’.

This event, then, has a curriculum, across two tracks, designed to address specific needs. There’s also a different pedagogy than most conferences.We also have specific faculty, rather than presenters based upon submissions. Of course, there are tradeoffs. At least we can share our thinking.

The faculty are folks we know and trust to present evidence-based content. You won’t hear promotion for snake oil, like learning styles. We have a pretty impressive lineup, frankly, of people we think are world-class. This includes folks like Ruth Clark, Mirjam Neelen & Paul Kirschner, Karl Kapp, Julie Dirksen, Kat Koppett, Stella Lee, Nigel Paine, Will Thalheimer, and Thiagi. On top of, of course, Matt and myself. Reality means that a few folks we would’ve liked to have couldn’t commit, but this is a a broad and reputable group.

The tracks are basics and advanced. We want to be able to serve multiple audiences. The intent is that the basic track has the core knowledge an L&D person should know. As best we can, as we negotiate with the faculty, of course. Then, the advanced topics are things that are emergent and need addressing. Of course, there’s no commitment that you have to stay in one or another. As with other conferences, you can pick and choose what to view.

We’re also not just having presentations; we’ve asked the faculty to provide development. That is, we’re intending several rounds of content, activity, and feedback, spread out over several days or weeks. We don’t want people to hear good ideas, and maybe take them back. We want folks to take action! We’re also designing in the opportunity for mentoring.

Of course, there’ll be some social events, and other ways to not only hear content and apply it, but to mingle with faculty and other attendees. We want to foster some community. Also, we’re intending to somewhat front load stuff so that we can adapt. If we hear that we need to do something we haven’t planned, we’re looking to have leeway to address it. The nice thing about being small is the ability to be flexible!

None of this is saying you don’t get much of the same from conferences (except, perhaps, the design). I’ve been on conference program committees, and know conference organizers as well. They typically get more proposals than they can accept, so they can choose a suite that reflect things for various ranges of experience and cover important topics. They may not, however, know all the submitters, and take chances on a few. I laud that, actually, because we can’t know if a new approach or person is worthwhile without experimentation. Still, there is the chance for gaps, and for bad presentations/presenters. They’re also, except for the pre-conference workshops (e.g. my Make It Meaningful one at the upcoming DevLearn), one-off events.

We’re taking a chance on our format, too. We haven’t done it before. It may not work, though we have good reasons to believe it will. So, we hope to see you at the Learning & Development Conference, Oct 10 – Nov 18, if the above thinking about designing a conference sense. We think it does, we hope you do, too.

Projects That Didn’t Fly

by Leave a Comment

I’ve had the pleasure of leading the design of a number of projects that have had some impact. These include a mobile app a company could point to. Also a game that helped real kids. Even a context-sensitive performance support system that was worth a patent. Then, of course, are the projects that didn’t, for whatever reason, see the light of day.  So here are some reflections on a few projects that didn’t fly.

Back in the mid-90s, I was part of a government-sponsored initiative in online learning, and we were looking for a meaningful project. We made a connection to two folks with a small company that taught about communicating to the press. They could’ve come out with a book, but they wanted to do something more interesting. We collaborated on an online course on speaking to the media. I partnered with an experienced digital producer, and backstopped with a university-based media team. We had a comic skit writer, and cartoonists, to augment our resources. The result was technically sophisticated, educationally sound, and engaging both visually and in prose. It never flew, however, as we didn’t partner it with a viable business model. Which was reflective of the times.

Then, at the end of the 90’s, I was asked to lead a team developing an adaptive learning system. The charge was to help learners understand themselves as learners. I had a stellar team: software engineer, AI expert, psychometrician, learning science guru, visual designer, and an interface designer. The model was to do an initial profile, then present you with learning elements (concepts, examples, practice, etc) and update your model based on your performance. There was even a machine learning component to improve the models as we went along. We actually got a first draft up and running (10 elements in the student model), before ego and greed undermined and killed it. The lessons learned, of course, have continued to inform me, including, for instance, my calls for content systems.

Then, around the mid-2000s, I was given the task to devise a content model for a publisher.  They wanted to develop once and populate a variety of business products. Drawing on previous experience, I developed a robust model, which started from individual elements and supplemented and aggregated them in a systematic way. This also ended sadly. In this case, the software side never reached fruition.

There are lots of reasons good intentions can go awry.  In my case, it wasn’t going to be on a lack in the learning design ;). What I’ve learned, however, is that learning design isn’t the only element that matters. There’s vision, and execution, and partners, and more. All are ways in which things can go wrong. Yet, that doesn’t mean we shouldn’t try. It just means that we should, to the extent of our abilities, also try to ensure the success of the other comments. It’s worth exploring projects that didn’t fly so as to see how future ones might.

Small thoughts about Smalltalk

by Leave a Comment

A couple of weeks ago, The Computer History Museum hosted a panel session on Smalltalk, which I watched via video. Alan Kay (who’s vision for the Dynabook drove Smalltalk) came in via recorded video. Dan Ingall (the technical guru) joined by live video link. Adele Goldberg (who documented and tested it), showed up live. John Markoff, well known Silicon Valley documenter, hosted. All to talk about Smalltalk. It prompted some small thoughts about Smalltalk.

I was a regular Byte magazine reader, back in the day. I had created my own major in Computer-Based Education, and was designing and programming educational computer games. I’d done academic research as part of the degree requirements, so I was aware of the work at Xerox PARC. (In fact, I flunked a job interview there because I didn’t know what ‘protocol analysis’ meant, though it turns out that’s what I’d been doing!) So, when the Byte issue in Aug 1981 on Smalltalk came out (I checked the date), I was enchanted.

Smalltalk is an object-oriented language that is dynamic, in that you can edit and immediately run it again; it’s not compiled. It was also reflective, in that make itself visible and operate on itself, like Lisp. In Smalltalk, you model your world in objects and they communicate by messages. It has windows, icons, and interactions comes from the mouse as much or more than by the keyboard. You can edit the objects while running and they change. While it wasn’t available to me, I was a fan of the concept.  (Machines running Smalltalk were what Steve Jobs saw on his PARC visit that led to the Lisa and then the Macintosh.)

It’s ironic that between then and when I ended up teaching in a school of computer science, I somehow lost that focus.  I’d gone to grad school to get a grounding in cognitive science in just such a place. After a post-doc looking at learner models, I ended up teaching interface design (and researching educational technology). Along the way I got involved in other issues, though I did get involved in HyperCard, which in many ways was Smalltalk Lite(tm ;).

In the talk, besides the enlightenment of the thinking behind it, there was also the practical aspects. While relatively lean, the language did take up memory and as a dynamic machine wasn’t blindingly fast. There apparently were also decisions about pricing and markets that were classic Xerox. Thus, while it was and is a fabulous modeling environment (still in use in a variety of markets), it didn’t take over the world.  When Steve Jobs built the NeXT computer, he took on the object-oriented model of Smalltalk, but used C as the core language for a variety of pragmatic reasons.

In the session, they talked about the vision of Seymour Papert and Logo, and how they wanted more. Alan Kay walked around with a cardboard model of what a Dynabook would look like, and people begged to buy one. Doug Englebart’s work also was an inspiration. It was a glorious flashback to the days when we dreamt bigger than our tech would support. These days, it seems, we’ve reversed that. I’ve heard that computing isn’t living up to the potential we have for digital technology to be an optimal augment for cognition, and I agree. We can do better, and should. So these are some small thoughts about Smalltalk. And get off my lawn!

Test and tune

by Leave a Comment

Conducting ScienceIt’s easy to believe that if we build solutions according to the principles, what we build will work. Certainly, that’s the case in many instances. But not, I suggest, when we’re talking about solutions for people? Why do we need to test and tune? Because we’re complex.

Materials, such as wood and steel have predictable properties. When you build a deck, you can follow the building codes for what you need to do for fixing posts to the ground, etc. However, people aren’t quite that predictable.

There’s a case to be made that the brain is the most complex thing in the known universe. In fact, we don’t fully know how it works! Thinking, then, that we can achieve a reliable change in the brain with simple mechanisms is kind of naive. We have to understand how the brain works, first, and then for complex changes, we need some detailed analysis and careful specification. However, we’re not done.

Once we’ve built it, we need to test and tune! Any solution isn’t guaranteed to be optimal or even effective, at least initially. Since people are complex, we can’t just design for the average (c.f. Todd Rose’s The End of Average). We can’t follow waterfall models, despite how appealing it is. Assuming we can is a path to boring and ineffective solutions. “If we build it, it is good” isn’t a useful assumption.

You see this in the best approaches. Michael Allen’s Successive Approximation (SAM), Megan Torrance’s Lot Like Agile Management Approach (LLAMA), David Merrill’s Pebble in the Pond, or Guy Wallace’s PACT approach (I’m not even going to try to deal with that acronym) all have iteration as a fundamental component.

You need some metrics, of course. You test against them, and then tune to get closer. The answer to “when do we stop iterating” is not “when we run out of time and money”. If you’re running out of time and money faster than you’re getting to your metrics, you need to explicitly consider some alternatives, like relaxing your goals, or investing more, or (horrors) abandoning where you’ve at. But it’s better to do it consciously!

To do this, we need to build ‘test and tune’ into our practices. We need to allocate time and money to it. Does this mean things will take longer and cost more? Possibly. The tradeoff is that we should be doing less courses overall, once we’re asking questions, and our solutions will be more effective. Going beyond knowledge dump courses that achieve no organizational benefit? I think that’s a fair exchange. I hope you do, too.

Help with breaking up

by 4 Comments

In a current engagement, we’re faced with the challenge of being given large goals. They can be learning, or performance support, or even just awareness. One of the things we’re wrestling with is how and when to break things up into smaller chunks. To be clear, this is about whether to put it into a separate entity, rather than how to segment within an entity. This isn’t an area of my expertise, I admit. As a consequence, I’ve done some scouring and pondering, and here’re some thoughts and my request for help with breaking up.

My initial reaction is that this is about curriculum. That is, the level above pedagogy: not how to teach, but what to teach. A term which made sense is ‘curriculum mapping‘. Which works for K12, but the advice I found wasn’t helpful in this instance. It seemed to be about iterating, which is good, but I was looking for some research-based principles. We don’t have external standards.

Asking around, my colleagues suggested it’s more like information architecture. I know a bit about information architecture, but not a lot. In general, I take it as organizing around the way users think about the content. Which is good for information, but not necessarily for learning.

Of course, your learning objectives should provide a guide. There should be a path from the learner’s initial state, through enabling objectives, to the final objective. That should define the scope of an experience. If it gets too big, then make one or more of the enabling objectives their own piece. In this case, we’re taking something already created and trying to make sense of it.

Really, I’m fine with that latter, but I just wonder if there are any evidence based principles to guide this thinking. Something besides seemingly sensible breakups. It all seems based upon perception (and iterative testing). However, I wonder if there are metrics, or a principled basis. Hence, my request. I’m asking for help with breaking up. What am I missing? Any pointers?

What’s New at Quinnovation

by Leave a Comment

Time for an update, as things have been happening, Some over the past year, some more recently. So here’s what’s new at Quinnovation.

So, as I’ve already noted, I was asked to join the Learning Development Accelerator last fall as Editor in Chief of the new LDA Press. That means, practically, serving as both acquisition and development editor. So far, I’m better in the latter than the former. We did publish our first, book, er, mine ;). I’ve also joined as co-0rganizer.

We’re doing some fun things with that. The first is the You Oughta Know series of Wed webinars, introducing people, well, that you oughta know. We’ve had an amazing run of guests. I’ll also be starting up a series of Very Opinionated Tech Takes. Yeah, it’s me going off on various techs, but I will be unpacking my thinking behind the takes. And I have had a number of decades of experience at it… Finally, there’s the forthcoming Learning & Development Conference, where we’re looking to do what we think should be done. That is, a focus on important specifics; we’ve two tracks, one more for beginners and one more advanced. Attendees can choose. We’ve got a good tight faculty, and are expecting a great development experience.

In addition to my LDA duties, I’m still continuing on with Quinovation, of course. That has meant some steady client work since the beginning of the year; including two projects on learning and performance strategy. One is likely to be continuing at least a little while longer. From that one is a topic I’ll be discussing as part of at DevLearn. I’ll also be running a ‘Make It Meaningful workshop there; a full day digging into the principles and tricks of making experiences emotionally engaging. (I’ll also be part of a panel of the Guild Masters.)

I’ve also taken on some other roles via Quinnovation. For one, I’ve been working with an initiative attempting to take K12 STEM into the 21st Century. Yes, there are plenty of efforts in this area, I just happen to be involved in this one that has an approach I’m getting to shape aligned with my perspectives. I’m also now on the advisory board of a startup with a platform for augmenting formal learning programs. Finally, I’ve signed on to assist an established elearning firm improve their approach as part of a growth initiative. All are focusing on applying learning science to make bigger and better impacts. Those are the types of things I can comfortably get behind.

So that’s what’s new at Quinnovation. Reckon I’ll be busier than the proverbial beaver, but that’s preferable to the alternative. Stay tuned!

Consumed by consumption?

by 2 Comments

I don’t listen to podcasts. Not that I have anything against them; I would if I had a driving commute where my eyes must be occupied while my mind isn’t fully. However, I don’t have such a situation, and otherwise I’d rather read (as with videos, I can generally read faster than they can talk, and I can also skim). However, pondering the increasing growth of podcasts and other media, I begin to wonder if we’re being consumed by consumption. 

To be clear, I like consuming information. I regularly am searching for information about new ideas I’ve come across, or going deeper in particular areas that have gained in importance in what I’m doing. I track Twitter and LinkedIn, and… However, I also take time for reflection. Which is important. If you look at Harold Jarche’s PKM model, you see that the middle section of ‘sense’ is where you process the information, synthesizing it. It’s an important part of learning, elaborating, experimenting, and internalizing.

Nothing wrong with consuming, but I’m suggesting it must be balanced with the synthesis, the integration, the reflection. If we only consume, I fear that we’re really not learning. We need to ponder what it means to us. Yet, in this increasingly fast-paced world, I worry that people feel they need to continually be consuming. I’m arguing that it shouldn’t be the only thing. It needs to be balanced. 

It’s not that I don’t think people reflect, I just want to make sure that they’re conscious of it, and don’t fall prey to a need to be distracted continually. We need to be present, with ourselves and our information, and process it. We need to avoid being consumed by consumption. Happy to believe that I’m wrong, but better to err on the side of over-communicating, eh?

A cautionary tale

by 2 Comments

A colleague mentioned me in calling out a post that was touting a myth. Which is bad enough, but as I’ve done before, I looked deeper at the post, and found more problems. It’s worth breaking down the problems, both to sensitize you to the potential problems, and provide some guidance about what’s right. So here’s a cautionary tale.

First, the post was promoting Dale’s Cone (referred to in the post as the Learning Retention Pyramid). You’ve probably seen it, saying people retain X% if they listen to a lecture, Y% if they read, … all the way to Z% if they teach others. It’s been resoundingly debunked. It is made up, Dale never added numbers, etc. So, we’re starting off on the wrong foot.

The post points to an article by an organization. As I’m wont to do, I went to the article. Not surprisingly, it wasn’t any better. It talked about the importance of looking at retention practice. That is, designing for retention; a good thing. Along with a definition were nine strategies for promoting this desirable feature. Which is where we start to go wrong.

There’s nothing wrong with an aggregation of strategies, but it helps if there’s any reason to believe this is a full and complete list, and maybe oriented by biggest impact. No, this appeared to have no rationale other than just a collection. Please, please, provide the rationale for any such collection!

Then, the very first one immediately jumped into an example that “generation Z’ would process. Ok, we’re onto myth number 2. Generations. Isn’t. A. Thing! Sigh, can we please stop thinking in terms of limiting categories.

Moreover, much of the recommendations seemed focusing on remembering information, not on applying it. Only one, “apply learning to the real world” talked at all about actually doing things. A side note: by and large, recalling information isn’t going to lead to meaningful behavior change.

Aside from some minor grammatical errors and odd statements (what does “large muscular people are dump” mean?), this seems like a random collection. It’s not that the strategies are wrong, but they’re unjustified, badly presented, and unfocused. To me, this is a cautionary tale. I would actively steer away from an org that used such marketing to promote its understanding of learning. (For the record, there’s no attribution to authorship, and though the original poster on LinkedIn was identified, they don’t appear in the ‘About’ page of the org.) Caveat emptor.

Templates as content model extensions

by 2 Comments

I’ve been touting content models for, well, years now. Interestingly, I’m currently doing some more concrete work on them, from the bottom-up. Instead of looking at top-down implementation of governance and structure, the focus is on guidance for creating resources at scale. Yet the two are related, and I think it’s worth looking at templates as content model extensions.

The notion of content models is that instead of creating full courses, we build content in chunks, and pull them together by rule. Or even more appropriately, deliver the appropriate chunk to the right person at the right time. It’s been happening for web design for years, but for some reason the notion of content management systems lags in L&D. Yes, there are entailments – governance, strategy, engineering – but the alternative is that lingering legacy content that’s out of date but no one can deal with.

That’s the top level focus. Underpinning this, of course, is getting the content right, and that means having some good definitions around the content. I’d done that many moons ago, and in a current engagement it’s reemerging. The situation is that there are a number of people all writing content around this particular initiative, and it’s uncoordinated (sound familiar?). The realization that clients are struggling is enough of a driver to look for a solution.

Without a content management system, as yet, it still makes sense to systematize the resources around a map of the space, ensuring they align to what we know about how people learn and perform. That latter is important, because many times they just need the answer now, not a full course.

What we’ve ended up doing is creating meta-content that tell how to develop content that meets particular needs. With entailments, such as assembling a representative team to determine what’s needed and the labels to use. It also involves drafting and testing these content guides, prior to broader use.

It’s the tactical step of a strategic goal to provide support for people to successfully meet their needs. And, to be clear, to reduce the reliance on the support staff. Leveraging the cognitive and learning sciences, we’re building templates as content model extensions. This is before there’s even the technology support available to be more proactive, but planning for the possible future is part of the strategy.

I’ll be presenting a session on this at the DevLearn conference in October. If you’re interested and going to be there, I welcome seeing you.

Emotion, or motivation, or…

by 4 Comments

I’ve been promoting the importance of the emotional ‘hook’. Which I was called out on, rightly. I freely admit that the phrase is less than fully accurate. So let’s explore the issue of whether we’re talking emotion, or motivation, or…

As context, one perspective from cognitive science is thinking of our ourselves as comprised of three components. One is cognitive, that is what we think and know. Which is malleable, as we can learn more! A second component is termed affective, that is who we are. Which isn’t malleable; these are our fixed characteristics, certainly personality (e.g. OCEAN/HEXACO). Finally, there’s our conative component. This is our intent: to learn, to act, and arguably what we value. 

This last bit, the conative, is what I suggest we neglect in our learning design. We address the cognitive, and there’s little to do on the affective side, but we too often basically assume that the learner is ready for what we’re presenting. Which I suggest is a mistake. 

There’s considerable evidence that we perform better when we’re engaged. When experiences are motivating, and anxiety is kept to a minimum, and our confidence is built, etc. Further, we can address these. We can help folks see the WIIFM, make it psychologically safe, and provide appropriate levels of challenge with useful feedback. (And we should!) However, too often we don’t.

So I use the shorthand term ‘emotion’ to address this.  I use the term to separate from the cognitive in a shorthand way. The problem, of course, is whether anxiety and motivation are truly ‘emotions’. Effectively, they’re not. My stance, however, is that I can’t be talking ‘conative’ to folks who aren’t aware of that concept!

I also resist just talking about motivation. Anxiety is an issue. So too is confidence. John Keller’ has his ARCS model (arguably the only ID theorist considering the conative aspect, though I don’t know if he uses that term ;). His model incorporates gaining/maintaining Attention, manifesting the Relevance of what’s being learned, building Confidence, and ensuring Satisfaction from the experience. Really, we want an umbrella term for these elements. 

There is the concept of engagement. However, it’s been trivialized. Claims that ‘click to see more’ is more engaging, that points & leaderboards are engaging, etc. have undermined the term. So I avoid it to avoid getting mired in that morass. Instead, I think talking about emotion as a shorthand way to address the non-cognitive. Yet…

I was called out for talking about an ‘emotional hook’ by a very learned client. Rightly so. The question is, do we have a better term? Is there a more appropriate and yet still accessible way ta talk about this? Obviously, I haven’t thought so as yet, but I want to keep learning. So if you’ve a better solution, please do let me know. (Feel free to also say that this approach is probably the only solution, at least for now.) Otherwise, I’ll keep looking for a better approach than emotion, or motivation, or…

I’ll be discussing ‘emotion’ in a session for the Learning Guild’s LXD conference, coming up at the beginning of next month. I’m also presenting a two half-days pre-event online workshop on LXD overall, integrating emotion with science. I’ll also explore ‘engagement’ in a full day face-to-face pre-con workshop at their DevLearn conference in October). Hope to see you at one of these!