Search Results for: align

Tools for LXD?

24 September 2019 by Clark Leave a Comment

I’ve been thinking on LXD for a while now, not least because I’ve an upcoming workshop at DevLearn in Lost Wages in October. And one of the things I’ve been thinking about are the tools we use for LXD. I’ve created diagrams (such as the Education Engagement Alignment), and quips, but here I’m thinking something else. We know that job aids are helpful; things like checklists, and decision trees, and lookup tables. And I’ve created some aids for the Udemy course on deeper elearning I developed. But here I want to know what you are using as tools for LXD? How do you use external resources to keep your design on track?

The simple rationale, of course, is that there are things our brains are good at, and things they’re not. We are pattern-matchers and meaning-makers, naturally making up explanations for things that happen. We’re also creative, finding solutions under constraints. Our cognitive architecture is designed to do this; to help us adapt to the first-level world we evolved in.

However, our brains aren’t particularly good at the second-level world we have created. Complex ideas require external representation. We’re bad at remembering rote and arbitrary steps and details. We’re also bad at complex calculations. This makes the case for tools that help scaffold these gaps in our cognition.

And, in particular, for design. Design tends to involve complex responses, in this case in terms of an experience design. That maps out over content, time, and tools. Consequently, there are opportunities to go awry. Therefore, tools are a plausible adjunct.

You might be using templates for good design. Here, you’d have a draft storyboard, for instance, that insures you’re including a meaningful introduction, causal conceptual model, examples, etc. Or you might have a checklist that details the elements you should be including. You could have a model course that you use as a reference.

My question, to you, is what tools are you using to increase the likelihood of a quality design, and how are they working for you? I’d like to know what you’ve found helpful as tools for LXD, as I look to create the best support I can. Please share!

Clear about the concept

19 September 2019 by Clark Leave a Comment

I went to hear a talk the other day. It was about competency-based education (CBE) for organizations. Ostensibly. And, while I’m now affiliated with IBSTPI, it’s not like I’m a competency expert. And maybe I expect too much, but I really hope for people to be clear about the concept. Alas, that’s not what I found.

So, it started out reasonably well, talking about how competencies are valuable. There were a number of points, and many made sense, although some were redundant. Maybe I missed some nuance? I try to be open-minded. It’s about creating clear definitions of performance, and aligning those with assessments. Thus, you’re working on very clear descriptions of what people should be doing.

It got interesting when the speaker decided to link CBE to Universal Design for Learning (UDL). And it’s a good program. UDL talks about using multiple representations to increase the likelihood for different learners to be able to comprehend and respond. This, in the talk, was mapped to three different segments: engaging the learners in multiple ways, communicating concepts in multiple ways, and allowing assessment in multiple ways. And this is good. For learning. Does it make sense for CBE?

To start, the argument was, you should make the rationale for the learning in multiple ways. While in general CBE inherently embodies meaningfulness in the nature of clear and needed skills, I don’t have a problem with this. I argue you should hook learners in emotionally and cognitively, and those can be separate activities. There was a brief mention of something like ‘learning styles’, but while now wary, I was ready to let it go.

However, the talk went on to make a case for multiple representations of content. And here the slide explicitly said ‘learning styles’ and used VARK. And don’t get me wrong, multiple representations and media are good, but not for learning styles! The current status is that there’s essentially no valid instrument to measure learning styles, and no evidence that even if you did, that it makes a difference. None. So, of course, I raised the issue. And we agreed that maybe not for learning styles, but multiple representations weren’t bad.

The final point was that there could be multiple forms of assessment. At this point, I wasn’t going to interrupt again, but at the end of the session raised the point that the critical element of CBE is aligning the assessment with the performance! You can’t have them do an interpretative dance about identifying fire hazards, for instance, you have to have them identify fire hazards! So, here the audience ultimately agreed that variability was acceptable as long as it measured the actual performance. Again, I don’t think the speaker was clear about the concept.

There were two major flaws in this talk. One was casually mashing up a couple of essentially incommensurate ideas. CBE and UDL aren’t natural partners. There can be overlapping concepts, but… The second, of course, is using a popular but fundamentally flawed myth about learning. If you’re going to claim authority, don’t depend on broken concepts.

To put it another way, I think it’s fair to expect speakers to be clear about the concept. (Either that, or maybe the lesson is that Clark shouldn’t be allowed to listen to normal speakers. ;) Please, please, know what you’re talking about before you talk about it. Is that too much to ask?

LXD Strategy

3 September 2019 by Clark Leave a Comment

In the continuing process of resolving what I want to do when I grow up (rest assured, not happening), I’ve been toying with a concept. And I’ve come up with the phrase: Learning Experience Design (LXD) Strategist. Which of course, begs the question of just what LXD strategy is. So here’s my thinking.

To me, LXD is about the successful integration of learning science and engagement. Yes, cognitive science studies both learning and engagement, but in my experience the two aren’t integrated specifically well. You either get something flashy but empty, or something worthwhile but dreary dull. I remember a particular company that produced rigorous learning that you’d rather tear your eyes out than actually consume. And, similarly, seeing an award winning product that was flashy, but underneath was just drill and kill. For something that shouldn’t be.

Learning experiences should emotionally hook you (e.g. ensuring you know that you need it, and that you don’t know it). Then it should take the necessary steps such as sufficient spaced meaningful practice resourced with appropriate models and examples and specifically feedback. Ultimately, it should transform the learner. Learners go from not having a clue to having a basic ability to do and how to continue to develop.

What is LXD strategy? Here I’m thinking about helping orgs restructure their design processes, and their org structure, to support delivering learning experience designs. This includes ensuring up front that this really does deserve learning instead of some other intervention, such as performance support. Then it includes how you work with SMEs, how you discern key decisions, wrap practice into contexts, etc. It’s also about using the tools – media and technology – to create a well-integrated experience. Note that the integration can include classrooms, ambient content and interactivity, and more. It’s about getting the design right, then implementing.

LXD strategy is about ensuring that resources and practices are aligned to create experiences that meet real org needs under pragmatic constraints. That’s what I’ve been doing in much of my work, and where my interests lead me as well. And it’s still a part of the performance ecosystem. Understanding that relationship is critical, when you start thinking about moving individuals from novices, through practitioners, to expertise. And the numbers of areas that will need this are going to increase.

LXD is, in my mind, the way we should be thinking about ID is now as LXD. And we need to not only think about what it is, and how to do it, but also how we organize to get it done. That, I think, is an important and worthwhile endeavor. So, what’s your thinking?

Level of polish?

22 August 2019 by Clark 4 Comments

A debate broke out amongst some colleagues the other day about the desirable level of polish in our elearning. One colleague was adamant that we were undermining our position by using low quality production. There was a lot of agreement. I had a slightly different view. Even after finding out he was talking more about external-facing content than internal, I still have some differences. After weighing in, I thought it required a longer response, and of course it has to go here.

So, the main complaint was that so much elearning looks dated and incomplete. And I agree! And others chimed in that this doesn’t have to be, while all agreed that it doesn’t need to approach game quality in effect. Then, in my mind, the question switches to “what is good enough?” And I think we do need an answer to that. And, it turns out, to also answer “and what does it take?”

What is good enough?

So, my first concern is the quality of the design. My mantra on design states that it has to be right first. Then you can implement it. If it isn’t right from the get-go, it doesn’t matter how you implement it. And the conversation took some time to sort this out. But let’s assume that the design’s right. Then, how much production values do you need?

The original complaint was that we’re looking slack by comparison. When you look at what’s being done in other, related, fields, our production values look last decade, if not last century! And I couldn’t agree more. But does that matter? And that’s where we start getting into nuances. My bottom line question is: “what’s the business case?”

So, I suggest that the investment in production values is based upon how important the ‘experience’ is. If it’s internal, and it’s a critical skill, the production values should be only enough to ensure that learners can identify the situation and perform appropriately (or get feedback). It needs a minimum level of professionalism, and that’s it. If you’re selling it to high-end customers and want to charge a premium price, you’ll need much more, of course.

The issue was that we’re losing credibility if we don’t approach a minimal level of competency. There were many arguments about the locus: fear of going out of bounds, managers oppression, low level tools, lack of skills, and more. And these all have validity. We should stipulate a minimal level. Perhaps the serious eLearning Design Manifesto? :) We can do better.

What does it take?

This was the other issue. It was pointed out that design teams in other disciplines work in layers: from concept to realization. Jesse James Garrett has a lovely diagram that represents this for information architecture. And others pointed out that there are multiple skills involved, from dialog writing, through media production and interface design (they’re conceptually separate), and the quality of the programming and more. The more you need polish, the more you need to invest in the appropriate skill sets. This again is a matter of marshaling the appropriate resources against the business case.

I think one of the issues is that we overuse courses when other solutions are more effective and efficient. Thus, we don’t have and properly allocate the resources to do the job right when it does positively absolutely has to be in the head. Thus, we do have a lot of boring, information dump courses. And we could be doing more with engaging practice, and less content presentation. That’s a design issue to begin, and then a presentation one.

Ultimately, I agree that bad elearning undermines our credibility. I do think, however, that we don’t need unnecessary polish. Gilded bad design is still bad design. But then we should align our investment with the professional reception we need. And if we have trouble doing that, we need to rethink our approaches. The right level of investment for the context is the right response; we need the right live of polish. But the assessment the context is complex. We shouldn’t treat is simplistically, but instead systemically. If we get that right, we have a chance to impress folks with our astute sense of doing the right thing with the right resources. Less than that is a path to irrelevancy, and doing more is a path to redundancy. Where do you want to go?

Little Whinging

6 August 2019 by Clark 1 Comment

Every once in a while, I have had enough of some things, and want to point them out. I do so not just to complain, but to talk about good principles that have implications beyond just the particular situation. So, here I go with a little whinging.

Services

Of late, when I call in for assistance, the phone system automatically asks me to verify some information. It can be an account number, or just to confirm some data like my house number. This is all good up until the point when I get connected to a live person, and they then ask me for that same data. Many times, as it’s escalated (“yes, it’s plugged in” and “yes, I’ve already tried rebooting it”), I get passed on to another person. And get asked for the same data again.

When pushed, “it’s our systems”. And that’s not good enough. What’s the lesson? You need your systems synched together. The employees need a performance ecosystem that’s integrated, if you’re going to be able to deliver a good customer experience. Reminded of the fact that Dominos is spending more money fighting to not have to be accessible than the estimate to actually make their system accessible!?!

This plays out in another way. So I’m having internet troubles. It’s intermittent (admittedly, that make it hard to diagnose), and it’s not disconnecting, it’s just slowing way down, and then going back to blazing fast.But it’s creating hiccups for my conference calls and webinars. I’m paying a pretty penny for this.

So, they do some remote stuff to the modem and say call back if it’s not better. And it’s not. So they send a tech. Who says it’s in the network, not the local connections and other techs will work on it, and I don’t have to be present, and they work 24/7 and it should be fixed in a couple of days. And then, I get a call which I return and am told it’ll be fixed by late this morning. And then it’s not. So I call again, and first, the person doesn’t seem to have access to the previous notes (which I’d made a point of), and asks me a bunch of questions. Which I’ve already answered previously in the same call. Then, they arrange to send a tech out! Isn’t that the definition of insanity, trying the same thing and expecting a different outcome?

The problem here is the lack of coordination between the different elements. The latest phone person said that they had the notes from the previous tech, and that this one has different skills, but the previous person had told a different story. It’s that that concerns me; the lack of consistency shatters my already-fragile confidence in them. They should have a good linked record (the ecosystem again), but be able to address obvious mismatches elegantly.

Products

Ok, so this one’s less obvious, but it’s relevant. Here’s my claim: I want products that aren’t just dishwasher-safe, I want them dishwasher-smart! What am I talking about? Look at these two glasses. It may be hard to see, but the one on the left has a three-lobed groove in the bottom. While there’s sufficient surface to stand steadily, it also drains. The one on the right, however, has a concavity in the bottom. So, when it goes in the dishwasher (or the dish drainer for that matter), water pools and it doesn’t dry efficiently. WHY?

Look, you should be designing products so the affordances (yeah, I said the ‘a’ word ;) work for consumers. I like my backup battery (thanks Nick and SealWorks) because it has a built-in cable! You don’t have to carry a separate one. This goes for learning experiences as well; make the desired behaviors obvious. Leave the challenges to the deliberate ones discriminating appropriate decisions from misconceived ones. And authoring tools should make it easy to do good pedagogy and difficult to do info dump and knowledge test! Ahem.

At core it’s about aligning product and service design with how we think, work, and learn. It should be in the products we purchase, and in the products we use. Heck, I can help if you want assistance in figuring this out, and baking it into your workflows. (I used to teach interface design, having had a Ph.D. advisor who is a guru thereof.). Do read Don Norman’s The Design of Everyday Things if you’re curious about any of this. It’s one of those rare books that will truly change the way you look at the world. For the better.

Design, whether instructional or industrial or interface or anything else that touches people needs to understand those people. Please ensure you do, and then use your powers for good.

Direct Instruction and Learning Experience Design

30 July 2019 by Clark Leave a Comment

After my previous article on direct instruction versus guided discovery, some discussion mentioned Engelmann’s Direct Instruction (DI). And, something again pointed me to the most comprehensive survey of educational effects. So, I tracked both of these down, and found some interesting results that both supported, and confounded, my learning. Ultimately, of course, it expanded my understanding, which is always my desire. So it’s time to think a bit deeper about Direct Instruction and Learning Experience Design.

Engelmann’s Direct Instruction is very scripted. It is rigorous in its goals, and has a high amount of responses from learners. Empirically, DI has great success, with some complaints about lack of teacher flexibility. It strikes me as very good for developing core skills like reading and maths. I was worried about the intersection of many responses a minute and more complex tasks, though it appears that’s an issue that has been addressed. I couldn’t find the paper that makes that case, however.

Another direction, however, proved fruitful. John Hattie, an educational researcher, collected and conducted reviews of 800+ meta-analyses to look at what worked (and didn’t) in education. It’s a monumental work, collected in his book Visible Learning. I’d heard of it before, but hadn’t tracked it down. It was time.

And it’s impressive in breadth and depth. This is arguably the single most important work in education. And it opened my eyes in several ways. To illustrate, let me collect for you the top (>.4) impacts found, which have some really interesting implications:

Reciprocal teaching (.74)
Providing feedback (.72)
Teaching student self-verbalization (.67)
Meta-cognition strategies (.67)
Direction instruction (.59)
Mastery learning (.57)
Goals-challenging (.56)
Frequent/effects of testing (.46)
Behavioral organizers (.41)

Reciprocal teaching and meta-cognition strategies coming out highly, a great outcome. And of course I am not surprised to see the importance of feedback. I have to say that I was surprised to see direct instruction and mastery learning coming out so high. So what’s going on? It’s related to what I mentioned in the afore-mentioned article, about just what the definition of DI is.

So, Hattie says: …”what the critics mean by direct instruction is didactic teacher-led talking from the front…” And, indeed, that’s my fear of using the label. He goes on to point out the major steps of DI (in my words):

Have clear learning objectives: what should the learner be able to do?
Clear success criteria (which to me is part of 1)
Engagement: an emotional ‘hook’
A clear pedagogy: info (models & examples), modeling, checking for understanding
Guided practice
Closure of the learning experience
Reactivation: spaced and varied practice

And, of course, this is pretty much everything I argue for as being key to successful learning experience design. And, as I suspected, DI is not what the label would lead you to believe (which I do think is a problem). As I mentioned in a subsequent post, I’ve synthesized my approach across many elements, integrating the emotional elements along with effective education practice (see the alignment). There’s so much more here, but it’s a very interesting result. Direct Instruction and Learning Experience Design have a really nice alignment.

And a perfect opportunity to remind you that I’ll be offering a Learning Experience Design workshop at DevLearn, which will include the results of my continuing investigation (over decades) to create an approach that’s doable and works. Hope to see you there!

Drink your own champagne?

25 July 2019 by Clark 2 Comments

I was talking with a vendor of a robust suite of tools. In the course of it, in my usual teasing way, I asked a question. And, while I wasn’t surprised at the answer, I was ‘concerned’. And so should you be. So I’m going to suggest you start asking of your vendors “Do you drink your own champagne?”

So, this was a manufacturer of an LMS (and some other, related, platforms). And they market their advanced capabilities. And, really, I have no problem with their tools; they seem pretty enlightened. So I asked whether they used their own tools.

And there are reasons to do so. For one, to have credibility, for sure. And, to truly know your own product. But the really important reason is to be able to understand the use experience and tune accordingly. Customer research is an important tool here as well, but it’s not the only one. You really need to use something to truly know what works and what doesn’t.

It’s a form of experimentation. I test myself by trying to apply my principles in my endeavors, and then try to take on new situations to try out my beliefs more broadly. And so should you, at the individual and organizational level. Using your own tool is a form of this. It is, essentially, testing your theory with research!

And I think most folks with services and such are likely to practice what they preach. And that could be for some bad things as well as good (thinking: learning styles vendors). But I was surprised when the answer was only “somewhat”. That’s not really good enough.

So, I’m going to suggest that this should be a question you ask of every vendor. I’m also going to suggest every vendor ensure that they do use their own tool. Internally, for their own work. Whether it’s authoring tools, a course management system, a portal, a web meeting tool, what have you. If you don’t drink your own champagne, you’re not only undermining trust, you’re losing a valuable source of information. Now, pass the bubbly, would you?

Dimensions of difficulty

11 July 2019 by Clark 1 Comment

As one of the things I talk about, I was exploring the dimensions of difficulty for performance that guide the solutions we should offer. What determines when we should use performance support, automate approaches, we need formal training, or a blend, or…? It’s important to have criteria so that we can make a sensible determination. So, I started trying to map it out. And, not surprisingly, it’s not complete, but I thought I’d share some of the thinking.

So one of the dimensions is clearly complexity. How difficult is this task to comprehend? How does it vary? Connecting and operating a simple device isn’t very complex. Addressing complex product complaints can be much more complex. Certainly we need more support if it’s more complex. That could be trying to put information into the world if possible. It also would suggest more training if it has to be in the head.

A second dimension is frequency of use. If it’s something you’ll likely do frequently, getting you up to speed is more important than maintaining your capability. On the other hand, if it only happens infrequently, it’s hard to try to keep it in the head, and you’re more likely to want to try to keep it in the world.

And a third obvious dimension is importance. If the consequences aren’t too onerous if there are mistakes, you can be more slack. On the other hand, say if lives are on the line, the consequences of failure raise the game. You’d like to automate it if you could (machines don’t fatigue), but of course the situation has to be well defined. Otherwise, you’re going to want lot of training.

And it’s the interactions that matter. For instance, flight errors are hopefully rare (the systems are robust), typically involve complex situations (the interactions between the systems mean engines affect flight controls), and have big consequences! That’s why there is a huge effort in pilot preparation.

It’s hard to map this out. For one, is it just low/high, or does it differentiate in a more granular sense: e.g. low/medium/high? And for three dimensions it’s hard to represent in a visually compelling way. Do you use two (or three) two dimensional tables?

Yet you’d like to capture some of the implications: example above for flight errors explains much investment. Low consequences suggest low investment obviously. Complexity and infrequency suggest more spacing of practice.

It may be that there’s no one answer. Each situation will require an assessment of the mental task. However, some principles will overarch, e.g. put it in the world when you can. Avoiding taxing our mental resources is good. Using our brains for complex pattern matching and decision making is likely better than remembering arbitrary and rote steps. And, of course, think of the brain and the world as partners, Intelligence Augmentation, is better than just focusing on one or another. Still, we need to be aware of, and assessing, the dimensions of difficulty as part of our solution. Am I missing some? Are you aware of any good guides?

Engaging Learning and the Serious eLearning Manifesto

9 July 2019 by Clark Leave a Comment

Way back in ’05, my book on games for learning was published. At its core was an alignment between what made an effective education practice and what makes engaging experiences. There were nine elements that characterized why learning should be ‘hard fun’. More recently, we released the Serious eLearning Manifesto. Here we had eight values that differentiated between ordinary elearning and serious elearning. So, the open question is how do these two lists match up? What is the alignment between Engaging Learning and the Serious eLearning manifesto?

The elements of the Serious eLearning Manifesto (SeM) are pretty straightforward. They’re listed as:

performance focused
meaningful to learners
engagement driven
authentic contexts
realistic decisions
real-world consequences
spaced practice
individualized challenges

The alignment (EEA: Effectiveness-Engagement Alignment) I found in Engaging Learning was based upon research I did on designing games for learning. I found elements that were repeated across proposals for effective education practice, and ones that were stipulated for engaging experiences. And I found a perfect overlap. Looking for a resolution between the two lists of elements looks something like:

clear goals
balanced challenge
context for the action
meaningful to domain
meaningful to learner
choice
active
consequences
novelty

And, with a little wordsmithing, I think we find a pretty good overlap! Obviously, not perfect, because they have different goals, but the important elements of a compelling learning experience emerge.

I could fiddle and suggest that clear goals are aligned to a performance focus, but instead that’s coming from making their learning be meaningful to the domain. I suggest that what really matters to organizations will be the ability to do, not know. So, really, the goals are implicit in the SeM; you shouldn’t be designing learning unless you have some learning goals!

Then, the balanced challenge is similar to the individualized challenge from the SeM. And context maps directly as well. As do consequences. And meaningfulness to learners. All these directly correspond.

Going a little further, I suggest that having choice (or appearance thereof) is important for realistic decisions. There should be alternatives that represent misconceptions about how to act. And, I suggest that the active focus is part of being engaging. Though, so too could novelty be. I’m not looking at multiple mappings but they would make sense as several things would combine to make a performance focus, as well as realistic decisions.

Other than that, on the EEA side the notion of novelty is more for engaging experiences than necessarily specific to serious elearning. On the SeM side, spaced practice is unique to learning. The notion of a game implies the ability for successful practice, so it’s implicit.

My short take, through this exercise, is to feel confident in both recommendations. We’re talking learning experience design here, and having the learning combine engagement as well is a nice outcome. I note that I’ll be running a Learning Experience Design workshop at DevLearn in October in Las Vegas, where’ll we’ll put these ideas to work. Hope to see you there!

The ITA Jay Cross Memorial Award for 2019: Michelle Ockers

5 July 2019 by Clark 1 Comment

Over a decade ago, my friend Jay Cross invited me to join the Internet Time Alliance. He had been touting the value of Informal Learning, and realized he was doing it alone. I was honored to join Jane Hart, Harold Jarche, and Charles Jennings, and have come to know and value them as colleagues and friends. When Jay passed away, we determined to honor his ideas by recognizing those who continue to carry the banner for informal learning. We announce the ITA memorial award on 5 July, Jay’s birthday. This year’s winner is Michelle Ockers.

I’ve only met Michelle once, when I was visiting Australia to deliver a keynote. She was kind enough to ask me to sign a copy of Revolutionize L&D. I didn’t know much about her work then, but have subsequently seen it in a variety of places. She’s active in social media, for instance. She also coordinated the Learning & Performance Institute capability map exercise that occurred last summer. She’s systematically demonstrated broad ranging interests and abilities around organizational learning.

I’m pleased that we can honor her and her work helping organizations work more productively and fluidly. The official announcement is on the ITA site. Congratulations, Michelle on the 2019 ITA Jay Cross Memorial Award!

« Previous Page