Learnlets

I was asked a somewhat challenging question the other day, and it led me to reflect. As usual, I‘m sharing that with you. The question was “How can IDs keep up with everything, feel competent and confident in our work” It‘s not a trivial question! So I‘ll share my response to overworked IDs.

There was considerable context behind the question. My interlocutor weighed in with her tasks:  

“sometimes I wonder how to best juggle everything that my role requires: project management, design and ux/ui skills, basic coding, dealing with timelines and SMEs and managers. Don‘t forget task analysis and needs assessment skills, making content accessible and engaging. And staying on top of a variety of software.”  

I recognize that this is the life of overworked IDs, particularly if you‘re the lone ID (which isn‘t infrequent), or expected to handle course development on your own. Yet it is a lot of different competencies. In work with IBSTPI, where we‘re defining competencies, we‘re recognizing that different folks cut up roles differently. Regardless, many folks wear different competency requirements that in other orgs are handled by different teams. So what‘s a person to do?

My response focused on a couple of things. First, there‘re the expectations that have emerged. After 9/11, when we were avoiding travel, there was a push for elearning. And, with the usual push for efficiency, rapid elearning became the vogue. That is, tools that made it easy to take PDFs and PPTs and put it up online with a quiz. It looked like lectures, so it must be learning, right?

One of the responses, then, is to manage expectations. In fact, a recent post addressed the gap between what we know and what orgs should know. We need to reset expectations.

As part of that, we need to create better expectations about what learning is. That was what drove the Serious eLearning Manifesto [elearningmanifesto.org], where we tried to distinguish between typical elearning and serious elearning. Our focus should shift to where our first response isn‘t a course!  

As to what is needed to feel competent and confident, I‘ve been arguing there are three strands. For one (not surprisingly ;), I think IDs need to know learning science. This includes being able to fill in the gaps in and update on instructional design prescriptions, and also to be able to push back against bad recommendations. (Besides the book, this has been the subject of the course I run for HR.com via Allen Academy, will be the focus of my presentation at ATD ICE this summer, and also my asynchronous course for the LDC conference.)  

Second, I believe a concomitant element is understanding true engagement. Here I mean going beyond trivial approaches like tarting-up drill-and-kill, and gamification, and getting into making it meaningful. (I‘ve run a workshop on that through the LDA, and it will be the topic of my workshop at DevLearn this fall.)

The final element is a performance ecosystem mindset. That is, thinking beyond the course: first to performance support, still on the optimal execution side of the equation. Then we move to informal learning, facilitating learning. Read: continual innovation! This may seem like more competencies to add on, but the goal is to reduce the emphasis (and workload) on courses, and build an organization that continues to learn. I address this in the  Revolutionize L&D book, and also my mobile course for Allen Interactions (a mobile mindset is, really, a performance ecosystem mindset!).

If you‘re on top of these you should prepared to do your job with competence and confidence. Yes, you still have to navigate organizational expectations, but you‘re better equipped to do so. I‘ll also suggest you stay tuned for further efforts to make these frameworks accessible.  

So, there‘re my responses to overworked IDs. Sorry, no magic bullets, I‘m afraid (because ‘magic‘ isn‘t a thing, sad as that may be). Hopefully, however, a basis upon which to build. That‘s my take, at any rate, I welcome hearing how you‘d respond.

As has become a pattern, someone recently asked me how to evaluate soft skills. And without being an expert on soft skill or evaluation, I tried to answer on principle. So I thought about the types of observable data you should expect to find. And that yielded an initial answer. Then I watched an interesting video of a lecture by a scholar and consultant, and it elaborated the challenges. So, there‘s a longer answer too. So here‘s an extended riff on evaluating soft skills.

I started with wondering what performance outcomes would you expect for soft skills. Coupled, as well, with how could you find evidence of these observable differences. As a short answer, I suggested that there should be 3(+) outcomes from effective soft skills training.  

0) the learner should be able to perform in soft skills scenarios (c.f. Will Thalheimer’s LTEM). This is the most obvious. Put them in the situation and ask them to perform. This is the bit that gets re-addressed further down.  

1) the learner should be aware of an improvement in their ability to perform. However, asking immediately can lead to a misapprehension of ability. So, as Will Thalheimer advises in his Performance-Focused Smile Sheets, ask them 3 months later. Also, ask about behavior, not knowledge.   E.g. “Are you using the <> model in your work, and do you notice an improvement in your ability”

2) The ‘customers’ of the learner should notice the improvement. Depending on whether that’s internal or external, it might show up (at least in aggregate) in either 360 eval scores, or some observable metric like customer sat scores. It may be harder to collect this data, but of course it‘s also more valuable.  

3) Finally, their supervisors/managers should notice the improvement, whether observationally or empirically.They should be not only prepared to support the change over time, but asked to look for evidence (including as a basis to fine tune performance).  

All together, triangulating on this should be a way to establish the validity.  

Now, extending this, Guy Wallace tweeted a link to a lecture by Neil Rackham. In it, Neil makes the case that universities need to change to teaching core skills, in particular the 4 C‘s: critical thinking, creativity, communication, and collaboration. He also points out how hard it is to evaluate these without a labor-intensive effort of an individual observing performance. This is a point that others have made, that these skills have hard to observe criteria.  

There‘s some argument about so-called 21C skills, and yet I can agree that these four things would be good. The question is how to assess them reliably. Rackham argues that perhaps AI can help here. Perhaps, but at this point I‘d argue for two things. First, help students self-evaluate (which has the benefits of them understanding what‘s involved). Second, instrumenting environments (say, for instance, with xAPI) in which these activities are performed. There will be data records that can be matched to behaviors, initially for human evaluation, but perhaps ultimately for machine evaluation.  

Of course, this requires assigning meaningful activities that necessarily involve creativity, critical thinking, communication, and/or collaboration. This means project based work, and I‘ve long argued that you can‘t learn such skills without a domain. Actually, to create transferable versions, you‘d need to develop the skills across domains.  

When I teach, I prefer to give group work projects that do require these skills. It was, indeed, hard to mark these extra skills, but I found that scaffolding it (e.g. a ‘how to collaborate‘ document) facilitated good outcomes. Being explicit about the best thinking practices isn‘t only a good idea, it‘s a demonstrably useful approach in general.  

So I think developing skills is important. That means we need a means to be evaluating soft skills. We know it when we see it, but it‘s hard to necessarily find the opportunity, but if we can assign it, we can evaluate and develop these skills more readily. That, I think, is a desirable goal. What think you?

A colleague asked “is anyone talking about how deep learning requires time, attention, and focus” He was concerned with “the trend that tells us everything must be short.”   He asked if I‘d written anything, and I realize I really haven‘t. Well, I did make a call  for “slow learning” once upon a time, but it‘s probably worth doing it again.   So here‘s a riff on deep learning and expertise.

First, what do we mean by deep learning? Here, I‘m suggesting that the goal of deep learning is expertise. We‘ve automated enough of the component elements that we can use our conscious processes to make expert judgments in addressing performance requirements. This could be following a process, making strategic decisions such as diagnoses and prescriptions, and more. It can also require developing pre-conscious responses, such as we train airline pilots to respond to emergencies.  

Now, these responses can vary in their degree of transfer. Making decisions about how to remedy a piece of machinery that‘s misbehaving is different than deciding how to prioritize the new product improvements. The former is more specific, the latter is more generic. Yet, there are certain things that are relevant to both.  

Another issue is how often it needs to be performed. You can develop expertise much quicker with lots of opportunities to apply the knowledge. It‘s more challenging to achieve when there aren‘t as many times it‘s relevant in the course of your workflow. The aforementioned pilots are training for situations they never hope to see!

Before we get there, however, there‘s one other issue to address: how much has to go in the head, and how much can be in the world?   In general, getting information in the head is hard (if we‘re doing it right), and we should try to avoid it when possible. I argue  for backwards design, starting with what the performance looks like if we‘ve focused on IA (intelligence augmentation ), that is, looking for the ideal combination of smarts between technology (loosely defined) and our heads. As Joe Harless famously said “iInside every fat course there‘s a thin job aid crying to get out.”  

Once we‘ve determined that we need human expertise, we also need to acknowledge that it takes time! I put it this way: the strengthening of connections (what learning is at the neural level) can only be done so much in any one day before the strengthening function fatigues; you literally need sleep before you can learn more. And only so much strengthening can happen in that one day. So to develop strong connections, e.g. strong enough that it will be triggered appropriately, is going to have to be spaced out over time.  

This does depend on the pre-existing knowledge of the learner, but it was Anders Ericsson who posited the approximately 10K hours of practice to achieve expertise. That‘s both not quite accurate and not quite what he said, but as a rule of thumb it may be helpful. The important thing is that not just any practice will work. It takes what he called ‘deliberate practice‘, that is the right next thing for this learner. Continued, over time, as the learners‘ ability increases new practice focuses are necessary.

All that can‘t come from a course (no one is going to sit through 10000 hours!). Instead, if we follow the intent of the 70:20:10 framework, it‘s going to take some initial courses, then coaching, with stretch assignments and feedback, and joining a relevant community of practice, and….

We also can‘t assume that our learners will develop this as efficiently as possible. Unless we‘ve trained them to be good self-learners, it will take guided learning across their experience. Even if it‘s only at a particular point; most people who are pursuing a sport, hobby, what have you, eventually will take a course to get past their own limitations and accelerate development.

The short answer is that deep expertise doesn‘t, can‘t, come from a short learning experience. It comes from an extended learning experience, with spaced, deliberate, and varied practice with feedback. If you want expertise, know what it takes and do it. That‘s true whether you‘re doing it for yourself or you‘re in charge of it for others. Deep learning and expertise comes with hard work. (Also, let‘s make that ‘hard fun‘ ;).  

Asked about why I used the word pedagogy instead of andragogy, I think it’s worth elaborating (since I already had in my reply ;) and sharing. In short, I think it‘s a false dichotomy. So here‘s my analysis of andragogy vs pedagogy.

Looking at Knowles‘ andragogy, I think it‘s misconstrued. What he talks about for adults is really true for all learners, taking into account their relative cognitive capability and amount of experience. So I fear that using andragogy will perpetuate the myth that pedagogy is a different learning approach (and keep kids in classrooms listening to lectures and answering rote questions). Empirically, direct instruction works (tho‘ it‘s interpretation is different than the name might imply, I once pointed out how it and constructivism properly construed both really say the same thing ;).  

There was an article  that posited five differences, and I see a major confound; the article‘s talking about andragogy as self-directed learning, and pedagogy as formal instruction. That‘s apples and oranges. It really is more about whether you‘re a novice or a practitioner level and the role of instruction. Age is an arbitrary element here, not a defining factor. Addressing each point:

1. Adults are self-directing learners. No, in things they know they need, they can be, but also they may have their bosses or coaches pointing them to courses. Plus, for areas where the adults are novices, they still need guided instruction. Also, owing to our bad K12 and higher ed, we’re not really enabling learners to be effective and efficient self-directed learners. Further, kids are self-directed about things they‘re interested in. But we make little effort to ground what we do (particularly K6) in any reason why this is on the syllabus.  

2. The role of learner experience. Yes, this matters, but it‘s a continuum. Also, you always want to base instruction on learner experience, because elaboration requires connecting to and building on existing knowledge. Yes, we do tend to do give kids abstract problems (particularly in math), which is contrary to good learning science. “Only two things wrong in education these days, the curriculum and the pedagogy, other than that we‘re fine.” Ahem. We teach the wrong things, badly.  

3. Adults generate interest in useful information. So does everyone, but that‘s not a matter of developmental level. Kids also prefer stuff that‘s relevant. We‘ve developed a curriculum for kids that is out of date, and we don‘t motivate it. Everyone has a curriculum, and there are degrees of self-direction, but it‘s not a binary division.

4. Adult readiness to learn is triggered by relevance (yeah, kind of redundant).Kids also learn better when there‘s a reason. Hence problem-based, service-based, and other such philosophy‘s of learning. Even direct instruction posits meaningful problems. Again, the article‘s comparing an ideal human learning model compared to a broken school model.  

5. What motivates learners are real life outcomes. Really, we‘ve covered this, everyone learns better when there‘s motivation. Children learn for grades because no one‘s made it meaningful for them to care!   Kids will pursue their learning when it makes sense to them. John Taylor Gatto made the case that kids could learn the entire K6 curriculum in 100 hours if they cared! Kids do learn outside of what‘s forced on them from schooling, be it Pokemon, polka, or porcupines.  

Thus, in the comparison between andragogy vs pedagogy, I come down on the side of pedagogy. It‘s the earlier term, and while ped does mean ‘kid‘, I still think it‘s really about learning design. Learning design should be aligned to our brains, not differentiated between child and adult. Yes, there are developmental differences, but they‘re a continuum and it‘s more a matter of capacity, it‘s not a binary distinction. That‘s my take, what‘s yours?

I stumbled across the Elements of UX diagram again, and happened to wonder if it would map to LXD. Here’s my stab:

And the text, as usual.

---

In a justifiably well-known image (PDF), Jesse James Garrett (JJG) detailed the elements of (web) user experience. I‘ve been involved in the parallel development of UX and ID (and cross-fertilized them), so I wondered what the LXD version would be. So, of course, I took a stab at levels of LXD design.

To start with, JJG‘s diagram works from the bottom up. The five levels, in order, are:

1. The original objectives and user needs.
2. That leads to content requirements and/or functional specifications.  
3. The next level is an information architecture or interface design that is structured to meet those needs.  
4. Those semantic structures are then rendered as an information design with navigation or interface design.
5. The top level is the visual design, what the user actually sees or experiences.

This systematic breakdown has been well recognized as a useful development framework. The development from need to semantics to implementation syntax suggests a logical development flow. As an aside, no one‘s claiming we should develop in a linear manner, and there tends to be more up and down action in actual practice. Drilling down and then working from the bottom up as well is a well-known cycle of design!  

The learning equivalent, then, should similarly have a structured flow. We want to go from our needs, through various levels of representation, until we reach the learner experience.  

Given that we should be driven not by the goals for the interface but learner needs, I‘ll suggest we start with the performance objectives.   Then, in parallel with user needs, I‘ll stipulate that the other top-level definition comes from the user characteristics. These match the initial level stipulated.  

At the next level, I‘ll suggest that the performance objectives drive assessment specifications, and the other decision at this level is for the pedagogical approach. We need to know what learners need to able to do, and how we‘ll get them there.

As an intermediate representation equivalent to UX‘s information architecture or interface design, I suggest from the assessment we determine the necessary practice activities required, and these are coupled with the necessary content requirements: models and examples, as well as the introduction and closing. Here we‘re still at what‘s required, not how it manifests.  

The next level is where we start getting concrete. We need to pick an overall theme or look and feel, and the flow of the experience. We‘ll also, of course, need to make a consistent interface to support navigation and taking action. We know what we need to have, but we haven‘t actually rendered it yet.  

Finally, we must render the necessary media. This will be the videos, audios, text, diagrams, images, and more that comprise the experience. This includes the actions to be taken and the associated consequences of each choice.  

That‘s the equivalent structure I‘m suggesting are the different levels of LXD design. Of course, this is a thought exercise, and so I may well have made some interpretations you could disagree with. For instance, I may have slavishly followed JJG’s levels too closely. Let me know! Also, it‘s not clear whether this is a useful representation, so far it‘s sort of a ‘because it‘s there‘ effort ;). You can let me know your thoughts on that, too!  

Here’s a story about where performance support would’ve made a task much easier.

And, as always, the text.

---

The other day, I had a classic need for performance support. Of course, it didn‘t exist. So here‘s a cognitive story about when and where a job aid would help.

Our Bosch dishwasher stopped near the beginning of the cycle, and displayed an icon of a water tap. The goal was to get the dishwasher running again. What with the layer of undrained water, we figured there was some sort of problem with the drain, clogged or the pump broken. M‘lady had cleaned the drain, but the icon persisted. What now? Of course we could call a service person, but trying to be handy and frugal (and safe), we wanted to find out if it was something I could deal with. So, off to the manual.

Well, in this case, since I didn‘t know where the manual was, I went online. I accessed the site and downloaded the manual. Only to find no guide to what the icons mean. What?!? This violates what we know about our brains, in this case that our memory is limited. The support section of the site did list the error codes, but numerically, not by icon.  So, I had an indication I couldn’t map to a problem, let alone a  solution.  

This is a real flaw! If you‘re gonna use icons, provide a guide!  Don’t assume they’re interpretable. (This had happened once before with this same appliance, with an impenetrable icon and no clue.) As a result, I had to call the service line. That wait took awhile (with more people staying home, they‘re using their dishwashers more, and the appliances are therefore breaking down more). Once, the call dropped. The second time I had to stop because I had an upcoming call. The third time, however, I got through.

And a perfectly nice person listened, asked some questions, and then instructed me through a process. After hitting cancel (which automatically tries to drain everything and reset to zero) by simultaneously pressing two buttons linked by a line on the control panel, I heard noises in the sink like it was draining. After a minute, I was told to go ahead and open it up (yep, drained), turn it off and on, and then try running the cleaning cycle again. And, voila, it worked! (Yay!)

So, what‘s wrong with this picture? First of all, there should be a clear explanation of what the icon means, as indicated above. Second, it should be clearly tied to a process to address the problem, including intermediate steps.This is so common, I am quite boggled that the great engineers that made our (very good) dishwasher aren‘t complemented with a great technical communications team who write up a useful manual to support. It. Is. Just. Silly!

Note: this isn‘t a learning experience. It‘s just fine that I don‘t recall what the last time‘s icon was or what it meant, and maybe what this icon meant and what I should do. It should be infrequent enough that it‘d be unreasonable for me to have to recall. Instead, I should be able to look it up. Put information in the world!  In the long term, this should save them buckets of money because most people could self help. Clearly, they‘ve gone to numeric codes, but they could‘ve just added in the associated icons, or given a mapping from icon to numeric code. Something to help folks who have the pics.  

This is just bad design, and it‘s so obvious how to ameliorate it. People will self-help many times, but only if they can!   Just as you shouldn‘t be creating a training course when a job aid will do, you can save a help call when a job aid can address most of the problems. Use performance support when it makes sense, and doing so comes from understanding how we actually think, work, and learn. When you do, you can design solutions that meet real needs. And that‘s what we want to do, no?