Learnlets

Secondary

Clark Quinn’s Learnings about Learning

New recommended readings

by Leave a Comment

My Near Book ShelfOf late, I‘ve been reading quite a lot, and I‘m finding some very interesting books. Not all have immediate take homes, but I want to introduce a few to you with some notes. Not all will be relevant, but all are interesting and even important. I‘ll also update my list of recommended readings. So here are my new recommended readings. (With Amazon Associates links: support your friendly neighborhood consultants.)

First, of course, I have to point out my own Learning Science for Instructional Designers. A self-serving pitch confounded with an overload of self-importance? Let me explain. I am perhaps overly confident that it does what it says, but others have said nice things. I really did design it to be the absolute minimum reading that you need to have a scrutable foundation for your choices. Whether it succeeds is an open question, so check out some of what others are saying. As to self-serving, unless you write an absolute mass best-seller, the money you make off books is trivial. In my experience, you make more money giving it away to potential clients as a better business card than you do on sales. The typically few hundred dollars I get a year for each book aren‘t going to solve my financial woes! Instead, it‘s just part of my campaign to improve our practices.

So, the first book I want to recommend is Annie Murphy Paul‘s The Extended Mind. She writes about new facets of cognition that open up a whole area for our understanding. Written by a journalist, it is compelling reading. Backed in science, it’s valuable as well. In the areas I know and have talked about, e.g. emergent and distributed cognition, she gets it right, which leads me to believe the rest is similarly spot on. (Also her previous track record; I mind-mapped her talk on learning myths at a Learning Solutions conference). Well-illustrated with examples and research, she covers embodied cognition, situated cognition, and socially distributed cognition, all important. Moreover, there‘re solid implications for the redesign of instruction. I‘ll be writing a full review later, but here‘s an initial recommendation on an important and interesting read.  

I‘ll also alert you to Tania Luna‘s and LeeAnn Renninger‘s Surprise. This is an interesting and fun book that instead of focusing on learning effectiveness, looks at the engagement side. As their subtitle suggests, it‘s about how to Embrace the Unpredictable and Engineer the Unexpected. While the first bit of that is useful personally, it‘s the latter that provides lots of guidance about how to take our learning from events to experiences. Using solid research on what makes experiences memorable (hint: surprise!) and illustrative anecdotes, they point out systematic steps that can be used to improve outcomes. It‘s going to affect my Make It Meaningful  work!

Then, without too many direct implications, but intrinsically interesting is Lisa Feldman Barrett‘s How Emotions Are Made. Recommended to me, this book is more for the cog sci groupie, but it does a couple of interesting things. First, it creates a more detailed yet still accessible explanation of the implications of Karl Friston‘s Free Energy Theory. Barrett talks about how those predictions are working constantly and at many levels in a way that provides some insights. Second, she then uses that framework to debunk the existing models of emotions. The experiments with people recognizing facial expressions of emotion get explained in a way that makes clear that emotions are not the fundamental elements we think they are. Instead, emotions social constructs! Which undermines, BTW, all the facial recognition of emotion work.

I also was pointed to Tim Harford‘s The Data Detective, and I do think it‘s a well done work about how to interpret statistical claims. It didn‘t grip me quite as viscerally as the afore-mentioned books, but I think that‘s because I (over-)trust my background in data and statistics. It is a really well done read about some simple but useful rules for how to be a more careful reviewer of statistical claims. While focused on parsing the broader picture of societal claims (and social media hype), it is relevant to evaluating learning science as well.  

I hope you find my new recommended readings of interest and value. Now, what are you recommending to me? (He says, with great trepidation. ;)

The case for model answers (and a rubric)

by 4 Comments

Human body modelAs I‘ve been developing online workshops, I‘ve been thinking more about the type of assessment I want. Previously, I made the case for gated submissions. Now I find another type of interaction I‘d like to have. So here‘s the case for model answers (and a rubric).

As context, many moons ago we developed a course on speaking to the media. This was based upon the excellent work of the principals of Media Skills, and was a case study in my  Engaging Learning book. They had been running a face to face course, and rather than write a book, they wondered if something else could be done. I was part of a new media consortium, and was partnered with an experienced CD ROM developer to create an asynchronous elearning course.  

Their workshop culminated in a live interview with a journalist. We couldn‘t do that, but we wanted to prepare people to succeed at that as an optional extra next step. Given that this is something people really fear (apocryphally more than death), we needed a good approximation. Along with a steady series of exercises going from recognizing a good media quote, and compiling one, we wanted learners to have to respond live. How could we do this?

Fortunately, our tech guy came up with the idea of a programmable answering machine. Through a series of menus, you would drill down to someone asking you a question, and then record an answer. We had two levels: one where you knew the questions in advance, and the final test was one where you‘d have a story and details, but you had to respond to unanticipated questions.  

This was good practice, but how to provide feedback? Ultimately, we allowed learners to record their answers, then listen to their answers and a model answer. What I‘d add now would be a rubric to compare your answer to the model answer, to support self-evaluation. (And, of course, we’d now do it digitally in the environment, not needing the machine.)

So that‘s what I‘m looking for again. I don‘t need verbal answers, but I do want free-form responses, not multiple-choice. I want learners to be able to self-generate their own thoughts. That‘s hard to auto-evaluate. Yes, we could do whatever the modern equivalent to Latent Semantic Analysis is, and train up a system to analyze and respond to their remarks. However, a) I‘m doing this on my own, and b) we underestimate, and underuse, the power of learners to self-evaluate.  

Thus, I‘m positing a two stage experience. First, there‘s a question that learners respond to. Ideally, paragraph size, though their response is likely to be longer than the model one; I tend to write densely (because I am). Then, they see their answer, a model answer, and a self-evaluation rubric.  

I‘ll suggest that there‘s a particular benefit to learners‘ self-evaluating. In the process (particularly with specific support in terms of a mnemonic or graphic model), learners can internalize the framework to guide their performance. Further, they can internalize using the framework and monitoring their application to become self-improving learners.

This is on top of providing the ability to respond in richer ways that picking an option out of those provided. It requires a freeform response, closer to what likely will be required after the learning experience. That‘s similar to what I‘m looking for from the gated response, but the latter expects peers and/or instructors to weigh in with feedback, where as here the learner is responsible for evaluating. That‘s a more complex task, but also very worthwhile if carefully scaffolded.  

Of course, it‘d also be ideal if an instructor is monitoring the response to look for any patterns, but that‘s outside the learners‘ response. So that‘s the case for model answers. So, what say you? And is that supported anywhere or in any way you know?

How to be an elearning expert

by 3 Comments

I was asked (and have been a time or two before): “What’s the one most important thing you’d like to tell to be successful Ed Tech industry leader” Of course there wasn‘t just one ;). Still, looking at colleagues who I think fit that characterization, I find some commonalities that are worth sharing. So here‘s one take on how to be an elearning expert.

Let‘s start with that ‘one thing‘.   Which is challenging, since it‘s more than one thing! Still, I boiled it down into two components: know your stuff, and let people know.   That really is the core. So let‘s unpack that some more.   The first thing is to establish credibility. Which means demonstrating that you track and promote the right stuff.  

Some folks have created a model that they tout. Cathy Moore has Action Mapping, Harold Jarche has PKM, Con Gottfredson has the 5 moments of need, and so on.   It‘s good having a model, if it‘s a good, useful one (there are people who push models that are hype or ill-conceived at best). Note that it‘s not necessarily the case that these folks are just known for this model, and most of these folks can talk knowledgeably about much more, but ‘owning‘ a model that is useful is a great place to be. (I occasionally regret that I haven‘t done a good job of branding my models.) They understand their model and its contribution, it‘s a useful one, and therefore they contribute validly that way and are rightly recognized.

Another approach like this is owning a particular domain. Whether gaming (e.g. Karl Kapp), visuals (Connie Malamed), design (Michael Allen), mixed realities (Ann Rollins), AI (Donald Clark), informal (Jane Hart), evaluation (Will Thalheimer), management (Matt Richter), and so on, they have deep experience and a great conceptual grasp in a particular area. Again, they can and do speak outside this area, but when they talk about these topics in particular, what they say is worthy of your attention!

Then there are other folks who don‘t necessarily have a single model, but instead reliably represent good science. Julie Dirksen, Patti Shank, Jane Bozarth, Mirjam Neelen, and others  have established a reputation for knowing the learning science and interpreting it in accurate, comprehensible, and useful ways.  

The second point is that these folks write and talk about their models and/or approaches. They‘re out there, communicating. It‘s about reliably saying the important things again and again (always with a new twist). A reputation doesn‘t just emerge whole-cloth, it‘s built step by step. They also practice what they preach, and have done the work so they can talk about it. They talk the talk and walk the walk. Further, you can check what they say.  

So how to start? There are two clear implications. Obviously, you have to Know. Your. Stuff! Know learning, know design, know engagement, know tech. Further, know what it means in practice!   You can focus deeply in one area, or generate one useful and new model, or have a broad background, but it can‘t just be in one thing. It‘s not just all your health content for one provider. What you‘re presenting needs to be representative and transferable.  Further, you need to keep up to date, so that means continually learning: reading, watching, listening.

Second, it‘s about sharing. Writing and speaking are the two obvious ways. Sure, you can host a channel: podcast, vlog, blog, but if you‘re hosting other folks, you‘re seen as well connected but not necessarily as the expert. Further, I reckon you have to be able to write and speak (and pretty much all of these folks do both well).   So, start by speaking at small events, and get feedback to improve. Study good presentation style. Then start submitting for events like the Learning Guild, ATD, or LDA (caveats on all of these owing to various relationships, but I think they‘re all scrutable). I once wrote about how to read and write proposals, and I think my guidance is still valid.

Similarly, write. Learning Solutions or eLearn Mag are two places to put stuff that‘s sensibly rigorous but written for practitioners.   Take feedback to heart, and deliberately improve. Make sure you‘re presenting value, not pitching anything. What conferences and magazines say about not selling, that your clear approach is what sells, is absolutely true.  

Also, make sure that you have a unique ‘voice’. No one needs the same things others are saying, at least in the same way. Have a perspective, your own take. Your brand is not only what you say, but how you say it.

A related comment: track some related fields. Most of the folks I think of as experts have some other area they draw inspiration from. UX/UI, anthropology, software engineering, there are many fields and finding useful insight from a related one is useful to the field and keeps you fresh.

Oh, one other thing. You have to have integrity. People have to be able to trust what you say. If you push something for which you have a private benefit, or something that‘s trendy but not real, you will lose whatever careful credibility you‘ve built up. Don‘t squander it!  

So that‘s my take on how to be an elearning expert. So, what have I missed?

Overworked IDs

by 2 Comments

I was asked a somewhat challenging question the other day, and it led me to reflect. As usual, I‘m sharing that with you. The question was “How can IDs keep up with everything, feel competent and confident in our work” It‘s not a trivial question! So I‘ll share my response to overworked IDs.

There was considerable context behind the question. My interlocutor weighed in with her tasks:  

“sometimes I wonder how to best juggle everything that my role requires: project management, design and ux/ui skills, basic coding, dealing with timelines and SMEs and managers. Don‘t forget task analysis and needs assessment skills, making content accessible and engaging. And staying on top of a variety of software.”  

I recognize that this is the life of overworked IDs, particularly if you‘re the lone ID (which isn‘t infrequent), or expected to handle course development on your own. Yet it is a lot of different competencies. In work with IBSTPI, where we‘re defining competencies, we‘re recognizing that different folks cut up roles differently. Regardless, many folks wear different competency requirements that in other orgs are handled by different teams. So what‘s a person to do?

My response focused on a couple of things. First, there‘re the expectations that have emerged. After 9/11, when we were avoiding travel, there was a push for elearning. And, with the usual push for efficiency, rapid elearning became the vogue. That is, tools that made it easy to take PDFs and PPTs and put it up online with a quiz. It looked like lectures, so it must be learning, right?

One of the responses, then, is to manage expectations. In fact, a recent post addressed the gap between what we know and what orgs should know. We need to reset expectations.

As part of that, we need to create better expectations about what learning is. That was what drove the Serious eLearning Manifesto [elearningmanifesto.org], where we tried to distinguish between typical elearning and serious elearning. Our focus should shift to where our first response isn‘t a course!  

As to what is needed to feel competent and confident, I‘ve been arguing there are three strands. For one (not surprisingly ;), I think IDs need to know learning science. This includes being able to fill in the gaps in and update on instructional design prescriptions, and also to be able to push back against bad recommendations. (Besides the book, this has been the subject of the course I run for HR.com via Allen Academy, will be the focus of my presentation at ATD ICE this summer, and also my asynchronous course for the LDC conference.)  

Second, I believe a concomitant element is understanding true engagement. Here I mean going beyond trivial approaches like tarting-up drill-and-kill, and gamification, and getting into making it meaningful. (I‘ve run a workshop on that through the LDA, and it will be the topic of my workshop at DevLearn this fall.)

The final element is a performance ecosystem mindset. That is, thinking beyond the course: first to performance support, still on the optimal execution side of the equation. Then we move to informal learning, facilitating learning. Read: continual innovation! This may seem like more competencies to add on, but the goal is to reduce the emphasis (and workload) on courses, and build an organization that continues to learn. I address this in the  Revolutionize L&D book, and also my mobile course for Allen Interactions (a mobile mindset is, really, a performance ecosystem mindset!).

If you‘re on top of these you should prepared to do your job with competence and confidence. Yes, you still have to navigate organizational expectations, but you‘re better equipped to do so. I‘ll also suggest you stay tuned for further efforts to make these frameworks accessible.  

So, there‘re my responses to overworked IDs. Sorry, no magic bullets, I‘m afraid (because ‘magic‘ isn‘t a thing, sad as that may be). Hopefully, however, a basis upon which to build. That‘s my take, at any rate, I welcome hearing how you‘d respond.

What about books | conferences?

by Leave a Comment

Responding to a frequent question  yet again, I decided to post an answer to the “what about books | conferences?” question.

And, as usual, the transcript.

Once again, after talking about how learning requires meaningful practice, I was asked the seemingly timeless question: “but what about books” Similarly, I regularly get “what about conferences”   So, for the record, let me say when and why books and lectures make sense. And when not. Hopefully I won‘t have to answer another “what about books | conferences” question.

To start, learning is action and reflection. That is, learning ‘outside‘ formal instruction. We act in the world and reflect on it to cement the lesson. It‘s slightly more complicated, because certain things, e.g. Geary‘s biologically primary things, may not really need reflection. Further some things may be really challenging to learn on your own even with reflection. But basically, doing things and reflecting (which can be reading, experimenting, writing/representing), etc is the way we learn on our own.  

Which, as I‘ve argued before, suggests that instruction  be designed action and guided reflection. That is, instructors should be choosing meaningful activities and scaffolding reflection around it. When we‘re designing for novices [link], in particular, when the learner doesn‘t know what‘s important nor why, we need to do the whole enchilada (darn, now I‘m hungry).

Which also means that when we‘ve segued beyond novice to practitioner (and beyond), we begin to know what‘s important and why, and we just need it. We want resources that can fill in the gaps. We want support for reflection.

So now we can explain why we can attend conferences, read books and articles, and the like. When we‘re deeply engaged in something, whether work or a passion, reading a book, listening to someone tell their story, and the like, serves as the necessary adjunct to our activity! They provide the complement to our own endeavors; the reflection to our action!

Now, hopefully, we‘ll never again need to discuss this. Realistically, we can point people here when we‘ get “what about books | conferences”? At least, that‘s my story, what‘s yours?  

A message to CxOs 2: about org learning myths

by 2 Comments

When I wrote my last post on a message to CxOs about L&D myths, I got some pushback. Which, for the record, is a good thing; one of us will learn something. As a counter to my claim that L&D often was it’s own worst enemy, there was a counter. The claim was that there are folks in L&D who get it, but fight upward against wrong beliefs. Which absolutely is true as well. So, let‘s also talk about what CxOs need to know about the org learning myths they may believe.  

First, however, I do want to say that there is evidence that L&D isn‘t doing as well as it could and should. This comes from a variety of sources. However, the question is where does the blame lie. My previous post talked about how L&D deludes itself, but there are reasons to also believe in unfair expectations. So here‘s the other side.  

  1. If it looks like schooling… I used this same one against L&D, but it‘s also the case that CxOs may believe this. Further, they could be happy if that‘s the case. Which would be a shame just as I pointed out in the other case. Lectures, information dump & knowledge test, in general content presentation doesn‘t lead to meaningful change in behavior in the absence of activity. Designed action and guided reflection, which looks a lot more like a lab or studio than a classroom, is what we want.
  2. SMEs know what needs to be learned. Research tells us to the contrary; experts don’t have conscious access to around 70% of what they  do (tho’ they do have access to what they know). Just accepting what a SME says and making content around that is likely to lead to a content dump and lack of behavior change. Instead, trust (and ensure) that your designers know more about learning than the SME, and have practices to help ameliorate the problem.
  3. The only thing that matters is keeping costs low.  This might seem to be the case, but it reflects a view that org learning is a necessary evil, not an investment. If we’re facing increasing change, as the pundits would have it, we need to adapt. That means reskilling. And effective reskilling isn’t about the cheapest approach, but the most effective for the money. Lots of things done in the name of learning (see above) are a waste of time and money. Look for impact first.
  4. Courses are the answer to performance issues.  I was regaled with a tale about how sales folks and execs were  insisting that customers wanted training. Without evaluating that claim. I’ll state a different claim: customers want solutions. If it’s persistent skills, yes, training’s the answer. However, a client found that customers were much happier with how-to videos than training for most of the situations. It’s a much more complex story.
  5. Learning stops at the classroom. As is this story. One of the reasons Charles Jennings was touting 70:20:10 was not because of the numbers, but because it was a way to get execs to realize that only the bare beginning came from courses, if at all. There’s ongoing coaching with stretch assignments and feedback, and interacting with other practitioners…don’t assume a course solves a problem. A colleague mentioned how her org realized that it couldn’t create a course without also creating manager training, otherwise they’d undermine the outcomes instead of reinforcing them.
  6. We‘ve invested in an LMS, that‘s all we need. That’s what the LMS vendors want you to believe ;)!  Seriously, if all you’re doing is courses, this could be true, but I’m hoping the above
  7. Customers want training.  Back to an earlier statement, customers want solutions. It is cool to go away to training and get smothered in good food and perks. However, it’s  also known that sometimes that  goes to the manager, not  the person who’ll actually be doing the work! Also, training can’t solve certain types of problems.  There are many types of problems customers encounter, and they have different types of solutions. Videos may be better for things that occur infrequently, onboard help or job aids may meet other needs to unusual to be able to predict for training, etc. We don’t want to make customers happy, we want  to make them successful!
  8. We need ways to categorize people. It’s a natural human thing to categorize, including people. So if someone creates an appealing categorization that promises utility, hey that sounds like a good investment. Except, there are many problems! People aren’t easy to categorize, instruments struggle to be reliable, and vested interests will prey upon the unwary.  Anyone can create a categorization scheme, but validating it, and having it be useful, are both surprisingly big hurdles. Asking people questions about their behavior tends to be flawed for complex reasons. Using such tools for important decisions like hiring and tracking have proven to be unethical. Caveat emptor.
  9. Bandwagons are made to be jumped on. Face it, we’re always looking for new and better solutions. When someone links some new research to a better outcome, it’s exciting. There’s a problem, however. We often fall prey to arguments that appear to be new, but really aren’t. For instance, all the ‘neuro’ stuff unpacks to some pretty ordinary predictions we’ve had for yonks. Further, there are real benefits to machine learning and even artificial intelligence. Yet there’s also a lot of smoke to complement the sizzle. Don’t get misled. Do a skeptical analysis.  This holds doubly true for technology objects. It’s like a cargo cult, what’s has come down the pike must be a new gift from those magic technologists! Yet, this is really just another bandwagon. Sure, Augmented Reality and Virtual Reality have some real potential. They’re also being way overused. This is predictable, c.f. Powerpoint presentations in Second Life, but ideally is avoided. Instead, find the key affordances – what the technology uniquely provides – and match the capability to the need. Again, be skeptical.

My point here is that there can be misconceptions about learning  within  L&D, but it can also be outside perspectives that are flawed. So hopefully, I’ve now addressed both. I don’t claim that this is a necessary and complete set, just certain things that are worth noting. These are org learning myths that are worth trying to overcome, or so I think. I welcome your thoughts!

A message to CxOs: about L&D myths

by 3 Comments

If you’re a CEO, COO, CFO, and the like, are you holding L&D to account? Because much of what I see coming out of L&D doesn’t stand up to scrutiny. As I’ve cited in books and presentations, there’s evidence that L&D isn’t up to scratch. And I think you should know a few things that may be of interest to you. So here’re some L&D myths you might want to watch out for.

  1. If it looks like school, it must be learning. We’ve all been to school, so we know what learning looks like, right? Except, do you remember how effective school actually was? Did it give you many of the skills you apply in your job now?  Maybe reading and writing, but beyond that, what did you learn about business, leadership, etc? And how  did you learn those things? I’ll bet not by sitting and listening to lectures presented via bulletpoints. If it looks like schooling, it’s probably a waste of time and money. It should look more like lab, or studio.
  2. If we’re keeping our  efficiency in line with others, we’re doing good. This is a common belief amongst L&D: well, our [fill in the blank: employees served per L&D staff member | costs per hour of training | courses run per year | etc.] is the same or better than the industry average, so we’re doing good. No, this is all about efficiency, not effectiveness. If they’re not reporting on measurable changes in the improvement of business metrics, like sales, customer service, operations,e tc, they’re not demonstrating their worth. It’s a waste of money.
  3. We produce the courses our customers need. Can they justify that? It’s a frequent symptom that the courses that are asked for have little relation to the actual problem. There are many reasons for performance problems, and a reliable solution is to throw a course at it. Without knowing whether it’s truly a function of lack of skill. Courses can’t address problems like the wrong incentives, or a lack of resources. If you’re not ensuring that you’re only using courses when they make sense, you’re throwing away money.
  4. Job aids aren’t our job.  Performance should be the job, not just courses. As Joe Harless famously said: “Inside every fat course there‘s a thin job aid crying to get out.” There are many times when a job aid is a better solution than a course. To believe otherwise is one of the classic L&D myths. If they’re avoiding taking that on, they’re avoiding a cheaper and more effective solution.
  5. Informal learning isn’t our job. Well, it might not be if L&D truly doesn’t understand learning, but they should. When you’re doing trouble-shooting, research, design, etc., you don’t know the answer when you start. That’s learning too, and there is a role for active facilitation of best principles. Assuming people know how to do it isn’t justifiable. Informal learning is the key to innovation, and innovation is a necessary differentiation.
  6. Our LMS is all we need. Learning management systems (which is a misnomer, they’re course management systems) manage courses well. However, if they’re trying to also be resource portals, and social media systems, and collaboration tools, they’re unlikely to be good at all that. Yet those are also functions that affect optimal performance and continual innovation (the two things I argue  should be the remit of L&D). Further, you want the right tool for the job. One all-singing, all-dancing solution isn’t the way to bet for IT in general, and that holds true for L&D as well.
  7. Our investment in evaluation instruments is valuable. If you’re using some proprietary tools that purport to help you identify and characterize individuals, you’re probably being had. If you’re using it for hiring and promotion, you’re also probably violating ethical guidelines. Whether personality, or behavior, or any other criteria, most of these are methodologically and psychometrically flawed. You’re throwing away money. We have a natural instinct to categorize, but do it on individual performance, not on some flawed instrument.
  8. We have to jump on this latest concept.  There’re a slew of myths and misconceptions running around that are appealing and yet flawed. Generations, learning styles, attention spans, neuro-<whatever> and more are all appealing, and also misguided. Don’t spend resources on investing in them without knowing the real tradeoffs and outcomes.These are classic L&D myths.
  9. We  have to have this latest technology. Hopefully you’re resistant to new technologies unless you know what they truly will do for your organization. This holds true for L&D as well. They’re as prone to lust after VR and AR and AI as the rest of the organization. They’re also as likely to spend the money without knowing the real costs and consequences. Make sure they’re coming from a place where they know the unique value the technology brings!

There’s more, but that’s enough for now. Please, dig in. Ask the hard questions. Get L&D to be scrutable for real results, not platitudes. Ensure that you’re not succumbing to L&D myths. Your organization needs it, and it’s time to hold them to account as you do the rest of your organization. Thanks, and wishing you all the best.

Something that emerged from a walk, and, well, I had to get it off my chest. I welcome your thoughts.

Evaluating soft skills

by 3 Comments

As has become a pattern, someone recently asked me how to evaluate soft skills. And without being an expert on soft skill or evaluation, I tried to answer on principle. So I thought about the types of observable data you should expect to find. And that yielded an initial answer. Then I watched an interesting video of a lecture by a scholar and consultant, and it elaborated the challenges. So, there‘s a longer answer too. So here‘s an extended riff on evaluating soft skills.

I started with wondering what performance outcomes would you expect for soft skills. Coupled, as well, with how could you find evidence of these observable differences. As a short answer, I suggested that there should be 3(+) outcomes from effective soft skills training.  

0) the learner should be able to perform in soft skills scenarios (c.f. Will Thalheimer’s LTEM). This is the most obvious. Put them in the situation and ask them to perform. This is the bit that gets re-addressed further down.  

1) the learner should be aware of an improvement in their ability to perform. However, asking immediately can lead to a misapprehension of ability. So, as Will Thalheimer advises in his Performance-Focused Smile Sheets, ask them 3 months later. Also, ask about behavior, not knowledge.   E.g. “Are you using the <> model in your work, and do you notice an improvement in your ability”

2) The ‘customers’ of the learner should notice the improvement. Depending on whether that’s internal or external, it might show up (at least in aggregate) in either 360 eval scores, or some observable metric like customer sat scores. It may be harder to collect this data, but of course it‘s also more valuable.  

3) Finally, their supervisors/managers should notice the improvement, whether observationally or empirically.They should be not only prepared to support the change over time, but asked to look for evidence (including as a basis to fine tune performance).  

All together, triangulating on this should be a way to establish the validity.  

Now, extending this, Guy Wallace tweeted a link to a lecture by Neil Rackham. In it, Neil makes the case that universities need to change to teaching core skills, in particular the 4 C‘s: critical thinking, creativity, communication, and collaboration. He also points out how hard it is to evaluate these without a labor-intensive effort of an individual observing performance. This is a point that others have made, that these skills have hard to observe criteria.  

There‘s some argument about so-called 21C skills, and yet I can agree that these four things would be good. The question is how to assess them reliably. Rackham argues that perhaps AI can help here. Perhaps, but at this point I‘d argue for two things. First, help students self-evaluate (which has the benefits of them understanding what‘s involved). Second, instrumenting environments (say, for instance, with xAPI) in which these activities are performed. There will be data records that can be matched to behaviors, initially for human evaluation, but perhaps ultimately for machine evaluation.  

Of course, this requires assigning meaningful activities that necessarily involve creativity, critical thinking, communication, and/or collaboration. This means project based work, and I‘ve long argued that you can‘t learn such skills without a domain. Actually, to create transferable versions, you‘d need to develop the skills across domains.  

When I teach, I prefer to give group work projects that do require these skills. It was, indeed, hard to mark these extra skills, but I found that scaffolding it (e.g. a ‘how to collaborate‘ document) facilitated good outcomes. Being explicit about the best thinking practices isn‘t only a good idea, it‘s a demonstrably useful approach in general.  

So I think developing skills is important. That means we need a means to be evaluating soft skills. We know it when we see it, but it‘s hard to necessarily find the opportunity, but if we can assign it, we can evaluate and develop these skills more readily. That, I think, is a desirable goal. What think you?

Deep learning and expertise

by 3 Comments

A colleague asked “is anyone talking about how deep learning requires time, attention, and focus” He was concerned with “the trend that tells us everything must be short.”   He asked if I‘d written anything, and I realize I really haven‘t. Well, I did make a call  for “slow learning” once upon a time, but it‘s probably worth doing it again.   So here‘s a riff on deep learning and expertise.

First, what do we mean by deep learning? Here, I‘m suggesting that the goal of deep learning is expertise. We‘ve automated enough of the component elements that we can use our conscious processes to make expert judgments in addressing performance requirements. This could be following a process, making strategic decisions such as diagnoses and prescriptions, and more. It can also require developing pre-conscious responses, such as we train airline pilots to respond to emergencies.  

Now, these responses can vary in their degree of transfer. Making decisions about how to remedy a piece of machinery that‘s misbehaving is different than deciding how to prioritize the new product improvements. The former is more specific, the latter is more generic. Yet, there are certain things that are relevant to both.  

Another issue is how often it needs to be performed. You can develop expertise much quicker with lots of opportunities to apply the knowledge. It‘s more challenging to achieve when there aren‘t as many times it‘s relevant in the course of your workflow. The aforementioned pilots are training for situations they never hope to see!

Before we get there, however, there‘s one other issue to address: how much has to go in the head, and how much can be in the world?   In general, getting information in the head is hard (if we‘re doing it right), and we should try to avoid it when possible. I argue  for backwards design, starting with what the performance looks like if we‘ve focused on IA (intelligence augmentation ), that is, looking for the ideal combination of smarts between technology (loosely defined) and our heads. As Joe Harless famously said “iInside every fat course there‘s a thin job aid crying to get out.”  

Once we‘ve determined that we need human expertise, we also need to acknowledge that it takes time! I put it this way: the strengthening of connections (what learning is at the neural level) can only be done so much in any one day before the strengthening function fatigues; you literally need sleep before you can learn more. And only so much strengthening can happen in that one day. So to develop strong connections, e.g. strong enough that it will be triggered appropriately, is going to have to be spaced out over time.  

This does depend on the pre-existing knowledge of the learner, but it was Anders Ericsson who posited the approximately 10K hours of practice to achieve expertise. That‘s both not quite accurate and not quite what he said, but as a rule of thumb it may be helpful. The important thing is that not just any practice will work. It takes what he called ‘deliberate practice‘, that is the right next thing for this learner. Continued, over time, as the learners‘ ability increases new practice focuses are necessary.

All that can‘t come from a course (no one is going to sit through 10000 hours!). Instead, if we follow the intent of the 70:20:10 framework, it‘s going to take some initial courses, then coaching, with stretch assignments and feedback, and joining a relevant community of practice, and….

We also can‘t assume that our learners will develop this as efficiently as possible. Unless we‘ve trained them to be good self-learners, it will take guided learning across their experience. Even if it‘s only at a particular point; most people who are pursuing a sport, hobby, what have you, eventually will take a course to get past their own limitations and accelerate development.

The short answer is that deep expertise doesn‘t, can‘t, come from a short learning experience. It comes from an extended learning experience, with spaced, deliberate, and varied practice with feedback. If you want expertise, know what it takes and do it. That‘s true whether you‘re doing it for yourself or you‘re in charge of it for others. Deep learning and expertise comes with hard work. (Also, let‘s make that ‘hard fun‘ ;).  

Andragogy vs Pedagogy

by 24 Comments

Asked about why I used the word pedagogy instead of andragogy, I think it’s worth elaborating (since I already had in my reply ;) and sharing. In short, I think it‘s a false dichotomy. So here‘s my analysis of andragogy vs pedagogy.

Looking at Knowles‘ andragogy, I think it‘s misconstrued. What he talks about for adults is really true for all learners, taking into account their relative cognitive capability and amount of experience. So I fear that using andragogy will perpetuate the myth that pedagogy is a different learning approach (and keep kids in classrooms listening to lectures and answering rote questions). Empirically, direct instruction works (tho‘ it‘s interpretation is different than the name might imply, I once pointed out how it and constructivism properly construed both really say the same thing ;).  

There was an article  that posited five differences, and I see a major confound; the article‘s talking about andragogy as self-directed learning, and pedagogy as formal instruction. That‘s apples and oranges. It really is more about whether you‘re a novice or a practitioner level and the role of instruction. Age is an arbitrary element here, not a defining factor. Addressing each point:

1. Adults are self-directing learners. No, in things they know they need, they can be, but also they may have their bosses or coaches pointing them to courses. Plus, for areas where the adults are novices, they still need guided instruction. Also, owing to our bad K12 and higher ed, we’re not really enabling learners to be effective and efficient self-directed learners. Further, kids are self-directed about things they‘re interested in. But we make little effort to ground what we do (particularly K6) in any reason why this is on the syllabus.  

2. The role of learner experience. Yes, this matters, but it‘s a continuum. Also, you always want to base instruction on learner experience, because elaboration requires connecting to and building on existing knowledge. Yes, we do tend to do give kids abstract problems (particularly in math), which is contrary to good learning science. “Only two things wrong in education these days, the curriculum and the pedagogy, other than that we‘re fine.” Ahem. We teach the wrong things, badly.  

3. Adults generate interest in useful information. So does everyone, but that‘s not a matter of developmental level. Kids also prefer stuff that‘s relevant. We‘ve developed a curriculum for kids that is out of date, and we don‘t motivate it. Everyone has a curriculum, and there are degrees of self-direction, but it‘s not a binary division.

4. Adult readiness to learn is triggered by relevance (yeah, kind of redundant).Kids also learn better when there‘s a reason. Hence problem-based, service-based, and other such philosophy‘s of learning. Even direct instruction posits meaningful problems. Again, the article‘s comparing an ideal human learning model compared to a broken school model.  

5. What motivates learners are real life outcomes. Really, we‘ve covered this, everyone learns better when there‘s motivation. Children learn for grades because no one‘s made it meaningful for them to care!   Kids will pursue their learning when it makes sense to them. John Taylor Gatto made the case that kids could learn the entire K6 curriculum in 100 hours if they cared! Kids do learn outside of what‘s forced on them from schooling, be it Pokemon, polka, or porcupines.  

Thus, in the comparison between andragogy vs pedagogy, I come down on the side of pedagogy. It‘s the earlier term, and while ped does mean ‘kid‘, I still think it‘s really about learning design. Learning design should be aligned to our brains, not differentiated between child and adult. Yes, there are developmental differences, but they‘re a continuum and it‘s more a matter of capacity, it‘s not a binary distinction. That‘s my take, what‘s yours?