Learnlets

Secondary

Clark Quinn’s Learnings about Learning

You are here: Home / Archives for strategy

What to evaluate?

by 2 Comments

In a couple of articles, the notion that we should be measuring our impact on the business is called out. And being one who says just that, I feel obligated to respond.  So let’s get clear on what I’m saying and why.  It’s about what to evaluate, why, and possibly when.

So, in the original article, by my colleague Will Thalheimer, he calls the claim that we should focus on business impact ‘dangerous’!  To be fair (I know Will, and we had a comment exchange), he’s saying that there are important metrics we should be paying attention to about what we do and how we do it. And no argument!  Of course we have to be professional in what we do.  The claim isn’t that the business measure is all we need to pay attention to. And he acknowledges that later. Further, he does say we need to avoid what he calls ‘vanity metrics’, just how efficient we are. And I think we do need to look at efficiency, but only after we know we’re doing something worthwhile.

The second article is a bit more off kilter. It seems to ignore the value of business metrics all together. It talks about competencies and audience, but not impacting the business. Again, the author raises the importance of being professional, but still seems to be in the ‘if we do good design, it is good’, without seeming to even check to see if the design is addressing something real.

Why does this matter?  Partly because, empirically, what the profession measures are what Will called ‘vanity’ measures. I put it another way: they’re efficiency metrics. How much per seat per hour? How many people are served per L&D employee?  And what do we compare these to?  Industry benchmarks. And I’m not saying these aren’t important, ultimately. Yes, we should be frugal with our resources. We even should ultimately ensure that the cost to improve isn’t more than the problem costs!  But…

The big problem is that we’ve no idea if that butt in that seat for that hour is doing any good for the org.  We don’t know if the competency is a gap that means the org isn’t succeeding!  I’m saying we need to focus on the business imperatives because we aren’t!

And then, yes, let’s focus on whether our learning interventions are good. Do we have the best practice, the least amount of content and it’s good, etc. Then we can ask if we’re efficient. But if we only measure efficiency, we end up taking PDFs and PPTs and throwing them up on the screen. If we’re lucky, with a quiz. And this is not going to have an impact.

So I’m advocating the focus on business metrics because that’s part of a performance consulting process to create meaningful impacts. Not in lieu of the stuff Will and the other author are advocating, but in addition. It’s all too easy to worry about good design, and miss that there’s no meaningful impact.

Our business partners will not be impressed if we’re designing efficient, and even effective learning, if it isn’t doing anything.  Our solutions need to be targeted at a real problem and address it. That’s why I’ll continue to say things like “As a discipline, we must look at the metrics that really matter… not to us but to the business we serve.”  Then we also need to be professional. Will’s right that we don’t do enough to assure our effectiveness, and only focus on efficiency. But it takes it all, impact + effectiveness + efficiency, and I think it’s dangerous to say otherwise.  So what say you?

Redesigning Learning Design

by 1 Comment

Of late, a lot of my work has been designing learning design. Helping orgs transition their existing design processes to ones that will actually have an impact. That is, someone’s got a learning design process, but they want to improve it. One idea, of course, is to replace it with some validated design process. Another approach, much less disruptive, is to find opportunities to fine tune the design. The idea is to find the minimal set of changes that will yield the maximal benefit. So what are the likely inflection points?  Where am I finding those spots for redesigning?  It’s about good learning.

Starting at the top, one place where organizations go wrong right off the bat is the initial analysis for a course. There’s the ‘give us a course on this’, but even if there’s a decent analysis the process can go awry. Side-stepping the big issue of performance consulting (do a reality check: is this truly a case for a course), we get into working to create the objectives. It’s about how you work with SMEs. Understanding what they can, and can’t, do well means you have the opportunity to ensure that you get the right objectives to design to.

From there, the most meaningful and valuable step is to focus on the practice. What are you having learners do, and how can you change that?  Helping your designers switch to good assessment writing is going to be useful. It’s nuanced, so the questions don’t seem that different from typical ones, but they’re much more focused for success.

Of course, to support good application of the content to develop abilities, you need the right content!  Again, getting designers to understand what the nuances of useful examples from just stories isn’t hard but rarely done. Similarly knowing why you want models and not just presentations about the concept isn’t fully realized.

Of course, making it an emotionally compelling experience has learning impact as well. Yet too often we see the elements just juxtaposed instead of integrated. There are systematic ways to align the engagement and the learning, but they’re not understood.

A final note is knowing when to have someone work alone, and when some collaboration will help.  It’s not a lot, but unless it happens at the right time (or happens at all) can have a valuable contribution to the quality of the outcome.

I’ve provided many resources about better learning design, from my 7 step program white paper to my deeper elearning series for Learnnovators.  And I’ve a white paper about redesigning as well. And, of course, if you’re interested in doing this organizationally, I’d welcome hearing from you!

One other resource will be my upcoming workshop at the Learning Solutions conference on March 25 in Orlando, where we’ll spend a day working on learning experience design, integrating engagement and learning science.  Of course, you’ll be responsible for taking the learnings back to your learning process, but you’ll have the ammunition for redesigning.  I’d welcome seeing you there!

Thinking Strategically

by Leave a Comment

Repurposed from another use.

In today’s increasing rate of change and competition, coupled with growing ambiguity and uncertainty, L&D just can’t be about delivering courses on demand. Optimal execution, the result of formal learning, is only the cost of entry, and continual innovation will be the necessary element for organizations to thrive. Organizations have to move faster, be more agile, and adapt more effectively.  And it’s here that L&D has a true opportunity, and imperative, to contribute. It’s about thinking strategically.

That means, intrinsically, that L&D has to start thinking about how to move forward..  People are learning on their own more and more. The tools to access information are quite literally in the palm of their hands.  L&D can no longer be about controlling content.  Instead a new role is needed.

Rethinking Formal

How does L&D cope? The answer involves a couple of major shifts, from familiar to challenging. The first is that courses go from an event model to an approach that better reflects how we actually learn. We need to have spaced, distributed practice to truly master our skills.  This is harder than the ‘information dump and knowledge test’ that too often characterizes organizational learning, which brings up two issues: 1) formal learning should be reserved for when it absolutely, positively has to be in the head, and 2) putting information in the world when possible. 

That latter is referring to performance support, the first step in broadening the L&D perspective. The point is that we too often use courses when cognitive skills are not the problem. Performance consulting is a process to identify the real problem and cause, and provide appropriate solutions. Performance support is often a solution we can use instead of a course!  Note that this is a first step out of the comfort zone, as it means engaging with our stakeholders, the business units we are tasked to assist. But it’s past time!

Beyond Formal

Doing courses the right way, coupled with performance support, are the key to optimizing execution. But that’s just the starting point.  The key to organizational improvement will be the ability to learn. And that should be L&D’s role.  But this means we have to again step out of our comfort zone. 

We need to branch out into informal and social learning.  Employees do learn on their own, but the evidence suggests that they’re not particularly good at it. There are lots of folk stories about what works that just aren’t aligned with what science tells us!  Assisting the individuals and the organization to learn, independently and collectively, is the new opportunity. Assisting the organization to innovate means moving to the core of competitive advantage. And that’s a valuable place to be.

Wishful thinking isn’t the answer. It takes both knowing the bigger picture, the performance ecosystem, and working strategically to get from here to there. That’s what’s on the table. It might be scary, but the opportunity offers a brighter future for L&D.  I’m excited about the prospects, and hope you’ll be making the move.  I’d welcome the opportunity to assist, as well.

Learning Experience Portals?

by Leave a Comment

What is a learning experience platform?  Suddenly the phrase seems ubiquitous, but what does it mean?  It’s been on my mental ‘todo’ list for a while, but I finally spent some time investigating the concept. And what I found as the underlying concept mostly makes sense, but I have some challenges with the label.  So what am I talking about?

It’s ImPortal!

Some background: when I talk about the performance ecosystem, it’s not only about performance support and resources, but finding them.  Ie, it includes the need for a portal. When I ask audiences “how many of you have portals in your org”, everyone raises their hands. What also emerges is that they have bunches of them. Of course, they’re organized by the business unit offering them. HR, product, sales, they all have their own portals. Which doesn’t make sense. What does make sense is to have a place to go for thing organized by people’s roles and membership in different groups.

A user-centered way of organizing portals makes sense then. People need to be able to see relevant resources in a good default organization, have the ability to reorganize to a different default, and search.  Federate the portal and search over all the sources of resources, not some subset.  I’ve suggested that it might make sense to have a system on top of the portals that pulls them together in a user-centric way.

An additional issue is that the contents of said portal should be open, in the sense that all users should be able to contribute their curated or created resources, and the resources can be in any format: video, audio, document, even interactive. In today’s era of increasing speed of change and decreasing resources for meeting the learning needs, L&D can no longer try to own everything. If you create a good culture, the system will be self-policing.

And, of course, the resources aren’t all about learning. Performance support is perfectly acceptable. The in-the-moment video is as needed as is the course on a new skill. Anything people want, whether learning resources from a library to that quick checklist should be supported.

The Learning Experience Platform(?)

As I looked into Learning Experience Platforms (LXP), (underneath all the hype) I found that they’re really portals; ways for content to be aggregated and made available. There are other possible features – libraries, AI-assistance, paths, assessments, spaced delivery – but at core they’re portals. The general claim is that they augment an LMS, not replace it. And I buy that.

The hype is a concern: microlearning for instance (in one article that referred to the afore-mentioned in-the-moment video, glossing over that you may learn nothing from it and have to access it again). And of course exaggerated claims about who does what.  It appears several LMS companies are now calling themselves LXPs. I’ll suggest that you want such a tool designed to be a portal, not having it grafted onto to another fundamental raison-d’être. Similarly, many also claim to be social. Ratings would be a good thing, but also trying to be a social media platform would not.

Ultimately, such a capability is good. However, if I’m right, I think Learning Experience Platform isn’t the right term, really they’re portals. Both learning and experience are wrong; they can be perform in the moment, and generally they’re about access, not generating experiences. And I could be wrong.

Take-home?

Ecosystems should be integrated from best-of-breed capabilities. One all-singing, all-dancing platform is likely to be wrong in at least one if not more of the subsidiary areas, and you’re locked in.  I think a portal is a necessary component, and the LXPs have many performance & development advantages for over generic portal tools.

So I laud their existence, but I question their branding. My recommendation is always to dig beneath the label, and find the underlying concept. For instance, each of the concepts underpinning the term microlearning is valuable, but the aggregation is problematic. Confusion is an opening for error. So too with LXP: don’t get it confused with learning or creating experiences.  But do look to the genre for advanced portals.  At least, that’s my take: what’s yours?

Experimentation specifics

by Leave a Comment

I’m obviously a fan of innovation, and experimentation is a big component of innovation. However, I fear I haven’t really talked about the specifics.  The details matter, because there are smart, and silly, ways to experiment. I thought I’d take a stab at laying out the specifics of experimentation.

First, you have to know what question you’re trying to answer. Should we use a comic or a video for this example?  Should we use the content management system or our portal tool to host our learning and performance support resources?  What’s the best mechanism for spacing out learning?

An important accompanying question is “how will we know what the answer is?”  What data will discriminate?  You need to be looking for a way to tell, we know, we can’t know, or we need to revise and do again.

Another way to think about this is: “what will we do differently if we find this?” and “what will we do differently if it turns out differently?” The point is to know not just what you’ll know, but what it means.

You want to avoid random experimentation. There are the ‘lets try it out’ pilots that are exploratory, but you still want to know what question your answering. Is it “what does it take to do VR” or “let’s try using our social media platform to ‘show our work'”.

Then you need to design the experiment. What’s the scope? How will you run it? How will you collect data? Who are your subjects?  How will you control for problems?

One of the claims has regularly been “don’t collect any data you don’t know what you’ll do with”.  These days, you can run exploratory data analysis, but still, accumulating unused data may be a mistake.

The after-experiment steps are also important. Major questions include: “what did we learn”, “do we trust the results”, and “what will we do as a result”. Then you can followup with the actions you determined up front that would be predicated on the outcomes you discover.

Experimentation is a necessary component of growth. You have to have a mindset that you learn from the experiment, regardless of outcome. You should have a budget for experimentation and expect a degree of failure. It’s ok to lose, if you don’t lose the lesson!  And share your learnings so others don’t have to make the same experiment.  So experiment, just like I did here; is this helpful?  If not, what would it need to be useful?

Citations

by 2 Comments

Following on my thoughts on writing yesterday, this was a topic that didn’t fit (the post got too long ;).  So here we go..  Colleagues have written that citations are important. If you’re making a claim, you should be able to back it up. On the other hand, if you’re citing what you think is ‘received wisdom’, do you need to bother?  Pondering…

Now, citations can interfere with the flow, I believe. If not the reading, they can interfere with the flow of my writing! (And, I’ve been accused of ‘name dropping‘, where instead I believe it’s important to both acknowledge prior work and show that you know what’s been done.) Still, it’s important to know what to cite, and when.

I admit that I don’t always cite the claims I make. Because, I take it as a given.  I may say something like “we know” or otherwise presume that what I’m saying is accepted premise. One problem, of course, is that I don’t know what others know (and don’t). And, of course, that this isn’t an official article source, this is my blog ;). Still, when I’m talking about something new to me (like thoughts from books), I will cite the locus.

Articles are different. When I write those, I try to provide sources. In both cases I generally don’t go to the extent of journal article links, because I’m not expect that folks have easy access to them, and so prefer to cite more commonly available resources, like books that have ‘digested’ the research.

And when I write ‘take down’ articles, I don’t cite the offender. It’s to make the point, not shame anyone. If you’re really curious, I’m sure you can track it down.

And, realize I don’t have easy access to journals either. Not affiliated with an institution, I don’t have access to the original articles behind a pay wall. I tend to depend on people who summarize including books and articles that summarize. Still, I’ve a grounding for over a decade in the original materials and am able to make inferences. And of course occasionally I’ll be wrong. Sometimes, I’ll even admit it ;).

The issue really is when do you need to make a citation. And I reckon it’s when you’re stating something that folks might disagree with. And I can’t always anticipate it. So I’ll try to consistently point to the basis for any claims I think might be arguable, or state that it’s my (NSH :) opinion.  And you can always ask!  Fair enough?

Trends for 2019?

by 2 Comments

It’s already started!  Like Christmas (which morally shouldn’t be even be thought about before Thanksgiving), requests for next year’s trends should be on hold until at least December.  Still, a request came in for my thoughts. Rather than send them off and await their emergence; I toss them out here, with a caveat: “It’s tough to make predictions, particularly about the future.”

1. What, on your opinion, are the main Digital Learning (DL) trends for 2019?

I think the main trend will be an increasing exploration of alternatives to ‘courses’.  This will include performance support, and social networks. Similarly, models for formal learning will shift from the ‘event’ model to a more sustained and distributed framework that segues from spaced learning through coaching.

I sincerely hope that we’ll be paying more attention to aligning learning with cognition, and pursue ‘shiny objects‘ only after we establish a solid foundation. Instead of looking for the magic bullet, we’ll recognize that our brain architecture means we need a drip-irrigation model, not a flood.

This may be wishful thinking, but I believe we’re beginning to see some positive signs. We’re seeing more interest in learning science, growing awareness of myths, and more. Hopefully there’s an accompanying shift from being fascinated by technology to being interested in what technology can do for better learning outcomes!

2. What are the main threats and obstacles, then?

The main threats and obstacles are several. For one, our own lack of understanding of the foundations of our industry hampers us. When we don’t really understand learning, we can be swayed by well-designed distractors.  That’s the second factor: there are those who are happy selling us the latest fad.

Coupled with this is a lack of business awareness in our own practices. We measure the wrong things, e.g. efficiency – such as cost/seat/hour. And we’re reluctant to talk to the stakeholders in the business. We should be worried about impact: are we reducing costs, increasing profitability or customer satisfaction?

Overall, we’re hampered by a true lack of professionalism. We learn the tools, and crank stuff out, but we’re not concerned enough about whether it’s demonstrably the right stuff.

3. Do you believe in the AI and DL robotization? When does this bright future come?

I believe in increasing use of AI to support functions that shouldn’t involve humans. It’s silly to have people doing rote things we can teach computers to do. That includes responding to knowledge requests, and filtering, and a few other tasks.  However, I think we need to recognize that not all the things needed in learning, such as evaluating complex work products, should be done by machines. I think we should look for when we can automate, and when we want people in the loop.

So I’m more interested in IA (not AI): Intelligence Augmentation.  That is, what is the right distribution of tasks between machine and people?  There are things that computers do well, but they’re remarkably brittle; as of yet they don’t handle edge cases, or make good inferences in the grey areas very well. That’s when you want people. I think our design discipline needs to be smart about when to use each, and how they complement each other.

The future of IA is already underway, as is AI. We’re seeing, and will see more, uses of AI to filter, to answer questions, and to take over rote tasks.  These behaviors are not yet ready to be termed ‘bright’, however. Some success stories are emerging, but I suspect we don’t hear much yet about the money being wasted.  The time of consistency in effective synergies is still a few years off.

4. Your advice to the market for 2019.

Work smarter!  Get smart about learning science, about business, and about what technology can (and can’t) do. I’d like to see: staff pushing more for real impact via metrics, leaders asking for business cases not order taking, vendors pushing solutions not resource savings, and buyers asking for real evidence.  I’d like to see smarter purchasing, and the snake oil sales folks’ business withering away.  We can do better!

And I realize that my proposed trends are more wishful thinking than predictions. One of my favorite quotes is by Alan Kay: “The best way to predict the future is to invent it” so I keep pushing this agenda. My goal is simple: to make this a field I’m truly proud to be working in. The folks in L&D, I think, are some of the nicest folks; they’re here because they want to help others (you don’t go to L&D to become rich ;). I think there’s a promising future, but it doesn’t start with AI or ML or DL, it starts with getting down to the realities of how we learn, and how we can support it. When we do that, I think our future will be one which will help our organizations and our people thrive. Our future can be bright, and it’s up to us to make it so.

Content Confusion

by 1 Comment

I read, again and again, about the importance of ‘content’ in learning. And I don’t disagree, but…I think there’s still a problem about it. And where I get concerned is about what is meant by the term.  Just what do mean by ‘content’?  And why should we care about the distinction?

My worry is twofold. For one, I get concerned that talking about content foregrounds ‘information’. And that’s a problem. I’ve been concerned for a while about how it’s too easy to allow knowledge to dominate learning objectives. Know, understand, etc are generally not meaningful objectives. Objectives should be ‘able to use to ___’.  Talking about content, as I’ve talked about before, leads us down a slippery slope to curriculum being defined as content.

My second concern is related. It’s about content being meant to include concepts, examples, and practice. Yet, if we don’t separate out interactivity separate from consumption, we can make nonsensical learning interactions instead of meaningful applications of concepts to contexts. Recognition is not powerful learning.

Look, I get it.  From a technical point of view, e.g. a content management system perspective, it’s all content. It’s addressable files. They may just report access, or they can report success/failure, or many other things. However, again, this view can make it easy to do bad things. And, as the book Nudge I just read suggests, we want to make it easy to do the right things, and make you have to work to do things inappropriately.

So I may be being a pedant about this, but I have a reason. It won’t be when we’re all on the same page about good learning design, practice foregrounded and concepts and examples as learning resources, not the goal. But I don’t think we’re there yet.  And language matters in shaping thinking. It may not be the Whorfian Hypothesis, but it does influence how we think and what we do.

For principled and practical reasons, I think we want to distinguish between content (concepts and examples) and interactives (practice).  At least as designers. Others can focus differently, but we have our own language for other things (I’d argue our use of the term ‘objectives’ is different than business folks, for example), and I argue we should do so here as well.  What say you?

And the myths go on

by Leave a Comment

Yet another silly post I stumbled upon.  And last week at a conf someone said they liked my take-downs. If you disagree, let me know, but otherwise here’s yet another bunch of marketing hype.  Hopefully no one uses this for any real decisions!

This one talks about ‘generation Z’, and implications for L&D. Ok, so we’re off on the wrong foot from the get-go.  These are listed as 1995-2014. (Er, um, as Jessica Kriegel pointed out last week, isn’t the whole point of the millennial label that they’re ‘2000’? )  However, there’s no evidence to point to reliable generational differences. What differences there are can be attributed to age, and it’s still a form of age discrimination, how about treating people by how they individually behave?

So there’s a list of differentiators, sourced from elsewhere. You go to the elsewhere, and it’s preferences, and anecdotal. Neither one are good bases for making broad claims. There are several cites in the list, as well. From marketing sites. So the author clearly doesn’t understand good data.  What are they talking about? Here’s a subset:

  • Digital multitaskers: well, we know that’s inefficient, but haven’t we seen that taken up by device, not age group? It’s certainly true for millennials as well, and seems to be true for everyone who’s gotten on to mobile devices.
  • Secretly social: (wth?) they share, but with control. As do most astute folks beyond high school.
  • Diverse: er, yes, so’s the whole US. And, more and more, the world. How is this definitional? And do you think they really don’t still have biases?
  • Quick Information Processors/Communicators: dealing with chunks, quickly but not necessarily accurately. Isn’t that, er, just kind of human?

The recommendations list is similarly silly:

  • Update job descriptions: make sure they’re up-to-date.  Really?  This isn’t just good practice?
  • Expunge bias: ditto
  • Go where the talent is: use appropriate social media. C’mon, already; any other statements of the obvious?
  • Benefits: emphasize the WIIFM. Can you imagine?

The overarching theme here is ‘do good things’.  Why isn’t this appropriate for every job search?  And the same thing continues when recommendations for your courses:

  • Digital and Visual Content: Use media? Really?  Who’d have thought of it?
  • Reassess your Library and Curricula: you don’t need diversity, but you do need soft skills. Here I think there is bad advice, instead of the generally ‘best principles argued for the wrong reasons’.  Just because you hear more messages of tolerance (yay!), doesn’t mean you know how to be inclusive, and are aware of unconscious bias. (That’s why it’s unconscious!)

And the same overall pattern of good advice pretending to be specific to a generation holds true for the final list.   (I’m paraphrasing the advice here):

  • Embrace diversity
  • Provide social connection tools
  • Give them the ability to contribute
  • Include them
  • Don’t try to ‘own’ their time

Tell me if you think any of these should be not true for other folks than these new folks?  I think this approach is a bad idea, overall. You’re providing decent advice (er, mostly), but doing so through a myth-perpetuating framing. That’s still myth-perpetuating!

Ok, so this was from a company that’s trying to flog their services. It still seems like it’s written by a person more focused on marketing than matter. And I think we need to unpack these, and push back. Generation Z is just as discriminatory as millennials,  gender, and other differences that are attempts to avoid dealing with people as individuals.  If we don’t kick up our heels, we won’t get better efforts. And we should.

Intellectricity

by Leave a Comment

Many years ago, I met a guy who worked for Apple. They were allowed to have their own job titles, and his was “Intellectrician”. I thought that was a very nice turn of phrase. And, as I just ordered new business cards, I put Intellectricity as the tagline instead of “Learning & Performance Strategy” or other permutations with Technology and such in the title.  Why?

The goal, of course, is to have a phrase that folks will read it and go “what’s that about?”, as some of my colleagues helped me remember. If you can spark a conversation, you have a chance to do a little evangelism/education. (And maybe some business interest?)  Also, I think it actually captures what I believe and like to do pretty well.

You’ve likely heard or read me harp many times on how companies aren’t well aligned with how we think, work, and learn. The cognitive violations are many, from how we design our learning, to design our workplaces, policies, tool use, and culture. If we redesigned what we’re doing, creating strategies to get better practices in place, we’d be unleashing the organizational intellect!  Hence, ‘intellectricity’.

And this is pretty much what I’m on about, in several ways:

  • knowing what formal learning really looks like, and designing our design processes accordingly
  • recognizing what facilitates informal learning in the short term (the ‘solve this’ type of problem-solving’)
  • facilitating long-term informal learning by practices and tools suites
  • fostering a culture where innovation thrives

This is a partial list that goes fractal really quickly with practices and principles around each area. The point is that these elements are key to organizational ‘thrival’.  Overall, they’re about optimizing the intellectual activity of the organization, learning quickly to be agile.

We’ll see if this tactic works to generate conversations and then new thinking. As Jay Cross used to say “conversations are the stem cells of learning”.  Practicing what we preach.  So here’s to Intellectricity: more conversations and more learning.