Archives for 2019

Learning Lessons

16 May 2019 by Clark 1 Comment

So, I just finished teaching a mobile learning course online for a university. My goal was not to ‘teach’ mobile so much as develop a mobile mindset. You have to think differently than what the phrase ‘mobile learning’ might lead you to think. And, not surprisingly, some things went well, and some thing didn’t. I thought I’d share the learning lessons, both for my own reflection, and for others.

As a fan of Nilson’s Specifications Grading, I created a plan for how the assessment would go. I want lots of practice, less content. And I do believe in checking knowledge up front, then having social learning, and a work product. Thus, each week had a repeated structure of each element. It was competency based, so you either did it or not. No aggregation of points, but instead: you get this grade if you do: this many assignments correct, and write a substantive comment in a discussion board and comment on someone else’s this many times, and complete this level on this many knowledge checks. And I staggered the deadlines through the week, so there’d be reactivation. I’ve recommended this scheme on principle, and think it worked out good in practice, and I’d do it again.

In many ways it ‘teacher proofs’ the class. For one, the students are giving each other feedback in the discussion question. The choice of discussion question and assignment both were designed to elicit the necessary thinking, which makes the marking of the assignment relatively easy. And the knowledge checks set a baseline background. Designing them all as scenario challenges was critical as well.

And I was really glad I mixed things up. In early weeks, I had them look at apps or evaluated ones that they liked. For the social week, I had them collaborate in pairs. In the contextual week, they submitted a video of themselves. They had to submit an information architecture for the design week. And for the development week, they tested it. Thus, each assignment was tied to mobile.

It was undermined by a couple of things. First, the LMS interfered. I wrote careful feedback for each wrong answer for each question on the knowledge checks. And, it turns out, the students weren’t seeing it! (And they didn’t let me know ’til the 2nd half of the abbreviated semester!) There’s a flag I wasn’t setting, but it wasn’t the default! (Which was a point I then emphasized in the design week: start with good defaults!)

And, I missed making the discussions ‘gradeable’ until late because of another flag. That’s at least partly on me. Which meant again they weren’t getting feedback, and that’s not good. And, of course, it wasn’t obvious ’til I remedied it. Also, my grading scheme doesn’t fit into the default grading schema of the LMS anyways, so it wasn’t automatically doable anyways. Next time, I would investigate that and see if I could make it more obvious. And learn about the LMS earlier. (Ok, so I had some LMS anxiety and put it off…)

With 8 weeks, I broke it up like this:

Overview: mobile is not courses on a phone. The Four C’s.
Formal Learning: augmenting learning.
Performance Support: mobile’s natural niche
Social: connecting to people ‘on the go’
Contextual: the unique mobile opportunity
Design: if you get the design right…
Development: practicalities and testing.
Strategy: platform and policy.

And I think this was the right structure. It naturally reactivated prior concepts, and developed the thinking before elaborating.

For the content, I had a small set of readings. Because of a late start, I only found out that I couldn’t use my own mLearning book when the bookstore told me it was out of print (!). That required scrambling and getting approval to use some other writings I’d done. And the late start precluded me from organizing other writings. No worries, minimal was good. And I wrote a script that covered the material, and filmed myself giving a lecture for each week. Then I also provided the transcript.

The university itself was pretty good. They capped the attendance at 20. This worked really well. (Anything else would’ve been a deal breaker after a disaster many years ago when an institution promised to keep it under 32 and then gave me 64 students.) And there was good support, at least during the week, and some support was available even over the weekend.

Overall, despite some hiccups and some stress, I think it worked out (particularly under the constraints). Of course, I’ll have to see what the students say. One other thing I’d do that I didn’t do a good job of generally (I did with a few students) was explain the pedagogy. I’ve learned this in the past, and I should’ve done so, but in the rush to wrestle with the systems, it slipped through the cracks.

Those are my learning lessons. I welcome your feedback and lessons!

Shaming, safety, & misconceptions

14 May 2019 by Clark 1 Comment

Another twitter debate, another blog post. As an outgrowth of a #lrnchat debate, a discussion arose around whether making errors in learning could be a source of shaming. This wasn’t about the learners, however, being afraid of being shamed. Instead it was about whether the designers would feel proscribed from making real errors because of their expectation of learner’s emotions. And, I have strong beliefs about why this is an important issue. Learners should be making errors, for important reasons. So, we need to make it safe!

The importance of errors is in the fact that we’d rather make them in practice than when it counts. Some have argued that we literally have to fail to be ready to learn. (Perhaps almost certainly if the learners are overconfident.) The importance to me is in misconceptions. Our errors don’t tend to be random (there is some randomness), but instead are patterned. They come from systematic ways of perceiving the situation that are wrong. They come from bringing in the wrong models in ways that seem to make sense. And it’s best to address them by being able to make that choice, and getting feedback about why that’s wrong.

Which means learners will have to fail. And they should be able to make mistakes. (Guided) Exploration is good. Learners should be able to try things out, see what the consequences are, and then try other approaches. It shouldn’t be a free-for-all, since learners can not explore systematically. Instead, as I’ve said, learning should be designed action and guided reflection. And that means we should be designing in these alternatives to the right action as options, and provide specific feedback.

So, if they’re failing, is that shaming? Not if we do it right. It’s about making failing okay. It’s about making the learning experience ‘safe‘. Our feedback should be about the decision, and why it’s wrong (referring to the model). We might not give them the right answer, if we want them to try again. But we don’t make it personal, just like good coaching. It’s about what they did, not who they are. So our design should prevent shaming, but by making it safe to fail, not preventing failure.

The one issue that emerged was that there was fear that the designers (or other stakeholders) might have fear that this could be emotionally damaging, perhaps from fears of their own. Er, nope! It’s about the learning, and we know what research tells us works. We have to be responsible to be willing to do what’s right, as challenging as that may be for any reason. Time, money, emotions, what have you. Because, if we want to be responsible stewards of the resources entrusted to us, we should be doing what’s known to be right. Not chasing shiny objects. (At least, until we get the core right. ;)

So, let’s not shame ourselves by letting irrelevant details cloud our judgment. Do the right thing. For the right reasons. We know how to be serious about our learning. Make it so.

Facilitate is the new train

9 May 2019 by Clark Leave a Comment

Ok, so I’m being provocative with the title, since I’m not advocating the overthrow of training. The main idea is that a new area for L&D is facilitation. However, this concept also updates training. It’s part of what I was arguing when I suggested that the new term for L&D should be P&D, Performance & Development. So let’s start with that. We need to facilitate in several directions!

The driver behind the suggested nomenclature change is that the focus of L&D needs a shift. The revolutionary point of view is that organizations need both optimal execution and continual innovation (read: learning). In this increasingly chaotic time, the former is only the cost of entry, but it can’t be ignored. The latter is also becoming more and more critical!

A performance focus is the key to execution. You want to ensure people are doing what’s known about what’s need to be done. That’s the role of instruction and performance support. Performance consulting is the way to work backwards from the problem and determine the best interventions to do that optimization.

However, learning science is pushing us to recognize that we can do better. Information dump and knowledge test isn’t going to lead to any change in behavior. If you want people to be able to do, you have to have them do in practice. Which means the focus is on the practice and the feedback. That latter is facilitation. The clichéd switch from sage on the stage to guide on the side does capture it. So even here we see the need for facilitation.

It’s in the latter, however, where facilitation really comes to the fore. When we talk about development, we’re going beyond developing the individual. We are addressing the organization’s learning. And, as I’ve said, innovation is learning, just a different sort. What’s needed is informal learning.

And informal learning, while natural, isn’t always optimal. Habits, misconceptions, culture, and more can intrude. This is why facilitation may be even more key to success for organizations.

And, again, L&D should be the most knowledge about learning, because learning underpins both performance and development. Thus, if L&D is going to adapt, learning how to facilitate learning will be core. Facilitate really will be the new ‘train’.

Hub or spoke?

7 May 2019 by Clark 2 Comments

How are learning design teams are distributed (or not) in an organization? I’ve seen both totally separate teams in organizations (spoke), and totally central ones (hub), and of course gradations in between. While size of the organization is one driver, there are tradeoffs in efficiencies and effectiveness. And, I think tech can help. How?

So, to start, this has been an ongoing debate. I cynically (who, me? :) suspect that when a new manager comes in, whatever it is that’s been done, they have to do the opposite. Something must be done, right away! More seriously, there are strengths to either.

Distributed teams as closer to their partners. They have greater internal knowledge, and can be more responsive. Central teams make it easier to maintain quality. You don’t get driven as much by differing team cultures and can maintain a bastion of quality. Similarly, you can often find efficiencies from scale and lack of redundancy. And sometimes, you can have distributed teams taking advantage of some shared resources such as video production.

However, I was pondering how we can use technology to help break through the tradeoffs. As we build a community around the design of learning, the teams can be distributed as long as they’re continuing to learn together. If the community is continuing to learn together, showing their work and lessons learned, and regularly connecting whether through lunch-and-learns, offsites, or what have you, the shared learnings don’t need to come from physical proximity.

Building culture is hard, but as I’ve argued elsewhere, L&D really should take ownership of the new ways of working first, before proselytizing it elsewhere. Thus, L&D should be practicing the principles of a learning culture. Then, it really doesn’t matter if you’re hub or spoke, or anything in-between, because you are a community.

Competencies for L&D Processes?

1 May 2019 by Clark Leave a Comment

We have competencies for people. Whether it’s ATD, LPI, IBSTPI, IPL, ISPI, or any other acronym, they’ve got definitions for what people should be able to do. And it made me wonder, should there be competencies for processes as well? That is, should your survey validation process, or your design process, also meet some minimum standards? How about design thinking? There are things you do get certified in, including such piffle as MBTI and NLP. So does it make sense to have processes meet minimum standards?

One of the things I do is help orgs fine-tune their design processes. When I talk about deeper elearning, or we take a stand for serious elearning, there are nuances that make a difference. In these cases, I’m looking for the small things that will have the biggest impact. It’s not about trying to get folks to totally revamp their processes (which is a path to failure). Yet, could we go further?

I was wondering whether we should certify processes. Certainly, that happens in other industries. There are safety processes in maintenance, and cleanliness in food operations, and so on. Could and should we have them for learning? For performance consulting, instructional design, performance support design, etc?

Could we state what a process should have as a minimum requirement? Certain elements, at least, at certain way points? You could take Michael Allen’s SAM and use it as a model, for instance. Or Cathy Moore’s Action Mapping. Maybe Julie Dirksen’s Design For How People Learn could be created as such. The point being that we could stipulate some way points in design that would be the minimum to be counted as sufficient for learning to occur. Based upon learning science, of course. You know, deliberate and spaced practice, etc.

Then the question is, should we? Also, could we agree? Or, of course, people could market alternative process certifications. It appears this is what Quality Matters does, for instance, at least K12 and higher ed. It appears IACET does this for continuing education certification. Would an organization certification matter? For customers, if you do customer training? For your courses, if you provide them as a product or service? Would anyone care that you meet a quality standard?

And it could go further. Performance support design, extended learning experience design (c.f. coaching), etc. Is this something that’s better at the person level than the process level?

Should there be certification for compliance with a competency about the quality of the learning design process? Obviously in some areas. The question is, does it matter for regular L&D? On one hand, it might help mitigate against the info dump/knowledge test courses that are the bane of our industry. On the other hand, it might be hard to find a workable definition that could suit the breadth of ways in which people meet learning needs.

All I know is that we have standards about a lot of things. Learning data interchange. Individual competencies. Processes in education. Can and should there be for L&D processes? I don’t know. Seriously. I’m just pondering. I welcome your thoughts.

Surprise and safety

30 April 2019 by Clark Leave a Comment

As I reflect further on the improved surprise model, I realize there’s one thing I missed. The model gives a motivation for learning, and an implication for design. But there’s one thing more in the model, and one more implication for design. And this has to do with safety.

So, first, the initial model says that we learn to minimize surprise. We’re driven to remove the mismatch between what we expect and what occurs. This could lead to a desire to do nothing, or as little as possible, but a further elaboration says we also want to maximize outcomes. Thus, we won’t just sit around, but explore.

That means that helping learners know 2 things: that they want to know this (it’s to optimize what they care about), and that they don’t know it (the gap they have to minimize). If we do that, they’re ready to learn. But there’s one more thing.

We won’t explore other alternatives to see if they’re a better solution if the consequences are high. We’ll only explore if the cost of this exploration isn’t higher than the benefits we gain if it’s better. So that one other things is safety. If it isn’t safe, we’ll stick with the known solution.

Which means that we need to make it safe to explore in our learning. And, that includes both formal learning and informal. Mistakes in learning must be expected and accepted. In formal learning, mistakes are learning opportunities. Have alternatives that represent reliable ways folks go wrong, and it’s ok if they choose those because you have feedback specifically for that selection. And informally, mistakes (not the same ones, or obvious ones, there’s accountability too) are fine when the lesson’s learned.

Understanding how, and why, we learn is critical to optimizing learning. And I think that’s a valuable goal. It’s too important to leave to chance, or old habits. It’s time to be alert to what we know, and put it into practice.

What’s the next buzzword?

24 April 2019 by Clark 2 Comments

I was perusing an old list of potential column topics, and came across one that asked about MOOCs. Now, you probably recognize that the term is pretty much evaporated from any list of top L&D concerns. That’s kind of funny, to me (ok, so you may question my sense of humor). And it makes me wonder what topics are current and are on the horizon. What is the coming buzzword?

I talked in a column about the problem with chasing shiny objects. In short, it’s easy to get swayed by the latest hot topic, and want to be seen to be on top of things. But, as I’ve said repeatedly, a gilded dud is still a dud. If we get the core right first, then we can move on to see what’s real. And, of course, we need to dig into the real affordances, not just the hype (PowerPoint in Second Life, anyone?).

So there are some buzzwords already on the wane. Such as MOOCs. A good sign is if someone’s trademarked it, it’s jumped the shark. Frankly, that already characterizes microlearning. And we’ve had someone recently claim to have invented workflow learning (though it’s been talked about for years). When they’re fighting about ownership, it’s done.

What’s waxing as opposed to waning? How about ‘bots? That’s the topic du jour! Often, as part of AI; as is Machine Learning, Deep Learning, and so on. Also Analytics (I think Big Data is already in the last paragraph’s category). Not necessarily bad, but part of this phenomenon is a lack of clarity about what we mean when we use any of these terms. So, maybe it is like AI: if you know what you’re talking about, it’s no longer new and shiny! And of course, AR and VR are very much now. And personalized and adaptive! (Time for some ownership moves!)

So here’s the question: what’s the next buzzword? Would that it were learning science! Ok, there’s been a bit of a resurgence (time to plug the coming Science of Learning Summit, with the usual caveat), but not near enough. C’mon, folks, lets get together and work on taking your design approaches and tuning them up! Of course, I could wish we’re talking IA instead of AI, too. What else? Contextual. Content Systems. Those are my thinking (and I’ve been talking about these things for years; maybe it’s time).

So, what’s on your list? What’s next? What’s ready for primetime? Wearables? Post-AI? (I just made that up.) I look forward to hearing your thoughts!

Reflection on reflection

23 April 2019 by Clark Leave a Comment

Of late, there’ve been a few dialogs on Twitter. As I opined in the recent podcast I was interviewed in, using Twitter for a dialog is kind of new. I’m not talking about a tweet chat like #lrnchat (which I think is a great thing), but a out-loud dialog with others weighing in. And it’s fun, and informative, but occasionally I need to go deeper. So here’s a reflection on reflection.

In that podcast interview, I opined, as I often do, about action and reflection. The starting point is a claim is that our own learning action and then reflection. What I mean is that we act in the world, and if we reflect on it we can learn.

One of the pushbacks was that we can learn without reflection. And, yes, I agree. We can learn without conscious feedback. In fact, in Kathy Sierra’s insightful Badass, she talks about chicken sexing, a task which no one’s been able to make consciously accessible. Things can go below consciousness.

This was related to another pushback: do we really learn differently from chickens and rats? And the answer is no, but what we learn is different. And, further, what we can learn is different. I’ve yet to see rats sending rockets up to the moon to see if it’s made of cheese.

Conscious representations facilitate learning, particularly for things we learn that aren’t strongly tied to our evolved survival. Learning about cognition itself, for instance, the ability to think about our own thinking, is just something that separates us fundamentally. And, to do that well, conscious artifacts facilitate it.

We’ve found that creating conscious frameworks to facilitate our understanding and acquisition are helpful. So, specifically, models and examples are two things that help us develop skills. We use models to guide and review our performance, to guide us. M0dels are conceptual relationships that we can compare to our performance. Examples show how those models play out in particular contexts.

There’s a followup: if learning is action and reflection, then instruction should be designed action and guided reflection. That is: do, get feedback, but also more. To me, models and examples are that additional reflection. We can present them ahead of time (but see Problem-Based Learning), but we should use them as part of the feedback, pointing out how flaws in performance didn’t align with the models, and further examples that illustrate those nuances.

Ok, so I may be playing fast and loose with the notion of reflection here, lumping in models and examples and feedback. However, my point is to try to keep learning not being information dump and knowledge test. We know that won’t lead to meaningful change. If I label it action and reflection, we have a better chance to push for an application-based instruction.

So, I’ll stick to my claim about (designed) action and (guided) reflection, with the caveat that my ‘reflection’ is more than just noodling. And, yes, it’s for learning goals beyond ‘hitting your head on rocks hurts’. But the goals I’m focusing on are the types of goals that will make a difference in individual and organizational success in our society. If I’m pushing too far and too hard, let me know.

Exploration & Surprise

17 April 2019 by Clark 2 Comments

Some weeks back, I posted about surprise. That is, a new model that says that that our brains work to minimize surprise. We learn so as not to be wrong. And that made sense in one way, but left another gap. Another article explains (well, partly; the mathematics are more than I want to wade into) further, and that gives me a new handle on thinking about designing transformative experiences. It’s about the value of exploration to accompany surprise.

The problem with the original story of us just minimizing surprise is that this leads to another inference. Why wouldn’t we want to just hang in a dark warm room? The notion of minimizing surprise did explain people who don’t seem keen to learn, but many of us are. And, as Raph Koster told us in A Theory of Fun, the drive to play games seems to be learning! We want exploration, and the outcomes aren’t certain. This is in conflict.

The new article posits that there’s another factor, the expectation of value. We also want the optimal outcome. The theory says that we’ll be willing to try several relatively equal predicted value outcomes to learn which to choose in the future (if I’ve understood the article correctly). So we will explore even under uncertainty if there’s a benefit to learning.

This doesn’t quite explain things to me. I think it’s missing some emotional aspect. Why would we do things like try out Escape Rooms or The Void (as I’ve done with colleagues)? There’s no real outcome, except perhaps to know about such experiences. But horror movies or thrillers? All we know is that we’ll have our emotions raised and then settled. But maybe that fits into a good outcome.

Still, this gives me a new handle. When I was preparing the Learning Experience Design workshop I gave at Learning Solutions last month, I was talking about ensuring surprise. That is, the learning experience should make learners aware that they didn’t know what the outcome would be. But I knew, and suggested, that there had to be more. They had to care about the outcome. And now we have the hook.

They care about the outcome, because it’ll be a higher value situation once they do! If we do this right, we let them know that they care about the outcome, and they can’t do it now (either they know already, or we have them fail). Then, we can offer them the path to achieve this outcome. If they explore, they’ll learn? If we’ve got a truly meaningful outcome (you’ll now be able to do X) that they truly care about (you do want to be able to do X), you’re now set with emotionally ready learners. Cognitive science models suggest that this should work! :)

To turn it around. the point is that you should create a goal that they should desire, and then demonstrate that they don’t already know it. It’s simplistic, but I think it’s part of creating a transformative experience, one where they are not just ready for the outcome, but eager. And I think that’s desirable. What do you think?

Quinnovations

16 April 2019 by Clark Leave a Comment

I was talking with my lass, and reminiscing about a few things. And, it occurs to me, that I may not have mentioned them all. Worse, I confess, I’m still somewhat proud of them. So, at the risk of self-aggrandizement, I thought I’d share a few of my Quinnovations. There’s a bigger list here, but this is the ‘greatest hits’ list, with some annotation. (Note, I’ve already discussed the game Quest for Independence, one of my most rewarding works.)

One project was a game based upon my PhD topic. I proposed a series of steps involved in analogical reasoning, and tested them both alone and then after some training. I found some improvement (arguing for the value of meta-learning instruction). During my post-doc, a side project was developing a game that embedded analogical reasoning in a story setting. I created a (non-existent) island, and set the story in the myths of the voodoo culture on it. The goal was a research environment for analogical reasoning; the puzzles in the game required making inferences from the culture. Most players were random, interestingly, at a test, but a couple were systematic.

With a colleague, Anne Forster, we came up with an idea for an online conference to preface a face-to-face event. This was back circa 1996, so there weren’t platforms for such. I secured the programming assistance of a couple of the techs in the office I was working for (Open Net), and we developed the environment. In it, six folks reknown in their area conducted overlapping conversations around their topic. This set up the event, and saw vibrant discussions.

A colleague at an organization I was working for, Access Australia CMC, had come up with the idea of competition for school kids to create websites about a topic. With another colleague, we brainstormed a topic for the first running of the event. In it, we had kids report on innovations in their towns that they could share with other towns (anywhere). I led the design and implementation of the competition: site and announcements, getting it up and running. It ended up generating vibrant participation and winning awards.

Upon my return to the US, I led a team to generate a learning system that developed learners’ understanding of themselves as learners. Ultimately, I conceived of a model whereby we profiled learners as to their learning characteristics (NB: not learning styles) and adapted learning on that basis. There was a lot to it: a content model, rules for adaptation, machine learning for continuing improvement, and more. We got it up and running, and while it evaporated in 2001 (as did the organization we worked for), it’s legacy served me in several other projects. (And, while they didn’t base it on our system, to my knowledge, it’s roughly the same architecture being seen in Newton.)

Using the concept of that adaptive system, with one of my clients we pitched and won the right to develop an electronic performance support system. It ended up being a context-sensitive help system (which is what an EPSS really is ;). I created the initial framework which the team executed against (replacing a help system created by the system engineers, not the right team to do it). The design wrote content into a framework that populated the manual (as prescribed by law) and the help system. The client ended up getting a patent on it (with my name on too ;).

Last one I’ll mention for now, a content system for a publisher. They were going to the next generation of their online tool, and were looking for a framework to: incorporate their existing texts, guide the next generation of texts, and support multiple business models. Again pulling on that content structure experience, I gave them a structured content model that met their needs. The model was supposed to be coupled with a tech platform, and that project collapsed, meaning my model didn’t see the light of day. However, I was pleased to find out subsequently that it had a lasting impact on their subsequent works!

The point being that, in conjunction with clients and partners, I have been consistently generating innovations thru the years. I’m not an academic, tho’ I have been and know the research and theories. Instead, I’m a consultant who comes in early, applies the frameworks to come up with ideas that are both good and unique (I capitalize a lot on models I’ve collected over the years), and gets out quickly when I’m no longer adding value. Clients get an outcome that is uniquely appropriate, innovative, and effective. Ideas they likely wouldn’t have come up with on their own! If you’d like to Quinnovate, get in touch!

« Previous Page