Archives for September 2021

Meta-ethics of learning design

28 September 2021 by Clark 1 Comment

I‘ve addressed ethics elsewhere, but I‘m looking at it a different way now. I’m thinking from the perspective of situated cognition, and recognizing that there are certain things we can do. For better or worse. Further, these choices have ramifications beyond the initial impact. I think we need to be aware of the possibilities, and then consider the meta-ethics of learning design.

My particular concern sparks from the notion of how we are contextually sensitive. To make this concrete, think of the research by Beth Loftus. Iâ€ll characterize a whole suite of research with a simply paraphrased experiment. So, she had folks watch a video of an accident. Then, she prompted recall of the amount of damage they witnessed. For one, she just asked them. For another group, the recall was prompted by “Besides the broken glass,…â€. The latter group recalled worse damage. And. There. Was. No. Broken. Glass!

The point here is that context can influence our thinking and memory. Which is what I see worry about with videos. They can take a statement as if it‘s fact, and then continue on with that as received wisdom. It‘s a classic cognitive approach, making a statement as if it‘s assumed.The bad part is that there‘s a narrative flow, and it‘s hard to stop and reflect. Versus, say, reading.

On the other hand, of course, we can build in reflection time. Sure, learners can use the pause button, but there are times they may not. For instance, if the learner is following confirmation bias, that is looking at things that align with what you believe.

We have the option to use coercive techniques for good, of course. However, we can also choose to use legitimate presentation techniques. I believe we should. Part of the development of metacognition is to see the pedagogy being used and internalizing it. If our pedagogy is visible (as it should be), it needs to be scrutable if we want our learners to adopt appropriate skills.

Our designs, and our meta-designs, need to be ethically designed, both to effectively achieve our ends, and to develop our learners. We need to support the meta-ethics of learning design, as well as the ethics themselves.

Complexity in Learning Design

21 September 2021 by Clark Leave a Comment

$a fractal$ I recently mentioned that one of the problems with research is that things are more interconnected than we think. This is particularly true with cognitive research. While we can make distinctions that simplify things in useful ways (e.g. the human information processing system model*), the underlying picture is of a more interactive system. Which underpins why it makes sense to talk about Learning Experience Design (LXD) and not just instructional design. We need to accommodate complexity in learning design. (* Which I talk about in Chapter 2 of my learning science book, and in my workshops on the same topic through the Allen Academy.)

We’re recognizing that the our cognition is more than just in our head. Marcia Conner, in her book Learn More Now mentioned how neuropeptides passed information around the body. Similarly, Annie Murphy Paul’s The Extended Mind talks about moving cognition (and learning) into the world. In my Make It Meaningful workshops (online or F2F at DevLearn 19 Oct), I focus on how to address the emotional component of learning. In short, learning is about more than just information dump and knowledge test.

Scientifically, we’re finding there are lots of complex interactions between the current context, our prior experience, and our cognitive architecture. We’re much more ‘situated’ in the moment than the rational beings we want to believe. Behavioral economics and Daniel Kahneman’s research have made this abundantly clear. We try to avoid the hard mental work using shortcuts that work sometimes, but not others. (Understanding when is an important component of this).

We get good traction from learning science and instructional design approaches, for sure. There are good prescriptions (that we often ignore, for reasons above) about what to do and how. So, we should follow them. However, we need more. Which is why I tout LXD Strategy! We need to account for complexity in learning design approaches.

For one, our design processes need to be iterative. We’ll make our best first guess, but it won’t be right, and we’ll need to tune. The incorporation of agile approaches, whether SAM or LLAMA or even just iterative ADDIE, reflects this. We need to evaluate and refine our designs to match the fact that our audience is more complex than we thought.

Our design also needs to think about the emotional experience as well as the cognitive experience. We want our design processes to systematically incorporate humor, safety, motivation, and more. Have we tuned the challenge enough, and how will we know? Have we appropriately incorporated story? Are our graphics aligned or adding to cognitive load? There are lots of elements that factor in.

Our design process has to accommodate SMEs who literally can’t access what they do. Also learner interests, not just knowledge. We need to know what interim deliverables, processes for evaluation, times when we shouldn’t be working solo, and tools we need. Most importantly, we have to do this in a practical way, under real-world resource constraints.

Which is why we need to address this strategically. Too many design processes are carry-over from industrial approaches: one person, one tool, and a waterfall process. We need to do better. There’s complexity in learning design, both on the part of our learners, and ourselves as designers. Leveraging what we know about cognitive science can provide us with structures and approaches that accommodate these factors. That’s only true, however, if we are aware and actively address it. I’m happy to help, but can only do so if you reach out. (You know how to find me. ;) Here’s to effective and engaging learning!

Reading Research?

14 September 2021 by Clark Leave a Comment

I was honored to have a colleague laud my Myths book (she was kind enough to also promote the newer learning science book), but it was something she said that I found intriguing. She suggested that one of the things in it includes “discussing how to read research”. And it occurs to me that it’s worth unpacking the situation a wee bit more. So here’s a discussion about how we (properly) develop learning science that informs us in reading research.

Caveat: I haven’t been an active researcher for decades, serving instead to interpret and apply the research, but it’s easier to say ‘we’ than “scientists”, etc.

Generally, theory drives research. You’ve created an explanation that accounts for observed phenomena better than previous approaches. What you do then is extend it to other predictions, and test them. Occasionally, we do purely exploratory studies just to see what emerges, but mostly we generate hypotheses and test them.

We do this with some rigor. We try to ensure that the method we devise removes confounding variables, and then we use statistical analysis to remove the effects of other factors. For instance, I created a convoluted balancing approach to remove order effects in my Ph.D. research. (So complicated that I had to analyze a factor or two first, to ensure it wasn’t a factor, so I could remove it from the resulting analysis!). We also try to select relevant subjects, design uncontaminated materials, and carefully control our analysis. Understanding the ways in which we do this requires an ability to know about experiment design, which isn’t common knowledge.

Moreover, we then need to share this with our colleagues so that they can review what we’ve done. We need to do it in unambiguous language, using the specific vocabulary of our field. And we need to make it scrutable. Thus, we publish in peer-reviewed journals which mean others have looked at our work and deemed it acceptable. However, the language is deliberately passive, unemotional, and precise, as well as focused on a very narrow topic. Thus, it’s not a lot of fun to read unless you really care about the topic!

There are problems with this. Increasingly, we’re finding that trying to isolate independent variables doesn’t reflect the inherent interactions. Our brains actually have a lot of complexity that hinder simple explanations. We’ve also found that it’s difficult to get representative subjects, when what’s easy to get are higher education students in the developed world. There are also politics involved, sad to say, so that it can be hard for new ideas to emerge if they challenge the entrenched views. Yet, it’s still the best approach we have. The scientific method has led to more advances in understanding than anything else!

There are things to worry about as a consumer of science. For one, there are people who fake results. They’re few, of course. There’s also research that’s kept proprietary, for financial reasons. Or is commissioned. As soon as there’s money involved, there’s the opportunity for corruption (think: tobacco, and sugar). Companies may have something that they tout as valid, but the research base isn’t publically available. Caveat emptor!

Thus, being able to successfully read research isn’t for everyone. You need to be able to comprehend the studies, and know when to be wary. The easy thing to do is to look for translations, and translators, who have demonstrated a trustworthy ability to help sort out the wheat from the chaff. They exist.

I hope this illustrates what reading research requires. You can take some preliminary steps: give it the ‘sniff’ test, see if it applies to you, and see who’s telling you this (and if anyone else is agreeing or saying to the contrary) and what their stake in the game is. If these steps don’t answer a question, however, maybe you want to look for good guidance. Make sense?

Coping with Change: A Book Review of Flux by April Rinne

9 September 2021 by Clark 1 Comment

How do we cope with change? There’s a myth that we resist change, but Peter de Jaeger busted that in a talk I heard where he pointed out that we make changes all the time. We get married, take a different job, have kids, all of which are changes. The difference is that these are changes we choose! However, in this era of increasing change, we’re likely going to face more and more changes we didn’t expect. Can we improve our ability for coping with change? Yes, says April Rinne in her book Flux: 8 Superpowers for Thriving in Constant Change.

And here’s a caveat: I am part of a group she put together to talk about Flux while writing the book. I’m in the acknowledgements.

April, faced with a heavy unchosen change in her teens, carried that with her. It’s driven her interest in change and how we can learn to cope. Given that we’re in an era of increasing change, she recognized that we would benefit from having some approaches to improve our reslience. She looked at a wide variety of inputs, and has distilled her learnings into 8 mental frameworks that assist.

The underlying focus is on a flux mindset, that is, a stance that change is coming and to be accepting, not resisting. The eight different ways of looking at the world are deliberately provocative, but also apt:

Run Slower
See What’s Invisible
Get Lost
Start with Trust
Know Your ‘Enough’
Create Your Portfolio Career
Be All the More Human
Let Go of the Future

Each gets a chapter, with illustrations of the challenge, and practical ways to enact. You may find, like I did, that some are familiar, others are more challenging. Each comes from either or both of ancient wisdom and practical experience. The ones that were new I find to be all the more interesting. And useful!

That’s the real key. It’s very much aligned with what we know about how our brains work (a big issue with me, as this audience has probably learned ;). Some areas I feel like I’ve a handle on (e.g. run slower), and others are things are more challenging (e.g. see what’s invisible). There are bound to be areas of work for you. The upside of that work, however, is likely to be a better ability to ‘be’.

This is a book that you’ll want your loved ones to read, because what it provides aligns with a view of the world as it could and should be. It’s a guide for coping with change that addresses not only individuals, but organizations and society as a whole. Highly recommended.

Iterating and evaluating

7 September 2021 by Clark Leave a Comment

I’ve argued before about the need for evaluation in our work. This occurs summatively, where we’re looking beyond smile sheets to actually determine the impact of our efforts. However, it also should work formatively, where we’re seeing if we’re getting closer. Yet there are some ways in which we go off track. So I want to talk about iterating and evaluating our learning initiatives.

Let’s start by talking about our design processes. The 800 lb gorilla of ADDIE has shifted from a water flow model to a more iterative approach. Yet it still brings baggage. Of late, more agile and iterative approaches have emerged, not least Michael Allen’s SAM and Megan Torrance’s LLAMA. Agile approaches, where we’re exploring, make more sense when designing for people, with their inherent complexity.

Agile approaches work on the basis of creating, basically, Minimum Viable Products, and then iterating. We evaluate each iteration. That is, we check to see what need to be improved, and what is good enough. However, when are we done?

In my workshops, when talking about iteration, I like to ask the audience this question. Frequently, the answer is “when we run out of time and money”. That’s an understandable answer, but I maintain it’s the wrong answer.

If we iterate until we run out of time and money, we don’t know that we’ve actually met our goals. As I explained about social media metrics, but applies here too, you should be iterating until you achieve the metrics you’ve set. That means you know what you’re trying to do!

Which requires, of course, that you set metrics about what your solution should achieve. That could include usability and engagement (which come before and after, respectively), but most critically ‘impact’. Is this learning initiative solving the problem we’re designing it to achieve? Which also means you need to have a discussion of why you’re building it, and how you know it’s working.

Of course, if you’re running out of time and money faster than you’re getting close to your goal, you have to decide whether to relax your standards, or apply for more resources, or abandon your work, or…but at least you’re doing so consciously. Yet this is still better than heuristically determining that three iterations is arbitrarily appropriate, for example.

I do recognize that this isn’t our current situation, and changing it isn’t easy. We’re still asked to make slide decks look good, or create a course on X, etc. Ultimately, however, our professionalism will ask us to do better. Be ready. Eventually, your CFO should care about the return on your expenditures, and it’ll be nice to have a real answer. So, iterating and evaluating should be your long term approach. Right?