I was thinking about what too often we see as elearning. That is, the usual content dump and knowledge test. There’s good reason to believe that it isn’t effective. So, why are we seeing it continue? Is it a placebo effect?

I tend to view this as a superstition. That is, the belief that information presentation will lead to behavior change is held implicitly. I think it originates from a legacy perspective that we’re logical, and therefore new information will yield impact. (Not.) Regardless, it exists.

I was inclined to wonder if, really, it’s a placebo. That is, doing something with a hope that things change, but the onus is on the individual, not the intervention. There’s not going to be any actual effect, but it makes people feel better. Of course, the role is different here; the placebo makes the doctor feel better! (Or the health system? I’m muddling my metaphor…:)

It may not be that in practice, of course. There is a ‘faith’ that “if we build it, it is good”. So, biz units can ask for a course, and get one. They’ve provided content and access to SMEs. However, they push back when asked “what’s the actual problem”, let alone asked for measures.  It’s like they think the job can be done with information. They are happy with the appearance of a solution, because it’s easy, and no one’s checking.

We, of course, have to change this perception. If we continue to let folks believe they can give us content and we’ll deliver meaningful change, shame on us. Of course they don’t care about the measures, and they want things to be easy. We have to, however.  It may as well be a placebo effect, because the ultimate impact is a likely null as sugar pills, unless the patient wants to change. It’s probably not a great metaphor, but somehow it still seems apt. Thoughts?

Ok, so I’m going to go out on a limb here, and talk a wee bit about what I’ve been learning about designing curricula. I care about doing it right (and probably haven’t always). It’s not the average course that’s the issue, but big ones, or multiple courses addressing skill gaps. It’s been challenging to find a systematic approach, which is why I’m teetering on a curriculimb.

So, the issue is how to develop a curriculum. I know in higher ed (I was there once) it tends to be a process of figuring out what content they need, and distributing across courses. It’s probably more art than science, where you move stuff around until it feels like you’ve got the right sized amount of content for each subject and it covers the ‘right stuff’. How people meet the criteria can vary. In a more research institution, I could design my HCI course my way. In more teaching-focused institutions, people may actually be given course syllabi to teach to!

My problem is when I have an uncertain amount of content, say for a large domain, and I want to develop specific capabilities. On principle, we should work backwards from the final performance. Which might include some very rich types of capabilities, so we might have a lot of concepts and practice involved. We’d need to create a large map. We might even break it up into conceptual stages (e.g. with programming: learning conditionals and then loops), and addressing them separately.

You probably also need to provide some practice to deal with misconceptions. That is, where are folks likely to get off track and maybe discouraged? Then you want to create practice for that. The things you’d rather they learned before it matters.

When I looked for good principles around this, it seemed like most of what I found basically said it’s iterative, there are no overarching principles (except work backwards and iterate ;). Which was less than satisfying, and some evidence-based practice would be nice.

Now, one of the things I was pondering in the dark of the night was how AI could help. I’ve been hearing how it can parse content and create maps. However, I also realized that to do so, it needs well-structured content. Kind of a circular argument. I think we need people to define it then AI can align it.

Again, right now it seems more like an art than a science. And I get that; it’s a lot like designing in engagement: create a first best guess and then test. Still, there are some solid results in engagement that give us some grounds for the first pass. I feel less like that at the next level up. So, I’m out on a curriculimb, and welcome help getting down!

For the LDA debate this month, Ruth Clark talked to Matt Richter and I about aesthetics in learning. Ruth, you should know, is the co-author of eLearning and the Science of Instruction, amongst other books, a must-have which leverages Rich Mayer’s work on multimedia learning. Thus, she’s knowledgeable about what the research says. What emerged in the conversation was a problem about tradeoffs in aesthetics, that’s worth exploring.

So, for one thing, we know that gratuitous media interferes with learning. From John Sweller’s work on cognitive load theory, we know that processing the unnecessary data reduces cognitive resources available to support learning. There’s usually enough load just with the learning materials. Unless the material materially supports learning, it should be avoided.

On the other hand, we also know that we should contextualize learning. The late John Branford’s work with the Cognitive Technology Group while at Vanderbilt, for instance, demonstrated this. As the late David Jonassen also demonstrated with his problem-based learning, we retain and transfer better with concrete problems. Thus, creating a concrete setting for applying the knowledge is of benefit to learning.

What this sets up, of course, is a tradeoff. That is, we want to use aesthetics to help communicate the context, but we want to keep them minimal. How do we do this? Even text (which is a medium), can be extraneous. There really is only one true response. We have to create our first best guess, and then we test. The testing doesn’t have to be to the level of scientific rigor, mind you. Even if it just passes the scrutiny of fellow team members, it can be the right choice, though ideally we run it by learners.

What we have to fight is those who want to tart it up. There will be folks who want more aesthetics. We have to push back against that, particularly if we think it interferes with learning. We need to ensure that what’re producing doesn’t violate what’s known. It’s not always easy, and in situations we may not always win, but we have to be willing to give it a go.

There are tradeoffs in aesthetics, so we have to know what matters. Ultimately, it’s about the learning outcomes. Thus, focusing on the minimum contextualization, and the maximum learning, is likely to get us to a good first draft. Then, let’s see if we can’t check. Right?

In conversations, I’ve begun to suspect that one of the biggest, if not the biggest, problem facing designers wishing to do truly good, deep, design, is client expectations. That is, a belief that if we’re provided with the appropriate information, we can crank out a solution. Why, don’t you just distribute the information across the screen and add a quiz? While there are myriad problems, such as lack of knowledge of how learning works, etc, folks seem to think you can turn around a course in two weeks. Thus, I’m led to ponder if time is the biggest problem.

In the early days of educational technology, it was considered technically difficult. Thus, teams worked on instantiations: instructional designers, media experts, technologists. Moreover, they tested, refined, and retested. Over time, the tools got better. You still had teams, but things could go faster. You could create a draft solution pretty quickly, with rapid tools. However, when people saw the solutions, they were satisfied. It looks like content and quizzes, which is what school is, and that’s learning, right? Without understanding the nuances, it’s hard to tell well-produced learning from well-designed and well-produced learning. Iteration and testing fell away.

Now, folks believe that with a rapid tool and content, you can churn out learning by turning the handle. Put content into the hopper, and out comes courses. This was desirable from a cost-efficiency standpoint. This gets worse when we fail to measure impact. If we’re just asking people whether they like it, we don’t really know if it’s working. There’s no basis to iterate! (BTW, the correlation for learner assessment of learning quality, and the actual quality, is essentially zero.)

For the record, an information dump and knowledge test is highly unlikely to lead to any significant change in behavior (which is what learning we are trying to accomplish). We need is meaningful practice, and to get that right requires a first draft, and fine tuning. We know this, and yet we struggle to find time and resources to do it, because of expectations.

These expectations of speed, and unrealistic beliefs in quality, create a barrier to actually achieving meaningful outcomes. If folks aren’t willing to pay for the time and effort to do it right, and they’re not looking at outcomes, they will continue to believe that what they’re spending isn’t a waste.

I’ve argued before that what might make the biggest impact is measurement. That is, we should be looking to address some measurable problem in the org and not stop until we have addressed it. With that, it becomes easier to show that the quick solutions aren’t having the needed impact. We need evidence to support making the change, but I reckon we also need to raise awareness. If we want to change perception, and the situation, we need to ensure that others know time is the biggest problem. Do you agree?

In a recent conversation, I was talking about good design. Someone asked a question, and I elaborated that there was more to consider. Pressed again, I expanded yet more. I realized that when talking good learning design, it’s complex. However, knowing how it’s complex is a first step. Also, there are good guidelines. Still, we will have to test.

I’m not alone in suggesting that, arguably, the most complex thing in the known universe is the human brain. I jokingly ask whether bullet points are going to lead to sustained changes in behavior in such a complex organism? Yet, I also tout learning science design principles that help us. Is there a resolution?

The complexity comes from a number of different issues. For one, the type, quantity, challenge, and timing of practice depends on multiple factors. Things that can play a role include how complex the task is, how frequently it’s performed, and how important the consequences are. Similarly, the nature of the topic, whether it’s evolutionarily primary or secondary can also have an influence. The audience, of course, makes a difference, as does the context of practice. Addressing the ‘conative’ element – motivation, anxiety, confidence – also require some consideration.That’s a lot of factors!

Yet, we know what makes good practice, and we can make initial estimates of how much we need. Likewise, we can choose a suite of contexts to be covered to support appropriate transfer. We have processes as well as principles to assist us in making an initial design.

Importantly, we should not assume that the first design is sufficient. We do, unfortunately, and wrongly. Owing to the complexity of items identified previously, even with great principles and practices, we should expect that we’ll need to tune the experience. We need to prototype, test, and refine. We also need to build that testing into our timelines and budgets.

There is good guidance about testing, as well. We know we should focus on practice first, using the lowest technology possible. We should test early and often. Just as we have design guidance, these are practices that we know assist in iterating to a sufficient solution. Similarly, we know enough that it shouldn’t take much tuning since we should be starting from a good basis.

Using the cognitive and learning sciences, we have good bases to start from on the way to successful performance interventions. We have practices that address our limitations as designers, and the necessities for tuning. We do have to put these in practice in our planning, resourcing, and executing. Yet we can create successful initiatives reliably and repeatedly if we follow what’s known, including tuning. It’s complex, but it’s doable. That’s the knowledge we need to acknowledge, and ensure we possess and apply.