Learnlets

Secondary

Clark Quinn’s Learnings about Learning

The plusses and minuses of learning science research

25 August 2020 by Clark 1 Comment

A person who I find quite insightful (and occasionally inciteful ;) is Donald Clark. He built and sold Epic, an elearning company, and now he leads a learning AI company, Wildfire. He’s knowledgeable (for instance, having read up and summarized centuries of learning theorists), willing to call out bad learning, and he’s funny. And so, when he reported on a new study, I of course looked into it. And I find that it points out the plusses  and  minuses of learning science research.

To be clear, this is about his product, so there’s a vested interest. However, he’s got integrity; he’s not going to sully his reputation with a bad study. And, it’s a good study. It rightly demonstrates an important point. It’s just that it stops short of what we need for full  learning.

So, his product does something pretty amazing. You give it content, and it can not only answer questions about the content (as, for instance, some chat tools do), it can turn the tables and ask  you questions about the content. That is, it can serve as a sort of tutor. Which is all to the good.

What it can’t do, of course, is design meaningful practice. As Van Merriënboer’s Four Component Instructional Design (4C/ID) points out, you need to know the information, and you also need practice applying it. And I reckon we’re still far from that. So, while this is part of a whole solution (and Donald knows this), it’s not the full solution. He’s subsequently let me know it can do language tasks, which is impressive. I’m thinking more of contextualized scenarios, however.

The study demonstrates, as you might expect, that breaking up a video into reasonable chunks, and having system-generated questions asked in-between, led to 61% better retrieval, going from getting 8 to 14 questions right. That is a big improvement. it’s also impressive, since it’s generating those questions from video! That is, it parses the video, establishes a transcript, and then uses that to generate a knowledge base. Very cool.

And it’s a well-designed study. It’s got a control group, and a  reasonable number of subjects. It uses the same test material, for an AB comparison. Presumably, the video chunking was done by hand, into four pieces. The chunking and break might account for the difference, which wasn’t controlled for, but it’s still a big improvement. Granted, we know that watching a video alone isn’t necessarily going to improve retention (except, perhaps, over some other non-interactive way of dumping content). But still, this is good as it’s an improvement and a lot of work was saved.

What I quibble about, however, is the nature of the retrieval. The types of questions liable to be asked (and it’s not indicated), are knowledge questions. As suggested above, knowledge is a necessary component. But using that knowledge to make decisions in context is typically what our goals are. And to achieve such goals, you basically have to practice making decisions in context. (Interestingly, the topic here was equality and diversity, a topic he has complained about!)

Knowledge about a topic isn’t likely to impact your ability to apply it. What will  make a difference are actually doing things about it, like calling it out, having consequences, and actively working to remedy imbalances. And that requires separate practice. Which he’s acknowledged in the past, and rightly points out that his solution means you can devote more resources to that end.

Thus, the plusses of learning science research are we nibble away at the questions we need to answer, and find answers about the questions we ask. The minus, of course, is not necessarily asking the most important questions. It’d be easy to see this and say: “we’ve improved retention, and we’re done”. However, it won’t necessarily lead to reducing the behaviors being learned about, or building ability to deal with it.  There are plusses and minuses of learning science research, and we need to know the strengths, and limitations, of it when we hear it.

Top 10 Tools for Learning 2020

18 August 2020 by Clark Leave a Comment

It’s time, once again, for Jane Hart’s excellent Top 10 Tools for Learning survey. And, so, it’s time once again for my reflections. Here are my take on the top 10 tools that support my learning.

The first way I learn is to process what I’ve seen. That, toolwise, is largely about representing and communicating.

Processing Tools

1-2. Writing is arguably the top way I reflect. And, so that’d put Microsoft Word at the top of my list. That’s where I write books and articles first. And, of course WordPress is how I write my blog (e.g. here!).   Writing is a way to sort out how I think about things. As I say, things that end up in presentations and books tend to show up on blog first. Well, one of the main ways.

3-4. Besides writing, two ways I sort out my understandings are to diagram and to outline. I use OmniGraffle as a general purpose diagramming tool because, well, it largely works the way I want to think about it. Diagrams, mind maps, even recently as sort of posterboard. And I use OmniOutliner to do, well, outlines. Another way to map out structures. I’d use a less costly tool, but…the columns feature is really helpful for annotation. Both, unfortunately, are Mac only (and sadly quite dear).

5. Keynote is how I create presentations, another way I do, and then share, my thinking. Diagrams are a big part of my talks, punctuated with stock photos to represent concepts (from Pixabay and occasionally Unsplash). I believe (and don’t have evidence for) that using an image that relates to the concept but doesn’t exactly communicate it leaves open some curiosity that then gets connected. And that this leads to better comprehension (I avoid bullet points in live presos, and save them for handouts). Anyone got that data?

The second thing I do is see what other people are pointing to and have to say, and ask them questions   as well. So the second category is about interacting with others.

Social tools

6. Twitter is a regular feature of how I see what people are pointing to, as well as pointing to things I’ve found as well. Chats there are fun, too. Like Jane, Tweetdeck is my tool of course on my Mac. I have to use the Twitter client on iPad/iOS, since they’ve taken away Tweetdeck on the iPad (grr).

7. I like FeedBlitz as a way to sign up for blogs, as it brings them into my inbox, instead of me needing a separate app. Reading a select list of blogs is one of my tactics. That’s how people can sign up to get my blog in email, too.

8. Slack has also been a major component of getting things done, mostly with IBSTPI. It’s a handy way to get things done with others.

9-10. Social networks are a big part of my learning, which means that Facebook and LinkedIn also play big roles. Facebook’s more personal, ie less about work, but I learn about   many societal things there. And LinkedIn is a place for learning as well, professionally as opposed to personally.

And…

Honorable Mention: to round out the picture (10 is such an arbitrary number ;), sharing collaborative documents, e.g. Google Docs, is a major way to collaboratively process and learn together. Also socially, Zoom and BlueJeans (the latter’s almost the same, and what ISBTPI uses) are used a lot to discuss and negotiate understandings. And email, of course (using the Mac Mail client) is a major way I learn, e.g. blogs appear there, and it’s a major way I interact.

DuckDuckGo has become my goto search engine (and Brave as my browser,  at least on my Mac, awaiting cross-device sync), because I don’t need to spread my data any further than necessary. And searching is a big part of my learning.

As an aside, owing to the pandemic, like everyone else I’ve been doing much more with Zoom to interact with colleagues than I had in the past. And I find, interestingly, that the ways I reach out are more opportunistic: I’ll use FB Messenger, or a Twitter DM, or a LinkedIn message, or an email depending on who, why, and what tool I’m in at the time. There may be some method to the madness, but I’m not confident on that point ;).

So, there’re my Top 10 Tools for Learning. I hope you’ll post or send your list to Jane too, so we can continue to see what emerges.

 

Experimenting with conference design

13 May 2020 by Clark 2 Comments

As part of coping in this time of upheaval, I’m trying different things. Which isn’t new, but there seem to be more innovations to tap into. In addition to teaching a course on mobile learning, I’m one of the speakers at a new online event. And, what’s nice, is that they’re experimenting with conference design, not just moving straight online.

To be fair, the Learning Guild has had a continual practice of trying different things at their conferences, and it’s been good. And, so too, was the most recent TK by ATD.   But this is different. Two of my colleagues organized it as a response to our ‘new normal’, Will Thalheimer and Matt Richter. And their stated goal is changing the way we conference.

The key, of course, is to leverage what’s different, and possible, online. It’s running from June 22 – July 31. That’s not a typo, it’s all of July and the tail end of June. That’s a long time!   They’ve recruited a suite of experts from around the world (they’re really trying to do this across boundaries include time and geography). And, to let you know, I’m one (so take my comments with the appropriate caveats ;).

They’re also tossing out traditional ideas and open to new ones. Speakers are expected to build an experience that’s spread out over the time. Yet also designed so that you can come in late, or early, and drill into what you want when you want. They’re also planing on having synchronous events – debates, panels, socializing – again using technology.

Note that it’s not free. There are some free conferences being put on, mostly webinars. And those are good. This is different. It’s deeper. It’s a stab at looking afresh. And I’m not sure it could even have come from any existing framework.

And, we won’t know if it all will work. We’re designing this in the time between now and launch. There’re bound to be hiccups. Which, of course, means there’re bound to be learnings. I know I want to talk about Learning Science 101. And something else. Lots I could (I welcome suggestions). I’m inclined to think it might be Emotion and Learning. But it could also be LXD. (There are all linked, of course.)

But it’s a high quality group (er, mostly…they did let me in). AND, importantly, it’s focused on evidence-based content. There may be sponsors, or even an exhibit hall, but every presenter is honor-bound not to push anything that’s not legit. Most importantly, there’s enough quality that overall it’s bound to be worth it.

I’m excited, frankly. I have to come up with some different ideas. And I like that. I’m glad that they’re experimenting with conference design. We all win, regardless! It’s part of learning, challenging yourself. So, do yourself a favor. Check it out. It may not be for you, but keep an open mind!

 

 

How to be a world-class educational technologist

25 February 2020 by Clark 4 Comments

On LinkedIn, I was asked: “I would like to ask sir, how can I be a world-class educational technologist?” And I thought that was a very interesting question. (Of course, my immediate response should be “how should I know?” ;) But I thought I’d do a bit better. So here’s a recast of my response.

First, I get requests about how to get started as an instructional designer (particularly offers to come work for me). And, well, I’m an independent consultant, and just haven’t been a business builder. But I want to respond helpfully, and it’s one of those things that happen enough that I have this canned response:

If you want a bootstrap, working volunteer for a not-for-profit (NFP) foundation is a good step if you can.   Areas of specialization? Depends on what you like: kids – K12 or NFP, higher ed, adult – organizational L&D. They differ. As to skills, make sure you know the major authoring tools, e.g. Lectora, Captivate, and/or Storyline. And of course have some background in instructional design/learning science.   If you haven’t covered performance consulting, look into it so you don’t design a course when there’s a better/simpler solution. Make sure you have a portfolio of work. Good luck!

In this case, I also pointed him to a previous post, where I’d outlined some roles for learning experience design.

Then, thinking at the bigger scale of not just getting going as a new ID, but persisting, I added this:

Overall, you have to have the passion, it’s a long road. Have a good understanding of learning science, a fundamental grasp of technology, a mind for both design and process, and then put it to work doing real projects! Continue to read, reflect, and then as you start getting your mind around it, start sharing your thinking and get feedback (and listen to it!). Start local, work outward to sharing regionally, nationally, and internationally. If you learn, adapt and improve, and persist, you can get there.

I think that’s the path to improvement, regardless. In short. There’s more: I have just finished reading Eric Barker’s Barking Up the Wrong Tree  (affiliate link);  thanks to an ATD Sacramento event attendee, and found it having very interesting recommendations. Things like setting goals, giving, getting mentored, and more.

I think aiming to be a world-class educational technologist is a noble goal. Even if you don’t succeed, you’re liable to be better than if you just go through the motions. Now, I’m sure you’ve found things I’ve missed, so have at it!

Content systems not content packages

17 December 2019 by Clark 1 Comment

In a conversation last week (ok, an engagement), the topic of content systems came up. Now this is something I’ve argued for before, in several ways. For one, separate content from how it’s delivered. And, pull content together by rules, not hardwired. And it’s also about the right level of granularity. It’s time to revisit the message, because I thought it was too early, but I think the time is fast coming when we can look at this.

This is in opposition to the notion of pre-packaged content. MOOCs showed that folks want to drill in to what they need. Yet we still pull everything together and launch it as a final total solution. We are moving to smaller chunks (all for the better; even if it is burdened with a misleading label). But there’s more.

The first point is about content models. That we should start designing our content into smaller chunks. My heuristic is the smallest thing you’d give one person or another. My more general principle is that resolves to breaking content down by it’s learning role: a concept model is different than an example is different than a practice.

This approach emerged from an initiative on an adaptive learning system I led. It now has played out as a mechanism to support several initiatives delivering content appropriately. For one, it was supporting different business products from the same content repository. For another it was about delivering the right thing at the right time.

Which leads to the second point, about being able to pick and deliver the right thing  for the context.  This includes adaptive systems for learning, but also context-based performance support. With a model of the learner, the context, and the content, you can write rules that put these together to optimally identify the right thing to push.

You can go further. Think of two different representatives from the same company visiting a client. A sales person and a field engineer are going to want different things in the same location. So you can add a model of ‘role’ (though that can also be tied to the learner model).

There’s more, of course. To do this well requires content strategy, engineering, and management. Someone put it this way: strategy is what you want to deliver, engineering is how, and management is overseeing the content lifecycle.

Ultimately, it’s about moving from hardwired content to flexible delivery. And that’s possible and desirable. Moreover, it’s the future. As we see the movement from LMS to LXP, we realize that it’s about delivering just what’s needed when useful. Recognizing that LXPs are portals, not about creating experiences, we see the need for federated search.

There’s more: semantics means we can identify what things are (and are not), so we can respond to queries. With chatbot interfaces, we can make it easier to automate the search and offering to deliver the right thing to the right person at the right time.

The future is here; we see it in web interfaces all over the place. Why aren’t we seeing it yet in learning? There are strong cognitive reasons (performance support, workflow learning, self-directed and self-regulated learning).  And the technology is no longer the limitation. So let’s get on it. It’s time to think content systems, not content packages.

 

Sophia the Robot #DevLearn Keynote Mindmap

23 October 2019 by Clark 1 Comment

DevLearn opened with a keynote from Sophia the Robot. With an initially scripted presentation, and some scripted questions from host David Kelly, Sophia addresses the differences between AI and robots, with a bit of wit. The tech used to make the illusion was explored, but the technology was put to the test with some unscripted questions, and the responses were pretty good. An interesting start!

Tools for LXD?

24 September 2019 by Clark Leave a Comment

I’ve been thinking on LXD for a while now, not least because I’ve an upcoming workshop at DevLearn in Lost Wages in October. And one of the things I’ve been thinking about are the tools we use for LXD. I’ve created diagrams (such as the Education Engagement Alignment), and quips, but here I’m thinking something else. We know that job aids are helpful; things like checklists, and decision trees, and lookup tables. And I’ve created some aids for the Udemy course on deeper elearning I developed. But here I want to know what  you are using as tools for LXD? How do you use external resources to keep your design on track?

The simple rationale, of course, is that there are things our brains are good at, and things they’re not. We are pattern-matchers and meaning-makers, naturally making up explanations for things that happen. We’re also creative, finding solutions under constraints. Our cognitive architecture is designed to do this; to help us adapt to the first-level world we evolved in.

However, our brains aren’t particularly good at the second-level world we have created. Complex ideas require external representation. We’re bad at remembering rote and arbitrary steps and details. We’re also bad at complex calculations.  This makes the case for tools that help scaffold these gaps in our cognition.

And, in particular, for design. Design tends to involve complex responses, in this case in terms of an experience design. That maps out over content, time, and tools. Consequently, there are opportunities to go awry. Therefore, tools are a plausible adjunct.

You might be using templates for good design. Here, you’d have a draft storyboard, for instance, that insures you’re including a meaningful introduction, causal conceptual model, examples, etc. Or you might have a checklist that details the elements you should be including. You could have a model course that you use as a reference.

My question, to you, is what tools are you using to increase the likelihood of a quality design, and how are they working for you?  I’d like to know what you’ve found helpful as tools for LXD, as I look to create the best support I can. Please share!

Level of polish?

22 August 2019 by Clark 4 Comments

A debate broke out amongst some colleagues the other day about the desirable level of polish in our elearning. One colleague was adamant that we were undermining our position by using low quality production. There was a lot of agreement. I had a slightly different view. Even after finding out he was talking more about external-facing content than internal, I still have some differences. After weighing in, I thought it required a longer response, and of course it has to go here.

So, the main complaint was that so much elearning looks dated and incomplete. And I agree!  And others chimed in that this doesn’t have to be, while all agreed that it doesn’t need to approach game quality in effect. Then, in my mind, the question switches to “what is good enough?” And I think we do need an answer to that. And, it turns out, to also answer “and what does it take?”

What is good enough?

So, my first concern is the quality of the design. My mantra on design states that it has to be right first. Then you can implement it. If it isn’t right from the get-go, it doesn’t matter  how you implement it. And the conversation took some time to sort this out. But let’s assume that the design’s right. Then, how much production values do you need?

The original complaint was that we’re looking slack by comparison. When you look at what’s being done in other, related, fields, our production values look last decade, if not last century!  And I couldn’t agree more. But does that matter?  And that’s where we start getting into nuances. My bottom line question is: “what’s the business case?”

So, I suggest that the investment in production values is based upon how important the ‘experience’ is. If it’s internal, and it’s a critical skill, the production values should be only enough to ensure that learners can identify the situation and perform appropriately (or get feedback).  It needs a minimum level of professionalism, and that’s it.  If you’re selling it to high-end customers and want to charge a premium price, you’ll need much more, of course.

The issue was that we’re losing credibility if we don’t approach a minimal level of competency. There were many arguments about the locus: fear of going out of bounds, managers oppression, low level tools, lack of skills, and more. And these all have validity. We should stipulate a minimal level. Perhaps the serious eLearning  Design Manifesto? :) We can do better.

What does it take?

This was the other issue. It was pointed out that design teams in other disciplines work in layers: from concept to realization. Jesse James Garrett has a lovely diagram that represents this for information architecture. And others pointed out that there are multiple skills involved, from dialog writing, through media production and interface design (they’re conceptually separate), and the quality of the programming and more. The more you need polish, the more you need to invest in the appropriate skill sets.  This again is a matter of marshaling the appropriate resources against the business case.

I think one of the issues is that we overuse courses when other solutions are more effective and efficient. Thus, we don’t have and properly allocate the resources to do the job right when it does positively absolutely has to be in the head. Thus, we do have a lot of boring, information dump courses. And we could be doing more with engaging practice, and less content presentation. That’s a design issue to begin, and then a presentation one.

Ultimately, I agree that bad elearning undermines our credibility. I do think, however, that we don’t need  unnecessary polish. Gilded bad design is still bad design. But then we should align our investment with the professional reception we need. And if we have trouble doing that, we need to rethink our approaches. The right level of investment for the context is the right response; we need the right live of polish. But the assessment the context is complex. We shouldn’t treat is simplistically, but instead systemically. If we get that right, we have a chance to impress folks with our astute sense of doing the right thing with the right resources. Less than that is a path to irrelevancy, and doing more is a path to redundancy. Where do  you want to go?

Graham Roberts #Realities360 Keynote Mindmap

26 June 2019 by Clark Leave a Comment

Graham Roberts kicked off the 2nd day of the Realities 360 conference talking about the Future of Immersive Storytelling. He told about their experiences and lessons building an ongoing suite of experiences. From the first efforts through to the most recent it was insightful. The examples were vibrant inspirations.

Stephanie Llamas #Realities360 Keynote Mindmap

25 June 2019 by Clark Leave a Comment

Stephanie Llamas kicked off the Realities 360 conference by providing an overview of VR & AR industry. As a market researcher, she made the case for both VR and AR/MR. With trend data and analysis she made a case for growth and real uses. She also suggested that you need to use it correctly. (Hence my talk later this day.)

Keynote Mindmap

« Previous Page
Next Page »

Clark Quinn

The Company

Search

Feedblitz (email) signup

Never miss a post
Your email address:*
Please wait...
Please enter all required fields Click to hide
Correct invalid entries Click to hide

Pages

  • About Learnlets and Quinnovation

The Serious eLearning Manifesto

Manifesto badge

Categories

  • design
  • games
  • meta-learning
  • mindmap
  • mobile
  • social
  • strategy
  • technology
  • Uncategorized
  • virtual worlds

License

Previous Posts

  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021
  • May 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • August 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • September 2009
  • August 2009
  • July 2009
  • June 2009
  • May 2009
  • April 2009
  • March 2009
  • February 2009
  • January 2009
  • December 2008
  • November 2008
  • October 2008
  • September 2008
  • August 2008
  • July 2008
  • June 2008
  • May 2008
  • April 2008
  • March 2008
  • February 2008
  • January 2008
  • December 2007
  • November 2007
  • October 2007
  • September 2007
  • August 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007
  • March 2007
  • February 2007
  • January 2007
  • December 2006
  • November 2006
  • October 2006
  • September 2006
  • August 2006
  • July 2006
  • June 2006
  • May 2006
  • April 2006
  • March 2006
  • February 2006
  • January 2006

Amazon Affiliate

Required to announce that, as an Amazon Associate, I earn from qualifying purchases. Mostly book links. Full disclosure.

We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.Ok