Clark Quinn's Learnings about Learning
(The Official Quinnovation blog)

15 December 2016

(When) Is pattern-matching enough?

Clark @ 8:09 am

In the course of my research, I came across the project shown here, as represented by the accompanying video.  In the video, they show (and tout) the value of their approach to developing pattern recognition around  mathematics.  Further, they argue that it’s superior to the typical rule presentation and practice. And I can buy that, but with many caveats that I want to explore.

So it’s clear that we learn by abstracting patterns across our experiences. We can provide models that guide, but ultimately it’s the practice that works. An extreme example is chicken-sexing (mentioned in the transcript); determining the gender of new-born chicks.  Here, no one can articulate the rationale, it’s merely done by attempts and correct/incorrect feedback!  And the clear implication is that by having learners do repetitive tasks of looking for patterns, they get better at it.

And, yes, they do.  But the open question is what is the learning benefit of that.  Let’s be clear, there are plenty of times we want that to happen. As I learned during my graduate studies, pilots are largely trained to react before their brains kick in: the speed at which things happen are faster than conscious processing.  When speed and accuracy is important, nay critical, we want patterned responses. And it does work for component skills to more complex ones in well-defined domains.  But…

When we need transfer, and things are complex, and we aren’t needing knee-jerk responses, this doesn’t work.  I would like to train myself to recognize patterns of behavior and ways to deal with them effectively, for instance (e.g. in difficult presentation situations, or negotiations).   On the other hand, in many instances I want to preclude any immediate responses and look for clues, ponder, explore, and more.

The important question is when we want rote performance and when we don’t .  Rote ability to do math component skills I’m willing to accept.  But I fear a major problem with math instruction in schools is about doing math, not about thinking like a mathematician (to quote Seymour Paper).  And I don’t want students to be learning the quadratic equation (one of Roger Schank’s most vivid examples) instead of how math can be used a problem-solving tool. The nuances are subtle, to be sure, but again I’m tired of us treating learning like color-by-numbers instead of the rocket science it should be.

Look, it’s great to find more effective methods, but let’s also be smart about the effective use of them. In my mind, that’s part of learning engineering. And I’m by no means accusing the approach that started this discussion of getting it wrong, this is my own editorial soapbox ;).  There’s much we can and should be doing, and new tools are welcome. But let’s also think about when they make sense.  So, does this make sense?

13 December 2016

Improving design processes

Clark @ 8:02 am

Recently, I had a chance to catch up with a client who’d used my services to review their design processes. Per my approach, I’d generated a report with a large number of suggestions. What happened, not surprisingly, is that a subset of them actually were implemented.  Still, it was gratifying to hear of the changes, and I think they’re worth exploring as an opportunity to show how small changes can yield big improvements.

One of the major areas was to work on how they use SMEs. Instead of just having a contracted expert, they’re moving to work with folks who have ‘on the ground’ expertise to couple with their domain expertise.  I argue for triangulating on the real objectives with several perspectives, and this approaches that ideal.

A second thing was that there was a bit of a ‘waterfall model’, and instead I suggest having collaboration at critical points along the way. In this case, they have moved to more collaboration at an important juncture point, with more roles involved to be more innovative on the possibilities.

Also, in their particular case, they have to abide by certain constraints, such as amount of time.  As a result of looking at the opportunities, they are moving to meeting their requirements by adding more practice, not more content.  This is a big win for the learners, and the learning.

They are also looking at applying their refined understanding of the nuances of learning. This is embryonic, I was told, but they are moving to looking to redesigning their content to better align with what is known about learning science. At this point it may not be instituted in their processes, but it’s already affecting the mindset they bring to the task.

There are a couple of other things they’re beginning, but I reckon these are some big wins for their audience and the outcomes. With only small changes in what they are doing, they are increasing the likelihood of effective learning.  And that’s the point: for any design process, there are inflection points where better outcomes can be achieved with minimal impact on the overall processes. That is, the processes may change, but they’re different, not hugely larger.

There’s much we can and should be doing to improve our learning design processes, for our learners, and the learning. We know what the opportunities are, now we just need to marshal the will to make changes.  Are you ready?

7 December 2016

Learning Strategy Issues

Clark @ 8:01 am

Online Educa logoThe other thing that I was involved in at Online Educa in Berlin was a session on The Flexible Worker.  Three of us presented, each addressing one particular topic.  One presentation was on collaborating to produce an elearning course on sleeping better, with the presenter’s firm demonstrating expertise in elearning, while the other firm had the subject matter expertise on sleep health. A second presentation was on providing tools to trainers to devolve content development locally, addressing a problem with centrally-developed content.  My presentation was on the gaps between what L&D does and how our brains work, and the implications. And, per our design, issues emerged.

The format was interesting: our presentations were roughly 10 minutes each.  And we were using a tool (sli.do) to collect and rank questions. Then we had the audience work at their tables (in the round) to come up with their answers to the top questions, which we then collected and the panelists riffed on the outcomes.  We got through three questions as a group, and I thought the outcomes were quite interesting. In short, as a rapporteur at the closing business session, I suggested that the topic ended up being about flexible work, not flexible working.

The top question that emerged had to do with how to support effective search (after I expounded on problem with the notion that it all had to be in the head).  The sourced answers included crowd-sourcing the tags for finding objects, using a controlled vocabulary, and auto-analyzing the content to determine tags.  I suggest a hybrid solution, in general. The interesting thing here was the audience picking up on the need to go beyond courses and start looking at resources.

The next question was how to move from a training to a performance culture.  And it was another exciting development to hear them thinking this way. The solutions offered included coaching, supporting the importance of self-learning (meta-learning, yay!), and working both top-down and bottom-up. I also suggested that measuring was a likely catalyst that could begin to draw attention to outcomes (just as  I reckon competencies are the lever in higher-ed).

The third question was about ensuring quality in a localized learning environment (e.g. user-generated content).  The concern was that the knowledge of learning design wouldn’t necessarily be widespread.  Suggestions included making the content editable for collaborative improvements, or using rankings, and scaffolding of improvement through the community. Here too, a focus on learning itself could assist.

What’s encouraging to me is that each of these questions was really about moving to a transformative viewpoint.  The audience was clearly thinking ‘beyond the course’.  They were focusing on supporting performers in learning, and resources, and leveraging the community, all activities consonant with the revolution.

An interesting aside came in the closing session. Several folks were mentioning a need for change, and an audience member asked “why?”  He was a consultant, and his clients already seemed to be moving forward. I suggested he was seeing the best, and that many folks were not there (mentioning the Towards Maturity data as well as the problems I identified in the beginning of the Revolution book.  And it’s a problem that too many people don’t yet see the missed opportunities and don’t feel the pain (and are frankly not looking).

So, there are opportunities to start taking small steps in the direction of taking on a bigger perspective and making the role of L&D more strategic.  It first takes an awareness of the problems (my old line: “L&D isn’t doing near what it could and should, and what it is doing it is doing badly, other than that things are fine” :), and then a strategy to move forward. The strategy depends on where the organization is to begin with, but there are systematic principles to guide progress.  That’s what I do, after all!  It’s nice to see awareness growing. So, are you ready to start taking some positive steps?


6 December 2016

Aligning Learning

Clark @ 8:09 am

Online Educa logoLast week, at Online Educa in Berlin, I gave a tutorial on deeper elearning as a pre-conference event. In it, I talked about getting more meaningful objectives, writing practice that actually develops meaningful outcomes, and content (concepts & examples) aligned to support effective practice. I also talked about emotional engagement and social learning, before talking about revising design processes to incorporate these deeper elements in an effective and not-too-different approach.  In short, I was talking about aligning our designs, and our design processes, to how we think and learn.

This is something that most organizations should be thinking about.  I was pleasantly surprised that the audience included folks from universities, not-for-profits, and government agencies as well as businesses.  The challenges are different in some respects, but there are shared elements. Education tends to be about long term learning relationships (typically at least a half year to several years), versus the short-term relationships (e.g. an hour to several days) in organizational learning.  Yet the need to respect how our brains work is a continuum. Our brains learn in particular ways that are unaffected by the curricular needs.  Learning solutions for performance and for education both still need to respect our neural and cognitive architecture.

And too little of what we do reflects what we know.  As a recent commenter noted, there’s a conflict between the de-facto practices and what research says.  As she also noted, our tools are also focused on supporting wrong approaches.  It’s not that tools prevent doing meaningful learning, it’s just that you have to get your design right first and then make the tool conform (as opposed to the alternative). And our limitations as designers flow from the same source, our brain, as our limitations as learners.  Thus we need to be as aware of cognition in our designing as in our design.

I’ll be talking about the problems this engenders in a special webinar tomorrow. There’re still a few slots left. If you’re committed to trying to improve your learning design, and you have the resources to do so, this is an opportunity to get started.  There isn’t a lot of pressure yet, but it’s time to be proactive before people start asking questions about the business impact of what we’re doing. What do you think?

30 November 2016

Content aggregation

Clark @ 8:05 am

At the recent DevLearn conference, I had a chance to present in xAPI Base Camp on Content Systems.  This is a topic I’ve been thinking about since working on an adaptive learning system back in 1999-2000. And I think it’s now an even bigger big opportunity.  In the course of responding to an interested attendee, I aggregated this list of my (non-blog) publications on the process, and I thought it might be useful to share them here (from most recent to earliest):

Talking about  lifecycle and opportunities in the Litmos blog

Perspective from visiting the Intelligent Content conference in eLearnMag

The possibilities and necessities of adapting to context in Learning Solutions magazine

New ways of looking at learning content in Learning Solutions magazine

Moving from text to experience for publishers in eLearnMag

A discussion paper including granularity and tagging in the IFETS journal

contentelementsWhat you see is a transition in focus.  From thinking about content objects at the right granularity and with the right tagging, I transition to thinking about strategy for content publishers. Ultimately, I have shifted to looking at the learning industry and opportunities to adapt learning to context (which includes location, current task, current role, and more).

While I think that adaptivity is a ways away, I also believe that the initial efforts in getting more rigorous about content strategy, engineering, and governance are a worthwhile investment now. The benefits are in less redundancy, tighter design, tracking, and greater flexibility.  The long term benefits will come from analytics, adaptivity, and contextualization.  I reckon that’s an opportunity that’s hard to ignore. I’ll suggest that it’s past time to be thinking about it, and time to start acting.  What about you?  Are you ready?

29 November 2016

Thoughts on Learning Design Strategy

Clark @ 8:10 am

learning design strategy questionsAt the DevLearn conference, I ran a Morning Buzz on Learning Design Strategy. I’m happy to say that the participants threw in lots of ideas, and I thought they were worth capturing. I started with a set of questions to address, so I’ll go through their comments in roughly that order (though we didn’t exactly follow this structure):

What is learning design strategy?

I had in mind the approach taken by an organization to their learning design.  Attendees suggested it’s your goals and approach, ensuring you are delivering effectively.  It’s also your review approach, and metrics.  These are all elements that indeed contribute to strategy.

What gaps are we seeing in learning design strategy?

The participants offered up a suite of places that were problems, including aligning with organizational goals and access to support measuring impact, both of which are indeed strategic issues.  They also raised problems with prioritization of the demands, the need to move beyond just courses, and the lack of learning design knowledge. All are real problems.

What do we need to be able to improve?

The audience offered up a number of suggestions.  For one, there was a desire for strategies (probably more tactics) for doing beyond ‘the event’.  Support for selling changes in the way of doing things was mentioned as well.  The shift to self-learning was mentioned, leading to concern over how to support this. Attendees also mentioned a need of awareness in designing ‘backwards‘. Finally, a culture of learning was expressly discussed.

What are possible solutions?

The participants offered a suite of suggestions. One was adopting a learn-apply-perform model, which another termed a learn-practice-demo. Both were getting at the need for active practice and an ability to actually demonstrate performance.  There was also a mention of looking to social networks and peer recommendations to lower the demand and facilitate self-learning.  A culture shift was suggested, supported by the methods used to teach! A final solution was to move quickly to mentoring, which implicitly suggests including mentoring in the design.

Steps to take to move forward?

I also wanted to know what how they might move forward, and what they needed.  Two clear suggestions emerged.  One was for examples, and I reckon both of better learning designs, and approaches to implement those learning designs in organizations.  The other was for tools. Here it was clear that they weren’t talking about tools to develop learning, but tools to support them doing good design, and following processes.

At the end I left with mixed feelings. It’s good to know that the problems I see are reflected in what the practitioners reports; we see the same problems  It’s also sad that these problems exist.  I do believe that the Serious eLearning Manifesto is one piece of support.  And I’ve written on practices (e.g. with SMEs), but it’s clear that some practical scaffolding would help. I’ve worked with a few organizations, but I’m struggling to find ways to help more.  (Maybe this is the topic of my next book?) So, what ideas do you have?

(I’m offering a webinar next week that will address these issues, if you’re serious about making changes.)

23 November 2016

Special Webinar on Learning Design Strategy

Clark @ 8:03 am

Learning, properly, should have an impact. It’s about systematically changing behavior, developing new skills to meet ever-changing needs. That’s why we invest in learning: training or elearning. If elearning doesn’t make an impactwho cares how accessible or affordable is seems? It’s actually undermining your goals for increasing expertise, effectiveness and productivity.

Too much of what is done under this umbrella isn’t sufficient. It’s quite simply not effective. Here’re some signs that your elearning might not be working:

  • Your learning unit develops courses on demand
  • You work from PDFs and PPTs to develop your course
  • You have a knowledge quiz at the end
  • You use libraries to add graphics & interactions
  • Your learners avoid the courses
  • You track completion
  • You evaluate impact by learner feedback

burning moneyIf you’re spending money to develop elearning, and it has any of the above features, there’s a strong chance you’re burning money.

The good news is that it’s not as hard to change as you think! The design processes you are using now, the ones that reflect the above issues, are not that far removed from ones that offer real outcomes. Yes, there are changes, but they’re changes within the process, not fundamental. You can be making changes that will make a marginal impact on the development measures, but a real impact on the learning outcomes.  It’s not trivial, but it is doable.

If this is of interest, I’m offering a free webinar to talk through the issues. It’s not for everyone: if you don’t have the authority or the resources to make a change, there are other posts that talk through the opportunities. This webinar is for those who really want to explore the possibilities. If that’s something you want to be thinking about, if you’d really like to consider consider how you get from here to there, I encourage you to keep reading.

Webinar: 7 Unbelievable eLearning Mistakes

Date: December 7

Time: 10 AM to 11AM Pacific

Via: Zoom Conference Service

Have you been concerned about your learning design: whether your designs are actually producing results? Ultimately, is your elearning changing behaviors, developing skills, and increasing capacity? And how do you know?  There are a lot of reasons to believe that most elearning is not delivering on the promise. And yet elearning has the potential to be a powerful tool for organizational excellence. The barriers are not unsurmountable; we know the problems, and steps to change.  However, as has been said: “When all is said and done, more is said than done.” I want to give you the chance to take the steps. In this complimentary webinar we’ll explore barriers to getting measurable results from elearning.

This isn’t your usual webinar, however. Here we are specifically talking to organizations that want their elearning to actually have an impact. If you don’t have the resources and position to make a change, this really isn’t for you. If you want your organization to take it more seriously, invite your boss ;).

This is for you, if:

  • You’re ready to look at your elearning with a serious eye
  • You want to ensure that you are getting value for your investment
  • You need to operate in the real world, under real constraints
  • You’re willing to invest for a real change that has impact

This isn’t for you if:

  • You are happy with the status quo
  • You haven’t the authority or the resources to make a change

If you are ready to take a serious look at your elearningI invite you to sign up.  There’s a limit to how many can attend, so please register early. When you register I’ll send you the necessary details. Hope to see you!

The webinar was run on schedule, and signups are now closed.

22 November 2016

Thoughts on story, games, and VR

Clark @ 8:09 am

story games VRAs luck would have it, I found out about an event on Storytelling Across Media being run in the city, and attended a couple of the panels: half of one on interactive design and Telltale Games, one on story and games, and one on story and VR.  There were interesting quotes from each about story, games, and VR that prompted reflection, and I thought I’d share my thoughts with  you.

Story and Games

The first quote that struck home was “nonlinear storytelling strikes a balance between narrative and choice”. This is the challenge that I and I think all game designers struggle with. So, I subsequently asked “How do you integrate storytelling with experience design?”  The panelists acknowledged that this was the ongoing challenge. Another comment was that “stories are created in your imagination”.  That’s key, I think, to create experiences that the player will end up writing as a story they can tell.

I found myself  thinking about story machines versus experience engines.  It appears to me that, ala Sid Maier’s “a good game is a series of interesting decisions”, that it’s all about the decisions you make.  It’s easier to tell a good story when you put a game ‘on rails’; it’s harder when you want to have an open world and still ramp up the tension across the board.  Having rules and timers give you the opportunity.  For serious games, however, not commercial ones, I reckon it’s more ok for the story to be somewhat linear.

Another interesting comment was about how things are going transmedia.  An issue that emerged was the business of transmedia, how you might start with a comic to build interest and revenue to fund adding in a game, or a movie.  Telling stories across media is an interesting challenge, and could have real opportunity for learning. I have been a fan of Andrea Phillips Transmedia Storytelling and Koreen Pagano’s Immersive Learning, which I think give good clues about how this might go.  I’m also thinking about the movie The Game (Michael Douglas & Sean Penn), and how it’s a great example of an alternate reality game. I’d love to do something like that, but serious. We did a demo once about sales that captured some of the opportunity, but…

Also, I’ve looked at many instances of experience design: movies, theatre, amusement parks, games, etc.  And I’ve advocated that those interested in making experience engaging, particularly learning, should similarly explore this. It’s hard work, you know ;).  However, one of the panelists commented on ‘circus design’. That’s something I had never thought to explore, so it’s now on my ‘todo’ list!

There were also several mentions of a theatre experience in New York called Sleep No More.  It involves two intersecting stories: Macbeth and a lady looking for someone. There’s no dialog, and it plays out across several venues. The interesting thing is that you, as an audience member, choose where to go, who to follow, and what to watch.  Now I need to find a way to experience this! (Wish I’d heard about it before my keynote there in June.)


The other theme was VR, and there were some very interesting comments made. It was repeatedly made clear by the practitioners that this was a field  still very much in development. The tools and technologies had become good and cheap enough to allow tinkering and exploration, but the business models and viable experiences were still being explored.

One quote that was interesting was a response to the issue of what the ‘frame is’.  In computer games, the frame is the screen. But in VR there’s no ‘screen’, you’re surrounded.  A response to this was “the player is the ‘frame’ in VR”.  That’s an interesting perspective.  I might reframe it as “the player’s attention is the ‘frame’ in the game”, and manipulating that may be the key.  To ponder.

Another interesting comment was “proximity breeds empathy”.  I was reminded of the phrase “familiarity breeds contempt”, but I can see that an experiential approach may help generate sympathy and comprehension.  Can you actually share someone else’s experience?  Certainly, immersion has yielded concrete learning improvements, and successful behavioral interventions.

Which brings up a response on the question of where the future of VR is (that seemed to be reflected by the other panelists) is that shared VR is the future.  Clearly, social has big benefits for learning, and can be the basis of strong emotions (sometimes negative!)

There are clearly times when VR has unique and valuable advantages for learning, though I continue to think that AR may provide the greater overall opportunity, when it’s done right.  It might be like the difference between courses and mentoring.  That is, VR to make a step change, and the AR for continual development.  Where do ARGs fit in?  Perhaps more for developing the ability to deal with the unexpected?

One of the panelists mentioned Magic Leap, and I was reminded that that type of experience will be where we can really get opportunities for transformative experiences. I think that’s where Google Glass was going, and they’re right to hold off and get it right, but when we can really start annotating the world, combining it with ARGs, there will be real potential.  We can start designing now, but it’ll definitely be some time before tools and technologies hit the ‘experimentation’ phase VR has reached.

Lots of fodder for thinking!

18 November 2016

Karen McGrane #DevLearn Keynote Mindmap

Clark @ 12:27 pm

Karen McGrane closed the DevLearn conference talking about adaptive content. She had addressed mLearnCon in the past, a great presentation, so my expectations were high.  Plus, given that I riffed on background integration in my ELearning strategy pre-con and then content strategy as a session in the xAPI camp the next day, this is a talk I was eager to hear (congrats to the eLearning Guild for putting the topic on the table).

In this entertaining and illuminating session, she made the point that responsive is better than customizing to screen, and adapting is hard, so responsive is a good starting point.

Karen McGrane Keynote Mindmap

17 November 2016

Tony DeRose #DevLearn Keynote Mindmap

Clark @ 9:41 am

Tony DeRose opened the second day of DevLearn with a geeky (and intriguing) presentation on the links between math and story in making animation.  With clips and anecdotes he showed how it works, and inspired about how they’re connecting this to STEM.

Tony DeRosa Keynote Mindmap

« Previous PageNext Page »

Powered by WordPress