meta-learning Archives - Page 25 of 57

Reflection ‘out loud’

12 July 2017 by Clark Leave a Comment

I am a fan of Show Your Work and Work Out Loud, but I’m wondering about whether they could mislead. Not that that’s the intent, of course, but they don’t necessarily include reflection, a critical component. I believe they care about it, but the phrase don’t implicitly require annotating your thoughts. And I think it’s important.

The original phrase that resonated for me was ‘narrate your work’, which to me was more than just showing it. When teachers told you to show your work, they just wanted intermediate steps. But Alan Schoenfeld’s research has documented that’s valuable to show the thinking behind the steps. What’s your rationale for taking this step?

Teachers would be able to identify where you went wrong, but that doesn’t necessarily say why you went wrong. On things as simple as multi-column subtraction, the answer would tell you whether they borrowed wrong or reversed the number or other specific misconceptions that would reveal themselves in the result. But on more complex problems, the intermediate steps may not preserve the rationale.

The design rationale approach emerged on complex projects for just this reason. New people could question earlier decisions and if they weren’t documented, you’d revisit them unnecessarily. It’s important to capture not only the decision, but the criteria used, the others considered, etc. It is this thinking about what drove decisions that helps people understand your thinking and thereby improve it.

I don’t really think “reflection out loud” is the right term. I like ‘narrate your work’ or ‘share your thinking’ perhaps better. And I do believe that those talking about working out loud and sharing your work do intend this, it’s just that too often I’ve seen people take the surface implications of a phrase and skip the real importance (*cough* Kirkpatrick’s levels *cough*). So, worst case, I’ve confused the terminology space, but hopefully I’ve also helped illuminate the valuable underpinning. And practiced what I’m preaching ;). Your thoughts?

Jay Cross Memorial Award 2017: Marcia Conner

5 July 2017 by Clark 2 Comments

The Internet Time Alliance Jay Cross Memorial Award is presented to a workplace learning professional who has contributed in positive ways to the field of Real Learning and is reflective of Jay‘s lifetime of work. Recipients champion workplace and social learning practices inside their organization and/or on the wider stage. They share their work in public and often challenge conventional wisdom. The Jay Cross Memorial Award is given to professionals who continuously welcome challenges at the cutting edge of their expertise and are convincing and effective advocates of a humanistic approach to workplace learning and performance.

We announce the award on 5 July, Jay‘s birthday. Following his death in November 2015, the partners of the Internet Time Alliance (Jane Hart, Harold Jarche, Charles Jennings, and myself) resolved to continue Jay‘s work. Jay Cross was a deep thinker and a man of many talents, never resting on his past accomplishments, and this award is one way to keep pushing our professional fields and industries to find new and better ways to learn and work.

The Internet Time Alliance Jay Cross Memorial Award for 2017 is presented to Marcia Conner. Marcia was an early leader in the movement for individual and social learning, and an innovator. As a Senior Manager at Microsoft, she developed new training practices and wrote an accessible white paper on the deeper aspects of learning design. She subsequently was the Information Futurist at PeopleSoft. She also served as a co-founder and editor at Learnativity, an early online magazine.

Marcia co-organized and co-hosted the Creating a Learning Culture conference at the Darden School of Business at the University of Virginia, leading to a book of the same title. As an advocate for the power of learning, alone and together, she wrote Learn More Now and co-wrote The New Social Learning (now in it‘s second edition) with Tony Bingham of the Association for Talent Development. She also was the instigator who organized the team for the twitter chat #lrnchat, which continues to this day.

Marcia‘s a recognized leader, writing for Fast Company, and keynoting conferences around the world. She currently helps organizations go beyond their current approaches, changing their culture. She‘s also in the process of moving her focus beyond organizations, to society. In her words, “I‘m in pursuit of meaningful progress, with good faith and honesty, girded by what I know we are capable of doing right now. When we assemble all that is going on at the edges of culture, technology, and (dare I say) business, we find a wildly hopeful view of the future. People doing extraordinary things, on a human scale, that has the potential to change everything for the better.â€

Marcia was a friend of Jay‘s for many years (including organizing the creation of his Wikipedia page), and we‘re proud to recognize her contributions.

Helen Blunden was the inaugural award winner in 2016.

FocusOn Learning reflections

27 June 2017 by Clark Leave a Comment

If you follow this blog (and you should :), it was pretty obvious that I was at the FocusOn Learning conference in San Diego last week (previous 2 posts were mindmaps of the keynotes). And it was fun as always. Here are my reflections on what happened a bit more, as an exercise in meta-learning.

There were three themes to the conference: mobile, games, and video. I’m pretty active in the first two (two books on the former, one on the latter), and the last is related to things I care and talk about. The focus led to some interesting outcomes: some folks were very interested in just one of the topics, while others were looking a bit more broadly. Whether that’s good or not depends on your perspective, I guess.

Mobile was present, happily, and continues to evolve. People are still talking about courses on a phone, but more folks were talking about extending the learning. Some of it was pretty dumb – just content or flash cards as learning augmentation – but there were interesting applications. Importantly, there was a growing awareness about performance support as a sensible approach. It’s nice to see the field mature.

For games, there were positive and negative signs. The good news is that games are being more fully understood in terms of their role in learning, e.g. deep practice. The bad news is that there’s still a lot of interest in gamification without a concomitant awareness of the important distinctions. Tarting up drill-and-kill with PBL (points, badges, and leaderboards; the new acronym apparently) isn’t worth significant interest! We know how to drill things that must be, but our focus should be on intrinsic interest.

As a side note, the demise of Flash has left us without a good game development environment. Flash is both a development environment and a delivery platform. As a development environment Flash had a low learning threshold, and yet could be used to build complex games. As a delivery platform, however, it’s woefully insecure (so much so that it’s been proscribed in most browsers). The fact that Adobe couldn’t be bothered to generate acceptable HTML5 out of the development environment, and let it languish, leaves the market open for another accessible tool. And Unity or Unreal provide good support (as I understand it), but still require coding. So we’re not at an easily accessible place. Oh, for HyperCard!

Most of the video interest was either in technical issues (how to get quality and/or on the cheap), but a lot of interest was also in interactive video. I think branching video is a real powerful learning environment for contextualized decision making. As a consequence the advent of tools that make it easier is to be lauded. An interesting session with the wise Joe Ganci (@elearningjoe) and a GoAnimate guy talked about when to use video versus animation, which largely seemed to reflect my view (confirmation bias ;) that it’s about whether you want more context (video) or concept (animation). Of course, it was also about the cost of production and the need for fidelity (video more than animation in both cases).

There was a lot of interest in VR, which crossed over between video and games. Which is interesting because it’s not inherently tied to games or video! In short, it’s a delivery technology. You can do branching scenarios, full game engine delivery, or just video in VR. The visuals can be generated as video or from digital models. There was some awareness, e.g. fun was made of the idea of presenting powerpoint in VR (just like 2nd Life ;).

I did an ecosystem presentation that contextualized all three (video, games, mobile) in the bigger picture, and also drew upon their cognitive and then L&D roles. I also deconstructed the game Fluxx (a really fun game with an interesting ‘twist’). Overall, it was a good conference (and nice to be in San Diego, one of my ‘homes’).

Nathalie Nahai #FocusOnLearn Keynote Mindmap

21 June 2017 by Clark Leave a Comment

Nathalie Nahai opened the second day of the FocuOn Learning conference. In a rapid fire presentation, she covered 7 principles that engage individuals into behaviors. With clear examples from familiar online experiences, she portrayed how these things work. Admirably, she finished with a call to ethical behavior.

Liza Donnelly #FocusOnLearn Keynote Mindmap

20 June 2017 by Clark Leave a Comment

Liza Donnelly opened the FocusOn Learn conference with stories from her career as a cartoonist. With a very personal and compelling story illustrated by her cartoons and some live drawing, she unpacked creativity and innovation. With lessons about commitment and meaning, it was a really nice kickoff to the event.

A learning meta-story

31 May 2017 by Clark Leave a Comment

Been thinking about how to generate meaningful learning in optimal (read: concise but effective) ways. And a lot of what I’ve been thinking about involves contextualized meaningful practice (no surprise there, eh?). So how might this play out? Thought I’d use a story to convey the experience I’m thinking of:

Pat logs on to the system, and notes that it’s time to take a crack at the next assignment. In it is a setup with a role for Pat to play. The story details a business situation: the organization, it’s current status, and a situation that’s occurred that requires an action. The details are exaggerated, so it’s a dire situation with a lot riding on the outcome. The instructions are phrased in the form of an email directly from the CEO, with pointers to some folks to talk to for assistance.

The necessity is for Pat to create a plan to address the need. In this case, it’s a marketing plan for a new product that has been the focus of most of the organization’s effort. With old products facing receding sales, this product has to succeed. The existing plan, legacy of a departed individual, is ‘old school’ and an up-to-date approach is needed. The indicated need is heavily aligned with this week’s topic of social-media marketing.

Pat starts work to create a document to send to the CEO. This includes making ‘calls’ (viewing videos of quick messages from the various roles involved including the product manager, the financial officer) to find out the parameters which are in play and to get expert knowledge. There are also some marketing materials available.

In previous assignments there were support tools about creating documents and about marketing plans, but this time such support isn’t available. Pat realizes that this being a more advanced cut through the topic, it’s time to start taking ownership of the process. The CEO has asked for an interim plan report before creating the entire marketing plan, and Pat uses previous materials and adapts them to create the plan.

Pat will get feedback from the CEO to incorporate in the plan before putting together the final submission. Ultimately, the success of the plan will be presented, and then feedback on the details of Pat’s submission. The document creation will be evaluated separately and in the context of previous documents required across this particular topic and previous ones, while the marketing plan itself will be evaluated in terms of it’s response to the context.

Several things to note here. The contextualized performance requirement isn’t unique, of course. This very much draws upon similar work seen in Roger Schank’s Story-Centered Curriculum and Goal-Based Scenarios. It differs in that subsequent assignments might use totally separate story settings. It’s similar also to work like Bransford, et al’s Anchored Instruction. The notion of embedding performance in context reflects research that shows abstract instruction doesn’t transfer as well. My own proposal (research, anyone?) is that the story should complete before the conceptual feedback is presented, or indeed that the story outcome includes the conceptual feedback in an intrinsic way.

The second important thing is that the document creation details are assessed separately, and tracked across other such assignments that might appear anywhere. The point is to develop meta-skills like digital document creation (and others such as presentations, working in groups, research, etc) as well as the domain skills.

I believe that we need learners to create complex work products that are challenging to auto-mark, because the outcomes are necessary. This means that you need people in the learning loop; totally asynchronous isn’t going to work to develop rich capabilities. I’m trying to figure out ways to approximate that with as little human intervention as possible because pragmatically we have more learning to achieve than we have resources to achieve that (at least until we get our priorities right ;).

Grappling with Groups

24 May 2017 by Clark 4 Comments

I’m a fan of the power of social learning. When people get together (and the process is managed right), the outcomes of a negotiated understanding can be powerful. However, in designing learning, working in groups has some real negative perceptions and realities. The open question is: what to do?

The problems are well-known. As my kids complained, on group projects some team members will reliably slack, letting the most driven student do the work. Even with a commitment, there can be differences in working style: getting started early versus preferring to do it under pressure.

Some things have been tried. When I assigned group projects, I told my students I expected them to do equal work, and would grade accordingly. If it didn’t end up being the case, they were to each write up a report on what each team member did, including themselves. Others require this, regardless, and that sounds like a smart way to make concrete a requirement for contribution.

One thing to be addressed is invigilation. Is the work being tracked in any way? If they’re working in a collaborative environment that tracks contributions via versioning or some other way, then there’s a trail of work that can be scrutinized. Extra work, to be sure, but it’d serve as a tie-breaker if there was some question about contribution.

Another issue is support for working in groups. When I first assigned group work, it became clear that they didn’t know how (?!?!). So I wrote up a little guide to doing group work, and those problems subsided. Working together is a skill that shouldn’t be taken for granted. There should be some explicit statement of expectations if you can’t determine whether there’s reliable prior experience. (Certainly, it seems that the teachers weren’t providing guidance or oversight, in the case of my kids.)

As an aside: make sure the students know why you’re asking them to work in groups. I’ve learned that learners will be much more willing to undertake what you assign if you explain the rationale that justifies your choice!

Then there’s them question of just when group work makes sense. Given that the value-added benefit is the negotiated understanding, it would make sense to do that when the material is complex, and there’s a risk of an individual taking a unique, incomplete, and or imperfect understanding. At times when you want to assess an individual’s ability to deliver, you wouldn’t want a group project!

There’s also the issue of the nature of the task. Are you just having them come to a shared understanding in representing their thinking (e.g. a response to a question) or actually produce a work product of some sort (a video, presentation, report, etc). If you can get what you need with less effort, you shouldn’t assign a more complex project.

Which brings up the issue of the scope of the work. I would expect that the more imposing the total amount of work is, the more it would invoke those with time or effort concerns to be lulled to the lazy side. Keeping the scope small might contribute to a greater willingness to participate.

Breaking up the deliverables is one way to manage student effort. If you have interim deliverables, it helps manage the process and the time. Certainly, early in a curriculum, you could provide this scaffolding (and make it explicit), and then gradually hand off responsibility for the learners to internalize the self-management. (Meta-learning!)

Breaking it up can also manage to address the contribution. If individual submissions are required before group ones, you can at least have the learners having had to contribute thought before sharing and creating a greater understanding.

Finally, there’s the issue of group work in an independent schedule. In a cohort model (scheduled timetable) it’s easy, but otherwise, how do you do it? If there’s ‘critical mass’, you can have learners arrange to meet with anyone available. If there’re more, you could even have them indicate working style preferences: quick, early, what media channels. Otherwise, it’s more challenging (or a non-issue, just don’t do it).

There are lots of issues and potential solutions for addressing group work. I can’t say I’ve found an easy solution, despite having wrestled with it. I think it’s important, so I’m curious what you’ve tried and found out!

To show or not to show (and when)

2 May 2017 by Clark Leave a Comment

At an event the other evening, showing various career technology tools, someone said something that I thought was just wrong. I asked afterwards, and then explained why I thought it was wrong. The response was “well, there can be different ways to go about it”. And frankly, there really can’t. Think for yourself about why I might say so, and then let me show you why.

The trigger was a design program talking about their design courses. And the representative was saying that once a learner had created a project, it was shown to everybody. Which sounds good, since ‘sharing is caring’, or at least it’s a good example of working out loud. And, in general, this is a good idea. But I think it’s not in learning.

In brainstorming (e.g. informal learning), we know that sharing before others have had their chance to think, it can color their output. This limits the exploration of the total possible space of opportunities that would come from a diverse team. Hearing another response likely will limit that spaces that might get explored. Instead, the goal is to diverge before converging.

And so, too, in learning. I’ve argued for assignment submission systems that only allow you to see the other submissions once you’ve submitted your own. Until you’ve struggled yourself with the challenge, you won’t get the most out of seeing how others have solved the situation.

If you immediately share the first submission, it may affect those who aren’t that far along yet. Some may even end up holding off to see what others do! This undermines the integrity of the assignment. One explanation that was given was to provide guidance to others, but that, to me, is the role of the assignment specification.

There is, however, real value in seeing the other submissions once you’ve completed yours. Seeing other approaches helps broaden the understanding. Better yet is to have discussion on them, as when critiquing others (constructively) you internalize the monitoring. This discussion also provides the opportunity to experiment with working out loud that eventually develops good working habits.

(I’ve similarly argued, by the way, that ‘rollover’ questions -where the answer is shown once you move your pointer over the question- don’t lead to any meaningful learning. If you haven’t made the mental effort to commit to a response, it won’t stick as well.)

So I believe that, if you’re developing people’s ability to do, you have a responsibility to do so in the most advantageous way. That includes making effort to use the best approach to sharing assignments. I was surprised (and dismayed) to see someone arguing to the contrary! I implore you to do the details on the approaches you work, for your learners’, and the learning’s, sake.

Innovation Thoughts

27 April 2017 by Clark Leave a Comment

So I presented on innovation to the local ATD chapter a few weeks ago, and they did an interesting and nice thing: they got the attendees to document their takeaways. And I promised to write a blog post about it, and I’ve finally received the list of thoughts, so here are my reflections. As an aside, I’ve written separate articles on L&D innovation recently for both CLO magazine and the Litmos blog so you can check those out, too.

I started talking about why innovation was needed, and then what it was. They recalled that I pointed out that by definition an innovation is not only a new idea, but one that is implemented and leads to better results. I made the point that when you’re innovating, designing, researching, trouble-shooting, etc, you don’t know the answer when you start, so they’re learning situations, though informal, not formal. And they heard me note that agility and adaptation are premised on informal learning of this sort, and that the opportunity is for L&D to take up the mantle to meed the increasing need.

There was interest but some lack of clarity around meta-learning. I emphasize that learning to learn may be your best investment, but given that you’re devolving responsibility you shouldn’t assume that individuals are automatically possessed of optimal learning skills. The focus then becomes developing learning to learn skills, which of needs is done across some other topic. And, of course, it requires the right culture.

There were some terms they heard that they weren’t necessarily clear on, so per the request, here are the terms (from them) and my definition:

Innovation by Design: here I mean deliberately creating an environment where innovation can flourish. You can’t plan for innovation, it’s ephemeral, but you can certainly create a felicitous environment.
Adjacent Possible: this is a term Steven Johnson used in his book Where Good Ideas Come From, and my take is that it means that lateral inspiration (e.g. ideas from nearby: related fields or technologies) is where innovation happens, but it takes exposure to those ideas.
Positive Deviance: the idea (which I heard of from Jane Bozarth) is that the best way to find good ideas is to find people who are excelling and figure out what they’re doing differently.
Hierarchy and Equality: I’m not quite sure what they were referring to hear (I think more along the lines of Husband’s Wirearchy versus hierarchy) but the point is to reduce the levels and start tapping into the contributions possible from all.
Assigned roles and vulnerability: I’m even less certain what’s being referred to here (I can’t be responsible for everything people take away ;), but I could interpret this to mean that it’s hard to be safe to contribute if you’re in a hierarchy and are commenting on someone above you. Which again is an issue of safety (which is why I advocate that leaders ‘work out loud’, and it’s a core element of Edmondson’s Teaming; see below).

I used the Learning Organization Dimensions diagram (Garvin, Edmondson & Gino) to illustrate the components of successful innovation environment, and these were reflected in their comments. A number mentioned psychological safety in particular as well as the other elements of the learning environment. They also picked up on the importance of leadership.

Some other notes that they picked up on included:

best principles instead of best practices
change is facilitated when the affected individual choose to change
brainstorming needs individual work before collective work
that trust is required to devolve responsibility
the importance of coping with ambiguity

One that was provided that I know I didn’t say because I don’t believe it, but is interesting as a comment:

“Belonging trumps diversity, and security trumps grit”

This is an interesting belief, and I think that’s likely the case if it’s not safe to experiment and make mistakes.

They recalled some of the books I mentioned, so here’s the list:

The Invisible Computer by Don Norman
The Design of Everyday Things by Don Norman
My Revolutionize Learning and Development (of course ;)
XLR8 by John Kotter (with the ‘dual operating system‘ hypothesis)
Teaming to Innovate by Amy Edmondson (I reviewed it)
Working Out Loud by John Stepper
Scaling Up Excellence by Robert I. Sutton and Huggy Rao (blogged)
Organize for Complexity by Niels Pflaeging (though they heard this as a concept, not a title)

It was a great evening, and really rewarding to see that many of the messages stuck. So, what are your thought around innovation?

Human Learning is Not About to Change Forever

26 April 2017 by Clark 1 Comment

In my inbox was an announcement about a new white paper with the intriguing title Human Learning is About to Change Forever. So naturally I gave up my personal details to download a copy. There are nine claims in the paper, from the obvious to the ridiculous. So I thought I’d have some fun.

First, let’s get clear. Our learning runs on our brain, our wetware. And that’s not changing in any fundamental way in the near future. As a famous article once had it: phenotypic plasticity triumphs over genotypic plasticity (in short, our human advantage has gained via our ability to adapt individually and learn from each other, not through species evolution). The latter takes a long time!

And as a starting premise, the “about to” bit implies these things are around the corner, so that’s going to be a bit of my critique. But nowhere near all of it. So here’s a digest of the nine claims and my comments:

Enhanced reality tools will transform the learning environment. Well, these tools will certainly augment the learning environment (pun intended :). There’s evidence that VR leads to better learning outcomes, and I have high hopes for AR, too. Though is that a really fundamental transition? We’ve had VR and virtual worlds for over a decade at least. And is VR a evolutionary or revolutionary change from simulations? Then they go on to talk about performance support. Is that transforming learning? I’m on record saying contextualized learning (e.g. AR) is the real opportunity to do something interesting, and I’ll buy it, but we’re a long way away. I’m all for AR and VR, but saying that it puts learning in the hands of the students is a design issue, not a technology issue.
People will learn collaboratively, no matter where they are. Um, yes, and…? They’re already doing this, and we’ve been social learners for as long as we’ve existed. The possibilities in virtual worlds to collaboratively create in 3D I still think is potentially cool, but even as the technology limitations come down, the cognitive limitations remain. I’m big on social learning, but mediating it through technology strikes me as just a natural step, not transformation.
AI will banish intellectual tedium. Everything is awesome. Now we’re getting a wee bit hypish. The fact that software can parse text and create questions is pretty impressive. And questions about semantic knowledge aren’t going to transform education. Whether the questions are developed by hand, or by machine, they aren’t likely on their own to lead to new abilities to do. And AI is not yet to the level (nor will it be soon) where it can take content and create compelling activities that will drive learners to apply knowledge and make it meaningful.
We will maximize our mental potential with wearables and neural implants. Ok, now we’re getting confused and a wee bit silly. Wearables are cool, and in cases where they can sense things about you and the world means they can start doing some very interesting AR. But transformative? This still seems like a push. And neural implants? I don’t like surgery, and messing with my nervous system when you still don’t really understand it? No thanks. There’s a lot more to it than managing to adjust firing to control limbs. The issue is again about the semantics: if we’re not getting meaning, it’s not really fundamental. And given that our conscious representations are scattered across our cortex in rich patterns, this just isn’t happening soon (nor do I want that much connection; I don’t trust them not to ‘muck about’).
Learning will be radically personalized. Don’t you just love the use of superlatives? This is in the realm of plausible, but as I mentioned before, it’s not worth it until we’re doing it on top of good design. Again, putting together wearables (read: context sensing) and personalization will lead to the ability to do transformative AR, but we’ll need a new design approach, more advanced sensors, and a lot more backend architecture and semantic work than we’re yet ready to apply.
Grades and brand-name schools won‘t matter for employment. Sure, that MIT degree is worthless! Ok, so there’s some movement this way. That will actually be a nice state of affairs. It’d be good if we started focusing on competencies, and build new brand names around real enablement. I’m not optimistic about the prospects, however. Look at how hard it is to change K12 education (the gap between what’s known and what’s practiced hasn’t significantly diminished in the past decades). Market forces may change it, but the brand names will adapt too, once it becomes an economic necessity.
Supplements will improve our mental performance. Drink this and you’ll fly! Yeah, or crash. There are ways I want to play with my brain chemistry, and ways I don’t. As an adult! I really don’t want us playing with children, risking potential long-term damage, until we have a solid basis. We’ve had chemicals support performance for a while (see military use), but we’re still in the infancy, and here I’m not sure our experiments with neurochemicals can surpass what evolution has given us, at least not without some pretty solid understanding. This seems like long-term research, not near-term plausibility.
Gene editing will give us better brains. It’s alive! Yes, Frankenstein’s monster comes to mind here. I do believe it’s possible that we’ll be able to outdo evolution eventually, but I reckon there’s still not everything known about the human genome or the human brain. This similarly strikes me as a valuable long term research area, but in the short term there are so many interesting gene interactions we don’t yet understand, I’d hate to risk the possible side-effects.
We won‘t have to learn: we‘ll upload and download knowledge. Yeah, it’ll be great! See my comments above on neural implants: this isn’t yet ready for primetime. More importantly, this is supremely dangerous. Do I trust what you say you’re making available for download? Certainly not the case now with many things, including advertisements. Think about downloading to your computer: not just spam ads, but viruses and malware. No thank you! Not that I think it’s close, but I’m not convinced we can ‘upgrade our operating system’ anyway. Given the way that our knowledge is distributed, the notion of changing it with anything less than practice seems implausible.

Overall, this is reads like more a sci-fi fan’s dreams than a realistic assessment of what we should be preparing for. No, human learning isn’t going to change forever. The ways we learn, e.g. the tools we learn with are changing, and we’re rediscovering how we really learn.

There are better guides available to what’s coming in the near term that we should prepare for. Again, we need to focus on good learning design, and leveraging technology in ways that align with how our brains work, not trying to meld the two. So, there’re my opinions, I welcome yours.

« Previous Page