Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Search Results for: engag

Working and learning out loud

by 2 Comments

I’ve been thinking about some of the talk around the Future of Work, and in addition to the free flow of information I recently posted about  from the Coherent Organization, I think working out loud is another component.  Inspired by a post from my colleague ITA Harold Jarche,  this is how I see it, in actionable terms.  (And I expect this is also part of Jane Bozarth’s Show Your Work, but I’ve yet to get my mitts on a copy, mea culpa.)

The point is to make your work visible.  There’re two points: showing the actual goals, progress and status of your work, and showing the thinking that’s going on behind it.  And there are two dimensions: within the organization, and outside the organization.  Pragmatically, this yields some concrete and actionable elements.

First, there have to be mechanisms to share.  Ideally, you don’t have to work and then separately post it, but instead your tools automatically share.  This really means collaborative work tools, because you want people to be able to engage: at least commenting, and of course sometimes (maybe most of the time) you’ll be working in a team. Also tools that track contributions and changes.  And there may have to be permissions: so some people can edit, some can only comment, etc.

And I wouldn’t assume folks know what it means to ‘narrate your work‘ (aka learn out loud).  Here, I mean exposing the underlying elements.  This includes the context, assumptions, considerations, experiments, and reflections.  In User Interface Design, it was called Design Rationale, and it’s showing not only the current state, but how you got there.  Benefits include others’ experiences, not revising early decisions when new team members are added, and more.  The typical techniques of being explicit, modeling, evangelizing, promoting, etc, play a role here.

Then of course it has to be ‘safe’ to share, you can’t be working in a Miranda organization.  If your work will be held against  you in any way, you won’t want to share.  This means culture and policies and more. Basically, you need to be working in a Learning Organization, where the elements are aligned to facilitate optimal engagement.  This includes the practices about how to work out loud and learn out loud.

Another   issue is how far to share.  What can you share outside the organization?  Harold talks about the essential requirement of sharing outside the workplace, and that can be a big concern.  Obviously, proprietary work needs to be protected until it’s not business vulnerable. This means policies about what’s safe to share, and when.  Certainly, ‘sanitized’ work, where critical details are obscured but the thinking is shown should be supported in going out to communities of practice.  And the end result, when the business advantage isn’t threatened, could and should be shared through articles or webinars or conference presentations.

At the end, it’s a risk/reward tradeoff for any project at any time. What’s the benefit of getting feedback to improve versus what’s the downside of information getting out to competitors or exposing regulated data?  At any point, with any particular version of ‘anonymizing’, the balance may tip one way or another.   But the point is to be open to the benefit, and take advantage of it when and where you can.  Getting systematic at making it a regular part of any project is likely to be key.

It’s what I do here, and I encourage you to work out loud as much as you can.  So, any feedback so far?

#itashare

Conferencing reflections

by Leave a Comment

It turns out that I’ve been to a  lot of conferences this year (8, if my math is right) scattered through the year and around the globe. And over the past decade, I’ve hit a lot more.  And it’s given me some opportunities to contrast and compare some of the tradeoffs that can be made.  So I thought I’d share my thoughts with you.

Now, my perspective starts out a bit different. At these events, I’m speaker, so I see things from a different perspective.  However, I also try to go see sessions as an audience member as well, and I still see the same events.  So I am trying to write this from all perspectives: conference organizer, attendee, speaker, and vendor.  And let me be clear, I’m a learning technology strategist, and my passions are learning, technology, and how to use them together to make things better. So that colors my comments.

Here are the major elements and my thoughts on them:

Keynotes: I am tired of ‘inspirational’ keynotes. I really don’t need to hear some person who climbed a mountain or sailed a sea and their attempts to connect that to learning somehow. I’d rather hear about an issue that affects learning.  Topics about how we think, work, or learn are of interest. Let’s hear about the risks of technology, or some new ones or ways to use them.  Yes, I like compelling speakers, but please give me new thoughts, not random aspiration.

Speakers:  I think it’s unconscionable to have an unprepared speaker who can’t manage time.  It’s even worse on panels or shared sessions where one speaker runs over. It’s just not fair to the other speakers.  It’s also essential that the talk is not a sales pitch, but instead presents real value in ideas or experience. And they should be happy to chat afterward.  It boggles my little mind when someone gets up and clearly hasn’t practiced and checked their timing. It’s appears  I’m somewhat unusual, but I really don’t necessarily feel the critical need to spend most of the time conversing with others. I don’t mind, and even can recommend some interaction, but I want to hear something substantive as well.

Schedule: I like events that have a clear and comprehensible schedule. I want to know exactly what things are at the same time, so I can choose and then vote with my feet if the first choice isn’t working for me.  Having different tracks have different schedules doesn’t work.  And as a speaker and an attendee,  I don’t like short sessions.  Give me at least an hour as a speaker to set the tone, present the topic, talk about the issues and tradeoffs, and talk about the way forward.  Similarly as an audience member, I want suitable depth.  30 minutes just isn’t enough.

Breaks: And then I want a break.  The break should be long enough to potentially chat  with the speaker at the preceding event, get out and find some sustenance, use the facilities, have a conversation or two, and get to the next event.  Workshop breaks can be shorter, as you’re with a group for a half or full day, but for separating concurrent sessions, they need to be sufficiently long.

Events: I love having social events, as a way to have those important serendipitous  conversations. An evening reception after the first day is mandatory.  I like sufficient nibbles to fend off the need to escape to dinner, or dinner actually provided. And for the end of the day, I like social lubricant.  Preferably on demand, not via a limited ration.  It doesn’t have to be a broad selection, but not having to worry about  logistics means my mind is free to focus on conversations.  I assume lunch is provided, of course, and it doesn’t have to be fancy or rich, just healthy, substantive, and reasonably tasty.  Other events, such as mid afternoon treat breaks, and mid morning snack breaks are great.  I really like it if some form of breakfast is available as well.  I think I’m not the only one who prefers to eat little bits over time, not big meals.

Expo Hall: I like to have an exhibition. I like to see what’s around. Yes, I don’t like walking past and being grabbed, but I do like it if I can go up, have an intelligent conversation about the problem solution, and not feel pressured. I like to see the alternatives, and take the temperature of the market. And I like people who might have real needs to be able to explore real solutions.  Having events  in  the expo area  makes sense to me and the vendors.

App: I used to get a PDF of the program and put it on my tablet.  Now I am happy using an app, and it’s become a must-have. I like it when I can choose sessions for my schedule and have reminders.  I like having a stream of information, though it could be via Twitter. I like having an expo map if the expo is of any size at all.  And I don’t really care for  gamification to reward participation.  While I like the engagement of users, it leads to too many frivolous posts.  I really like it if presentation material is  available through the app, and happy to do evaluations that way.

Bookstore: I think a bookstore is important, for several reasons.  For one, you can get a heads up on a speaker before you see them. Or if you miss a session, you can graze what you might have missed.  You might also want to get the works of someone who you really were intrigued by. It’s also a way to see what’s happening in the field.

Rest areas: I don’t really need speaker prep.  Sometimes it may be handy if the event is really big, but the main things is, instead, having good connection to event staff. And I think that’s true for all, not just speakers.  Having places to sit for all attendees means that  anyone needing a break whether social or physical can achieve that end.

People:  The staff makes quite a big difference when they’re knowledgeable and helpful.  This has almost always been the case, but it’s nice to have informed people ready and willing to help.  This is true for vendors as well, having friendly and knowledgeable people trumps having shills who can chat you up but can’t really answer questions.

So, what have I forgotten to address?

I realize that there are different audiences,  purposes, and business models for these events, and so not all things are comparable.  And this is also my opinion, and your motives may differ, but I hope I’ve laid out some of the thinking to help you think about what works for you.  And I hope to see you at a conference sometime!

Scenarios and Conceptual Clarity

by 5 Comments

I recently came across an article ostensibly about branching scenarios, but somehow the discussion largely missed the point.  Ok, so I can be a stickler for conceptual clarity, but I think it’s important to distinguish between different types of scenarios and their relative strengths and weaknesses.

So in my book  Engaging Learning, I was looking to talk about how to make engaging learning experiences.  I was pushing games (and still do) and how to design them, but I also wanted to acknowledge the various approximations thereto.  So in it, I characterized the differences between what I called mini-scenarios, linear scenarios, and contingent scenarios (this latter is what’s traditionally called branching scenarios).  These are all approximations to full games, with various tradeoffs.

At core, let me be clear, is the need to put learners in situations where they need to make decisions. The goal is to have those decisions closely mimic the decisions they need to make  after the learning experience. There’s a context (aka the story setting), and then a specific situation triggers the need to make a decision.  And we can deliver this in a number of ways. The ideal is a simulation-driven (aka model-driven or engine-driven) experience.  There’s  a model of the world underneath that calculates the outcomes of your action and determines whether you’ve yet achieved success (or failure), or generates  a new opportunity to act.  We can (and should) tune this into a serious game.  This gives us deep experience, but the model-building is challenging and there are short cuts.

MiniScenarioIn  mini-scenarios, you put the learner in a setting with a situation that precipitates a decision.  Just one, and then there’s feedback.   You could use video, a graphic novel format, or just prose, but the game problem is a setting and a situation, leading to choices. Similarly, you could have them respond by selecting option A B or C, or pointing to the right answer, or whatever.  It stops there. Which is the weakness, because in the real world the consequences are typically more complex than this, and it’s nice off the learning experience reflects that reality.  Still, it’s better than knowledge test.  Really, these are  just a better written multiple choice question, but that’s at least a start!

LinearScenarioLinear scenarios are a bit more complex. There are a series of game problems in the same context, but whatever the player  chooses, the right decision is ultimately made, leading to the next problem. You use some sort of sleight of hand, such as “a supervisor catches the mistake and rectifies it, informing you…” to make it all ok.  Or, you can terminate out and have to restart if you make the wrong decision  at any point.  These are a step up in terms of showing the more complex consequences, but are a bit unrealistic.  There’s some learning power here, but not as much as is possible.  I have used them as sort of multiple mini-scenarios with content in between, and  the same story is used for the next choice, which at least made a nice flow. Cathy Moore  suggests  these  are valuable for novices, and I think it’s also useful if everyone needs to receive the same ‘test’ in some accreditation environment to be fair and balanced (though in a competency-based world they’d be better off with the full game).

BranchingScenarioThen there’s the full branching scenario (which I called contingent scenarios in the book, because the consequences and even new decisions are contingent on your choices).  That is, you see different opportunities depending on your choice. If you make one decision, the subsequent ones are different.  If you don’t shut down the network right away, for instance, the consequences are different (perhaps a breach) than if you do (you get the VP mad).  This, of course, is much  more like the real world.  The only difference between this and a serious  game is that the contingencies in the world are hard-wired in the branches, not captured in a separate model (rules and variables). This  is easier, but it gets tough to track if you have too many  branches. And the lack of an engine  limits the replay and ability to have randomness.  Of course, you can make several of these.

So the problem I had with the article  that triggered this post is that their generic model looked like a mini-scenario, and nowhere did they show the full concept of a real branching scenario. Further,  their example was really a linear scenario, not a branching scenario.  And I realize this may seem like an ‘angels dancing on the head of a pin’, but I think it’s important to make distinctions when they affect the learning outcome, so you can more clearly make a choice that reflects the goal you are trying to achieve.

To their credit, that they  were pushing for contextualized decision making at all is a major win, so I don’t want to quibble too much.  Moving our learning practice/assessment/activity to more contextualized performance is a good thing.  Still, I  hope this elaboration is useful  to get more nuanced solutions.  Learning design really can’t be treated as a paint-by-numbers exercise, you really should know what you’re doing!

Evidence for benefits: Towards Maturity Report

by 1 Comment

An organization that I cited in the Revolution book, Towards Maturity, has recently released their 2015-2016 Industry Benchmark Report, and it’s of interest to individuals and organizations looking for real data on what’s working, and not, in L&D.  Towards Maturity has been collecting benchmarking data on L&D practices for over a decade, and what they find bolsters the case to move L&D forwards.

The report has a number of useful sections, including documenting the current state of the industry, guidance for business leaders on expectations, on listening to learners, and on rethinking  the L&D team.  Included are some top level pointers for executives and L&D.  And while the report is  biased towards Europe, respondents cover the globe including Asia, Americas, and more.

Overall, they’re finding a 19% average in  technology spending out of L&D budgets (and this has been essentially flat for 3 years). This seems light;  given that technology is a key enabler of performance and development, such a figure doesn’t seem appropriate.  Of course, given that 55% of formal learning is still delivered face-to-face, this isn’t surprising.

A more interesting outcome is comparing what they call  Top Deck organizations; those in the top 10% of their Towards Maturity Index. These organizations are characterized by four elements that are tied to success:

  • Learning aligned to need
  • Active learner voice
  • Design beyond the course
  • Proactive in connecting

Here we see key elements of the revolution. For one, learning isn’t done on demand, but is coupled to organizational improvements.  For another, the learner is engaged in the processes of determining what solutions make sense.  One that intrigues me is that the solutions go beyond courses, looking at performance support and more. And finally, L&D is reaching out across silos to engage in conversations.  These are all key to achieving results from 6 – 8 times the average organization.

The advice to business leaders also echoes the revolution. The call is to focus on performance, not on courses.  It’s not about learning, it’s about outcomes.  The recommendation  is to break down silos so as to achieve the conversations that will achieve meaningful impact.

The advice goes on: understand how learners are learning, create a participatory culture, and use  real business metrics.  All grounded in what successful organizations are doing.  The point here is not to recite all the outcomes, but instead to list highlights and encourage you to have a look at the report.  Going forward, you might even consider benchmarking your own organization!

Benchmarking is best practices, and of course I encourage best principles, but the frameworks they use are grounded in the best principles, and measuring yourself against the framework and improving is really more important than comparing yourself to others.  I will suggest that  measuring yourself and evaluating your progress is a valuable investment of time in conjunction with a strategy.

What I really like, of course, is that the data support the position posited by principles that I derived from both practical experience and relevant conceptual models. The evidence is converging that there are positive steps L&D can, and should, take.  The revolution provides the roadmap, and their data provides a way to evaluate progress.  Here’s to improving L&D!

A Competent Competency Process

by 3 Comments

In the process of looking at ways to improve the design of courses, the starting point is good objectives. And as a consequence, I‘ve been enthused about the notion of competencies, as a way to put the focus on what people do, not what they know. So how do we do this, systematically, reliably, and repeatably?

Let‘s be clear, there are times we need knowledge level objectives. In medicine or any other field where responses need to be quick and accurate, we need a very constrained vocabulary. SO drilling in the exact meanings of words is valuable, as an example. Though ideally, that‘s coupled with using that language to set context or make decisions. So “we know it‘s the right medial collateral ligament, prep for the surgery” could serve as a context, or we could have a choice to operate on the left or right atrial ventricle as a decision point. As Van Merriënboer‘s 4 Component Instructional Design talks about, we need to separate out the knowledge from the complex problems we apply it to. Still, I suggest that what‘s likely to make a difference to individuals and organizations is the ability to make better decisions, not recite rote knowledge.

So how do we get competencies when we want them? The problem, as I‘ve talked about before, is that SMEs don‘t have access to 70% of what they actually do, it‘s compiled away. We then need good processes, so I‘ve talked to a couple of educational institutions doing competencies, to see what could be learned. And it‘s clear that while there‘s no turnkey approach, what‘s emerging is a process with some specific elements.

One thing is that if you‘re trying to cover a whole college level course, you‘ve got to break it up. Break down the top level into a handful of competencies. Then you continue to take each of those apart, and perhaps another level, ‘til you have a reasonable scope. This is heuristic, of course, but with a focus on ‘do‘, you have a good likelihood to get here.

One of the things I‘ve heard across various entities trying to get meaningful objectives is working with more than one SME. If you can get several, you have a better chance of triangulating on the right outcomes and objectives. They may well disagree about the knowledge, but if you manage the process right (emphasize ‘do‘, lather, rinse, repeat), you should be able to get them to converge. It may take some education, and you may have to let them get the

Not just any SMEs will do. Two things are really valuable: on the ground experience to know what needs to be done (and doesn‘t), and the ability to identify and articulate the models that guide the performance. Some instructors, for instance, can teach to a text but really aren‘t truly masters of the content nor are experienced practitioners. Multiple helps, but the better the SME, the better the outcome.

I believe you want to ensure that you‘re getting both the right things, and all the things. I‘ve recommended to a client about triangulating not just with SMEs, but with practitioners (or, rather, the managers of the roles the learners will be engaged in), and any other reliable stakeholders. The point is to get input from the practice as well as the theory, identifying the models that support proper behavior, and the misconceptions that underpin where they go wrong.

Once you have a clear idea of the things people need to be able to do, you can then identify the language for the competencies. I‘m not a fan of Bloom‘s (unwieldy, hard to reliably apply), but I am a fan of Mager-style definitions (action, context, metric).

After this is done, you can identify the knowledge needed, and perhaps created objectives for that, but to me the focus is on the ‘do‘, the competencies. This is very much aligned with an activity-based learning model, whereby you immediately design the activities that align with the competencies before you decide the content.

So, this is what I‘m inferring. There would be good tools and templates you could design to go with this, identifying competencies, misconceptions, and at the same time also getting stories and motivations. (An exercise left for the reader. ;) The overall goal, however, of getting meaningful objectives is key to getting good learning design. Any nuances I‘m missing?

Non-invasive Brain Surgery

by Leave a Comment

Changing behavior is hard. The brain is arguably the most complex thing in the known universe. Simplistic approaches aren‘t likely to work. To rewire it, one approach is to try surgery. This is problematic for a several reasons: it‘s dangerous, it‘s messy, and we really don’t understand enough about it. What‘s a person to do?

Well, we do know that the brain can rewire itself, if we do it right. This is called learning. And if we design learning, e.g. instruction, we can potentially change the brain without surgery. However, (and yes, this is my point) treating it as anything less than brain surgery (or rocket science), isn‘t doing justice to what‘s known and what‘s to be done.

The number of ways to get it wrong is long. Information dump instead of skills practice. Massed practice instead of spaced. Rote knowledge assessment. Lack of emotional engagement. The list goes on. (Cue the Serious eLearning Manifesto.) In short, if you don‘t know what you‘re doing, you‘re likely doing it wrong and are not going to have an effect. Sure, you‘re not likely to kill anyone (unless you‘re doing this where it matters), but you‘ll waste money and time. Scandalous.

Again, the brain is complex, and consequently so is learning design. So why, in the name of sense and money, do we treat it as trivial? Why would anyone buy a story that we can achieve anything meaningful by taking content and adding a quiz (read: rapid eLearning)? As if a quiz is somehow going to make people do better. Who would believe that just anyone can present material and learning will occur? (Do you know the circumstances when that will work?) And really, throwing fuzzy objects around the room and ice-breakers will somehow make a difference? Please. If you can afford to throw money down the drain (ok, if you insist, throw it here ;), and don‘t care if any meaningful change happens, I pity you, but I can‘t condone it.

Let‘s get real. Let‘s be honest. There‘s a lot (a lot) of things being done in the name of learning that are just nonsensical. I could laugh, if I didn‘t care so much. But I care about learning. And we know what leads to learning. It‘s not easy. It‘s not even cheap. But it will work. It requires good analysis, and some creativity, and attention to detail, and even some testing and refinement, but we know how to do this.

So let‘s stop pretending. Let‘s stop paying lip-service. Let‘s treat learning design as the true blend of art and science that it is. It‘s not the last refuge of the untalented, it‘s one of the most challenging, and rewarding, things a person can do. When it‘s done right. So let‘s do it right! We‘re performing brain surgery, non-invasively, and we should be willing to do the hard yards to actually achieve success, and then reap the accolades.

OK, that‘s my rant, trying to stop what‘s being perpetrated and provide frameworks that might help change the game. What‘s your take?

The Polymath Proposition

by 3 Comments

At the recent DevLearn conference, one of the keynotes was Adam Savage.  And he said something that gave me a sense of validation. He was talking about being a polymath, and I think that’s worth understanding.

His point was that his broad knowledge of a lot of things was valuable.  While he wasn’t the world’s expert in any particular thing, he knew a lot about a lot of things.  Now if you don’t know him, it helps to understand that he’s one of the two hosts of Mythbusters, a show that takes urban myths and puts them to the test.  This requires designing experiments that fit within pragmatic constraints of cost and safety, and will answer the question. Good experiment design is an art as well as a science, and given the broad range of what the myths cover, this ends up requiring a large amount of ingenuity.

The reason I like this is that my interests vary broadly (ok, I’m coming to terms with a wee bit of ADD ;).  The large picture is how technology can be designed  to help us think, work, and learn.  This ends up meaning I have to understand things like cognition and learning (my Ph.D. is in cognitive psychology), computers (I’ve programmed and designed architectures at many levels), design (I’ve looked at usability, software engineering, industrial design, architectural design, and more), and organizational issues (social, innovation…). It’s led to explorations covering things like games, mobile, and strategy (e.g. the topics of my books).  And  more; I’ve led development of adaptive learning systems, content models, learning content, performance support, social environments, and so on.  It’s led me further, too, exploring  org change and culture, myth and ritual, engagement and fun, aesthetics and media, and other things I can’t even recall right now.

And I draw upon  models from as many fields as I can.  My Ph.D. research was related to the power of models as a basis for solving new problems in uncertain domains, and so I continue to collect them like others collect autographs or music.  I look for commonalities, and  try to make my understanding explicit by continuing to diagram and write about my reflections.  I immodestly think I draw upon a broad swath of areas. And  I particularly push learning to learn and meta-cognition to others because it’s been so core to my own success.

What I thrive on is finding situations where the automatic solutions don’t apply. It’s not just a clear case for ID, or performance support, or…  Where technology can be used (or used better) in systemic ways to create new opportunities. Where I really contribute is where it’s clear that change is needed, but what, how, and where to start aren’t obvious.  I’ve a reliable track record of finding unique, and yet pragmatic solutions to such situations, including the above named areas I’ve innovated  in.  And it is a commitment of mine to do so in ways that pass on that knowledge, to work in collaboration to co-develop the approach and share the concepts driving it, to hand off ownership to the client. I’m not looking for a sinecure; I want to help while I’m adding value and move on when I’m not.  And many folks have been happy to have my assistance.

It’s hard for me to talk about myself in this way, but I reckon I bring that  polymath ability of a broad background to organizations trying to advance.   It’s been  in assisting their ability to develop design processes that yield better learning outcomes, through mobile strategies and solutions that meet their situation, to overarching organizational strategies that map from concepts to system.  There’s a pretty fair track record to back up what I say.

I am  deep in a lot of areas, and have the ability to synthesize solutions  across  these areas in integrated ways. I may not be the deepest in any one, but when you need to look across them and integrate a systemic solution, I like to think and try to ensure that I’m your guy. I help organizations envision a future state, identify the benefits and costs, and prioritize the opportunities to define a strategy.  I  have operated independently or with partners, but I adamantly retain my freedom  to say what I truly think so that you get an unbiased response from the broad suite of  principles I have to hand.  That’s my commitment to integrity.

I didn’t intend this to be a commercial, but I did like his perspective and it made me reflect on what my own value proposition is.  I welcome your thoughts.  We now return you to your regularly scheduled blog already in progress…

Natalie Panek #DevLearn Keynote Mindmap

by Leave a Comment

To close off the DevLearn conference, Natalie Panek (@nmpanek) told of her learning journey to be a space engineer with compelling stories of challenging experiences.  With an authentic and engaging style, she helped inspire us to keep learning.