Search Results for: align

Solutions for Tight Cycles of Assessment

22 November 2017 by Clark 5 Comments

In general, in a learning experience stretching out over days (as spaced learning would suggest), learners want to regularly get feedback about how they’re doing. As a consequence, you want regular cycles of assessment. However, there’s a conflict. In workplace performance we produce complex outputs (RFPs, product specs, sales proposals, strategies, etc). These still typically require human oversight to evaluate. Yet resource limitations are likely in most such situations, so we prefer auto-marked solutions (read: multiple choice, fill-in-the-blank), etc. How do we reconcile meaningful assessment with realistic constraints? This is one of the questions I’ve been thinking about, and I thought I’d share my reflections with you.

In workplace learning, at times we can get by with auto-assessment, particularly if we use coaching beyond the learning event. Yet if it matters, we’d rather them practice things that matter before they actually are used for real work. And for formal education, we want learners to have at least weekly cycles of performance and assessment. Yet we also don’t want just rote knowledge checks, as they don’t lead to meaningful performance. We need some intermediate steps, and that’s what I’ve been thinking on.

So first, in Engaging Learning, I wrote about what I called ‘mini-scenarios’. These are really just better-written multiple-choice questions. However, such questions don’t ask learners to identify definitions or the like (simple recognition), but instead put learners in contextual situations. Here, the learner chooses between different decisions. Which means retrieving the information, mapping it to the context, and then choosing the best answer. Such a question has a story context, a precipitating situation, and then alternative decisions. (And the alternatives are ways learners go wrong, not silly or obviously incorrect choices). I suggest that your questions should be like this, but are there more?

Branching scenarios are another, rich form of practice. Here it’s about tying together the decisions (they do tend to travel in packs) and consequences. When you do so, you can provide an immersive experience. (When designed well, of course.) They’re a pragmatic approximation of a full game experience. Full games are really good when you need lots of practice (or can amortize over a large audience), but they’re an additional level of complexity to develop.

Another one that Tom Reeves presented in an article was intriguing. You not only have to make the right choice, but then you also choose the reason why you made that choice. It’s only an additional step, but it gets at the choice and the thinking. And this is important. It would minimize the likelihood of guessing, and provide a richer basis for diagnosis and feedback. Of course, no one is producing a ‘question type’ like this that I know of, but it’d be a good one.

An approach we used in the past was to have learners create a complex answer, but have the learner evaluate it! In this case it was a verbal response to a question (we were working on speaking to the media), but then the learner could hear their own answer and a model one. Of course, you’d want to pair this with an evaluation guide as well. The learner creates a response, and then is presented with their response, a good response, and a rubric about what makes a good answer. Then we ask the learner to self evaluate against the rubric. This has the additional benefit that learners are evaluating work with guidance, and can internalize the behavior to become a self-improving learner. (This is the basis of ‘reciprocal teaching’, one of the component approaches in Cognitive Apprenticeship.)

Each of these is auto-(or self-) marked, yet provides valuable feedback to the learner and valuable practice of skills. Which shouldn’t be at the expense of also having instructor-marked complex work products or performances, but can supplement them. The goal is to provide the learner with guidance about how their understanding is progressing while keeping marking loads to a minimum. It’s not ideal, but it’s practical. And it’s not exclusive of knowledge test as well, but it’s more applied and therefore is likely to be more valuable to the learner and the learning. I’m percolating on this, but I welcome hearing what approaches (and reflections) you have.

My Professional Learner’s Toolkit

21 November 2017 by Clark 7 Comments

My colleague, Harold Jarche, recently posted about his professional learning toolkit, reflecting our colleague Jane Hart’s post about a Modern Learner’s Toolkit. It’s a different cut through the top 10 tools. So I thought I’d share mine, and my reflections.

Favorite browser and search engine: I use Safari and Google, by default. Of course, I keep Chrome and Firefox around for when something doesn’t work (e.g. Qualtrics). I would prefer another search engine, probably DuckDuckGo, but I’m not facile with it, for instance finding images.

A set of trusted web resources: That’d be Wikipedia, pretty much. And online magazines, such as eLearnMag and Learning Solutions, and ones for my personal interests. I use Pixabay many times to find images.

A number of news and curation tools: I use Google News and the ABC (Oz, not US) in my browser, and the BBC and News apps on my iDevices. I also use Feedblitz to bring blogposts into my email. I keep my own bookmarks using my browser.

Favorite web course platforms: I haven’t really taken online courses. I’ve used Zoom to share.

A range of social networks: I use LinkedIn professionally, as well as Slack. And Twitter, of course. I stay in touch with my ITA colleagues via Skype. Facebook is largely personal.

A personal information system: I use both Notability and Notes to take notes. Notes more for personal stuff, Notability for work-related. I use Omnigraffle for diagrams and mindmaps. And OmniOutliner also helps when I want to think hierarchically.

A blogging or website tool: I use WordPress for Learnlets (i.e. here), and I use Rapidweaver for my sites: Quinnovation and my book sites.

A variety of productivity apps and tools: Calendar is crucial, and Pagico keeps me on track for projects. I use Google Maps for navigation. I use SplashID for passwords and other private data. I often read and markup documents on my iPad with GoodReader. CloudClip lets me share a multi-item clipboard across my devices. Reflection: this overlaps with the personal information system.

A preferred office suite: I don’t have a preferred suite, though I’d like to use the Apple Suite. I use Word to write (Pages hasn’t had industrial-strength outlining), Keynote to create presentations (e.g. one from each suite). I don’t create sheets often.

A range of communication and collaboration tools: I use Google Drive to collaborate on representations. I have used Dropbox to share documents as well. And of course Mail for email. Reflection: this overlaps with social networks.

1 or more smart devices: I’d be lost without my iPhone and iPad (neither of which is the latest model). I use the phone for ‘in the moment’ things, the iPad for when I have longer time frames.

So, that’s my toolkit, what’s yours?

Tom Reeves AECT Keynote Mindmap

10 November 2017 by Clark 1 Comment

Thomas Reeves opened the third day of the AECT conference with an engaging keynote that used the value of conation to drive the argument for Authentic Learning. Conation is the component of cognition that consists of your intent to learn, and is under-considered. Authentic learning is very much collaborative problem-solving. He used the challenges from robots/AI to motivate the argument.

Derek Cabrera AECT Keynote Mindmap

9 November 2017 by Clark 1 Comment

Derek Cabrera opened the second day of the AECT conference with an insightful talk about systems thinking and the implications for education. With humorous examples he covered the elements of systems thinking and why it means we need to switch pedagogies to a constructivist approach.

Marcy Driscoll AECT Keynote Mindmap

8 November 2017 by Clark Leave a Comment

Marcy Driscoll kicked off the Association for Educational Communications and Technology’s annual conference with a thoughtful keynote on leadership. She used her experience as a Dean to explore possibilities and suggestions for what this could and should mean.

Revisiting 70:20:10

7 November 2017 by Clark 14 Comments

Last week, the Debunker Club (led by Will Thalheimer) held a twitter debate on 70:20:10 (the tweet stream can be downloaded if you’re curious). In ‘attendance’ were two of the major proponents of 70:20:10, Charles Jennings and Jos Arets. I joined Will as a moderator, but he did the heavy lifting of organizing the event and queueing up questions. And there were some insights from the conversations and my own reflections.

To start, 70:20:10 is a framework, it’s not a specific ratio but a guide to thinking about the whole picture of developing organizational solutions to performance problems. In the book by Jos & Charles, along with their colleague Vivian Heijnen, on the topic, there’s a whole methodology that encompasses 5 roles and 28 steps. The approach goes from a problem to a solution that incorporates tools, formal learning, coaching, and more.

The numbers come from a study on leaders, who felt that 10% of what they learned to do their jobs came from formal learning, 20% came from working with others and coaching, and 70% they learned from trying and reflecting on the outcomes. The framework’s role is to help people recognize this, and not leave the 70 and 20 to chance. The goal is to help people along the learning curve, not just leave them to chance after the ‘event’.

First, my impression was that a lot of people like that the 70:20:10 framework provides a push beyond the event model of ‘the course’. Also, a number struggle with the numbers as a brand, because they feel that the numbers are misleading. And some folks clearly believe that good instructional design should include the social and the activity, so the framework is a distraction. A colleague felt that there were also some who feel that formal learning is a waste of time, but I don’t think that many truly ignore the 10, they just want it in the proper perspective (and I could be wrong).

Now, there are times when the ratio changes. In roles where the consequences of failure are drastic (read: aerospace, medical, military), you tend to have a lot more formal. It can go quite a ways up the learning curve. Ideally, we’d do this for every situation, but in real life we have to strike a balance. If we can do the job right in the 10, and then similarly ensure good practices around the 20 and the 70, we’ll get people up the curve.

Another issue, for me, is that 70:20:10 not only provides a push towards thinking of the whole picture, but like Kirkpatrick (and perhaps better) it serves as a design tool. You should start from what the situation looks like at the end and figure out what can be in the world and what has to be in the head, and then go backwards. You then design your tools, and then your training, and 70:20:10 suggests including coaching, etc. But starting with the 70 is one of the messages.

So, I like the realization of 70:20:10 (except typing all those redundant zeros and colons, I often refer to it as 721 ;): the focus on designing the full solution, including tools and coaching and more. I don’t see 70:20:10 being the full solution, as the element of continual innovation and a learning culture are separate, but it’s a good solution for the performance part of the picture, and the specific parts of the development.

Addressing Changes

25 October 2017 by Clark Leave a Comment

Yesterday, I listed some of the major changes that L&D needs to acknowledge. What we need now is to look at the top steps that need to be taken. As serious practitioners in a potentially valuable field, we need to adapt to the changing environment as much as we need to assist our charges to do so. So what’s involved?

We need to get a grasp on technology affordances. We don’t need to that the latest technology exists, whether AI, AR, or VR. Instead, we have to understand what they mean in the context of our brains. What key capabilities are brought? Can VR go beyond entertainment to help us learn better? How can AI partner with us? If we can make practical use of AR, what would we do with it?

In conjunction, we need to understand the realities about us. We need to take ownership and have a suitable background in how people really think, work, and learn. Further, we need to recognize that they’re all tied together, not separate things. So, for instance, we learn as we work, we think as we learn, etc.

For example, we need to understand situated and distributed cognition. That is, we need to grasp that we’re not formal logical thinkers, but instead very context dependent, and that our thinking is across our tools. As a consequence, we need to design solutions that recognize our individual situations, and leverage technology as an augment. So we want to design human/computer system solutions to problems, not just human or system solutions.

We also need to understand cultural elements. We work better when we are given meaningful work, freedom to pursue those goals, and get the necessary support to succeed. This is not micromanagement, but instead, is leadership and coaching. We also need an environment where it’s safe, expected even, to experiment and even to make mistakes.

We also need to understand that we work better (read: produce better results), when we work together in particular ways. Where we understand that we should allow individual thought first, but then pool those ideas. And we need to show our work and the underlying thinking. Moreover, again, it has to be safe to do so!

And, these are all tied together into a systemic approach! It can’t be piecemeal, because working together and out loud can’t be divorced from the technology used to enable these capabilities. And giving people meaningful work and not letting them work together, or vice-versa, just won’t achieve the necessary critical mass.

Finally, we also need to do this in alignment with the business. And, lets be clear, in ways that can be measured! We need to be understanding what are the critical performance needs of the organization, and demonstrate that we’re impacting them in the ways above.

This can be done, and it will be the hallmark of successful organization. We’re already seeing a wide variety of converging evidence that these changes lead to success. The question is, are you going to lead your organization forward into the future, or keep your head down and do what you’ve always done?

Stay Curious

18 October 2017 by Clark Leave a Comment

One of my ongoing recommendations to people grew out of a toss-off line, playing off an advertisement. Someone asked about a strategy for continuing to learn (if memory serves), and I quipped “stay curious, my friends”. However, as I ponder it, I think more and more that such an approach is key.

I was thinking of this trend the other day as “intellectual restlessness”. What I’m talking about is being intrigued by things you don’t understand that have persisted or recently crossed your awareness, and pursuing it. It’s not just saying “how interesting”, but recognizing connections, and pondering how it could change what you do. Even to the point of actually changing!

It also would include pointing interesting things to other people who would benefit. This doesn’t always have to happen, but in the spirit of cooperation (in the Jarche sense), we could and should contribute, curate, when we can. And, ideally, leaving trails of your explorations that others can benefit from. Writings, diagrams, videos, what have you, helps yourself as well as others.

I was reminiscing that more than 30 years ago, on top of my job designing educational computer games, I was already curious. I still have copies of the magazines containing reviews I did (one hardware, one software), as well as a journal article based upon undergraduate research I was fortunate to participate in.

And that persistence in curiosity has led to a trail of artefacts. You may have come across the books, book chapters, articles, presentations, etc. And, of course, this blog for the past decade and more. (May it continue!) However, I’m not here to tout my wares, but instead to point to the benefit of being curious.

As things change faster, a continuing interest is what provides an ongoing ability to adapt. All the news about the ongoing changes in jobs and work isn’t likely to lessen. Staying curious benefits you, your colleagues and friends, and I reckon society in general. You want to look at many sources of information, track tangential fields, and be open to new ideas.

This isn’t just your choice, of course, ideally your organization is supportive. These lateral inputs are a component of innovation, as is time to allow for serendipity and incubation. Orgs that want to be able to be agile will need this capabilities as well. I suppose organizations need to stay curious as well!

Radical Coherency

11 October 2017 by Clark Leave a Comment

Tied to my last post about insufficient approaches, I was thinking again about the Coherent Organization . Coherency is powerful, but it could be a limiting metaphor. So I want to explore it a bit further.

First, coherency is powerful. Lasers, for example, are just light, the same as comes from your lightbulbs. Except that the wavelengths are aligned and focused. When they‘re at the same frequency, in the same direction, suddenly you can cut steel!

However, an easy interpretation is that you get this right, and it‘s then sufficient. But that‘s no longer sufficient in organizations. As things change, you need coherency and agility. How do you get both?

I‘m suggesting that coherency has to be on many dimensions. So you have coherency with the organization‘s purpose, but people are coherent with each other, and with the customers, and with best principles. And that latter is important, as best practices won‘t transfer unless they‘re abstracted and recontextualized.

So what I‘m arguing for is a more radical coherency, a coherency that‘s in synchrony in an ecosystem perspective. Where people are communicating and collaborating in ways that apply best principles in an way that integrates them into an aligned whole that‘s greater than the sum of the parts.

This is a learning organization, but one that‘s integrating many disparate elements. That, I think, is a desirable and achievable goal, but it’s more than one program. It’s a campaign that needs an initial focus, and a plan to successfully integrate it into practice first, and then to scale it to both shift practice and culture. It’s non-trivial, but I think it’s more than worthwhile: it’s necessary. What do you think?

Mundanities

29 September 2017 by Clark Leave a Comment

This post is late, as my life has been a little less reflective, and a little more filled with some mundane issues. There’re some changes here around the Quinnstitute, and they take bandwidth. For a small update on these mundanities with some lessons:

First, I moved office from the side of the house back to the front. My son had occupied it, but he’s settled into an apartment for college, and I prefer the view out to the street (to keep an eye on the neighborhood). Of course, this entailed some changes:

My ergonomic chair stopped working, and it took several days to a) find out someone who’d repair it, b) get it there, wait for it to get fixed, and get it back. It’s worth it (a lot less than replacing) and ergonomics is important.

Speaking of which, I also now could get a standup desk, or in my case one of those convertible desks that lets you raise and lower your workspace. I’ve been wanting one since the research has come out on the problems with sitting. We’d previously constructed a custom desktop (with legs from Ikea!), for the odd shaped room, so it was desirable to just put it on top. So far, so good. Strongly recommended.

Also bought a used bookshelf (rather than move the one from the old office). Real wood, real heavy. Used those ‘forearm forklift’ straps to get it in. They work! And, this being earthquake country, had to strap it to the wall. Still to come: filling with books.

At the same time, fed up with all the companies that provide internet and cable television, we decided to change. (We changed mobile providers back in January.) As I noted previously, companies use policies to their advantage. One of the approaches is that they sell you a two year package, but then there’s no notification that the time’s up and the rate jumps up. And you can’t find just a low rate provider (I don’t even mind if it’s higher than the bonus deal). Everyone uses this practice. Sigh.

As I said, I can’t find anyone better, but just decided to change. That involved conversations, and research, and installation time, and turning off the old systems. At least we’re getting a) a lower rate, b) nicer DVR, and c) faster internet. For the time being. While the new provider promised to ping me before the plan runs out, the old provider says they can’t. See what I mean? Regardless, I’ve got a trigger before it expires to sign up anew. Or change again. That’s the lesson on this one.

And of course there are some conversations about some upcoming presentations. I was away last week presenting, and have one coming up next month (ATD China Summit, if you’re near Shanghai say hello) and several in November at AECT in Jacksonville. You’ve seen some of the AI reflections, more likely to come on the new topics.

And there’s been some background work. Reading a couple of books, and working on two projects. Stay tuned for a couple of new things early next year.

The lesson, of course, is trying to find time to reflect while you’re executing on mundanities is more challenging, but still a valuable investment. I fight to make time, I hope you do too!

« Previous Page