Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Pre-tests = learner abuse*

by 12 Comments

I thought I’d gone off about pre-tests here before, but apparently not (at least I can’t find it).   So let me do it now.   Pre-tests are learner-abusive. Period.   *OK, with one (rare) caveat…

First, let’s agree that quizzes are usually not an enjoyable experience.   Except when the outcome doesn’t matter, and provides valuable information (e.g. the ‘Cozmo quiz’ where you learn things about yourself).   However, when you don’t know the answers (by definition, or you wouldn’t need it), it’s just a tedious process in most cases.

There are two major arguments for pre-tests, which I’ll argue against.   One is that it helps the learner understand what’s coming, serving as an advance organizer, activating relevant knowledge. Yes, it will do that.   However, there are much less cruel ways to do it, such as dramatically or humorously exaggerating the consequences of not having the knowledge, drilling down from the larger context, etc.   Doing it through a random quiz, particularly when you’re not already expected to know the information, just leads to frustration and/or boredom.

The other reason used to justify pre-tests is to show the delta from before and after the learning experience.   This is also wrong, since you shouldn’t even be developing the learning unless you already know they don’t know the material.   Consequently, the only thing you should need to demonstrate is that they know can achieve your objectives. And beyond, that it leads to improved performance and better outcomes.

The only qualification to this is when the pre-assessment is used to allow the student to test-out. That is, by passing a pre-assessment, they can skip material they already know.   Even then, it might be a preference, rather than required.

So, please, don’t abuse your learners, and don’t give pre-tests unless it allows the learner to test-out of the learning (and only if they want to).

Performance Support & Performance Ecosystem

by 3 Comments

Jay Cross has an eloquent post talking about the history of Performance Support, ending of course calling for considering the learnscape. Tony Karrer comments on it in his own post looking at performance support and learning technology.

Interestingly, what Jay doesn’t really cover in his history is that Gloria came up with Performance Support to cover up bad interface design.   The systems were monolithic and essentially impermeable to change, so she wrapped a solution around it.   The interaction design field was a little put out about the whole performance support system notion, saying it was really just good interface design. And there’s still too little of that, sad to say (I used to teach interaction design, and it’s a component of the performance ecosystem solution).

What Jay points out, however, is that the learning designer needs to take responsibility for more than just courses, and it’s ok if information is the solution (“‘Information is not instruction.’ …if information gets the job done, it doesn‘t matter whether it‘s instruction”).

However, Jay starts lumping all of the web 2.0 tools into performance support, which is where Tony gets curious.   He thinks some of the tools fall more into the knowledge management category, but admits he may be getting definitional.   He is in agreement about the need to look at the larger picture and consider all these tools as playing a role in meeting ePerformance, a term he and I agree upon.

Jay cites Marc Rosenberg, and Marc certainly has been calling for us to include knowledge management, performance support, and eCommunity as part of our tools to go beyond eLearning.   Which is where we’re all in agreement.   Good reading, good thoughts, good work.

Distributed Learning

by 3 Comments

Distributed learning is an idea that I think offers an untapped potential, what with the new technologies we have.   I’m not talking here about distance learning, but instead a combination of slow learning with ubiquitous learning.   The idea is to combine learning on the go and on-demand with a long term relationship, personalized awareness, and mixed media.   Think of it as cloud computing for your learning goals.

There are strong reasons for spreading learning over time (Will Thalheimer‘s got a whole white paper on it) –   think drip irrigation.   We can use technology to do this in a contextually relevant way; not just random elements, but wrapped around the events in our life.   With some knowledge of our schedule, and our learning goals, a system could pop out little relevant bits of learning to develop us over time.

Imagine that you’ve learning goals about communication, and about coaching.   Assume, for the moment, an imaginary curriculum that places ‘authenticity’ after ‘understanding the other’s point’, and that you’ve completed the latter.   Then, before a business meeting with a potential new contact, you might get a message to “‘say what you mean, mean what you say, without being mean‘, after you ensure you’ve heard them”   that comes in right before the meeting.   After the meeting you might be connected to a coach/mentor, to see how it went.

Later that same day, you’ve got a review meeting with one of your reports, and as your coaching curriculum’s next topic is “focus on behavior, not person”, you get not only a relevant message beforehand, but a customized job aid to take with you (filled out with the individual’s last details and your particular area to work on), and a self-evaluation form afterward.

Which is not to say you don’t also have the opportunity to request particular information beforehand, so there might be a custom ‘pull’ portal available to you with things you’re likely to need (in addition to the general search tools you already have).   A smart system might recognize that it’s been too long since some knowledge has been applied, and choose to send you some challenge to keep the knowledge active, at least until it’s part of your internalized repertoire.

Why is this of interest?   It’s about developing people over time, in the ways they want (an individual should could choose their goals, though there could be ones also negotiated with an employer).   It’s about taking advantage of your life’s occurences, not removing oneself from it to learn.   It’s being contextualized sensitive to not only where you are, but ‘when’ you are.   It’s about being opportunistic, effective, and efficient, rather than intrusive, effortful, and minimally effective.   Which is not to say that there might not be more concerted chunks, particularly at the beginning, or at major inflection points, but it’s the optimal blend – an information model, not an industrial model.

We’ve got the capability (Clarke’s “any truly advanced technology is indistinguishable from magic”), but we need the will and the resources.   Anyone game?

Stories and Tools

by 2 Comments

BJSchone‘s tweet pointed me to Jay’s business assessment of web 2.0 tools, which somehow I’d missed.   A great little chart.   What got me going was his final entry, on stories.   He says:

memorable, natural way to spread values and goals; more sophisticated than text, oral tradition reinforces meaning

It started me thinking about the research (e.g. Schank, way back when) about our ‘scripts‘ (Schank’s equivalent to Minsky‘s Frames, and Rumelhart‘s Schemas, several co-emergent conceptualizations for thinking from cognitive science).   There’s been lots of recent interest in stories for business and organizations (e.g. Steven Denning), and there are sound reasons to do so.

The point that strikes me about why stories are such a compelling, memorable way to communicate is that our brains are hardwired to process them, they naturally contextualize the message, and (when well-done), help communicate both the solution and the underlying concept.   They can communicate messages about values, as Jay points out, as well as methods.   And they tap into human universals, as this article from Scientific American points out (sent to me after I’d written first draft of this; serendipity).

Which is why I’m a fan of stories in elearning.   They can be used up-front for what I call a motivating example, not a reference example but instead a visceral demonstration of why this knowledge is important.   And, of course, they can be used for reference examples where they link concept to context.   There are some nuances about how to do this that I talk about in my talks about Deeper eLearning (coming to DevLearn) and in my article on the 7 Step Program to better elearning (PDF).   Basically, worked steps, cognitive annotation, and backtracking & repair.   Solid research to back it up.

Of course, podcasts are a great way to use stories.   They are naturally an audio medium.   Then, you can augment stories with images, ala a narrated slideshow, or video. I remember we used to attend a series of travel movies at Carnegie Library in Pittsburgh; the filmmakers themselves would narrate the film live, and it was a very professional, entertaining, and enlightening experience. So vidcasts would also be useful.

And, as I’ve stated before, I think that comics/manga are a great and underexplored way to communicate, as they are stories, with the same ability to exaggerate.   They can take more time to produce but are more visceral (because they add visuals).   They also globalize easily (though may have trouble with accessibility?).   I can’t resist pointing again to Dan Pink’s new manga because it’s both good career advice and a good example!   I’d bet they’d work well on an iPhone, too.   Hmm…

So, look at the tool guide, think stories, and media.   Now, if we could only find a reliable and affordable way to get comics/manga done.

Top 10 Tools

by Leave a Comment

Jane Hart’s Top Tools list is a great resource, and she reminded me that my list might need updating.   Fortunately, Jay blogged about his list, which reminded me (some mail didn’t make the transition to the new environment, hence the need for a new list, as well as updating).   My list has some changes:

3. Firefox – my Web tool for searching, browsing, surfing: with the new engine, it’s fast, and has great plugins
4. Twitter – I’m using TwitterFox on Firefox, and Twittelator on my iPhone.   A whole new world…
5. Google – their search engine, their maps, their website tracking,…
7. iTunes – how I connect my iPhone to my Mac, download mobile apps, and more.
8. Mail – part of my move to centralize on Mac apps (iCal, Address Book) to accommodate iPhone; I use email a lot (e.g. RSS feeds from Feedblitz), but Mail’s missing some things I liked in Entourage

You’ll see a few changes precipitated by the iPhone (and some ways of rearranging).   Interestingly my top 10 mobile tools list was just pointed to, and I realize it’s out of date too!   I’m still playing with the iPhone, but the tools I use are:

Mail – email on the go

Twittelator – twittering about

Google Maps – location, location, location

Safari – mobile web browsing at it’s best (which still is only so-so :)

Contacts – who’s who?

Photos – easily loaded all my diagrams and portfolio pictures

I’m anticipating using Flashlight (literally), EccoNote (voice memo), AIM (one IM tool, maybe to get around SMS charges), UrbanSpoon (fun way to find restaurants, tho’ not yet here in WC/East Bay SF), Yelp (reviewed places), SplashID (all those passwords, protected), and FaceBook.

I’m cheap, so I’m mostly downloading free apps.   Recommendations?

Missed this…

by Leave a Comment

Maybe I was in the woods last summer when this came out, or maybe it passed under the radar?   On July 16 of   last year, the House of Representatives (on a voice vote, no record) passed House Resolution 487:

Recognizing the contribution of modeling and simulation technology to the security and prosperity of the United States, and recognizing modeling and simulation as a National Critical Technology.

Wow, what I do is critical!   And here I thought I was just having fun and doing good…   I found out about this when it was cited in an RFP we (partner organization and I) were responding to.   Not quite sure what this means, practically.   I can be somewhat naive of governmental subterfuge (while cynical of the whole thing), but item 7 caught my eye:

(7) encourages the development and implementation of ways to protect intellectual property of modeling and simulation enterprises.

So, maybe this is a way to give some legal ammunition to an organization that’s fighting to wall-off certain approaches?   If so, hopefully prior art will prevail (c.f. Blackboard).

At least it did prod a governmental organization to prioritize simulations (read: scenarios, better yet: tuned into games).   If we can get organizations, governmental and not, to think harder about deep practice (when needed, and this instance is a case of resistance and complex new procedures), we’re increasing the likelihood of effective outcomes. Which is what I really care about.

Hard to cross a chasm with baby steps

by Leave a Comment

As I indicated, I’m experimenting with Twitter (@Quinnovator).   I’m following a number of people who point to interesting things or make interesting observations.   The benefits I’m finding with Twitter, I note, are random interesting thoughts that juxtapose with my own thinking, in addition to quick answers to questions.   It reminds me of Dave Owens’ long ago system DYK, that randomly gave you some unix tidbits.   It seemed to work, as many times it was irrelevant but every once in a while it was just the right thing.   Same concept as the later ‘tool tips’ you could get on starting up PowerPoint.

This particular reflection was triggered by George Siemens tweeting a response to his presentation: “we’re taking small steps” problem is, small steps=falling behind. need that big leap :).   Mark Oehlert responded with the title of this post.   And it triggered the thought that I’ve heard Jay Cross articulate, that evolution isn’t going to keep pace with the rapid rate of change. (And yes, I’m name dropping, because you too can follow thought leaders through blogs and twitter!)

On the other hand, change is hard, big change exponentially so.   What’s an organization to do?   Yet change is coming faster. One of the ways we’ve thought to address this is an opportunity we’re offering through the Cafe’, getting some high-value external input on major issues in a lightweight way, a jumpstart to out-of-the-box thinking.

What I’ve been assisting organizations to do, and this is definitely a good thing, is to start with setting a long term vision (e.g. performance ecosystem), and making organization-specific short, medium, and long term plans to get there.   However, I’m thinking that, in parallel, what’s needed is doing some ‘out of the box’ exercises where some more disruptive stuff is trialed.   Maybe a ‘tiger team‘ for communication & innovation (and yes, I’m aware of the abuse of the phrase, but in lieu of another quick way to communicate the concept…).

The point is that you can’t just daydream crossing the chasm, and you can’t rethink in increments.   Sometimes you’ve got to take a major rethink, and put it in place and learn from it.   The best advice I recall on this is that you’ve got to have some experiments going on.   A few well-thought out gambles that you’re willing to have fail.   However, it’s not random mutation, but also not intelligent design; rather a hybrid.   Taking a calculated risk.

Like trying out new technologies: the current experimental space is social networking for me, as well as the iPhone.     But you’ve got to keep pushing your personal boundaries to have the awareness for pushing the organizational ones.   So take risks and experiment yourself, and get your organization doing the same.

Game Development Tools

by 7 Comments

The last topics in our 2 day game design workshop for the Guild (great group of attendees, great experience) were evaluation, production, and organizational issues.   On the production issue, the perennial topic of tools came up.   In thinking about it, I realized that we needed a map, so I started coming up with one (a diagram, of course :) ).   I ran it past Jeff (Johannigman, my co-conspirator on the workshop) in our taxi to the airport, to his general approval.

gametoolspace

The two dimensions are complexity of the scenario (only covering branching and model-driven), and the power (e.g. complexity) of the tool.   It’s a pretty linear map, and realize that small distances aren’t significant (so the clusters are roughly equivalent).

The impossible dream is that tool that everyone wants that makes it easy to develop model-driven interactions.   Sorry, I’m convinced it can’t exist, because to be flexible enough to cover all the different models that we’d want to represent, it’s got to be so general as to be essentially just a programming language.   QED (Quinn Ephemeral Decision).

This is a first stab, so feedback welcome.   If desired, I can create it in Gliffy and we can collaboratively develop it (though my first effort with that was underwhelming in participation…).   Thoughts?

eLearning 2.0

by 2 Comments

During the 1st day of Mark and Brent’s Collaboration summer seminar, they got folks active: starting blogs, wikis, webtops, etc.   They’re doing a great job:

c2sss

Naturally, in addition to the tactical questions (“how do I move this tab”?), the conceptual questions started: when do you use a blog versus a wiki, how do you make sense of all the options out there.

Now, as part of my performance ecosystem, I think blogs are a personal reflection or a history (as I told an attendee, it would be great for capturing a ‘war story’), whereas wikis are for collaboration to create a unified view of something (e.g. the way to tune a network).   I don’t think blogs support a rich discussion and aren’t that collaborative.   Also, one of the problems I see is that we often forget old tools in the excitement of new tools: discussion boards are a great way to have an ongoing conversation; you don’t need some new tool for this. Yet wikis are really good for capturing the output of a collaboration.

Also recently I’ve been having conversations with folks about integrating tools to meet larger needs.   Ning is a tool that provides ways for individuals to have profiles, to have forums, to list events, etc.   Increasingly, LMSs also have these capabilities.

The interesting thing is the great spate of tools out there: Google, Central Desktop, Zoho, Wetpaint, PBWiki, an ever growing list. There are suites of web meeting apps, web-based productivity tools, etc. How do you make sense of it? I think there’s another ‘bubble’ with these, and eventually a bunch will fail and out of the ashes a few will persist.   The good thing, I think, is that by getting your hands dirty with a variety of these, you’ll access some generalized skills.   And web apps are not going away.

I believe training anybody on any particular tool (even the seemingly ubiquitous Microsoft Office suite), is the wrong way to go.   Talk to them about the skill (writing, creating presentations, etc), and then give some assignments across a couple of different tools.   This gives you transferrable skills, which will equip you to communicate and collaborate regardless of the latest wave of tools.   And that’s what’s important, in this day of increasing change.

Motivation by Behavior Change

by Leave a Comment

Clive Shepherd blogs this idea from Richard Middleton, about two possible dimensions that might affect your learning goals: how motivated your learners are, and how ‘big’ the behavior change is:

The quick notion is that if you’ve a small change and high motivation, it can be very lean.   Lower motivation requires more engaging presentation, and once you start having big changes you’ll need lots of practice, and when the learner isn’t interested or is resistant, you’ll really have to ramp up the engagement (tuning it into a game).

There are lots of other dimensions (e.g. maybe it doesn’t even require rapid elearning, but just an information update), but this is certainly a good way to look more richly at the design task and how it might be addressed.   And looking richly at your learning task is where you get more creative learning solutions (read: learning experience designs).