Part 3 of a 4 part series:

We talked previously about our cognitive limitations.   Here I list a subset of the heuristics I‘ve discovered across design practices and from experience.   I‘m sure you‘ll find some that seem obvious, and have others of your own. However, I still see instances where these principles would have helped, so I‘m tossing them out there just in case.

Team Design: One problem is covering all the required areas of expertise.   As was discovered in the early days of multimedia, designers should recognize and acknowledge the spectrum of talent required to properly develop a project.   In particular, there should be expertise on the team for the content area, the educational design, the interface design, the programming environment, and each media area to be used.   The management style has to allow the contributions of these experts to the design, and to resolve any fundamental contradictions only upon assessment of each point of view.   The saying is that the room is smarter than the smartest person in the room, but my caveat is that is only true if you manage the process right.   But diversity helps, and you want to find a way to involve diverse viewpoints early on, so ensure a suite of talents on the design team.

Egoless Design: Hand in hand with the above is the requirement for egoless design.   Each team member has to feel comfortable contributing to the overall design and willing to offer and receive constructive criticism.   Egoless programming was the source of this approach, but it holds true fo all group endeavors. The members have to recognize and respect the contributions of each other.   Team leaders can facilitate this by displaying the same quality themselves.   For instance, the opportunity for deliberately silly ideas can support a willingness to take risks.   There should be explicit discussions of process as well as product, as this is likely to prove valuable in not only leading to a high quality of output but in leading to improved capabilities of the team members.

3 Strategies Design: While creativity in the design process can contribute to selecting a good design, all designs will benefit from testing and refinement of the design.   Three strategies from the field of ‘user-centered design’ can be used to characterize an appropriate approach.   This process is captured in the notion of iterative, formative, and situated design, where successive iterations of the design are tested with real data and real users and the results are used to further inform the design process.   It has been reliably demonstrated that single passes at design fail for reasons specifically related to lack of appropriate testing.   In short, the waterfall approach doesn‘t work.   We‘ll find questions we can‘t answer on the basis of existing information and we‘ll have differing opinions. Prototype and test!   Also…

(the Double) Double P’s: Once some designs are generated, it is necessary to start producing limited-capability versions, or prototypes (different from the use of the term for the evolutionary design model), for testing to feed back into the design process.   The prototypes that are created should be of increasing fidelity, but it is tempting to start prototyping them on the computer   However, this can lead to functional fixedness.   A colleague many years ago had a rule for his team that no programming should proceed until a complete ‘storyboard’ has been completed on paper.   This leads to a prescription for the Double P’s: Postponing Programming and Preferring Paper.   User experience work has found that paper can be extremely valuable for experimenting, prototyping, and testing.   Get your answers before you spend lots of money and time!

OK, one more list of heuristics coming up.   And I look forward to yours and your feedback.

First: https://blog.learnlets.com/2008/09/design-design-as-search/

Second: https://blog.learnlets.com/2008/09/design-our-barriers/

Fourth: https://blog.learnlets.com/2008/09/design-final-heuristics/  

Part 2 of a 4 part series:

Cognitive psychology has identified certain characteristics of our brains. When it comes to problem solving, (and we can think of design as problem-solving as well as search), we have certain behaviors that predispose us to certain types of solutions (in other words, prematurely limiting our search space). These include functional fixedness, set-effects, premature evaluation, and, not too surprisingly, social issues.

Functional fixedness is really just about not seeing new applications of a tool.   This can be seen when designers are familiar with a particular implementation tool or environment. The designs then tend to resemble what is easy to do with that tool (the old “when you have a hammer everything looks like a nail”).   Alternatively, they may push the tool beyond it‘s effective range trying to accomplish a particular outcome.   If the tool is the optimal one for the job this is not a problem, but in other cases the tool is familiar and so it is used despite a lack of relevance.   Even in robust design environments, projects often resemble what is easy to implement in that environment, not what the analysis would indicate.

Set effects are solving new problems in ways appropriate for previous problems, whether or not that‘s appropriate now.   In this approach, subjects who had solved several prior problems in a particular way would solve a new problem in that well-practiced way.   However, subjects who had not had that prior experience would find the simpler solution to the problem.   What was identified as “set effects” manifests itself in designers whose new solutions resemble prior solutions even when the old solutions are inappropriate.   While it is true that the consistency produced from set effects is often beneficial, that should be a conscious decision and not the accidental outcome of limitations of the designer or the design process.

Premature evaluation is where problem-solvers will settle on a solution before all possibilities have been considered.   In creativity, one of the hurdles has been when problem-solvers will pursue the first solution path that presents itself rather than delay solution while considering other potential solution paths that might be more effective.   The cognitive overhead in retaining a meta-level of strategy that considers multiple solution paths is often overwhelmed in the efforts to consider all the factors in the problem itself.   Considerable practice can be required to develop good strategic maintenance of strategies as well as solution.

A final set of problems in problem-solving are the social ones: the difficulty of publicly suggesting an idea that may not be accepted, the difficulty in sharing an idea that may not get credited, the difficulty in offering help due to a perception that it may be an intrusion or unwelcome, or the difficulty in accepting help from others whether to not be a burden or to resent intrusion.   Another set of social problems have to do with different domains of expertise.   Most learning technology projects these days require multiple skill sets: writing, media production, software engineering, learning design, management, etc.   Who gets listened to? How do you coordinate this?

I suspect you recognize these problems, but of course the issue is what to do. Coming up in the next two posts: Team Design, Egoless Design, No Limits Analysis, ‘Kitchen Sink’ Analysis, Systematic Creativity, 3 Strategy Design, the (double) Double P’s, and Full Spectrum Design.   What are your ideas?

First: https://blog.learnlets.com/2008/09/design-design-as-search/

Third: https://blog.learnlets.com/2008/09/design-the-first-heuristics/

Fourth: https://blog.learnlets.com/2008/09/design-final-heuristics/  

A four-part series on design, barriers, and some heuristics to improve outcomes:

I found my passion in learning technology.   I took most of a computer science degree (after flirting with biology, and…) before designing my own major combining CS with education.   I went back to grad school to learn a lot about cognition and learning.   I took a rather eclectic approach, going beyond cognitive approaches to include behavioral approaches, instructional design, social learning theories, constructivist learning theories, even machine learning.   (I continue that today.)

As I took very much an ‘applied‘ approach (not doing pure research, but interpreting research to meet real problems, now known as “design-based research” in the edtech field, but close to the more general ‘action research‘), and was teaching interaction design, I started looking at design processes in that same eclectic way. (NB: I‘ve also tracked ‘engagement‘, as those who‘ve read the book or heard me speak on games know).   I looked at interface design processes, and instructional design processes, heck, I looked at architectural, industrial, engineering, and graphic design processes. And I realized some things that I‘ve talked about in various places but haven‘t written about for over 10 years, and yet I think are still relevant.   I‘m going to talk here about design, our cognitive (and other) barriers to design, and my plan is to subsequently post about a suite of heuristics I‘ve come up with to minimize the consequences of those barriers.

Design can be characterized as a search of a ‘solution‘ space. Think of all the possible solutions as this n-dimensional space or cloud, and outside the cloud are designs that wouldn‘t solve the problem, like the old approach we were using; inside the cloud are possible solutions).   Sometimes we can evolve an existing design incrementally to solve the problem, and sometimes we combine previous solutions to create a new one (or so some theories say).   Another way to think about it is we have this infinite solution space, and then we start putting in constraints that limit the space (must cost less than $50K, must be doable in six weeks, must work in our technical environment, etc).   Constraints are actually good, as they limit the space we have to search.   However, too often when we consider all the constraints, we‘ve just made the space the null set; we‘ve excluded any solution. Then we have to relax constraints: reduce scope, increase budget, etc.

One of the problems is prematurely limiting our search. It turns out that our cognitive architecture has biases that may limit our search long before we consciously look at our options.   We may only search a limited space nearby our prior experience, and only achieve what‘s know in search as a ‘local maximum‘, as a good solution for part of the space but not the best solution overall (a ‘global maximum‘). We need to know the barriers, and then propose solutions around those barriers.   Coming up: Functional Fixedness, Set-Effects, Premature Evaluation, and the ever-dreaded Social issues.   Stay tuned!

Second: https://blog.learnlets.com/2008/09/design-our-barriers/

Third: https://blog.learnlets.com/2008/09/design-the-first-heuristics/

Fourth: https://blog.learnlets.com/2008/09/design-final-heuristics/  

The web is an interesting place, and Google is an interesting player.   They’re just announcing their new browser, Chrome, which uses the same WebKit engine as Safari (as opposed to Firefox’s Gecko engine).   The challenge is to Microsoft’s Internet Explorer (which is raising new anti-trust issues with the announcement that once you install it, you may not be able to remove it), but at the same time it’s also challenging Apple and Mozilla (owners of Safari and Firefox, respectively) with whom they’ve had a pretty good relationship.   Of course, Google’s Android OS for mobile phones also challenges Apple’s Mac OS X for the iPhone.   The point being that there are times when it’s hard to decide who’s doing what to who!

Which isn’t to say that Chrome isn’t a compelling vision, and the way they chose to reveal this vision is with a comic!   They got Scott McCloud (author of the must-read Understanding Comics) to write/draw it, and he clearly had access to folks who understood the underpinning ideas and manages to communicate them.     Granted, it helps if you understand the differences between threads and processes, but it really does help illuminate the underlying ideas.

Two lessons:

1. you’ve got to stay on top of the changes in the web space: the arguments for the new technical implementation may not be important to you (they are, but it’s about performance and if you’re not a mega-surfer yet you won’t really notice), but Google’s weight behind it is important from a market perspective.

2. Comics are an underused communication tool with great power.   We’re looking at them for a project we’re working on, fortunately the client’s enthused.   Start considering how you might employ them.

I thought I’d gone off about pre-tests here before, but apparently not (at least I can’t find it).   So let me do it now.   Pre-tests are learner-abusive. Period.   *OK, with one (rare) caveat…

First, let’s agree that quizzes are usually not an enjoyable experience.   Except when the outcome doesn’t matter, and provides valuable information (e.g. the ‘Cozmo quiz’ where you learn things about yourself).   However, when you don’t know the answers (by definition, or you wouldn’t need it), it’s just a tedious process in most cases.

There are two major arguments for pre-tests, which I’ll argue against.   One is that it helps the learner understand what’s coming, serving as an advance organizer, activating relevant knowledge. Yes, it will do that.   However, there are much less cruel ways to do it, such as dramatically or humorously exaggerating the consequences of not having the knowledge, drilling down from the larger context, etc.   Doing it through a random quiz, particularly when you’re not already expected to know the information, just leads to frustration and/or boredom.

The other reason used to justify pre-tests is to show the delta from before and after the learning experience.   This is also wrong, since you shouldn’t even be developing the learning unless you already know they don’t know the material.   Consequently, the only thing you should need to demonstrate is that they know can achieve your objectives. And beyond, that it leads to improved performance and better outcomes.

The only qualification to this is when the pre-assessment is used to allow the student to test-out. That is, by passing a pre-assessment, they can skip material they already know.   Even then, it might be a preference, rather than required.

So, please, don’t abuse your learners, and don’t give pre-tests unless it allows the learner to test-out of the learning (and only if they want to).

Clive Shepherd blogs this idea from Richard Middleton, about two possible dimensions that might affect your learning goals: how motivated your learners are, and how ‘big’ the behavior change is:

The quick notion is that if you’ve a small change and high motivation, it can be very lean.   Lower motivation requires more engaging presentation, and once you start having big changes you’ll need lots of practice, and when the learner isn’t interested or is resistant, you’ll really have to ramp up the engagement (tuning it into a game).

There are lots of other dimensions (e.g. maybe it doesn’t even require rapid elearning, but just an information update), but this is certainly a good way to look more richly at the design task and how it might be addressed.   And looking richly at your learning task is where you get more creative learning solutions (read: learning experience designs).

Well, if you happened to hit my blog between yesterday and today, you might’ve noticed some slight changes to the format.   Unintentional.   I’ve just started tweeting (using Twitter), and had seen how your tweets could appear in a window in your blog.   I wanted to do the same, but my blog template is old.   I wanted to look at another template so I clicked on it, and it installed the new theme, not just giving me information.   Which wiped out the header and some of the customization on my sidebar.

I pinged my ISP, who’s also a friend/colleague/mentor, and was my boss a couple of times.   Sky let me know that the themes are just different sub-directories that get swapped between, and if I just clicked on another theme I’d eventually get mine back.   Which worked, and it’s now back, but there’s a lesson in there.

First, I’m reasonably tech literate.   I programmed for a living (for Sky, actually) before I went back to grad school.   I’ve maintained a knowledge of what tech can do, though I no longer maintain fluency in any languages.   I’ve maintained, updated, and have customized my sites as well (this blog, my book site, and my company site).   However, my understanding is more conceptual these days; e.g. while I know what CSS is and why it’s good and you should use it, I’m pretty much at the crayon level with it.

However, the lesson is that having taught interface design (and studied with Don Norman) I know that the interface could be doing a better job of helping me build a conceptual model of how my blog site operates.   They recently changed the blog entry interface, and actually made it worse because the ‘tag’ interface is no longer on the screen initially, it’s hidden down below and you need to scroll to get to it (which means I forget sometimes).   But overall, I really don’t understand where and how they’re using files to compile this site.

Now, I also don’t know PHP (or javascript, or Python, or… the point being that there are so many different web technologies I can’t keep up with them all; it’s not how I add value to the endeavors I’m engaged in), but I’ve managed to muddle through adding things to the files like the Feedblitz signup (if you want to read via email, like I do).   The old interface made it really hard to find a file to edit that you hadn’t edited recently, and the new one’s better (some things get worse and others get better, but really things should steadily get better). I realize web interfaces are going to be weaker than application ones, but it doesn’t have to be this hard.   Of course, as far as I know it’s free, so I shouldn’t complain too much.

Still, one of the things we know is that if there is a conceptual model underlying how something’s implemented, making that conceptual model clear (or even available) will help people work with a system when they’re using it intermittently.   There’re no clues for me in WordPress.   Now, their working assumption for people who’ve installed their own copy is that they’re reasonably facile with PHP and probably more regularly generating code, so maybe I’m not one of their target users.   Still, there’s little to be lost, and a lot to be gained by making the underlying model clear.

I do recommend you read Don’s Design of Everyday Things book, which helps explain why mappings and models are powerful guides to action. Everyone who designs solutions for others should read it; it’s an easy and short read, and it will definitely change the way you look at the world.   In a positive way, and that’s a good thing, I reckon.   Oh, and do include conceptual models in your learning designs. It leads to much more persistent and flexible performance.