Archives for September 2008

Part 4 of the 4 part series:

Here‘re the final suite of heuristics I came up with many years ago as a result of looking at our design process and the barriers our cognitive architecture can put in our own way.

Full Spectrum Design: One of the most insidious problems observed in educational multimedia is a tendency to incorporate all the solution into the computer.   The system will be the repository of all the text, sound, graphics, etc, and the instruction.   Unfortunately, this does not properly reflect what’s known about reading text on screen and the role of the teacher.   In conjunction with the No Limits Analysis, another way to get the best design is to consider the full spectrum of media, particularly considering delivering text on paper, having the instruction accomplished by an instructor, etc.   The proper use of the term multimedia is to consider all the available media and their use and to distribute the instructional task across all of them.

No Limits Analysis: After assembling a team, the first step in the design process is analysis, and an important component is proper information gathering to ensure that all relevant possible sources of inspiration have been considered.   However, before we consider what others have done in the same, we should see what we come up with when we think as if there were no limits.   This occurs after the pedagogical problem has been identified but before other examples are considered.   The step is to consider how the problem would be addressed if there were no technological limitations, as if anything could be accomplished as if by magic. Arthur C. Clarke said “any truly advanced technology is indistinguishable from magic”, and we‘re really at the stage where the barriers are our imaginations, not the technology.   So stop and think what an ideal solution would be.   You may not be able to achieve what you imagine, but you certainly can‘t if you don‘t identify that option, and you‘ll prematurely limit the solution space.

Kitchen Sink Analysis: After the No Limits Analysis, comes the systematic consideration of other corners of the design space or other relevant prototypes for modification.   Lewis & Reiman suggested that “plagiarism” is an appropriate design strategy (as far as your lawyers would let you, as they cautioned), where ideas are lifted from existing designs rather than reinventing the wheel.   An expansion to this concept is to not only look at what others have done but to also consider how instruction might proceeded without computer support, what theory suggests as an approach, etc.   In short, the suggestion is to exhaustively search all potential sources of input into the design process, including the proverbial ‘kitchen sink’.   This process is both to help populate the design space and discover all constraints.

And let me add one other that I didn‘t explicitly include before:
Lateral input:   Research on brainstorming and creativity (cf D Bono), has shown that besides being systematic and covering all the known or plausible solutions, lateral thinking is valuable. After you‘ve been exhaustive within the box, find ways to get ‘outside the box‘.   Use random inspirations: play a game, doodle, get some random input!   Get silly!   It may not be politically correct anymore, but back when I worked for a learning game company, the CEO (hi, Sky!), used to bring in pizza and beer on Friday afternoon and have some idea sessions!   There are lots of tools and approaches, just make sure you make some concerted effort here.

OK, that‘s it for this series on design.   I hope these past few posts give you some useful guidance or ideas.   I welcome your own!

First: https://blog.learnlets.com/2008/09/design-design-as-search/

Second: https://blog.learnlets.com/2008/09/design-our-barriers/

Third: https://blog.learnlets.com/2008/09/design-the-first-heuristics/

Part 3 of a 4 part series:

We talked previously about our cognitive limitations.   Here I list a subset of the heuristics I‘ve discovered across design practices and from experience.   I‘m sure you‘ll find some that seem obvious, and have others of your own. However, I still see instances where these principles would have helped, so I‘m tossing them out there just in case.

Team Design: One problem is covering all the required areas of expertise.   As was discovered in the early days of multimedia, designers should recognize and acknowledge the spectrum of talent required to properly develop a project.   In particular, there should be expertise on the team for the content area, the educational design, the interface design, the programming environment, and each media area to be used.   The management style has to allow the contributions of these experts to the design, and to resolve any fundamental contradictions only upon assessment of each point of view.   The saying is that the room is smarter than the smartest person in the room, but my caveat is that is only true if you manage the process right.   But diversity helps, and you want to find a way to involve diverse viewpoints early on, so ensure a suite of talents on the design team.

Egoless Design: Hand in hand with the above is the requirement for egoless design.   Each team member has to feel comfortable contributing to the overall design and willing to offer and receive constructive criticism.   Egoless programming was the source of this approach, but it holds true fo all group endeavors. The members have to recognize and respect the contributions of each other.   Team leaders can facilitate this by displaying the same quality themselves.   For instance, the opportunity for deliberately silly ideas can support a willingness to take risks.   There should be explicit discussions of process as well as product, as this is likely to prove valuable in not only leading to a high quality of output but in leading to improved capabilities of the team members.

3 Strategies Design: While creativity in the design process can contribute to selecting a good design, all designs will benefit from testing and refinement of the design.   Three strategies from the field of ‘user-centered design’ can be used to characterize an appropriate approach.   This process is captured in the notion of iterative, formative, and situated design, where successive iterations of the design are tested with real data and real users and the results are used to further inform the design process.   It has been reliably demonstrated that single passes at design fail for reasons specifically related to lack of appropriate testing.   In short, the waterfall approach doesn‘t work.   We‘ll find questions we can‘t answer on the basis of existing information and we‘ll have differing opinions. Prototype and test!   Also…

(the Double) Double P’s: Once some designs are generated, it is necessary to start producing limited-capability versions, or prototypes (different from the use of the term for the evolutionary design model), for testing to feed back into the design process.   The prototypes that are created should be of increasing fidelity, but it is tempting to start prototyping them on the computer   However, this can lead to functional fixedness.   A colleague many years ago had a rule for his team that no programming should proceed until a complete ‘storyboard’ has been completed on paper.   This leads to a prescription for the Double P’s: Postponing Programming and Preferring Paper.   User experience work has found that paper can be extremely valuable for experimenting, prototyping, and testing.   Get your answers before you spend lots of money and time!

OK, one more list of heuristics coming up.   And I look forward to yours and your feedback.

First: https://blog.learnlets.com/2008/09/design-design-as-search/

Second: https://blog.learnlets.com/2008/09/design-our-barriers/

Fourth: https://blog.learnlets.com/2008/09/design-final-heuristics/  

Part 2 of a 4 part series:

Cognitive psychology has identified certain characteristics of our brains. When it comes to problem solving, (and we can think of design as problem-solving as well as search), we have certain behaviors that predispose us to certain types of solutions (in other words, prematurely limiting our search space). These include functional fixedness, set-effects, premature evaluation, and, not too surprisingly, social issues.

Functional fixedness is really just about not seeing new applications of a tool.   This can be seen when designers are familiar with a particular implementation tool or environment. The designs then tend to resemble what is easy to do with that tool (the old “when you have a hammer everything looks like a nail”).   Alternatively, they may push the tool beyond it‘s effective range trying to accomplish a particular outcome.   If the tool is the optimal one for the job this is not a problem, but in other cases the tool is familiar and so it is used despite a lack of relevance.   Even in robust design environments, projects often resemble what is easy to implement in that environment, not what the analysis would indicate.

Set effects are solving new problems in ways appropriate for previous problems, whether or not that‘s appropriate now.   In this approach, subjects who had solved several prior problems in a particular way would solve a new problem in that well-practiced way.   However, subjects who had not had that prior experience would find the simpler solution to the problem.   What was identified as “set effects” manifests itself in designers whose new solutions resemble prior solutions even when the old solutions are inappropriate.   While it is true that the consistency produced from set effects is often beneficial, that should be a conscious decision and not the accidental outcome of limitations of the designer or the design process.

Premature evaluation is where problem-solvers will settle on a solution before all possibilities have been considered.   In creativity, one of the hurdles has been when problem-solvers will pursue the first solution path that presents itself rather than delay solution while considering other potential solution paths that might be more effective.   The cognitive overhead in retaining a meta-level of strategy that considers multiple solution paths is often overwhelmed in the efforts to consider all the factors in the problem itself.   Considerable practice can be required to develop good strategic maintenance of strategies as well as solution.

A final set of problems in problem-solving are the social ones: the difficulty of publicly suggesting an idea that may not be accepted, the difficulty in sharing an idea that may not get credited, the difficulty in offering help due to a perception that it may be an intrusion or unwelcome, or the difficulty in accepting help from others whether to not be a burden or to resent intrusion.   Another set of social problems have to do with different domains of expertise.   Most learning technology projects these days require multiple skill sets: writing, media production, software engineering, learning design, management, etc.   Who gets listened to? How do you coordinate this?

I suspect you recognize these problems, but of course the issue is what to do. Coming up in the next two posts: Team Design, Egoless Design, No Limits Analysis, ‘Kitchen Sink’ Analysis, Systematic Creativity, 3 Strategy Design, the (double) Double P’s, and Full Spectrum Design.   What are your ideas?

First: https://blog.learnlets.com/2008/09/design-design-as-search/

Third: https://blog.learnlets.com/2008/09/design-the-first-heuristics/

Fourth: https://blog.learnlets.com/2008/09/design-final-heuristics/  

A four-part series on design, barriers, and some heuristics to improve outcomes:

I found my passion in learning technology.   I took most of a computer science degree (after flirting with biology, and…) before designing my own major combining CS with education.   I went back to grad school to learn a lot about cognition and learning.   I took a rather eclectic approach, going beyond cognitive approaches to include behavioral approaches, instructional design, social learning theories, constructivist learning theories, even machine learning.   (I continue that today.)

As I took very much an ‘applied‘ approach (not doing pure research, but interpreting research to meet real problems, now known as “design-based research” in the edtech field, but close to the more general ‘action research‘), and was teaching interaction design, I started looking at design processes in that same eclectic way. (NB: I‘ve also tracked ‘engagement‘, as those who‘ve read the book or heard me speak on games know).   I looked at interface design processes, and instructional design processes, heck, I looked at architectural, industrial, engineering, and graphic design processes. And I realized some things that I‘ve talked about in various places but haven‘t written about for over 10 years, and yet I think are still relevant.   I‘m going to talk here about design, our cognitive (and other) barriers to design, and my plan is to subsequently post about a suite of heuristics I‘ve come up with to minimize the consequences of those barriers.

Design can be characterized as a search of a ‘solution‘ space. Think of all the possible solutions as this n-dimensional space or cloud, and outside the cloud are designs that wouldn‘t solve the problem, like the old approach we were using; inside the cloud are possible solutions).   Sometimes we can evolve an existing design incrementally to solve the problem, and sometimes we combine previous solutions to create a new one (or so some theories say).   Another way to think about it is we have this infinite solution space, and then we start putting in constraints that limit the space (must cost less than $50K, must be doable in six weeks, must work in our technical environment, etc).   Constraints are actually good, as they limit the space we have to search.   However, too often when we consider all the constraints, we‘ve just made the space the null set; we‘ve excluded any solution. Then we have to relax constraints: reduce scope, increase budget, etc.

One of the problems is prematurely limiting our search. It turns out that our cognitive architecture has biases that may limit our search long before we consciously look at our options.   We may only search a limited space nearby our prior experience, and only achieve what‘s know in search as a ‘local maximum‘, as a good solution for part of the space but not the best solution overall (a ‘global maximum‘). We need to know the barriers, and then propose solutions around those barriers.   Coming up: Functional Fixedness, Set-Effects, Premature Evaluation, and the ever-dreaded Social issues.   Stay tuned!

Second: https://blog.learnlets.com/2008/09/design-our-barriers/

Third: https://blog.learnlets.com/2008/09/design-the-first-heuristics/

Fourth: https://blog.learnlets.com/2008/09/design-final-heuristics/