Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Game-based meta-cognitive coaching

by 1 Comment

Many years ago, I read of some work being done by Valerie Shute and Jeffrey Bonar that I later got a chance to actually play a (very small) role in (and even later got to work with Valerie, definitely world-class talent).   They had developed three separate tutoring environments (geometric optics, economics, electrical circuits), yet the tutoring engine was essentially the same across all three, not domain specific.   The clever thing they were doing was tutoring on exploration skills, varying one variable at a time, making reasonable increments in values to graph trends, etc.

Subsequent to that, I got involved again in games for learning. What naturally occurred to me was that you could put the same sort of meta-cognitive skill tutoring in a game environment, as you have to digitally create all the elements you’d need to track anyways for the game reasons, and it could be a layer on top.   While this would work in a single game (and we did put a small version into the Quest game), it would be even better on top of a game engine.   I even proposed it as a research project, but the grant reviewers thought that while   a good idea, it was too ambitious (ahead of my time and underestimated :).

The reinterest in so-called 21st century skills, the kind Stephen Downes so eloquently calls an Operating System for the Mind, reawakens the opportunity.   These skills are manifested in activity, and require an understanding of the activity to be able to infer approaches and provide feedback. In a well-defined arena like a designed game environment, we can know the goals and possible actions, and start looking for patterns of behavior.

Game engines, with their fixed primitives, make it easier to define what goals are and consequently to specify the particular goals and makes looking for patterns more generally definable.   Thus, in a game, we can see whether the learners’ exploration is systematic, whether their attempts are as informative as possible, and possibly more.

This is also true of virtual worlds, although only when designed with goals (e.g. from a simulation to a scenario, whether tuned into a game or not).   The benefit of a virtual world is, again, the primitives are fixed, simplifying the task of defining goals and actions.

Of course, building particular types of interaction (e.g. social), particular types of clues (e.g. audio versus visual) and looking for patterns can provide deeper opportunities.   Really, such performance is initially an assessment (one of the facets of what we were doing on the Intellectricity project was building a learner characteristic assessment as a game), and that assessment can trigger intervention as a consequence.   For any malleable skill, we have real opportunities.

Given that much of what is necessary are abilities to research , evaluate the quality of sources, design, experiment, create, and more, these environments are a fascinating opportunity.   I’m not in a situation to lead such an initiative, but I still think it’s a worthwhile undertaking.   Anyone ‘game’?

Context & learning environments

by Leave a Comment

I was talking with Gina Shreck, who I’d known through Twitter, at a Sun-sponsored happy hour about new learning environments. She’s been quite active in Virtual Worlds (VW), and I was describing an Augmented Reality Game (ARG), and it came to me that there are some really meaningful similarities.

We know from research like John Bransford’s Anchored Instruction and Brown, Collins, & Duguid’s Situated Cognition that learning works better in context (even if you spread across contexts to generalize).   What I realized is that both approaches are really using technology to bring context for learning into vivid relief.   I’ve been active in games for learning because it provided meaningful practice, and of course VW’s can be used to host games in (realizing that VW‘s aren’t inherently games, but instead are just environments), and so are ARG’s.

Even when designed for learning, the point is to try to enrich the context.   Web-based games are the easiest, but there are times when more full contextualization is necessary, and the different environments offer different affordances or capabilities.

Despite the overhead, VWs are immersive in that your avatar is totally ‘in world’, and you can design that world to be anyplace/anytime you want it to be.   You can design the contingencies the way you want.   While most valuable for 3D, it may also be important for when total difference is necessary.   Specific examples include building real world structures that must be explored or investigated, for learning purposes.

On the other hand, ARGs are set in the real world, but specific constraints can be introduced.   You can have specific events, materials, and people (real or virtual) appear in the world you want.   Again, you want to develop associated decision making for those explored contexts.

The reason to use an ARG is to develop the ability to develop the capability in situ, that is, as close to the real world context as possible, whereas VWs can add extra dimensions, or work for contexts that are too expensive or dangerous to do live.   That’s also true for non-VW games as well, of course.

The point is to minimize distance and maximize transfer from learning context to real world application.   The overhead to take advantage of these sorts of capabilities is dropping quite rapidly. The goal is to discover the degree and type of contextualization needed (as well as pocketbook, of course), and decide what environment offers the necessary depth and value to achieve the outcomes you need.   However, you need to understand the full repertoire of tools available, and their affordances, to optimally choose an approach.   So, game on!

Designing on demand

by 3 Comments

Yesterday I had the pleasure of working with a team on a grant to use games as a context to conduct high stakes cognitive assessments.   The cognitive tasks for assessment are remarkably abstract, e.g do this particular discrimination task (ie look at a string of four characters and signal if one is a vowel), sometimes while monitoring another situation or as an attention-distractor from another task, but the tasks that they are matched to range from very expensive to life-saving..   The goal is to establish a baseline, and then look for decrements at particular instances before a crucial task, indicating lack of readiness.

The interesting thing is the challenge of placing these tasks in a meaningful context.   It’s creative, and consequently fun.   It’s also collaborative, and when you get divergent contributors in a safe environment, you can really get productive synergy going.   We got together the night before for a meal and some social lubricants, and the next day spent hours in a conference room discussing, whiteboarding and generally designing.

One of the problems is that there have been diverse project specifications from the granting organization, and lack of access to the intended audience. We had some feedback that there should be minimal ‘story’, and very clearly that if the audience doesn’t perceive value, they can ignore the activity completely.   Also, there were some important constraints on how much we could change the core task without invalidating the deep research base.   Fortunately, a background in cog psych as well as having the minds behind the tests with us allowed a reasonable guess.   Still, a bit of a challenge.

We focused in early on the value, and I brought up that if the cognitive activity produces improvements in ability, that it’s training as well as assessment, and the audience cares very much about being able to do the job.   That hadn’t been determined to date, but may be available.   We also talked about marketing the value, and if the assessment can in this case (as it has in the past) serve as a very accurate predictor of performance (e.g. detecting a decrement in performance without prior knowledge), that may provide the necessary motivation.

When it comes to design, I’ve made a claim before that you can’t give me an objective I can’t design a game for (I reserve the right to raise the objective ‘high’ enough, but have yet to be proved wrong; it’s an outcome of the engaging learning framework), and this isn’t an exception. In fact, we came up with numerous possible settings, originally for what we were told of the mission, and then for a more near-term mission.   We also came up with relative degrees of abstraction from real (e.g. closely aligned to real task) to essentially arbitrary (like Tetris has little correlation to real time).   The fact of the matter is, you can embed meaningful tasks in appropriate contexts, and tune into a game no matter what the objective is.   It just takes systematic creativity (not an oxymoron), as in the heuristics I’ve talked about previously in two spots.

Since we don’t yet have access to the audience (though we know who they are), I suggested that we need to mock up several different plausible looks and trial them when they do get access (they’re working on that).   They had talked to some stakeholders, but that’s not reliable, for reasons I related to them.   In the process of designing the Quest game, we talked to the counselors who worked with these ‘at risk’ youth, who suggested this issue was smart shopping and cooking.   Fortunately, we then got to talk to some of the youth themselves, who responded “yeah, that’s important, but what’s really important is…” and proceeded to give us a set of relationships that then became key to the game.   Lesson: don’t just listen to the managers, or just the trainers, or just…all those are important, but they may not be right.

It was easy to consider a number of degrees of story ‘depth’, and visual styles to go with each.   At least, they’re in my head, but I went out and grabbed screenshots of various things that could serve as models.   You want the game mechanics to reflect the cognitive task, but you can wrap a number of different looks round that. We’ll pull together our notes, get some storyboards generated, but we managed to sketch out five separate games for what evidence suggests are likely to be the most important skills.

And that’s the real lesson, that it can be done, reliably and repeatedly.   And that’s important, because if you can’t, then it’s all well and good to talk about the value of games as learning environments, but it’s a waste of time if you don’t have an associated design process.   Fortunately, I can still comfortably say: “learning can, and should, be hard fun“.

Mythconceptions

by 6 Comments

Several things got up my nose yesterday (and I don’t mean literally :).   I listened in on the Corporate Learning Trends event in the morning, and in the evening participated in #lrnchat.   Don’t get me wrong, both events were great: great presentations organized by Tony Karrer, with examples coordinated by Judy Brown on mobile, Bob Mosher on performance support, Karl Kapp on games & simulations, and Tony on asynchronous elearning (all folks I know and respect); and a great lrnchat session as always with Marcia Conner coordinating fantastic participation by a whole host of great folks.   It’s just that several continuing beliefs surfaced that we’ve really got to address.

The first one was the notion that games and simulations are about tarted up quiz shows.   Let me be clear, these are a last resort!   When you’ve addressed the important decisions, and there’s still some knowledge that absolutely has to be memorized, not looked up, they’re ok.   But they’re not your starting point!   Games should be first thought of as your best practice environment for skills, not knowledge recitation.   What’s going to make a difference in learner (and organizational) performance is not rote knowledge, but meaningful decisions.   That is where games shine.

Ok, as Treena Grevatt pointed out, these ‘frame games’ may serve as the easiest entry point for organizational acceptance, but only if you ‘get it’ really, and are only using them as an entry point to do meaningful stuff.   Otherwise, it’s still lipstick on a pig.

The problem is, we already have a problem with our formal learning being too knowledge focused, and not skill focused, and a tool to make drill and kill easy isn’t going to help us remedy the problem.   So, please: first get that games are really deeply contextualized, immersive, challenging skill practice.   Then, when your analysis has addressed that and there still are knowledge components, bring in the quiz show games.   If you ‘get’ that, then you might use a stealth policy, but only then.

The second problem had to do with mobile learning.   There were still notions that mobile learning could be about courses on a phone   and that there’s not really an audience.   Look, depending on what metrics you pay attention to, the mobile workforce can be anywhere from 20-40% of your workforce.   Sales reps, telecommuters, field engineers, execs, the list goes on. And that doesn’t even tap into the folks who want access for convenience!

And it’s not about courses.   It has been, and can be done, but that’s not the real win.   As an adjunct to a course, absolutely.   Reactivate knowledge (developing learners), update it with podcasts (Chris von Koschembahr had a nice way to interview yourself, controlling the outcome :), review stories, solve problems, review with mentors, etc.

The real win, however (as Judy and Bob both pointed out), is performance support. This can include references, job aids, how to videos, connections to experts, and more.   This is huge, yet people don’t seem to be seeing this opportunity yet.

Mobile is ready for primetime. There are ways to deal with screen sizes, security, and cross-platform differences.   Next to social learning, I reckon it’s the greatest missed opportunity going.

Speaking of performance support, I do have to admit how surprised I was that people were thinking that single sourcing content to populate help systems, manuals, and training was a new idea.   This really isn’t a misconception, it’s just surprising.   I led a project developing such an approach years ago now, and it’s another big opportunity.   Still ahead of the curve, though, more so than the other two.

The point being, the more you tie these together, the greater the synergy: the whole is greater than the sum of the parts. And having been out saying these things for years, it continues to surprise me that the meme hasn’t propagated any further than it has.   And that’s my learning, that changing minds is a tough job.   But still an important one.   Evangelism, anyone?

Sims, Games, and Virtual Worlds

by 4 Comments

On last week’s #lrnchat, which I missed most of for my lad’s band concert, I tuned in during a break and saw that Marcia Conner (@marciamarcia) had asked a question I wanted to answer (but couldn’t in 140 chars :).   She asked: “Would someone explain diff between sims (often used well for ed) and VWs?”   She was concerned that some people were using them interchangeably, and I do think it’s important to have some clear definitions.

I stipulate (and would love to get agreement on) a definition that works like this:

  • A simulation is, technically, just a model.   It’s captures the relationships of some part of the world (real or virtual), typically not all.   It can be in any potential state, and be manipulated to any other valid state.
  • When we put that simulation into an initial state, and ask someone to take it to a particular goal state, I want to call that a scenario.   And, typically, we wrap a story around it.
  • We can tune that scenario into a game.   Not turn it, tune it.   A game is a scenario that’s been optimized to have just the right (subjective) level of challenge, a story learners care about, and a bunch of other elements that characterize an engaging experience.

So what’s a Virtual World?   In the above definition, it’s a simulation with the particular characteristics that it’s 3D, and typically also can host many individuals within it.   Now, the infamous World of Warcraft has been turned into a game by a) embedding a bunch of quests (initial states where you try to achieve certain goal states) and b) tuning the experience to be compelling (even addictive).

It gets interesting when we start talking about learning in the context of sims, games, and Virtual Worlds.   A simulation, for a motivated and effective self-learner, is a powerful learning environment.   They can explore the relationships to their desired level of understanding.   The only problem is that motivated and effective self-learners are unfortunately rare.   So, we more typically create scenarios.

When you choose an initial state, and properly choose the goal state, you can ensure that they can’t achieve the goal state until they fully have grasped the nuances of the relationships and can act upon them in specific ways.   That’s the essence of serious game design! This is, I argue, the best learning practice next to live performance with mentoring.   The benefits to scenarios, of course, are that live performance can have costly consequences (e.g. losing money, breaking things, or killing people) and individual mentoring doesn’t scale well.

Are there reasons to tune a scenario into a game?   I want to argue that there are.   First of all, there are the motivational aspects, keeping the learner’s interests.   Second, optimizing the challenge means that the learner is moving through in the minimal amount of time.   Finally, we can alter the storyline to make it more meaningful – exaggerating characters or motives or context – which actually brings the practice environment closer to the urgency likely to be felt in the real world, when it matters. Truly, learning can and should be ‘hard fun’!

How about learning in virtual worlds?   I’ve talked about this before, but certainly, I believe, if the learning objectives inherently support 3D reasoning, whether industrial plant arrangement and operation, molecular structure, or architecture, absolutely.

However, a virtual world is just a simulation, and if you want learning outcomes, you need either self-directed and motivated learners, or embedded scenarios.   Which is what I have been seeing, for example I have seen a very nice demonstration for insurance adjusting.

In addition, when social interaction matters, there are some interesting opportunities.   Individuals can represent themselves as they please, and can create the contexts they wish as well.   (However, I have also seen what are, essentially, slide presentations in a virtual world, and think that’s ridiculous.)

On the other hand, virtual worlds currently have some overhead issues: learning to be effective in them has a learning curve, and there are technical overheads as well.   Consequently, I have been loath to recommend them for many situations where they could be used, if there isn’t an inherently 3D rationale.

However, I do believe that a) the overheads are rapidly being dropped by advancements both UI and technical and b) that there are some ephemeral things that are still fully to be realized.   People I trust, including Joe Miller and Claudia L’Amoreaux of Linden Labs, Karl Kapp of Bloomsburg University, and Tony O’Driscoll of Duke University, continue to express not only the available, but also the untapped potential.

Still, I think the definitions are solid, and am comfortable with the current assessment of virtual worlds.   I’m willing to be wrong, on the latter :). I welcome your thoughts.

A wee bit o’ experience…

by 1 Comment

A personal reflection, read if you’d like a little insight into what I do, why and what I’ve done.

Reading an article in Game Developer about some of the Bay Area history of the video game industry has made me reflective.   As an undergrad (back before there really were programs in instructional technology) I saw the link between computers and learning, and it’s been my life ever since.   I designed my own major, and got to be part of a project where we used email to conduct classroom discussion, in 1978!

Having called all around the country to find a job doing computers and learning,   I arrived in the Bay Area as a ‘wet behind the ears’ uni graduate to design and program ‘educational’ computer games.   I liked it; I said my job was making computers sing and dance.   I was responsible for FaceMaker, Creature Creator, and Spellicopter (among others) back in 81-82.   (So, I’ve been designing ‘serious games’, though these were pretty un-serious, for getting close to 30 years!)

I watched the first Silicon Valley gold rush, as the success of the first few home computers and software had every snake oil salesman promising that they could do it too.   The crash inevitably happened, and while some good companies managed to emerge out of the ashes, some were trashed as well.   Still, it was an exciting time, with real innovation happening (and lots of it in games; in addition to the first ‘drag and drop’ showing up in Bill Budge’s Pinball Construction Set, I put windows into FaceMaker!).

I went back to grad school for a PhD in applied cog sci (with Don Norman), because I had questions about how best to design learning (and I’d always been an AI groupie :).   I did a relatively straightforward thesis, not technical but focused on training meta-cognitive skills, a persistent (and, I argue, important) interest.   I looked at all forms of learning; not just cognitive but behavioral, ID, constructivist, connectionist, social, even machine learning.   I was also getting steeped in applying cognitive science to the design of systems, and of course hanging around the latest/coolest tech.   On the side, I worked part-time at San Diego State University’s Center for Research on Mathematics and Science Education working with Kathy Fischer and her application SemNet.

My next stop was the University of Pittsburgh’s Learning Research & Development Center for a post-doctoral fellowship working on a project about mental models of science through manipulable systems, and on the side I designed a game that exercised my dissertation research on analogy (and published on it).   This was around 1990, so I’d put a pretty good stake in the ground about computer games for deep thinking.

In 1991 I headed to the Antipodes, taking up a faculty position at UNSW in the School of Computer Science, teaching interface design, but quickly getting into learning technology again.   I was asked, and I supervised a project designing a game to help kids (who grow up without parents) learn to live on their own. This was a very serious game (these kids can die because they don’t know how to be independent), around 1993.   As soon as I found out about CGIs (the first ‘state’-maintaining technology) we ported it to the web (circa 1995), where you can still play it (the tech’s old, but the design’s still relevant).

I did a couple other game-related projects, but also experimented in several other areas.   For one, as a result of looking at design processes,   I supervised the development of a web-based performance support system for usability, as well as meta-cognitive training and some adaptive learning stuff.

I joined a government-sponsored initiative on online learning, determining how to run an internet university, but the initiative lost out to politics.   I jumped to another, and got involved in developing an online course that was too far ahead of the market (this would be about 1996-1997).   The design was lean, engaging, and challenging, I believe (I shared responsibility), and they’re looking at resurrecting it now, more than 10 years later!   I returned to the US to lead an R&D project developing an intelligent learning system based on learning objects that adapted on learner characteristics (hence my strong opinions on learning styles), which we got up and running in 2001 before that gold rush went bust.   Since then, I’ve been an independent consultant.

It’s been interesting watching the excitement around serious games.   Starting with Prensky, and then Aldrich, Gee, and now a deluge, there’s been a growing awareness and interest; now there are multiple conferences on the topics, and new initiatives all the time.   The folks in it now bring new sensibilities, and it’s nice to see that the potential is finally being realized. While I’ve not been in the thick of it, I’ve quietly continued to work, think, and write on the issue (thanks to clients, my book, and the eLearning Guild‘s research reports).   Fortunately, I’ve kept from being pigeonholed, and have been allowed to explore and be active in other areas, like mobile, advanced design, performance support, content models, and strategy.

The nice thing about my background is that it generalizes to many relevant tasks: usability and user experience design and information design are just two, in addition to the work I cited, so I can play in many relevant places, and not only keep up with but also generate new ideas.   My early technology experience and geeky curiosity keeps me up on the capabilities of the new tools, and allows me to quickly determine their fundamental learning capabilities.   Working on real projects, meeting real needs, and ability to abstract to the larger picture has given me the ability to add value across a range of areas and needs.   I find that I’m able to quickly come in and identify opportunities for improvement, pretty much without exception, at levels from products, through processes, to strategy.   And I’m less liable to succumb to fads, perhaps because I’ve seen so many of them.

I’m incredibly lucky and grateful to be able to work in the field that is my passion, and still getting to work on cool and cutting edge projects, adding value.   You’ll keep seeing me do so, and if you’ve an appetite for pushing the boundaries, give me a holler!

Predictions for 2009

by 13 Comments

Over at eLearn Magazine, Lisa Neal Gualtieri gets elearning predictions for 2009, and they’re reliably interesting. Here’re mine:

The ordinary: Mobile will emerge, not as a major upheaval, but quietly infiltrating our learning experiences. We‘ll see more use of games (er, Immersive Learning Simulations) as a powerful learning opportunity, and tools to make it easier to develop. Social networking will become the ‘go to‘ option to drive performance improvements.

The extraordinary: Semantics will arise; we‘ll start realizing the power of consistent tagging, and start being able to meta-process content to do smart things on our behalf.   And we‘ll start seeing cloud-hosting as a new vehicle for learning services.

I’ve been over-optimistic in the past, for example continuing to believe mobile will make it’s appearance (and it is, but not in the big leap I hoped).   It’s quietly appearing, but interest isn’t matching the potential I’ve described in various places.   I’m not sure if that’s due to a lack of awareness of the potential, or perceptions of the barriers: too many platforms, insufficient tools.

I continue to see interest in games, and naturally I’m excited.   There is still a sadly-persistent view that it’s about making it ‘fun’ (e.g. tarted up drill and kill), while the real issue is attaching the features that drive games (challenge, contextualization, focus on important decisions) and lead to better learning.   Still, the awareness is growing, and that’s a good thing.

And I’ve been riffing quite a lot recently about social networking (e.g. here), as my own awareness of the potential has grown (better late than never :).   The whole issues of enabling organizational learning is powerful.   And I’ve also previously opined about elearning 3.0, the semantic web, so I’ll point you there rather than reiterating.

So there you have it, my optimistic predictions. I welcome your thoughts.

‘Novel’ learning about reading

by Leave a Comment

I like to read.   These days, I confess I seldom find time to read a full non-fiction book, but try to find the ‘readers digest’ condensed version on the web.   Time/money.   But I do still read novels, as enjoyment.   However, I’m reading differently than I used to.

As a Father’s Day present, my family took me to a used book store to load up on fun novels. I picked up a couple from recommended series of books, and two of them really were a revelation.   One was written in a very ‘street’ language, and very elliptical, and I had to work hard to understand (it also sort of presumed previous experience with the series). The other was a recent book from a familiar series, but was in the first person present, and also was hard work to read, requiring cognitive ‘leaps’ to make sense.   The revelation was that both books kept me to the end, not that I’d choose to have those experiences again.   It taught me a lot about how far we might be able to stretch our audience to stay engaged.   That is, if we’ve set up a compelling story line, or have other ways of ensuring motivation.

Another lesson comes from another series, where the protagonist’s reflections on society are revelatory to me.   It’s fiction, but the description of what the character sees resonates with what I see my partners doing in successful conversations with clients, and I’m always looking to learn to be better at what I do.

From the game design point of view, these are important reasons to read different genres of books (ok, so I’m lax on reading bodice rippers, I have to have some limits), but my learning here is that reading different author’s styles (and their stylistic explorations) give you two things: an exposure to the breadth of what will work, and some insight into how other people can parse the world.

So, as I tell my workshop attendees: “you have a tough assignment ahead of you, you’ve got to spend more time exploring the breadth and depth of entertainment to add to the repertoire of solutions you can bring”.   And there’s something to be said about a hot tub, a cold beer, and a good book…

Killer Game App?

by 5 Comments

Martine asked on the Serious Games list what the killer app was for ‘serious games’ list, and it prompted some thinking.

As I‘ve suggested before, Pine and Gilmore say that after the product and services economy is the ‘experience’ economy, that we’re in now, where we pay for experiences.   A good example is Apple, which makes buying and owning Apple products more than a product/service, but an experience (think also: themed restaurants/travel, amusement parks, etc).

They argue that the next phase is the ‘transformation’ economy, where experiences will transform us.   Read: learning.   I’ll suggest serious games is a component of that transformation experience, and the principles underlying ‘engaging learning’ (engaginglearning.com), designing learning games, are the principles for designing those transformative experiences.

However, it occurs to me that the killer app may just well be a game-based high-stakes assessment.   Why?   Assessment is important, and tough to do well.   Simulation is the closest thing to real performance, and consequently should provide the highest fidelity assessment.   You have to perform to succeed (read: win the game).

A number of years ago I was leading an R&D project building an intelligently adaptive learning system, using learner characteristics.   We started with a profiling instrument to develop the basis for adaptation, but intended to build a game-based environment to assess learner’s ‘styles’ (c.f. my rant on learning styles) as the basis for adaptation.   I think this idea could be extended for many important skills.

There’s no reason, for instance, that SimuLearn’s Virtual Leader couldn’t be a leadership assessment as well as a learning environment (if you buy their leadership model).   In fact, any assessment use would naturally (ala problem-based learning) serve as the basis for a learning experience as well.

So, my take on the killer app for games (besides games already being the killer app for elearning :), is high-stakes assessment.   This is a test: what do you think?

Missed this…

by Leave a Comment

Maybe I was in the woods last summer when this came out, or maybe it passed under the radar?   On July 16 of   last year, the House of Representatives (on a voice vote, no record) passed House Resolution 487:

Recognizing the contribution of modeling and simulation technology to the security and prosperity of the United States, and recognizing modeling and simulation as a National Critical Technology.

Wow, what I do is critical!   And here I thought I was just having fun and doing good…   I found out about this when it was cited in an RFP we (partner organization and I) were responding to.   Not quite sure what this means, practically.   I can be somewhat naive of governmental subterfuge (while cynical of the whole thing), but item 7 caught my eye:

(7) encourages the development and implementation of ways to protect intellectual property of modeling and simulation enterprises.

So, maybe this is a way to give some legal ammunition to an organization that’s fighting to wall-off certain approaches?   If so, hopefully prior art will prevail (c.f. Blackboard).

At least it did prod a governmental organization to prioritize simulations (read: scenarios, better yet: tuned into games).   If we can get organizations, governmental and not, to think harder about deep practice (when needed, and this instance is a case of resistance and complex new procedures), we’re increasing the likelihood of effective outcomes. Which is what I really care about.