Learnlets

Secondary

Clark Quinn’s Learnings about Learning

Top 10 tools for Learning 2023

by 3 Comments

Somehow I missed colleague Jane Hart’s annual survey of top 10 tools for learning ’til just today, yet it’s the last day! I’ve participated in the past, and find it a valuable chance for reflection on my own, as well as seeing the results come out. So here’s my (belated) list of top 10 tools for learning 2023.

I’m using  Harold Jarche’s Personal Knowledge Mastery framework for learning here. His categories of seek (search and feed), sense (interpret) and share (closely or broadly) seems like an interesting and relevant way to organize my tools.

Seek

I subscribe to blog posts via email, and I use Feedblitz because I use it as a way for people to sign up for Learnlets. I finally started paying so they didn’t show gross ads (you can now signup safely; they lie when they say the have ‘brand-safe’ ads), and fortunately my mail removes images (for safety, unless I ask), so I don’t see them.

I’m also continuing to explore Mastodon (@quinnovator@sfba.social). It has its problems (e.g. hard to find others, smaller overall population), but I do find the conversations to be richer.

I’m similarly experimenting with Discord. It’s a place where I can generally communicate with colleagues.

I’m using Slack as a way to stay in touch, and I regularly learn from it, too. Like the previous two, it’s both seek and share, of course.

Of course, web surfing is still a regular activity. I’ve been using DuckDuckGo as a search engine instead of more famous ones, as I like the privacy policies better.

Sense

I still use Graffle as a diagramming tool (Mac only). Though I’m intrigued to try Apple’s FreeForm, in recent cases I’ve been editing old diagrams to update, and it’s hard to switch.

Apple’s Keynote is also still my ‘goto’ presentation maker, e.g. for my LDA activities. I have to occasionally use or output to Powerpoint, but for me, it’s a more elegant tool.

I also continue to use Microsoft’s Word as a writing tool. I’ve messed with Apple’s Pages, but…it doesn’t transfer over, and some colleagues need Word. Plus, that outlining is still critical.

Share

My blog (e.g. what you’re reading ;) is still my best sharing tool, so WordPress remains a top learning tool.

LinkedIn has risen to replace Twitter (which I now minimize my use of, owing to the regressive policies that continue to emerge). It’s where I not only auto-post these screeds, but respond to others.

As a closing note, I know a lot of people are using generative AI tools as thinking partners. I’ve avoided that for several reasons. For one, it’s clear that they’ve used others’ work to build them, yet there’s no benefit to the folks whose work has been purloined. There are also mistakes.  Probably wrongly, but I still trust my brain first. So there’re my top 10 tools for learning 2023

Don’t use AI unsupervised!

by Leave a Comment

A recent post on LinkedIn dubbed me in. In it, the author was decrying a post by our platform host, which mentioned Learning Styles. The post, as with several others, asks experts to weigh in. Which, I’ll suggest, is a broken model. Here’s my take on why I say don’t use AI unsupervised.

As a beginning, learning styles isn’t a thing. We’ve instruments, which don’t stand up to psychometric scrutiny. Further, reliable research to evaluate whether they have a measurable impact comes up saying ‘no’. So, despite fervent (and misguided) support, folks shouldn’t promote learning styles as a basis to adapt to. Yet that’s exactly what the article was suggesting!

So, as I’ve mentioned previously, you can’t trust the output of an LLM. They’re designed to string together sentences of the most probabilistic thing to say next. Further, they’ve been trained, essentially, on the internet. Which entails all the guff as well as the good stuff. So what can come out of it’s ‘mouth’ has a problematically high likelihood of saying something that’s utter bugwash (technical term).

In this case, LinkedIn (shamefully) is having AI write articles, and then circulating them for expert feedback. To me that’s wrong for two reasons. Each is bad enough in it’s own right, but together they’re really inexcusable.

The first reason is that they’ve a problematically high likelihood of saying something that’s utter bugwash! That gets out there, without scrutiny, obviously. Which, to me, doesn’t reflect well on LinkedIn for being willing to publicly demonstrate that they don’t review what they provide. Their unwillingness to interfere with obvious scams is bad enough, but this really seems expedient at best.

Worse, they’re asking so-called ‘experts’ to comment on it. I’ve had several requests to comment, and when I review them, they aren’t suitable for comment. However, asking folks to do this, for free on their generated content, is really asking for free work. Sure, we comment on each other’s posts. That’s part of community, helping everyone learn. And folks are contributing (mostly) their best thoughts. Willing, also, to get corrections and learn. (Ok, there’s blatant marketing and scams, but what keeps us there is community.) But when the hosting platform generates it’s own post, in ways that aren’t scrutable, and then invites people to improve it, it’s not community, it’s exploitation.

Simply, you can’t trust the output of LLMs. In general, you shouldn’t trust the output of anything, including other people, without some vetting. Some folks have earned the right to be trusted for what they say, including my own personal list of research translators. Then,  you shouldn’t ask people to comment on unscrutinized work. Even your own, unless it’s the product of legitimate thought! (For instance, I usually reread my posts, but it is hopefully also clear it’s just me thinking out loud.)

So, please don’t use AI unsupervised, or at least until you’ve done testing. For instance, you might put policies and procedures into a system, but then test the answers across a suite of potential questions. You probably can’t anticipate them all, but you can do a representative sample. Similarly, don’t trust content or questions generated by AI. Maybe we’ll solve the problem of veracity and clarity, but we haven’t yet. We can do one or the other, but not both. So, don’t use AI unsupervised!

Web 3.0 and system-generated content

by Leave a Comment

Not quite 15 years ago, I proposed that Web 3.0 would be system-generated content. There was talk about the semantic web, where we started tagging things, even auto-tagging, and then operating on chunks by rules connecting tags, not hard wiring. I think, however, that we’ve reached a new interpretation of Web 3.0 and system-generated content.

Back then, I postulated that Web 1.0 was producer-generated content. That is, the only folks who could put up content had all the skills. So, teams (or the rare individual) who could manage the writing, the technical specification, and the technical implementation. They had to put prose into html and then host it on the web with all the server requirements. These were the ones who controlled what was out there. Pages were static.

Then, CGIs came along, and folks could maintain state. This enabled some companies to make tools that could handle the backend, and so individuals could create. There were forms where you could type in content, and the system could handle posting it to the web (e.g. this blog!). So, most anyone could be a web creator. Social media emerged (with all the associated good and bad). This was Web 2.0, user-generated content.

I saw the next step as system-generated content. Here, I meant small chunks of (human-generated) content linked together on the fly by rules. This is, indeed, what we see in many sites. For instance, when you see recommendations, they’re based upon your actions and statistical inferences from a database of previous action. Rules pull up content descriptions by tags and present them together

There is another interpretation of Web 3.0, which is where systems are disaggregated. So, your content isn’t hosted in one place, but is distributed (c.f. Mastodon or blockchain. Here, the content and content creation are not under the control of one provider. This disaggregation undermines unified control, really a political issue with a technical solution.

However, we now see a new form of system-generated content. I’ll be clear, this isn’t what I foresaw (though, post-hoc, it could be inferred). That is, generative AI is taking semantics to a new level. It’s generating content based upon previous content. That’s different than what I meant, but it is an extension. It has positives and negatives, as did the previous approaches.

Ethics, ultimately, plays a role in how these play out. As they say, Powerpoint doesn’t kill people, bad design does. So, too, with these technologies. While I laud exploration, I also champion keeping experimentation in check. That is, nicely sandboxing such experimentation until we understand it and can have appropriate safe-guards in place. As it is, we don’t yet understand the copyright implications, for one. I note that this blog was contributing to Google C4 (according to a tool I can no longer find), for instance. Also, everyone using ChatGPT 3 has to assume that their queries are data.

I think we’re seeing system-generated content in a very new way. It’s exciting in terms of work automation, and scary in terms of the trustworthiness of the output. I’m erring on the side of not using such tools, for now. I’m fortunate that I work in a place of people paying me for my expertise. Thus, I will continue to rely on my own interpretation of what others say, not on an aggregation tool. Of course, people could generate stuff and say it’s from me; that’s Web 3.0 and system-generated content. Do be careful out there!

The Role of a Storyboard?

by 1 Comment

In a recent conversation, the issue of storyboards came up. It was in the larger context of representations for content development. In particular, for communicating with stakeholders (read: clients ;). The concern was how do you communicate the content versus representing the experience. So, the open question is what is the role of a storyboard?

So, there are (at least) several elements that are important in creating a learning experience. One is, of course, the domain. What are the performance objectives? Moreover, how are you providing content support. Specifically, what models are you using, and what examples. Then, of course, there’s the practice. Ideally, practice aligns with the performance objectives, and the models and examples support the practice.

There’s also the experience flow. How are you hooking learners up front? We are concerned with balancing the quantity of content with the actual practice, keeping engagement throughout.

In both cases, we need to communicate these to the clients. Too often, of course, clients raise concerns about making sure ‘the content’ is covered. In many cases, they really don’t understand that less is better, and have a large amount they’ve heard from subject matter experts (SMEs). Not knowing, of course, that SMEs have access to what they know about the domain, but less about what they do! Thus, they’re looking to make sure the content’s covered.

There will also be concern about the experience. This, likely not ideally, comes after assurance about the content. I personally have experienced situations where stakeholders say ‘ok’ to a storyboard, but then balk at the resulting experience. Some (surprisingly high) proportion of folks can’t infer an experience from a storyboard.  This has been echoed by others.

The question is what is the role of a storyboard. In game design, there is a (dynamic) design document that captures everything as it develops. Is this the right representation to communicate the experience? It  communicates to developers, but is it good for clients? I argue that we want more iterative representations, for instance getting sign-off on what we’ve heard from the analysis and documenting what will be the focus of the design. We also want to separate out the domain from the experience.

Overall, I advocate representing the experience, for instance in (mocked up) screenshots with narration to represent a sample interaction. That can accompany the storyboard, but when folks have to sign-off on an experience, and they can’t get it from the usual representations, you’ll need an augment. I wonder whether we should fight against presenting the content that’ll be covered.

We should show the objectives, models, and examples, but fight against content ‘coverage’. Cathy Moore does a good job in her ‘Action Mapping’ to argue that what the minimum is to achieve success on appropriate performance tasks is a good goal. I agree, as does learning science. The role of a storyboard is to capture development for developers. It may not be the right communication tool for stakeholders. I welcome your thoughts.

Misconceptions?

by 6 Comments

Several books ago, I was asked to to talk about myths in our industry. I ended up addressing myths, superstitions, and misconceptions. While the myths persist, the misconceptions propagate, aided by marketing hype. They may not be as damaging, but they also are a money-sink, and contribute to the lack of our industry making progress. How do we address them?

The distinctions I make for the 3 categories are, I think, pretty clear. Myths are beliefs that folks will willingly proclaim, but are contrary to research. This includes learning styles, attention span of a goldfish, millennials/generations, and more (references in this PDF, if you care). Superstitions are beliefs that don’t get explicit support, but manifest in the work we do. For example, that new information will lead to behavior change. We may not even be aware of the problems with these! The last category is misconceptions. They’re nuanced, and there are times when they make sense, and times they don’t.

The problem with the latter category is that folks will eagerly adopt, or avoid, these topics without understanding the nuances. They may miss opportunities to leverage the benefits, or perhaps more worrying, they’ll spend on an incompletely-understood premise. In the book, I covered 16 of them:

70:20:10
Microlearning
Problem-Based Learning
7 – 38 – 55
Kirkpatrick
NeuroX/BrainX
Social Learning
UnLearning
Brainstorming
Gamification
Meta-Learning
Humor in Learning
mLearning
The Experience API
Bloom’s Taxonomy
Learning Management Systems

On reflection, I might move ‘unlearning’ to myths, but I’d certainly add to this list. Concepts like immersive learning, workflow learning, and Learning Experience Platforms (LXPs)  are some that are touted without clarity. As a consequence, people can be spending money without necessarily achieving any real outputs. To be clear, there are real value in these concepts, just not in all conceptions thereof. The labels themselves can be misleading!

In several of my roles, I’m working to address these, but the open question is “how?” How can we illuminate the necessary understanding in ways that penetrate the hype? I truly do not know. I’ve written here and spoken and written elsewhere on previous concepts, to little impact (microlearning continues to be touted without clarity, for instance). At this point, I’m open to suggestions. Perhaps, like with myths, it’s just persistent messaging and ongoing education. However, not being known for my patience (a flaw in my character ;), I’d welcome any other ideas!

Thinking artificially

by Leave a Comment

I finally put my mitts on ChatGPT. The recent revelations, concern, and general plethora of blather about it made me think I should at least take it for a spin around the block. Not surprisingly, it disappointed. Still, it got me thinking about thinking artificially. It also led me to a personal commitment.

What we’re seeing is a two-fold architecture. On one side is a communication engine, e.g. ChatGPT. It’s been trained to be able to frame, and reframe, text communication. On the other side, however, must be a knowledge engine, e.g. something to talk about. The current instantiation used the internet. That’s the current problem!

So, when I asked about myself, the AI accurately posited two of my books. It also posited one that as far as I know, doesn’t exist! Such results are not unknown. For instance, owing to the prevalence of the learning styles myth (despite the research), the AI can write about L&D and mention styles as a necessary consideration. Tsk!

The problem’s compounded by the fact that many potential knowledge bases, beyond the internet, have legacy problems. Bias has been a problem in human interactions, and records thereof can also therefore have bias. As I (with co-author Markus Bernhardt) have opined, there is a role for AI in L&D, but a primary one is ensuring that there’s good content for an AI engine to operate on. Another, I argue, is to create the meaningful practice that AI currently can’t, and is likely true for the foreseeable future. I also have yet to see an AI that can create a diagram (tho’ that, to me, isn’t as far-fetched, depending on the input).

I have heard from colleagues who find the existing ChatGPT very valuable. However, they don’t take what it says as gospel, instead they use it as a thinking partner. That is, they’ll prompt it with thoughts they’re having to see what comes up. The goal is to get some lateral input to consider (not take as gospel). It’s a way to consider ideas they may have missed or not seen, which is a valuable role.

At this point, I may or may not use AI in this way, as a thinking (artificially) partner. I’ll have to experiment. One thing I can confidently assert is that everything you read (e.g. here) that is truly from me (i.e. there’s the possibility I will be faked ) will be truly from me. I’m immodest enough to think that my writing is not in need of artificial enhancement. I may be wrong, but that’s OK with me. I hope it is with you, too!

Meta-reflections

by Leave a Comment

Lake reflectionI was recently pinged about a new virtual world, a ‘metaverse‘ inspired new place for L&D. It looked like a lot of previous efforts! I admit I was underwhelmed, and I think sharing why might be worthwhile. So here are some meta-reflections.

I’ve written before on virtual worlds. In short, I think that when you need to be social and 3D, they make sense. At other times, there’s a lot of overhead for them to be useful that can be met in other ways. Further, to me, the metaverse really is just another virtual world. Your mileage may vary, of course.

This new virtual world had, like many others, the means to navigate in 3D, and to put information around. The demo they had was a virtual museum. Which, I presume, is a nice alternative to trying to get to a particular location. On the other hand, if it’s all digital, is this the best way to do it? Why navigate around in 3D? Why not treat it as an infographic, and work in 2D, leading people through the story? What did 3D add? Not much, that I could see.

My take has, and continues to be, as they say, “horses for courses”. That is, use the right tool for the job. I complained about watching a powerpoint presentation in Second Life (rightly so). Sure, I get that we tend to use new technologies in old ways first until we get on top of the new capabilities. However, I also argue that we can short-circuit this process if we look at core affordances.

The followup message was that this was the future of L&D, and we’d get away from slide decks and Zoom calls, and do it all in this virtual world. I deeply desire this not to be true! My take is that slide decks, Zoom, virtual worlds, and more all have a place. It’s a further instance of get the design right first, then figure out how to implement it. I want an ecosystem of resources.

Sure, I get that such a meta verse could be an integrating environment. However, do you really want to do all your work in a virtual world? Some things you can’t, I reckon, machining materials, for instance. Moreover, we have benefits from being out in the world. There are other issues as well. You might be better able to deal with diversity, etc, in a virtual world, but it’ll disadvantage some folks. Better, maybe, to address the structural problems rather than try to cover them over?

As always, my takeaway is use technology to implement better approaches, don’t meld your approaches to your tech. Those are, at least, my meta-reflections. What are yours?

Web 3.0 and whither the LMS

by 3 Comments

At the recent DevLearn conference, I was part of a Guild Master panel on emerging technologies. It featured notables such as Julie Dirksen, Mark Lassoff, Megan Torrence, Ron Price, Chad Udell, Karl Kapp, and Jane Bozarth, all hosted by Mark Britz. Not surprisingly, I guess, the topic went to the future of the LMS. In a session the next day, Dr. Jen Murphy of QIC talked insightfully (as she does) about the Metaverse, and compared it to Web 3.0. The conjunction of discussion prompted me to reflect on the intersection, considering Web 3.0 and whither the LMS.

To start, I’m not one proposing that the LMS should or will wither. I’ve suggested that courses make sense, particularly for novices. That said, they’re not full development plans. So it’s worth looking, and thinking, deeper. The conversation on the panel suggested the evolution of the LMS, and I think that’s an apt way to think about it.

What prompted this was Dr. Murphy’s comparison of Metaverse to Web 3.0. She argued that Web 3.0 was about user-control of content. That is, it’s about things like P2P, e.g. blockchain, NFT, etc. I’ve had a different view (now over a decade old, admittedly), that we’d moved from producer-generated content, through user-generated content, and the next would be system-generated content. AI can parse content (that people have painstakingly hand-crafted). Then systems can use models and rules to individualize the experience. That’s what web content is doing already.

So, have things changed? The recognition I see is that folks are concerned with identity and rights. Which I applaud, to be clear. The statement is that by having clear documentation, we can reward individual contributions instead of someone owning all the transactions. The latter of which would be part of a ‘system-generated’ web, for sure. Maybe my 3.0 is really 2.5? Or maybe theirs should be 4.0. Not sure I care…

What does matter is what that implies for courses. Obviously, if courses aren’t enough, we need a bigger picture. An associated question is who should own it? I see a development path as having many components. Even courses should be broken up for spacing, and have a follow-on for ongoing feedback whether digitally delivered and/or a coach. There was an LMS that actually allowed you to mix things into your paths: so you could interview someone, or read a book, or…other things besides courses. Made sense.

The other part aligns more closely with the user-controlled vision. I believe (and have stated, not that I can find it) that I think that ultimately, the community should own the path into membership. That is, just as we should determine the path into membership of L&D, a group in sales should determine what the necessary component skills are. They may need facilitation of this, but us ‘owning’ it isn’t right. We should merely be supporting the endeavor.

Again, it doesn’t really matter whether it’s labeled Web 3.0 or not, but I think that having a mechanism to track development, owned by the associated community (or communities) is useful. It’s not really a Learning Management System (you can’t really ‘manage’ learning), but it can include courses, and it is worthwhile. So those are my thoughts on Web 3.0 and whither the LMS, what’re yours?

Kaku #DevLearn Keynote Mindmap

by Leave a Comment

Michio Kaku opened the second day of DevLearn with a keynote on the future of the mind. He portrayed extrapolations of current research to some speculative ideas of what our future could mean. He talked about research from physics (?!?) on MRI, AI, and more to provide new capabilities.

Fewer myths, please

by 5 Comments

I had the pleasure of being the opening keynote at the People Matters L&D conference in Mumbai this past week, with a theme of ‘disruption’. In it, I talked about some particular myths and their relation to our understanding of our own brains. Following my presentation, I sat through some other presentations. And heard at least one other myth being used to flog solutions. So, fewer myths, please.

My presentation focused on the evidence that we’re still operating under the assumption that we’re logical reasoners (which I pointed out, isn’t apt). I mentioned annual reviews, bullet points presos, unilateral decisions, and more. I also cited evidence that L&D isn’t doing well, so it is a worry. Pointing to post-cognitive frameworks like predictive coding, situated & distributed cognition, and more, I argued that we need to update our practices. I closed by urging two major disruptions: measurement, and implementing a learning culture in L&D before taking it out to the broader org.

In a subsequent presentation, however, the presenter (from a sponsoring org) was touting how leadership needed to accommodate millennials. I’m sorry, but there’s considerable evidence that ‘generation differences’ are a myth. The boundaries are arbitrary, there’re no significant differences in workplace values, and every effect is attributable to age and experience, not generation. (Wish I could find a link to the ‘eulogy for millennials myth’ two academics wrote.)

Another talk presented a lot of data, but ultimately seemed to be about supporting user preferences. Sorry, but user preferences, particularly for novices, aren’t a good guide. There was also a pitch for an ‘all-singing, all-dancing’ solution. Which could be appealing, if you’re willing to live with the tradeoffs. For instance, locking into whatever features your provider is willing to develop, and living without best-0f-breed for all components.

Yes, it’s marketing hype. However, marketing hype should be based on reality, not myths. I can get promising a bit more than you can deliver, and focusing on features you’re strong on. I can’t see telling people things that aren’t true. My first step in dealing with the post-cognitive brain is to know the cognitive and learning sciences, so you’ll know what’s plausible and what’s not. Not to PhD depth, but to have a working knowledge. That’s the jumping off point to much that’s the necessary disruption, revolution, that L&D needs to have. And fewer myths, please!