Archives for May 2025

What sorts of activities?

27 May 2025 by Clark Leave a Comment

When we do learning, we must be active. That is, it’s not enough to receive information. (Unless we’re actively practicing and attending presentations are reflection.) We must do! Then the question becomes one of doing ‘what’? I’m seeing too many of the wrong sorts of things in play, so it’s worth asking: what sorts of activities should we be doing?

Cognitively, we need to perceive information to get it into working memory. From there, to get into long-term memory – and be useful – we need to elaborate and practice retrieval. Elaboration is the process whereby we strengthen connections between the new material and the familiar. This increases the likelihood of activation in context. Then, we need to practice retrieving the knowledge for use. This strengthens our ability to retrieve and apply as we need.

One thing to note is that research shows that we don’t need to retrieve the fact-based knowledge before practicing retrieval to actually use. Our goal for organizational learning is to use information to make meaningful decisions. Better fact-recall isn’t likely to be what will help your organization thrive. Instead, what matters is acquiring the new skills that will define the ability to adapt.

For elaboration, what we increasingly hear is about ‘generative‘ learning activities. These are when you’re taking new information, and processing it more deeply. It can involve rephrasing, visualizing, and of course connecting it to your prior experience. These activities help strengthen the information into long-term memory.

An associated task it to practice using the information. That is, putting learners into situations where they need to use the new information to make decisions that they couldn’t before. The ideal situation, of course, is mentored live practice, but…there are limitations. Individual mentoring isn’t always cost-effective. Also, live practice may have consequences for wrong answers. In many cases, we use simulations. These can be programmed, or branching scenarios. Even mini-scenarios (e.g. better written multiple-choice questions) are a good option.

What we don’t need are fact-check questions. As above, there’s no real benefit. They may make us feel good, but they aren’t inclined to make us better at using the information. There are lots of bad practices around this. We can just use knowledge questions, thinking we’re helping learning (and not). Worse, I’ve seen many cases where they’re asking for arbitrary bits of information that aren’t highlighted. Also, too often we’re presenting way too much information than people can remember at one time (or at all).

So, if we’re to design effective learning, what sorts of activities is an important question. We don’t need fact-checks. We do benefit from processing, and retrieval. That’s worth practicing and performing. Review your work and look at what you’re having learners do. If it’s not elaboration and retrieval, you’re wasting learners’ time and your efforts. Why do that?

Software engineer vs programmer

20 May 2025 by Clark Leave a Comment

If you go online, you’ll find many articles that talk about the difference in roles between software engineers and programmers. In short, the former have formal training and background. And, at least in this day and age, oversee coding from a more holistic perspective. Programmers, on the other hand, do just that, make code. Now, I served in a school of computer science for a wonderful period of my life. Granted, my role was teaching interface design (and researching ed tech). Still, I had exposure to both sides. My distinction between software engineer vs programmer, however, is much more visceral.

Early in my consulting career, I was asked to partner with a company to develop learning. The topic was project management for non-project managers. They chose me because of my game design experience as well as learning science background, The company that contracted me was largely focused on visual design. For instance, the owner also was teaching classes on that. Moreover, their most recent project was a book on the fauna of a fictitious world in the Star Wars universe (with illustrations). He also had a team of folks back in India. Our solution was a linear scenario, quite visual, set in outer space both because of experience of their team and the audience of engineers.

After the success of the project, the client came back and asked for a game to accompany the learning experience. Hey, no problem, it’s not like we’ve already addressed the learning objectives or anything! Still, I like games! This was going to be fun. So I dug in, cobbling together a game design. We used the same characters from the previous experience, but now focused on making project management decisions and dealing with different personality types (the subtext was, don’t be a difficult person to work with).

The core mechanic was:

choose the next project
assess any problem
find the responsible person,
ask (appropriately) for the fix

Of course, the various rates of problems, stage of development and therefore person, stage and scope of the project, were all going to need tuning. In addition, we wanted the first n problems to deal with good people, to master the details, before beginning to deal with more difficult personality types.

So, from my development docs, they hired a flash programmer to build the game. And…when we tried to iterate, we got more bugs instead of improvement. This happened twice. I realized the coders were hard-wiring the parameters throughout the code, which meant that if you wanted to tune a value, they had to search throughout the code to change all the values. Now, for those who know, this is incredibly bad programming. It wasn’t untoward to develop a small Flash animation, but it didn’t scale to a full game program.

We had a discussion, and they finally procured someone who actually understood the use of constants, someone with more than just a programming background. Suddenly, tweaks were returning with short-turnaround, and we could tune the experience! Thus, we were able to create a game that actually was fun. We didn’t really get to know whether it was effective, because they hadn’t set any metrics for impact, but they were happy and touted the game in several venues. We took that as a positive outcome ;).

The take-home lesson, of course, is if you need tuning (and, for anything of sufficient size and user-facing, you will), you need someone who understands proper code structures. I’ll always ask for someone who understands software engineering, not just a programmer. There’s a reason that a) they’re known as ‘cowboy coders’, and b) there’s software process! That’s my personal definition of a software engineer vs programmer, and I realize it’s out of date in this era of increasingly complex software. Still, the value of structure and process isn’t restricted to software, and is ever more important, eh?

The illusion of (my) competency

13 May 2025 by Clark 2 Comments

I have a cobbled together tech environment. My monitor and printer are years old, and my laptop is relatively new. To make them fit together, particularly with the limited ports of a laptop, I’ve a hub. And, now a few weeks ago, things went wonky. In weird ways. And I still don’t know it all, but there are some lessons here, not least the illusion of (my) competency. We all have travails, but this one has some lessons.

So, the first symptom was my microphone suddenly not working. I’d gotten a reasonable one, but it died (and apparently wasn’t good enough anyway), so my co-director in the LDA, Matt Richter, got me a nice one. It was this fancy, heavy one, that stopped working. It’s USB, so has LEDs, which are blue normally but green when in use. Except what happened was that when I wanted to use it, it started not working and blinking between blue and green. Hold that thought.

Then, the camera in my monitor stopped working. I have one in my laptop that’s better/newer, but it’s a bit away (I prefer to work on the big monitor, of course). I use the laptop camera when I want to look impressive and show my book case beside me ;). But I prefer the monitor camera for work meetings and the like. Yet, I couldn’t. Now hold that thought, too!

The icing on the proverbial cake came when my backup drive stopped backing up. It would start, and say it was finding probs, but then would time out with an error. I tried to run diagnostics, which said it was corrupt. I also got SMART results saying it was fine. All very confused.

Finally, I decided I had to contact Apple (fortunately Matt’s also insisted I keep up my service plan). After a lot of shenanigans that I won’t bore you with, the question came: is my hub powered? As it was, the answer’s no. If memory serves (dodgy proposition), I got it for free when Amazon was experimenting with providing things to people who wrote reviews. Which would be a weird issue, as I’d been using it for several years this way.

Still, went and ordered a powered hub. Then ordered another, realizing I wanted the faster version. BTW, I had to pick up both, because the other was fulfilled before I got the second, and the solution seemed to be that I picked up both and returned the first. Which took an extra 5 minutes, apparently to defend against fraud. Why we can’t have nice things…(Lesson: allow people to cancel orders via chat or online, don’t make them come in, take and immediately return).

While I was dealing with the hub, I decided to call the maker of the microphone (remember?). I’d called them before and left a message, and also emailed, to no reply. (Lesson: return your customer queries.) I was leaving a voicemail when I got a call from them, so I switched. It turns out that the blinking means the mic’s muted. There’s a knob on the back that’s ‘gain’, BUT it turns out it’s also a button, and if you push it, it mutes the mic. Now, I’d gone to the site and the mic instructions, and it doesn’t talk about that at all! I mentioned it to the person on the phone, and they said that they’d tried to get it changed, to no avail. (Lesson: put all the information about operation in the <expletive deleted> documents!) Problem fixed.

Then, when I installed the powered hub, the video camera in the monitor started working again. It’s not clear why it suddenly stopped, but…it was fixed. The lesson here, and for me (tho’ feel free to take it to heart), is probably never to trust an unpowered hub to be sufficient. However, and in my defense, it had been working for months if not years. (And, if you could’ve told me that, I don’t want to hear it.)

Finally, we get back to the drive. The powered hub didn’t fix it. I spent an unreasonable amount of time trying to run diagnostics, both Apple’s and the manufacturers. Finally, I delisted it from the backup software, erased it, then reintroduced it. And, voila’, it’s working! Not sure what the problem was. (Lesson: provide more feedback to the user on what’s going on.)

As a weird aside, I asked for a support call, but when it came it shut off my wifi calling. I live in a slight depression and have a bad signal, so I use wifi. But by turning it off, they ensured that the call would get dropped. I can’t imagine why they would do such a thing, but I’ve had real trouble getting calls from them, and this time just happened to see that the wifi calling had stopped at the time of the call. I had to use chat. Very puzzling and unresolved.

All told, this took way too much time, and while I learned one lesson, there was too much the result of bad design, not incompetence. As Don Norman said in The Design of Everyday Things, if it’s difficult to use, blame the designer, not the user. We know how to do better, we just don’t do it frequently enough. (I also recommend Kathy Sierra’s Badass as a guiding light.) I’m willing to assume responsibility for my culpability, and admit to the illusion of (my) competency. But I also recognize that I’m not stupid, and better design would’ve limited my frustrations and time wasting.

Locus of intelligence

6 May 2025 by Clark 1 Comment

I’m not a curmudgeon, or even anti-AI (artificial intelligence). To the contrary! Yet, I find myself in a bit of a rebellion in this ‘generative‘ AI era. And I’m wondering why. The hype, of course, bugs me. But it occurs to me that a core problem may reside in where we put the locus of intelligence. Let me try to make it clear.

In the early days of the computer (even before my time!), the commands were to load memory into registers, conduct boolean operations on them, and to display the results. The commands to do so were at the machine level. We went a level above with a translation of that machine instructions into somewhat more comprehensible terms, assembly language. As we went along, we went more and more to putting the onus on the machine. This was because we had more processor cycles, better software etc. We’re largely to the point where we can stipulate what we want, and the machine will code it!

There are limits. When Apple released the Newton, they tried to put the onus on the machine to read human writing. In short, it didn’t work. Palm’s Pilots succeeded because Jeff Hawkins went for Graffiti as the language, which shared the responsibility between person and processor. Nowadays we can do speech and text recognition, but there are still limitations. Yes, we have made advances in technology, but some of it’s done by distributing to non-local machines, and there are still instances where it fails.

I think of this when I think of prompt engineering. We’ve trained LLMs with vast quantities of information. But, to get it out, you have to ask in the right way! Which seems like a case of having us adapt to the system instead of vice versa. You have to give them heaps more context than a person would need, and they still can hallucinate.

I’m reminded of a fictional exchange I recently read (of course I can’t find it now), where the AI user is being advised to know the domain before asking the AI. When the user queries why they would need the AI if they know the domain, they’re told they’re training the AI!

As people investigate AI usage, one of the results is that your initial intelligence indicates how much use you’ll get out of this version of AI. If you’re already a critical thinker, it’s a good augment. If you’re not, it doesn’t help (and may hinder).

Sure, I have problems with the business models (much not being accounted for: environmental cost, IP licensing, security, VC boosting). But I’m more worried about people depending too much on these systems without truly understanding what the limitations are. The responsible folks I know advocating for AI always suggest having a person in the loop. Which is problematic if you’re giving such systems agency; it’ll be too late if they do something wrong!

I think experimenting is fine. I think it’s also still too early to place a bet on a long-term relationship with any provider. I’m seeing more and more AI tools, e.g. content recommenders, simulation avatars, and the like. Like with the LMS, when anyone who could program a database would build one, I’m seeing everyone wanting to get in on the goldrush. I fear that many will end up losing their shirts. Which is, I suppose, the way of the world.

I continue to be a big fan of augmenting ourselves with technology. I still think we need to consider AI a tool, not a partner. It’s nowhere near being our intellectual equal. It may know more, but it still has limitations overall. I want to develop, and celebrate our intelligence. I laud our partnership with technologies that augment what we do well with what we don’t. It’s why mobile became so big, why AI has already been beneficial, and why generative AI will find its place. It’s just that we can’t allow the hype to blind us to the real locus of intelligence: us.