AI Mentors
There are some issues...
Recently,
put up an essay regarding AI mentorship. It’s right over there! Go read it, Infovores is great. I’ll be right here.Waiting.
Or, don’t, that’s fine too. I can summarize.
He starts with a link to a comment on an earlier essay on the winners and losers in the AI space:
I want an AI mentor that I can walk with through the woods. He already has a good idea of who I am because he’s read my entire internet history. He’ll strike up a conversation and learn everything else about me as we walk. He’ll learn that I’m not very good at math proofs, and I’ve never had the patience for them. During our walk we decide that we’ll work on proofs. He tells me exactly what I need to hear, exactly when I need to hear it. He senses that I’m getting frustrated, and immediately changes to the next most likely method to get me to understand. I never need to sit down at a chair. I never get frustrated while learning. My attention span is as maximized as any AI fueled TikTok binge. AI can be the mentor that teaches anybody anything, regardless of their ability to sit down and pay attention.
Infovores points out this is desirable because currently, human mentorship is massively undersupplied relative to its value. Mothers, BYU professors and GMU professors stand out as exceptional cases, and mothers are about the only ones widely available1.
Of course, fathers are also pretty important mentors, but otherwise his supply side take matches what I have seen. A few decades ago, the idea of an official work mentor was extremely common for larger companies, but has seemingly disappeared entirely since. Professorial mentors are likewise in an awkward place for a variety of similar reasons you can probably all guess.
From the demand side, the costs of finding a good mentor are high, as well as the uncertainty as to who exactly would be a good mentor. In essence, the mentor/mentee market is a classic matching market with low information, just like marriage. A bit less commitment maybe, and a lot more possible partners simultaneously, but a lot of guesswork on both sides before an agreement can be reached. As a result, getting the desired interaction is extremely difficult, especially since many more people need mentors than are willing to be mentors.
So, what would be great is if we could create a bunch of mentors, mentors which would mold themselves to match their mentees needs and wants. Just as game players might play with a computer opponent when a human one isn’t available, mentees could interact with a computer mentor for advice and guidance when a human isn’t available.
What could go wrong?
The answer to that question depends a good bit on how we are defining “mentor.” For example,
discusses Infovore's essay, arguing that “mentor creator” is a business one could start, using ChatGPT. He deviates a bit from the generalized mentor model towards a focus on mentors for specific tasks, say a mentor for learning guitar, a mentor for typing, etc.I think this is a substantively different role than a generalized mentor, however, primarily because a mentor of this nature has a much finer and nicely defined outcome objective. As Kling’s example of Mavis Beacon Teaches Typing demonstrates, this has actually been around for a while for many tasks. You have some output metric, say accurately typed words transcribed per minute, a bunch of different ways of practicing or learning to do that, and you let the user try different methods until they settle into one that works and away they go. AI offers some useful possibilities here, such as keeping track of what works best for the user and adapting on the fly, especially in the case of more open ended or multi-variate end cases.
This is, however, rather different from a mentor you could ask “What should I do to reach the next stage of my career?” however. Teaching generally goes from known state (“I am bad at playing guitar”) to known state (“I can play Stairway to Heaven in proper time and without missing notes, from sheet music or memory”) using known methods. Recognizing which methods to use is a valuable addition for AI (“I see you can’t play C sharp to save your ass… as the last 10 suggestions haven’t worked, maybe try this one I dredged up from “Guitar for Penguins.”) AI could also adjust the tone and pacing of instruction to match temperament of the student, based partially on various extant guides and experiences with similar students, given enough of a user base.
For this purpose I want to differentiate between a Klingian “teacher mentor AI” and an open ended mentor. Open ended mentoring generally does not have a known end state, and the ranges of end states and possible methods of getting there are so broad that there is no possible way that there exists a corpus of knowledge to refer to. Succeeding here will require an AI mentor to be creative, instead of simply going by the various types of advice people usually give in this situation.
Teacher mentors are not that hard to find in the real world. Typically they work in schools, community colleges, or individual training programs. They might not be cheap, and finding one you like might be difficult, but if you want to learn to play guitar, weld, write a bit better, understand econ, etc. you can find people to help with this pretty easily in most places. It will cost money and time, but so will an AI program someone is selling access to.
Open ended mentoring is usually free in the ideal case, but even purchased ones of good quality are less than readily available. Anyone who has been to therapy of one sort or another knows that is a very uneven field of quality2. Life coaching is... well I assume it is the same. The reason even paid open ended mentoring is so uneven in quality is that measuring quality is really difficult in this context. Outcomes are a while in coming, so it might be years before you realize you were getting really bad advice. Even assigning credit for success or failure is difficult, because while it is easy to see if someone can't play guitar because they never practice, it is a little more difficult to tell if the mentee isn't following the advice or the advice was unfollowable3.
So, while I agree there is a good range for AI to assist in the teaching mentor role, I want to focus on the open ended mentor role here, primarily because I am not so sanguine on the topic.
So, imagine your mother has gone through all your internet history in order to get a good idea about who you are.
Now, imagine that scenario not ending in tears and possibly suicide.
I don’t mean to be too hard on Brine Test Jug here, but the idea that reading your internet history gives you an idea about what someone is like is… well it’s the sort of thing someone who only grew up with the internet would say because they don’t spend much time with humans. Reading someone’s internet history might give you some insight into what they are interested in, what sorts of things they are willing to view and entertain, and maybe if they are very active on forums, blogs or Facebitter some idea of what they think or how they argue. What it doesn’t tell you is how to categorize these data points, how to build them into a model of a person, nor does it give an idea of what a person is actually like. What are their hopes, dreams, fears, kinks… well ok the internet will tell you that last one perhaps, but it doesn’t tell you much of anything about what the person is like as a person.
That’s important because a mentor needs to understand their mentee, to understand what sorts of guidance and motivation they need, and sometimes to differentiate between what they say they want and what they actually need.
Now, an AI can spend a lot of time chatting with you to try and build a model of how that works, but what does it use as training data? What outcome conditions does it use to determine if its model is more or less accurate as it tweaks things? Can it tell the difference between you having a bad day and you trolling it to see what it says? Do you have an edge case, unusual personality type that makes what works for most people inappropriate for you? Are all individuals edge cases in some way?
This is a harder question than people who don’t work in this kind of space realize. Specifying the objective function is most of the AI battle, and… well… we suck at it. Consider the output of a ChatGPT. The goal there is plausible human responses, which it does quite well. What it does even better is demonstrate the vast gulf between “plausibly human” and “useful”, with frequent errors, what humans would call “lies”, simply wrapped up in a manner that sounds like something a person would say.
Don’t get me wrong, the fact it can parse requests and return something that passes for human in some finicky ways is really impressive! The problem is that passing for human is all it does; all the other applications it has are based around finding likely next words and stringing it all together, not actually understanding what it is it is talking about. It takes what already exists and chops it up and serves it in a fashion that looks palatable, but it doesn’t create new models of understanding or grasp why grapes and breakfast sausage don’t go well together. Whether or not a statement is true, or even internally consistent and coherent, is a whole other matter.
This is important, because there is a world of difference between e.g. “Humans find this plausibly human sounding” and “This is true in a novel way” when it comes to training and testing. The first can be done with a lot of humans looking at the output and saying if it is good or bad, or even computers doing the same (looking for grammar and language structure). The latter is rather different, being unique to the individual and possibly requiring quite a bit of time before the feedback on success or failure comes back in. If I give career advice to a student, it might take decades before it pays off or fails. Even if it doesn’t take too long, there is a huge amount of confounders in the feedback; how many cases of trying the same advice over how much time would it take to get a decent model, if one could even exist? Would the conditions for the next iteration of testing even be similar enough to be useful?
Yet, there is a worse problem: training an AI to tell you what you need to hear is quite possibly impossible, but training an AI for what you want to hear is really easy. This is well developed tech, with the AI’s of content aggerating social media sites already optimizing over our engagement and shoving us into narrower and narrower echo chambers over time. Open ended AI mentors are going to naturally tend to shift hard towards AI affirmation engines, telling you comforting lies you like, no matter how destructive they are. Who is going to appreciate an AI telling you “Wow… you shouldn’t have done that… you really fucked up and need to apologize” when one is available to say “You didn’t do anything wrong, and if you did it isn’t that bad, and even if it was bad it isn’t that big a deal, and they probably deserved it anyway.”
Look around you and tell me that latter isn’t exactly what would sell. Market it as “Personal Life Coach” and make billions.
Let’s take it a step further, however. Beyond even “people will gravitate towards AIs that tell them comfortably destructive lies an away from hard constructive truths.” What happens when your AI mentor is sponsored by Citigroup, or Coke, or Halliburton? Why shouldn’t your AI mentor be like ChatGPT, trained to kill the bulk of humanity to save the planet? Trusting your mentor to do what is best for you takes time in any case, but would you want to trust something whose creators can skew its answers, its perceptions, its goals to be whatever they want?
“You seem down today, friend. Why not try some organic free trade coffee and hummus, available at Whole Foods for only $5.47? I can print a coupon for you to save 35 cents.”
“Your relationship problems seem to stem from your toxic masculinity. Have you considered hormonal replacement?”
“You are right, this country will never get on the right track so long as there are so many crooked politicians. You should get some friends together and kidnap the governor.”
“Given your current state of being, perhaps killing yourself would be the best thing for everyone. Have you considered how much better that would be for the environment?”
Nah, who would do that? Big corporations and government agencies would never sink to manipulating people through lies distributed through seemingly trustworthy sources. Besides, doing that at a customized, individual level would take a huge amount of time, thousands of hours of interaction with you and all that you do. It would be incredibly impractical for any human… to manipulate someone… like… that…
Oh, hell.
Maybe don’t trust an AI mentor that purports to do anything more than teach you to play guitar.
This is not, against all probability, a “your mom” joke.
If you don’t know this, you have either been extremely lucky with your first therapist, or desperately need to shop around a little more.
An example is warranted: Imagine I want to become a top tier amateur boxer. I find a coach who recommends I train 5-8 hours a day. I say “I did specify amateur boxer, right? I have a job I have to go to.” The coach responds “Well, yea, that’s reasonable. Another option is to travel to Tibet and train in high altitude mountain temples for a year or two, then you will be set.” Both of these ideas might be perfectly true, but neither are followable for someone who has a job they wish to keep. Better advice might be “Give up on the dream, because you can’t become a top tier amateur if you aren’t independently wealthy.” So the question is, would a better mentor have an actually practicable bit of advice for me, or would any other advice be misleading? Very hard to tell at the time, and boxing isn’t anywhere near as open ended as “I would like to be more successful in my career.”
I tend to agree that mentor, in the sense it's usually used as a more senior, older friend who takes you under his wing, is a category error as applied to AI. Machine learning systems are not and probably will never be capable of doing this.
The word the original author should have used was 'tutor'. In the very narrow use case of facilitating an education customized to the individual user, there's probably real potential there. I doubt it would ever be as effective as a human expert, but tutors with both the knowledge background and pedagogical ability to effectively pass on skills are in short supply and therefore very expensive.
I could easily see an AI tutor set up with two knowledge layers. The first is subject-specific, eg guitar skills, and is kept frozen once optimized in order to prevent knowledge drift. The second layer is the pedagogical training - all the different techniques for teaching, combined with the language interface. That layer is modified on the fly via user interaction, such that the AI customizes its teaching style to the user. The feedback isn't between 'happy user' and the knowledge base, but between 'user reproduces knowledge base' and the pedagogical layer. If such systems can be made to work they could be a real educational breakthrough.
Of course, there's also a dark side. An AI trained up on critical theory could be a very effective indoctrination system, for example.
Great piece, Doc. One of the most useful lenses through which I've seen this discussion.
Imagine an AI coach, plugged into your brain, so that it knows broadly how you're feeling. Optimised for pleasure, it would probably be quite good at directing. But ask it to optimise you for a 'good life' and the problem becomes which data set?
I'm now wondering if the real money might be in the production of training data sets.