我的征尘是星辰大海。。。
The dirt and dust from my pilgrimage forms oceans of stars...
-------当记忆的篇章变得零碎，当追忆的图片变得模糊，我们只能求助于数字存储的永恒的回忆
作者:黄教授
手机视频列表
大语言模型实际上是记忆模型
视频
音频
原始脚本
大语言模型的记忆逻辑像人一样分长短，藏着技术落地的底层密码。 很多人聊大语言模型，LLM，总觉得它是会说话的计算器，靠复杂算法堆砌出智能。 但当我们跳出参数多少、算力强弱的技术细节，会发现一个更贴近人类认知的真相。 大语言模型的核心能力，本质是对人类记忆模式的粗糙却高效的模拟。 预训练阶段对应长期记忆的沉淀，推理阶段对应短期记忆的调用，两者的联动逻辑和我们每天用过往 经验应对当下问题的过程，惊人的相似。 这种模拟不是刻意设计的巧合，而是技术为解决记忆存储与记忆调用矛盾找到的最优解。 甚至能帮我们重新理解为什么有些 AI 能举一反三，有些却只会机械重复。 背后都藏着长短记忆协同的底层逻辑。 一，预训练。 像人积累经验一样，把数据链成长记忆。 如果把大语言模型比作一个人，预训练阶段就是他从零开始学世界的过程，不是死记硬背每一本书每一句话，而是像我们从小到大积累经验那样，把海量文本数 据提炼成可复用的长期记忆。 这个过程最关键的是泛化而非复刻。 人类不会记住十年前某顿饭的每一粒米，但会记住吃饭能饱腹，热饭不能直接用手碰的规律。 大语言模型也不会记住训练数据里的每一个句子，而是通过学习亿万文本的语法逻辑常识，把猫是哺乳动物，下雨要打伞这类规律。 以参数权重的形式刻进模 模型里，这些权重不是零散的知识点，而是相互关联的记忆网络。 就像我们的长期记忆里，猫会和爪子、吃鱼、毛茸茸自动绑定。 大语言模型的参数里，国王也会和王冠、城堡形成关联权重。 这种长期记忆解决了两个核心问题。 一是存储效率，人类大脑没法记住每天的所有细节，大语言模型也没法存储亿万文本的原始数据，泛化提炼让它用有限参数承载了海量知识。 二是迁移能力，就像我们能把骑自行车的经验迁移到骑电动车，大语言模型也能把预训练学到的语言规律迁移到写文案、做翻译、答问题等不同任务中。 这正是长期记忆的价值，不局限于某一个场景，却能支撑所有场景。 二、推理像 人应对当下一样，用短记忆唤醒长记忆。 当我们和大语言模型对话时，它的思考过程其实是短期记忆调用长期记忆的过程。 就像我们看到眼前的雨，短期感知会立刻想起要打伞，长期记忆，大语言模型也会根据你当下的提问，短期上下文。 从预训练的长期记忆里调取相关知识来回应。 这里的短期记忆就是大语言模型的上下文窗口，比如 GPT 4的128K窗口。 它不是用来存储新知识的，而是用来临时承接当下任务的。 你问猫和狗哪个更粘人，模型会先把猫狗粘人这几个关键信息放进上下文窗口，再通过窗口里的信明星 去激活预训练时学到的猫的习性、狗的习性等长期记忆，最后整合出答案。 这个过程和人类的记忆调用高度相似。 我们聊天时不会把一辈子的记忆都翻出来，只会根据当下的话题、短期注意力唤醒相关的长期记忆。 大语言模型也 也不会把所有参数权重都重新计算一遍，只会根据上下文窗口里的信息，聚焦到相关的权重区域。 而且和人类一样，大语言模型的短期记忆也有局限，窗口满了就会忘事，截断早期对话，就像我们没法同时记住十件事的细节。 但也正因为有了短 短期记忆，它才能避免用所有长期记忆应对一个小问题的低效，让推理更聚焦、更快。 三，长短记忆协同，不是模仿人，而是不得不像人。 有人会问，大语言模型的长短记忆是刻意模仿人类吗？其实不是，它更像是技术在存 存储与效率之间找到的和人类记忆殊途同归的解决方案。 人类的记忆之所以分长短，是因为短期记忆的容量有限，必须靠长期记忆的泛化提炼来承载更多知识。 大语言模型之所以分预训练长记忆和推理短记忆，也是因为如果不做预训练，每次推理都要重新学所有数据，效率低到无法落地。 就像我们没法每次遇到问题都重新活一遍，大语言模型也没法每次回答都重新训练一遍。 长短记忆的协同是两者共同的最优解。 四、结语。 理解记忆逻辑，才懂 AI 的现在与未来。 很多人觉得 AI 的核心是算法，但从长短记忆的角度看，AI 的核心其实是如何像人一样高效的存储和调用知识 大语言模型的进步本质是让长记忆更泛化，能学更多领域的规律。 短记忆更精准，能更好承接当下任务。 两者联动更流畅，能更快从记忆里找答案。 而对我们普通人来说，理解这个记忆逻辑不只是搞懂一个技术概念，它能帮我们更清晰的判断 AI 能做什么。 不能做什么？能做的是用长期记忆解决有规律的问题，比如写文案、做翻译。 不能做的是像人类一样有私人记忆，比如记住你的专属喜好。 更重要的是，它能让我们明白，AI 的发展不是超越人类，而是在用机器的方式。 走人类认知走过的路，而记忆正是这条路上最基础，也最核心的一块基石。
修正脚本
大语言模型的记忆逻辑像人一样分长短，藏着技术落地的底层密码。 很多人聊大语言模型，LLM，总觉得它是会说话的计算器，靠复杂算法堆砌出智能。 但当我们跳出参数多少、算力强弱的技术细节，会发现一个更贴近人类认知的真相。 大语言模型的核心能力，本质是对人类记忆模式的粗糙却高效的模拟。 预训练阶段对应长期记忆的沉淀，推理阶段对应短期记忆的调用，两者的联动逻辑和我们每天用过往经验应对当下问题的过程，惊人地相似。 这种模拟不是刻意设计的巧合，而是技术为解决记忆存储与记忆调用矛盾找到的最优解。 甚至能帮我们重新理解为什么有些 AI 能举一反三，有些却只会机械重复。 背后都藏着长短记忆协同的底层逻辑。 一，预训练。 像人积累经验一样，把数据链成长记忆。 如果把大语言模型比作一个人，预训练阶段就是他从零开始学世界的过程，不是死记硬背每一本书每一句话，而是像我们从小到大积累经验那样，把海量文本数据提炼成可复用的长期记忆。 这个过程最关键的是泛化而非复刻。 人类不会记住十年前某顿饭的每一粒米，但会记住吃饭能饱腹，热饭不能直接用手碰的规律。 大语言模型也不会记住训练数据里的每一个句子，而是通过学习亿万文本的语法逻辑常识，把猫是哺乳动物、下雨要打伞这类规律，以参数权重的形式刻进模型里，这些权重不是零散的知识点，而是相互关联的记忆网络。 就像我们的长期记忆里，猫会和爪子、吃鱼、毛茸茸自动绑定。 大语言模型的参数里，国王也会和王冠、城堡形成关联权重。 这种长期记忆解决了两个核心问题。 一是存储效率，人类大脑没法记住每天的所有细节，大语言模型也没法存储亿万文本的原始数据，泛化提炼让它用有限参数承载了海量知识。 二是迁移能力，就像我们能把骑自行车的经验迁移到骑电动车，大语言模型也能把预训练学到的语言规律迁移到写文案、做翻译、答问题等不同任务中。 这正是长期记忆的价值，不局限于某一个场景，却能支撑所有场景。 二、推理像人应对当下一样，用短记忆唤醒长记忆。 当我们和大语言模型对话时，它的思考过程其实是短期记忆调用长期记忆的过程。 就像我们看到眼前的雨，短期感知会立刻激活长期记忆想起要打伞，大语言模型也会根据你当下的提问、短期上下文，从预训练的长期记忆里调取相关知识来回应。 这里的短期记忆就是大语言模型的上下文窗口，比如 GPT 4的128K窗口。 它不是用来存储新知识的，而是用来临时承接当下任务的。 你问猫和狗哪个更粘人，模型会先把猫狗粘人这几个关键信息放进上下文窗口，再通过窗口里的信息去激活预训练时学到的猫的习性、狗的习性等长期记忆，最后整合出答案。 这个过程和人类的记忆调用高度相似。 我们聊天时不会把一辈子的记忆都翻出来，只会根据当下的话题、短期注意力唤醒相关的长期记忆。 大语言模型也不会把所有参数权重都重新计算一遍，只会根据上下文窗口里的信息，聚焦到相关的权重区域。 而且和人类一样，大语言模型的短期记忆也有局限，窗口满了就会忘事，截断早期对话，就像我们没法同时记住十件事的细节。 但也正因为有了短期记忆，它才能避免用所有长期记忆应对一个小问题的低效，让推理更聚焦、更快。 三，长短记忆协同，不是模仿人，而是不得不像人。 有人会问，大语言模型的长短记忆是刻意模仿人类吗？其实不是，它更像是技术在存储与效率之间找到的和人类记忆殊途同归的解决方案。 人类的记忆之所以分长短，是因为短期记忆的容量有限，必须靠长期记忆的泛化提炼来承载更多知识。 大语言模型之所以分预训练长记忆和推理短记忆，也是因为如果不做预训练，每次推理都要重新学所有数据，效率低到无法落地。 就像我们没法每次遇到问题都重新活一遍，大语言模型也没法每次回答都重新训练一遍。 长短记忆的协同是两者共同的最优解。 四、结语。 理解记忆逻辑，才懂 AI 的现在与未来。 很多人觉得 AI 的核心是算法，但从长短记忆的角度看，AI 的核心其实是如何像人一样高效地存储和调用知识，大语言模型的进步本质是让长记忆更泛化，能学更多领域的规律。 短记忆更精准，能更好承接当下任务。 两者联动更流畅，能更快从记忆里找答案。 而对我们普通人来说，理解这个记忆逻辑不只是搞懂一个技术概念，它能帮我们更清晰地判断 AI 能做什么，不能做什么：能做的是用长期记忆解决有规律的问题，比如写文案、做翻译。 不能做的是像人类一样有私人记忆，比如记住你的专属喜好。 更重要的是，它能让我们明白，AI 的发展不是超越人类，而是在用机器的方式，走人类认知走过的路，而记忆正是这条路上最基础，也最核心的一块基石。
英文翻译
The memory logic of large language models is divided into long-term and short-term, just like humans, hiding the underlying secrets of technology implementation. Many people talk about large language models, LLMs, and think they are just calculators that can talk, piecing together intelligence through complex algorithms. But when we step back from technical details like parameter counts and computing power, we discover a truth closer to human cognition. The core capability of large language models is essentially a rough but efficient simulation of human memory patterns. The pre-training phase corresponds to the deposition of long-term memory, and the inference phase corresponds to the recall of short-term memory. The interplay between the two is strikingly similar to how we use past experience to deal with current problems every day. This simulation is not a coincidence of deliberate design, but the optimal solution technology has found to resolve the conflict between memory storage and memory retrieval. It can even help us rethink why some AIs can draw inferences from one instance, while others can only mechanically repeat. Behind it all lies the underlying logic of long- and short-term memory coordination. First, pre-training. Just like humans accumulate experience, data is woven into long-term memory. If we compare a large language model to a person, the pre-training phase is its process of learning the world from scratch. It doesn't memorize every book or sentence by rote, but like how we accumulate experience from childhood, it distills massive amounts of text data into reusable long-term memory. The key to this process is generalization, not replication. Humans don't remember every grain of rice from a meal ten years ago, but they remember the rule that eating fills the stomach and hot food shouldn't be touched directly with hands. Similarly, large language models don't remember every sentence in the training data. Instead, by learning the grammatical logic and common sense from billions of texts, they carve patterns like "cats are mammals" and "use an umbrella in the rain" into the model in the form of parameter weights. These weights are not isolated pieces of knowledge, but an interconnected memory network. Just as in our long-term memory, "cat" is automatically linked to "claws," "eating fish," and "furry," in a large language model's parameters, "king" is also associated with "crown" and "castle" with weighted connections. This long-term memory solves two core problems. One is storage efficiency: the human brain can't remember every detail of every day, and a large language model can't store the raw data of billions of texts. Generalization and distillation allow it to carry vast knowledge with limited parameters. The other is transferability: just as we can transfer the experience of riding a bicycle to riding an electric scooter, a large language model can transfer the language rules learned during pre-training to different tasks like writing copy, doing translation, and answering questions. This is precisely the value of long-term memory: it is not limited to one scenario, yet it supports all scenarios. Second, reasoning. Just like humans dealing with the present, short-term memory activates long-term memory. When we converse with a large language model, its thinking process is actually the process of short-term memory invoking long-term memory. It's like seeing rain in front of us: our short-term perception immediately activates long-term memory reminding us to take an umbrella. Similarly, a large language model, based on your current query and short-term context, retrieves relevant knowledge from the long-term memory formed during pre-training to generate a response. Here, short-term memory is the model's context window, such as GPT-4's 128K window. It is not used to store new knowledge, but to temporarily handle the current task. When you ask "Which is more clingy, a cat or a dog?" the model first places key information like "cat," "dog," and "clingy" into the context window, then uses the information in the window to activate long-term memories learned during pre-training—cat habits, dog habits—and finally integrates the answer. This process is highly similar to human memory recall. When we chat, we don't pull out our entire lifetime of memories; we only activate relevant long-term memories based on the current topic and short-term attention. Similarly, a large language model doesn't recalculate all parameter weights; it only focuses on the relevant weight regions based on the information in the context window. Moreover, like humans, a large language model's short-term memory has limitations. When the window is full, it starts to forget things, truncating earlier conversations—just as we can't remember the details of ten things at once. But precisely because of short-term memory, it avoids the inefficiency of using all long-term memory to handle a small problem, making reasoning more focused and faster. Third, long- and short-term memory coordination is not about imitating humans, but an inevitable resemblance. Some may ask: Is the long- and short-term memory of large language models deliberately imitating humans? Actually, it's not. It's more like technology has found a solution that arrives at the same destination as human memory, driven by the trade-off between storage and efficiency. Human memory is divided into long- and short-term because the capacity of short-term memory is limited, and we must rely on the generalization and distillation of long-term memory to carry more knowledge. Large language models also have pre-training (long memory) and inference (short memory) because, without pre-training, every inference would require learning from all data anew, making it too inefficient to be practical. Just as we can't live our lives over every time we encounter a problem, large language models can't be retrained every time they answer. The coordination of long- and short-term memory is the optimal solution for both. Fourth, conclusion. Only by understanding memory logic can we grasp the present and future of AI. Many think the core of AI is algorithms, but from the perspective of long- and short-term memory, the core of AI is actually how to store and retrieve knowledge efficiently, just like humans. The progress of large language models is essentially about making long-term memory more generalized, able to learn patterns across more domains, and short-term memory more precise, better at handling current tasks. The coordination between the two becomes smoother, allowing faster retrieval of answers from memory. For ordinary people, understanding this memory logic is not just about grasping a technical concept; it helps us more clearly judge what AI can and cannot do: what it can do is solve regular problems using long-term memory, such as writing copy or doing translation. What it cannot do is have personal memories like humans, such as remembering your specific preferences. More importantly, it makes us realize that AI development is not about surpassing humans, but about using machine-like methods to walk the path that human cognition has walked—and memory is the most fundamental and core cornerstone on that path.
back to top