我的征尘是星辰大海。。。
The dirt and dust from my pilgrimage forms oceans of stars...
-------当记忆的篇章变得零碎，当追忆的图片变得模糊，我们只能求助于数字存储的永恒的回忆
作者:黄教授
手机视频列表
为了给Gemini加上语音输入我踩了4天的坑
视频
音频
原始脚本
为了给 Gemini 加中文语音输入，我踩了4天坑，AI 加持才搞定。 作为一个重度 AI 用户，最近被 Gemini 的中文语音输入逼疯了。 明明支持语音交互，却只认英文，中文说破天也没反应。 本以为是个小问题，改改设置就能搞定。 没想到一脚踏进了本地化语音转文字的深坑，折腾了整整四天才算爬出坑。 过程之曲折，简直是技术宅的血泪史。 初衷有多简单，过程就有多折磨。 一开始的想法特别纯粹，找个工具把中文语音转成文字，再复制粘贴给 Gemini。 本来想省点事用云端 API，最先试了火山引擎，毕竟是字节家的，想着和豆包适配能顺畅点。 结果折腾了大半天全是坑。 技术文档过时， Access Token 的配置步骤前后矛盾，按教程操作始终连不上。 更离谱的是，问豆包怎么解决，他居然反复劝说我别用火山引擎，让我换其他公司的，简直让人无语。 后来又看了百度讯飞的语音识别，要么是免费额度用完就收费，要么得绑定手机号，配置一堆密钥，还有网络延迟。 一番折腾下来，果断放弃云端，转向本地模型。 Whisper 几百兆的体积，我这台带 RTX 4050M 6G显存的笔记本肯定吃得消。 而且本地化运行不用联网，不用花钱，响应还快，简直是完美方案。 从硬件到软件每一步都是地雷。 一、麦克风，闲置苹果耳机成救星。 最开始用笔记本自带的麦克风，录出来全是杂音，Whisper 识别出来全是乱码。 本以为是模型问题，换了好几个版本都没用。 最后才发现是硬件拖后腿，笔记本麦克风没降噪，连键盘敲击声都能录进去，模型分不清人声和噪音。 翻出自己闲置的苹果 USB 耳机一试，瞬间惊艳了，降噪效果直接拉满，录出来的声音干净的不像话，而且在 Linux 上居然完美适配。 其实我早就知道这耳机麦克风质量好，只是没想到能在这个场景派上用场。 但新的问题又来了，Linux 系统里音频设备一大堆，什么 HDA, Intel PHE, USB Audio Mono, 程序得能让用户手动选择设备。 不然默认选中无效设备就白折腾了，这又给代码加了一层复杂度。 二，多线程，豆包瞎写导致死锁，Debug 到头痛环境配置好后，本以为能顺利运行，结果新的坑又来了。 一开始用单线程采集音频，按回车键停止时，经常出现最后几个字录不上的情况。 明明说了大家好，我是叉叉叉，识别结果却是大家好，我是。 之所以想搞多线程，是因为单靠 CPU 解码 Media 模型延迟太高，跑起来卡顿明显。 这时候豆包自告奋勇要写多线程代码，结果他根本没吃透逻辑，瞎写了一堆 Atomic 标志量，最后写出了死锁。 主线程等着采集线程退出，采集线程等着主线程处理数据，互相卡着不动，不仅最后几秒录不到，还得强制关闭程序。 多线程 Debug 本来就是老大难问题，哪怕我有多年编程经验，面对 Producer Consumer 这种标准模型也头疼。 最后实在没办法，只能找 Gemini 帮忙排查，反复调整停止逻辑，才终于解决了死锁和数据丢失的问题。 这里面和 Gemini 反反复复的 Debug 也是一个长长的话题，但他毕竟是高手，最后解决了。 三、模型与 GPU，在准确率和速度间反复横跳选模型的时候走了不少弯路。 一开始下了 Whisper 的 small 模型，体积不大，但准确率太差，漏字瞎编的情况频发，根本没法用。 换成 Medium 模型后，准确率明显上来了，但非量化版单靠 CPU 跑还是卡顿。 又试了 Medium 的 Q40量化版，速度快了30%，可准确率又掉了下去，只能在两者间反复折腾。 最后还是 Gemina 建议，既然有 RTX 4050M显卡，直接开 GPU 加速啊，6G 显存跑 Medium 模型绰绰有余。 可配置 CUDA 的过程简直是噩梦，不仅要装 CUDA 库，还得匹配 NVCC 编译器、 CUDA Nan 等好几个 NVIDIA 组件。 版本之间必须严丝合缝，号称的向前向后兼容根本不靠谱。 而且 whisper 点 cpp 编译时得手动打开 GPU 支持，运行时还要通过 LD, library, path 指定 GPU 编译的动态库路径，一堆基本功操作，少一步就报错。 最后还是 Gemini 给了成熟的配置方案，才没在环境上耗更多时间。 四，采样率。 没想到高音质反而成累赘，还有个意外的学习收获。 现在麦克风都是高采样率，比如44.1K的高音质，但 Whisper 只认16K的低采样率，而且得是无压缩格式。 后来查资料才明白，语音识别根本不需要那么高的音质。 16K 采样率已经能覆盖人类语音的频率范围，还能减少数据量，提升识别速度。 这一步音频格式转换也得在代码里处理好，不然识别结果全是乱码。 除了这些大问题，还有很多小细节让我崩溃。 模型下载， Shading Face 官网打不开，普通下载链接全是403。 最后发现国内有 HF 镜像网站，才终于把模型下下来。 静默识别，纯静默时模型会胡说八道，比如输出，嗯，是的。 最后加了个能量校验，全程静默就跳过识别，避免无意义输出。 依赖冲突，Portaudio、FanPEG 等库版本不匹配，要么编译报错，要么运行时找不到文件，只能逐个排查适配。 这些问题单独看都很小，但放在一起就像温水煮青蛙，没几 解决一个，又冒出一个新的，折腾的人没脾气。 最终效果，AI 加持，4天搞定半月的工作，现在终于搞定了。 打开程序，选择苹果耳机麦克风，按回车键开始录制，说完话再按回车，1~2秒就能出识别结果。 准确率能到95%以上，连嗯、啊这种语气词都能准确识别。 而且开启 GPU 加速后，识别速度比纯 CPU 快了近两倍，全程本地化运行，不用联网，不用花钱。 把识别结果复制粘贴给 Gemini，终于能用中文语音间接交互了。 看着顺畅运行的程序，我最大的感慨不是自己技术多厉害，而是 AI 带来的效率革命。 如果放在以前，要搞定硬件适配、环境配置、多线程 Debug、GPU 加速这些跨领域的工作。 涉及大量软件硬件知识点，我估计得花十天半个月，甚至一个月都未必能搞定。 这次能4天完工，完全是豆包和 Gemini 通力协作的结果。 豆包帮忙搭建基础代码框架，虽然多线程写崩了，但至少省去了从零开始的麻烦。 Gemini 则在关键节点发力，不管是排查死锁配置 CUDA，还是优化模型选择，都给出了精准解决方案，实力比豆包高出好几个数量级。 说实话，豆包已经堪比有十几年经验的程序员了，但和谷歌的 Gemini 比起来还是有明显差距。 给同样有需求的朋友避坑，三个核心建议。 一，硬件优先选闲置苹果耳机，降噪效果好，Linux、Windows 都能原声适配，比大多数笔记本自带麦克风和普通 USB 麦克风靠谱，还不用额外花钱。 二、能开 GPU 就别死磕 CPU 有 NVIDIA 显卡的话，一定要配置库达，开 GPU 加速。 不仅速度翻倍，还能避免多线程折腾。 配置时直接找成熟方案，别自己硬刚版本匹配。 三、模型优先选 Medium 非量化版加 GPU。 Small 准确率不够，Large 体积太大，Medium 非量化版在 GPU 加持下，能兼顾准确率和速度，比量化版更省心。 这次的经历让我深刻体会到，现在的 AI 已经不是简单的辅助工具，而是能大幅降低技术门槛、提升效率的神队友。 如果没有 AI 加持，很多跨领域的小工程对普通人来说可能遥不可及。 而现在，只要找对方法，善用工具，三四天就能搞定以前半个月的活，这就是科技进步的魅力。 如果你也想给 AI 加中文语音输入，又不想用云端 API。 希望我的踩坑经历能帮你少走点弯路。
修正脚本
为了给 Gemini 加中文语音输入，我踩了4天坑，AI 加持才搞定。 作为一个重度 AI 用户，最近被 Gemini 的中文语音输入逼疯了。 明明支持语音交互，却只认英文，中文说破天也没反应。 本以为是个小问题，改改设置就能搞定。 没想到一脚踏进了本地化语音转文字的深坑，折腾了整整四天才算爬出坑。 过程之曲折，简直是技术宅的血泪史。 初衷有多简单，过程就有多折磨。 一开始的想法特别纯粹，找个工具把中文语音转成文字，再复制粘贴给 Gemini。 本来想省点事用云端 API，最先试了火山引擎，毕竟是字节家的，想着和豆包适配能顺畅点。 结果折腾了大半天全是坑。 技术文档过时， Access Token 的配置步骤前后矛盾，按教程操作始终连不上。 更离谱的是，问豆包怎么解决，他居然反复劝说我别用火山引擎，让我换其他公司的，简直让人无语。 后来又看了百度讯飞的语音识别，要么是免费额度用完就收费，要么得绑定手机号，配置一堆密钥，还有网络延迟。 一番折腾下来，果断放弃云端，转向本地模型。 Whisper 几百兆的体积，我这台带 RTX 4050M 6G显存的笔记本肯定吃得消。 而且本地化运行不用联网，不用花钱，响应还快，简直是完美方案。 从硬件到软件每一步都是地雷。 一、麦克风，闲置苹果耳机成救星。 最开始用笔记本自带的麦克风，录出来全是杂音，Whisper 识别出来全是乱码。 本以为是模型问题，换了好几个版本都没用。 最后才发现是硬件拖后腿，笔记本麦克风没降噪，连键盘敲击声都能录进去，模型分不清人声和噪音。 翻出自己闲置的苹果 USB 耳机一试，瞬间惊艳了，降噪效果直接拉满，录出来的声音干净得不像话，而且在 Linux 上居然完美适配。 其实我早就知道这耳机麦克风质量好，只是没想到能在这个场景派上用场。 但新的问题又来了，Linux 系统里音频设备一大堆，什么 HDA, Intel PHE, USB Audio Mono, 程序得能让用户手动选择设备。 不然默认选中无效设备就白折腾了，这又给代码加了一层复杂度。 二、多线程，豆包瞎写导致死锁，Debug 到头痛。环境配置好后，本以为能顺利运行，结果新的坑又来了。 一开始用单线程采集音频，按回车键停止时，经常出现最后几个字录不上的情况。 明明说了大家好，我是叉叉叉，识别结果却是大家好，我是。 之所以想搞多线程，是因为单靠 CPU 解码 Media 模型延迟太高，跑起来卡顿明显。 这时候豆包自告奋勇要写多线程代码，结果他根本没吃透逻辑，瞎写了一堆 Atomic 标志量，最后写出了死锁。 主线程等着采集线程退出，采集线程等着主线程处理数据，互相卡着不动，不仅最后几秒录不到，还得强制关闭程序。 多线程 Debug 本来就是老大难问题，哪怕我有多年编程经验，面对 Producer Consumer 这种标准模型也头疼。 最后实在没办法，只能找 Gemini 帮忙排查，反复调整停止逻辑，才终于解决了死锁和数据丢失的问题。 这里面和 Gemini 反反复复的 Debug 也是一个长长的话题，但他毕竟是高手，最后解决了。 三、模型与 GPU，在准确率和速度间反复横跳。选模型的时候走了不少弯路。 一开始下了 Whisper 的 small 模型，体积不大，但准确率太差，漏字瞎编的情况频发，根本没法用。 换成 Medium 模型后，准确率明显上来了，但非量化版单靠 CPU 跑还是卡顿。 又试了 Medium 的 Q40量化版，速度快了30%，可准确率又掉了下去，只能在两者间反复折腾。 最后还是 Gemini 建议，既然有 RTX 4050M显卡，直接开 GPU 加速啊，6G 显存跑 Medium 模型绰绰有余。 可配置 CUDA 的过程简直是噩梦，不仅要装 CUDA 库，还得匹配 NVCC 编译器、 cuDNN 等好几个 NVIDIA 组件。 版本之间必须严丝合缝，号称的向前向后兼容根本不靠谱。 而且 whisper.cpp 编译时得手动打开 GPU 支持，运行时还要通过 LD_LIBRARY_PATH 指定 GPU 编译的动态库路径，一堆基本功操作，少一步就报错。 最后还是 Gemini 给了成熟的配置方案，才没在环境上耗更多时间。 四、采样率。 没想到高音质反而成累赘，还有个意外的学习收获。 现在麦克风都是高采样率，比如44.1K的高音质，但 Whisper 只认16K的低采样率，而且得是无压缩格式。 后来查资料才明白，语音识别根本不需要那么高的音质。 16K 采样率已经能覆盖人类语音的频率范围，还能减少数据量，提升识别速度。 这一步音频格式转换也得在代码里处理好，不然识别结果全是乱码。 除了这些大问题，还有很多小细节让我崩溃。 模型下载， Hugging Face 官网打不开，普通下载链接全是403。 最后发现国内有 HF 镜像网站，才终于把模型下下来。 静默识别，纯静默时模型会胡说八道，比如输出，嗯，是的。 最后加了个能量校验，全程静默就跳过识别，避免无意义输出。 依赖冲突，Portaudio、FFmpeg 等库版本不匹配，要么编译报错，要么运行时找不到文件，只能逐个排查适配。 这些问题单独看都很小，但放在一起就像温水煮青蛙，每解决一个，又冒出一个新的，折腾得人没脾气。 最终效果，AI 加持，4天搞定半月的工作，现在终于搞定了。 打开程序，选择苹果耳机麦克风，按回车键开始录制，说完话再按回车，1~2秒就能出识别结果。 准确率能到95%以上，连嗯、啊这种语气词都能准确识别。 而且开启 GPU 加速后，识别速度比纯 CPU 快了近两倍，全程本地化运行，不用联网，不用花钱。 把识别结果复制粘贴给 Gemini，终于能用中文语音间接交互了。 看着顺畅运行的程序，我最大的感慨不是自己技术多厉害，而是 AI 带来的效率革命。 如果放在以前，要搞定硬件适配、环境配置、多线程 Debug、GPU 加速这些跨领域的工作。 涉及大量软件硬件知识点，我估计得花十天半个月，甚至一个月都未必能搞定。 这次能4天完工，完全是豆包和 Gemini 通力协作的结果。 豆包帮忙搭建基础代码框架，虽然多线程写崩了，但至少省去了从零开始的麻烦。 Gemini 则在关键节点发力，不管是排查死锁配置 CUDA，还是优化模型选择，都给出了精准解决方案，实力比豆包高出好几个数量级。 说实话，豆包已经堪比有十几年经验的程序员了，但和谷歌的 Gemini 比起来还是有明显差距。 给同样有需求的朋友避坑，三个核心建议。 一，硬件优先选闲置苹果耳机，降噪效果好，Linux、Windows 都能原生适配，比大多数笔记本自带麦克风和普通 USB 麦克风靠谱，还不用额外花钱。 二、能开 GPU 就别死磕 CPU，有 NVIDIA 显卡的话，一定要配置CUDA，开 GPU 加速。 不仅速度翻倍，还能避免多线程折腾。 配置时直接找成熟方案，别自己硬刚版本匹配。 三、模型优先选 Medium 非量化版加 GPU。 Small 准确率不够，Large 体积太大，Medium 非量化版在 GPU 加持下，能兼顾准确率和速度，比量化版更省心。 这次的经历让我深刻体会到，现在的 AI 已经不是简单的辅助工具，而是能大幅降低技术门槛、提升效率的神队友。 如果没有 AI 加持，很多跨领域的小工程对普通人来说可能遥不可及。 而现在，只要找对方法，善用工具，三四天就能搞定以前半个月的活，这就是科技进步的魅力。 如果你也想给 AI 加中文语音输入，又不想用云端 API。 希望我的踩坑经历能帮你少走点弯路。
英文翻译
To add Chinese voice input to Gemini, I spent 4 days in a pitfall and only got it working with AI assistance. As a heavy AI user, I’ve recently been driven crazy by Gemini’s Chinese voice input. It claims to support voice interaction, but only recognizes English. No matter how much Chinese you speak, it doesn’t respond. I thought it was a small issue that could be fixed with a setting change. Little did I know I’d step into the deep pit of localized speech-to-text, spending a full four days crawling out. The twists and turns were nothing short of a tech geek’s blood-and-tears saga. The simpler the original idea, the more torturous the process. At first, the idea was very pure: find a tool to convert Chinese speech to text, then copy and paste it into Gemini. I wanted to save effort by using cloud APIs. First, I tried Volcengine—after all, it’s from ByteDance, and I thought it would integrate smoothly with Doubao. But after a whole day of tinkering, it was nothing but pitfalls. The technical documentation was outdated; the steps for configuring Access Token conflicted with each other, and following the tutorial never succeeded in connecting. What’s more ridiculous is that when I asked Doubao how to fix it, it repeatedly advised me not to use Volcengine and suggested switching to other companies—utterly exasperating. Later, I tried Baidu and iFlytek’s speech recognition, but either the free quota ran out and started charging, or I had to bind a phone number, configure a bunch of keys, and deal with network latency. After all that hassle, I gave up on cloud solutions and switched to local models. Whisper is only a few hundred megabytes in size, and my laptop with an RTX 4050M 6GB GPU can definitely handle it. Plus, running locally means no internet, no cost, and fast response—a perfect solution. From hardware to software, every step was a landmine. 1. Microphone: An unused Apple earbud became the lifesaver. At first, I used the laptop’s built-in microphone. The recordings were full of noise, and Whisper outputted gibberish. I thought it was a model issue and tried several versions, but nothing worked. Finally, I realized it was a hardware problem: the laptop mic had no noise cancellation; even keyboard clicks were recorded, and the model couldn’t separate human voice from noise. I pulled out my unused Apple USB earbuds and gave them a try—I was instantly amazed. The noise cancellation was top-notch; the recorded voice was incredibly clean, and it worked perfectly on Linux. I actually knew the earbud mic was good quality, but I never expected it to be useful in this scenario. But then a new problem arose: Linux has a ton of audio devices—HDA, Intel PHE, USB Audio Mono, etc. The program needed to let the user manually select the device. Otherwise, if it defaulted to an invalid device, all the effort would be wasted. This added another layer of complexity to the code. 2. Multithreading: Doubao wrote bad code causing deadlocks, leading to headaches during debugging. After setting up the environment, I thought everything would run smoothly, but more pitfalls emerged. Initially, I used a single thread to capture audio. When I pressed Enter to stop, the last few words often failed to record. Even if I clearly said "Hello everyone, I am XXX," the recognition result was just "Hello everyone, I am." The reason I wanted multithreading was that decoding the Media model solely on the CPU had high latency, causing noticeable stuttering. At this point, Doubao volunteered to write multithreaded code. But it didn’t fully understand the logic and threw in a bunch of Atomic flags, eventually creating a deadlock. The main thread waited for the capture thread to exit, while the capture thread waited for the main thread to process data—they blocked each other. Not only did it fail to capture the last few seconds, but I also had to force-close the program. Debugging multithreading is notoriously difficult. Even with years of programming experience, I find standard models like Producer-Consumer painful. In the end, I had no choice but to ask Gemini for help. After repeatedly adjusting the stop logic, we finally resolved the deadlock and data loss. The back-and-forth debugging with Gemini is a long story in itself, but since it’s a pro, it eventually solved the problem. 3. Models and GPU: Constant trade-off between accuracy and speed. Choosing a model involved many detours. First, I downloaded Whisper’s small model. It was small in size, but its accuracy was terrible—frequent missing words and made-up content made it unusable. Switching to the medium model improved accuracy significantly, but running the non-quantized version solely on CPU was still laggy. Then I tried the Q4_0 quantized version of medium, which was 30% faster, but accuracy dropped again. I kept going back and forth between the two. Finally, Gemini suggested that since I had an RTX 4050M GPU, I should directly enable GPU acceleration. 6GB VRAM is more than enough for the medium model. But configuring CUDA was a nightmare. Not only did I need to install the CUDA library, but I also had to match the NVCC compiler, cuDNN, and several other NVIDIA components. All versions had to fit perfectly together. The so-called forward and backward compatibility was unreliable. Moreover, when compiling whisper.cpp, I had to manually enable GPU support, and at runtime, I had to specify the dynamic library path for GPU compilation via LD_LIBRARY_PATH. A bunch of basic operations, and missing even one step would result in errors. In the end, Gemini provided a mature configuration plan, saving me from wasting more time on the environment. 4. Sample rate. Unexpectedly, high audio quality became a burden, and I learned something new. Modern microphones have high sample rates, e.g., 44.1K high quality, but Whisper only accepts low sample rates like 16K, and it must be in uncompressed format. Later, I looked up information and realized that speech recognition doesn’t need such high quality. 16K sample rate already covers the frequency range of human speech, reduces data volume, and improves recognition speed. This audio format conversion also had to be handled in the code; otherwise, the recognition output would be all gibberish. Apart from these major issues, many small details drove me crazy. Model download: The Hugging Face website was inaccessible, and ordinary download links all returned 403. Finally, I found a domestic HF mirror site and managed to download the model. Silent recognition: When completely silent, the model would output nonsense, like "嗯，是的" (hmm, yes). So I added energy detection—if there is complete silence, skip recognition to avoid meaningless output. Dependency conflicts: Libraries like Portaudio and FFmpeg had version mismatches—either compilation errors or runtime file-not-found issues. I had to troubleshoot and adapt one by one. Individually, these problems seemed small, but together they felt like boiling a frog slowly. Every time I solved one, another emerged, wearing down my patience. Final result: With AI assistance, 4 days to complete work that would have taken half a month. Now it’s finally done. Open the program, select the Apple earbud microphone, press Enter to start recording, and press Enter again after speaking. The recognition result appears within 1-2 seconds. Accuracy is above 95%, even accurately capturing filler words like "嗯" and "啊." With GPU acceleration enabled, recognition speed is nearly twice as fast as pure CPU. Everything runs locally—no internet, no cost. Copy and paste the recognition result into Gemini, and I can finally use Chinese voice for indirect interaction. Looking at the smoothly running program, my biggest takeaway is not how skilled I am, but the efficiency revolution brought by AI. In the past, to handle cross-domain tasks like hardware adaptation, environment configuration, multithreading debugging, and GPU acceleration— involving a ton of software and hardware knowledge—I’d probably have spent ten days to half a month, or even a month, and might not have succeeded. Finishing in 4 days this time was entirely the result of Doubao and Gemini working together. Doubao helped build the basic code framework—even though it messed up multithreading, at least it saved me from starting from scratch. Gemini stepped in at critical points, providing precise solutions for debugging deadlocks, configuring CUDA, and optimizing model selection. Its capability is orders of magnitude above Doubao’s. To be honest, Doubao is already comparable to a programmer with over a decade of experience, but compared to Google’s Gemini, there’s still a clear gap. For others with similar needs, here are three core suggestions to help you avoid pitfalls. 1. Hardware: Use unused Apple earbuds. They have great noise cancellation and work natively on both Linux and Windows. They’re more reliable than most laptop built-in mics or ordinary USB mics, and you don’t have to spend extra. 2. If you can use GPU, don’t stick with CPU. If you have an NVIDIA GPU, definitely configure CUDA and enable GPU acceleration. Not only does it double speed, but it also avoids multithreading headaches. When configuring, directly follow mature solutions—don’t try to match versions on your own. 3. Model: Prefer the medium non-quantized version with GPU. Small lacks accuracy, Large is too big. Medium non-quantized, with GPU support, balances accuracy and speed, and is more hassle-free than quantized versions. This experience made me deeply realize that today’s AI is no longer a simple assistant, but a powerful teammate that significantly lowers technical barriers and boosts efficiency. Without AI, many cross-domain small projects might be unattainable for ordinary people. Now, with the right approach and proper tools, you can finish in three or four days what used to take half a month. That’s the charm of technological progress. If you also want to add Chinese voice input to AI but don’t want to use cloud APIs, I hope my pitfall experience can help you avoid some detours.
back to top