如何构建AI智能导诊助手?使用BERT模型精准推荐医院科室
[[]]你是否为医院的科室选择而感到烦恼呢?这篇文章会引领你去构建一个 AI 智能导诊助手。只要输入症状的描述,它就能精准地推荐相应的科室。并且,代码具备完全开箱即可使用的特点,你能够轻松地运行并进行体验,让你的导诊助手在你的设备上呈现出“活”的状态。
我们会运用 BERT 模型去识别以及理解用户的自然语言输入,同时依据症状来进行科室推荐。现在让我们一起去探寻这个技术魔法背后的奥秘吧!
效果演示
在我们深入代码之前,先看下成品效果(后端简单效果)吧:
用户输入症状描述,智能助手会快速返回推荐的科室:
<p><pre> <code class="prism language-plaintext">用户输入:头痛、发烧、咳嗽
系统推荐:内科
用户输入:腹痛、反酸
系统推荐:消化内科
</code></pre></p>
很酷吧?你只要输入简短的症状描述,系统就能够高效且准确地推荐合适的科室!随后,我们会一步步带你构建这个系统。
系统架构与实现思路
整个系统分为四个模块:
我们要准备一份标注数据,这份数据是症状与科室相对应的。利用 BERT 模型的语义理解能力,把症状描述进行分类,使其归入合适的科室。使用构建 API,让前端能够调用模型服务。将系统打包成镜像,以便于进行部署和分享。第一步就是准备数据。
我们先准备一个症状到科室的标注数据集。例如:
症状描述科室
胃痛、反酸、腹胀
消化内科
头晕、心悸、胸闷
心内科
发烧、咳嗽、喉咙痛
内科
皮疹、瘙痒、红肿
皮肤科
这些数据将成为我们训练 BERT 模型的根基。我们会利用这些数据,让模型能够理解不同症状所对应的科室。
将这些数据保存为 .csv 文件。
️ 第二步:微调BERT模型
我们会借助 BERT 所具备的强大能力去理解症状描述。你不必进行模型的从头开始训练,仅仅需要对其进行微调就可以了。以下是简略的代码:
<p><pre> <code class="prism language-python"><span class="token keyword">import</span> pandas <span class="token keyword">as</span> pd
<span class="token keyword">from</span> transformers <span class="token keyword">import</span> BertTokenizer<span class="token punctuation">,</span> BertForSequenceClassification<span class="token punctuation">,</span> Trainer<span class="token punctuation">,</span> TrainingArguments
<span class="token keyword">import</span> torch
<span class="token keyword">from</span> sklearn<span class="token punctuation">.</span>preprocessing <span class="token keyword">import</span> LabelEncoder
<span class="token comment"># 加载数据</span>
data <span class="token operator">=</span> pd<span class="token punctuation">.</span>read_csv<span class="token punctuation">(</span><span class="token string">'hospital_data.csv'</span><span class="token punctuation">)</span>
X <span class="token operator">=</span> data<span class="token punctuation">[</span><span class="token string">'症状描述'</span><span class="token punctuation">]</span>
y <span class="token operator">=</span> data<span class="token punctuation">[</span><span class="token string">'科室'</span><span class="token punctuation">]</span>
<span class="token comment"># 标签编码</span>
label_encoder <span class="token operator">=</span> LabelEncoder<span class="token punctuation">(</span><span class="token punctuation">)</span>
y_encoded <span class="token operator">=</span> label_encoder<span class="token punctuation">.</span>fit_transform<span class="token punctuation">(</span>y<span class="token punctuation">)</span>
<span class="token comment"># BERT分词器</span>
tokenizer <span class="token operator">=</span> BertTokenizer<span class="token punctuation">.</span>from_pretrained<span class="token punctuation">(</span><span class="token string">'bert-base-chinese'</span><span class="token punctuation">)</span>
<span class="token comment"># 分词</span>
https://img0.baidu.com/it/u=81352404,1730903007&fm=253&fmt=JPEG&app=138&f=PNG?w=798&h=500
<span class="token keyword">def</span> <span class="token function">tokenize_texts</span><span class="token punctuation">(</span>texts<span class="token punctuation">,</span> max_len<span class="token operator">=</span><span class="token number">128</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
<span class="token keyword">return</span> tokenizer<span class="token punctuation">(</span>
texts<span class="token punctuation">.</span>tolist<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
max_length<span class="token operator">=</span>max_len<span class="token punctuation">,</span>
padding<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span>
truncation<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span>
return_tensors<span class="token operator">=</span><span class="token string">"pt"</span>
<span class="token punctuation">)</span>
tokens <span class="token operator">=</span> tokenize_texts<span class="token punctuation">(</span>X<span class="token punctuation">)</span>
<span class="token comment"># BERT模型</span>
model <span class="token operator">=</span> BertForSequenceClassification<span class="token punctuation">.</span>from_pretrained<span class="token punctuation">(</span><span class="token string">"bert-base-chinese"</span><span class="token punctuation">,</span> num_labels<span class="token operator">=</span><span class="token builtin">len</span><span class="token punctuation">(</span>label_encoder<span class="token punctuation">.</span>classes_<span class="token punctuation">)</span><span class="token punctuation">)</span>
<span class="token comment"># 训练设置</span>
training_args <span class="token operator">=</span> TrainingArguments<span class="token punctuation">(</span>
output_dir<span class="token operator">=</span><span class="token string">'./results'</span><span class="token punctuation">,</span>
num_train_epochs<span class="token operator">=</span><span class="token number">3</span><span class="token punctuation">,</span>
per_device_train_batch_size<span class="token operator">=</span><span class="token number">8</span><span class="token punctuation">,</span>
evaluation_strategy<span class="token operator">=</span><span class="token string">"epoch"</span>
<span class="token punctuation">)</span>
<span class="token comment"># Trainer API</span>
train_dataset <span class="token operator">=</span> torch<span class="token punctuation">.</span>utils<span class="token punctuation">.</span>data<span class="token punctuation">.</span>TensorDataset<span class="token punctuation">(</span>tokens<span class="token punctuation">[</span><span class="token string">'input_ids'</span><span class="token punctuation">]</span><span class="token punctuation">,</span> tokens<span class="token punctuation">[</span><span class="token string">'attention_mask'</span><span class="token punctuation">]</span><span class="token punctuation">,</span> torch<span class="token punctuation">.</span>tensor<span class="token punctuation">(</span>y_encoded<span class="token punctuation">)</span><span class="token punctuation">)</span>
trainer <span class="token operator">=</span> Trainer<span class="token punctuation">(</span>model<span class="token operator">=</span>model<span class="token punctuation">,</span> args<span class="token operator">=</span>training_args<span class="token punctuation">,</span> train_dataset<span class="token operator">=</span>train_dataset<span class="token punctuation">)</span>
<span class="token comment"># 模型训练</span>
trainer<span class="token punctuation">.</span>train<span class="token punctuation">(</span><span class="token punctuation">)</span>
</code></pre></p>
以上代码会在你的数据集上训练一个模型,这个模型是 BERT 模型。利用 BERT 模型强大的语义理解能力,能够完成从症状到科室的分类。
第三步:构建高效API
有了模型之后,我们利用它来构建一个 API 接口,如此一来,你便能在各类应用中对这个导诊助手进行调用了。
<p><pre> <code class="prism language-python"><span class="token keyword">from</span> fastapi <span class="token keyword">import</span> FastAPI<span class="token punctuation">,</span> Request
<span class="token keyword">from</span> transformers <span class="token keyword">import</span> BertTokenizer<span class="token punctuation">,</span> BertForSequenceClassification
<span class="token keyword">import</span> torch
<span class="token keyword">import</span> joblib
app <span class="token operator">=</span> FastAPI<span class="token punctuation">(</span><span class="token punctuation">)</span>
<span class="token comment"># 加载模型和标签编码器</span>
model <span class="token operator">=</span> BertForSequenceClassification<span class="token punctuation">.</span>from_pretrained<span class="token punctuation">(</span><span class="token string">'./results'</span><span class="token punctuation">)</span>
tokenizer <span class="token operator">=</span> BertTokenizer<span class="token punctuation">.</span>from_pretrained<span class="token punctuation">(</span><span class="token string">'bert-base-chinese'</span><span class="token punctuation">)</span>
label_encoder <span class="token operator">=</span> joblib<span class="token punctuation">.</span>load<span class="token punctuation">(</span><span class="token string">'label_encoder' 对应的文件名为 joblib</span><span class="token punctuation">)</span>
<span class="token decorator annotation punctuation">@app<span class="token punctuation">.</span>post</span><span class="token punctuation">(</span><span class="token string">"/predict"</span><span class="token punctuation">)</span>
<span class="token keyword">async</span> <span class="token keyword">def</span> <span class="token function">predict</span><span class="token punctuation">(</span>request<span class="token punctuation">:</span> Request<span class="token punctuation">)</span><span class="token punctuation">:</span>
json_data <span class="token operator">=</span> <span class="token keyword">await</span> request<span class="token punctuation">.</span>json<span class="token punctuation">(</span><span class="token punctuation">)</span>
symptoms <span class="token operator">=</span> json_data<span class="token punctuation">[</span><span class="token string">'symptoms'</span><span class="token punctuation">]</span>
tokens <span class="token operator">=</span> tokenizer<span class="token punctuation">(</span>symptoms<span class="token punctuation">,</span> max_length<span class="token operator">=</span><span class="token number">128</span><span class="token punctuation">,</span> padding<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> truncation<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">,</span> return_tensors<span class="token operator">=</span><span class="token string">"pt"</span><span class="token punctuation">)</span>
<span class="token keyword">with</span> torch<span class="token punctuation">.</span>no_grad<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
outputs <span class="token operator">=</span> model<span class="token punctuation">(</span>tokens<span class="token punctuation">[</span><span class="token string">'input_ids'</span><span class="token punctuation">]</span><span class="token punctuation">,</span> attention_mask<span class="token operator">=</span>tokens<span class="token punctuation">[</span><span class="token string">'attention_mask'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
prediction <span class="token operator">=</span> torch<span class="token punctuation">.</span>argmax<span class="token punctuation">(</span>outputs<span class="token punctuation">.</span>logits<span class="token punctuation">,</span> dim<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">.</span>item<span class="token punctuation">(</span><span class="token punctuation">)</span>
department <span class="token operator">=</span> label_encoder<span class="token punctuation">.</span>inverse_transform<span class="token punctuation">(</span><span class="token punctuation">[</span>prediction<span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span>
<span class="token keyword">return</span> <span class="token punctuation">{</span><span class="token string">"department"</span><span class="token punctuation">:</span> department<span class="token punctuation">}</span>
</code></pre></p>
快速启动
将代码保存为 app.py。使用以下命令运行:
<p><pre> <code class="prism language-bash">使用 uvicorn 运行 app 中的 app ,并且开启重载功能,同时设置主机地址为<span class="token number">0.0</span>.0.0 --port <span class="token number">8000</span>
</code></pre></p>
现在,你可以通过POST请求来调用API:
<p><pre> <code class="prism language-bash"><span class="token function">curl</span> -X <span class="token string">'POST'</span> <span class="token punctuation">\</span>
<span class="token string">该链接为 http://localhost:8000/predict 。它指向一个特定的地址,用于进行某种预测相关的操作。在这个地址上,可能会有相应的程序或系统来处理和返回预测结果。</span> <span class="token punctuation">\</span>
-H <span class="token string">接受:应用程序/JSON 格式。</span> <span class="token punctuation">\</span>
-H <span class="token string">Content-Type 的类型为 application/json 。 它表明所传输的内容是 JSON 格式的数据。 这种类型常用于 Web 服务和 API 中,以方便数据的传输和解析。 当服务器接收到带有 'Content-Type: application/json' 的请求时,它会知道如何处理和解析该请求中的 JSON 数据。 同样,当客户端发送带有该类型的请求时,服务器也能准确地理解和处理请求的内容。</span> <span class="token punctuation">\</span>
-d <span class="token string">症状包括头痛,还有发烧,同时伴有咳嗽。</span>
</code></pre></p>
系统将返回类似如下的结果:
<p><pre> <code class="prism language-json"><span class="token punctuation">{</span>
<span class="token string">"department"</span><span class="token operator">:</span> <span class="token string">"内科"</span>
<span class="token punctuation">}</span>
</code></pre></p>
第四步:化部署
通过容器化,系统变得更加便于共享和部署。它在不同的环境中都能够无缝运行,只需几行命令就可以部署上线。
<p><pre> <code class="prism language-dockerfile">FROM python:3.9
WORKDIR /app
COPY . .
运行 pip 命令来安装 requirements.txt 文件中的依赖项。
EXPOSE 8000
CMD 包含 "uvicorn" 以及 "app:app" ,同时还包含 "--host" 、"0.0.0.0" ,另外还有 "--port" 和 "8000" 。
</code></pre></p>
创建.txt
<p><pre> <code>fastapi
torch
transformers
joblib
uvicorn
</code></pre></p>
构建和运行镜像
<p><pre> <code class="prism language-bash">使用 docker 进行构建操作,将其标记为 hospital-assistant 。<span class="token builtin class-name">.</span>
docker run -d -p <span class="token number">8000</span>:8000 hospital-assistant
</code></pre></p>
项目体验
你的智能导诊助手现在已经能够在任何地方运行啦!本地测试可以运行,云端部署也能运行。要是把这个智能导诊系统嵌入到你的应用里,用户只要输入症状描述,就能够获得推荐科室,这样能极大地改善他们的医疗体验呢!
是不是很激动?亲自试试吧,享受AI为生活带来的便捷!
页:
[1]