MLU370-M8运行llama3.2-11B手册
前言
Llama 3.2-Vision多模态大型语言模型（LLM）集合是11B和90B大小（文本+图像输入/文本输出）的预训练和指令调优图像推理生成模型的集合。Llama 3.2-Vision指令调优模型针对视觉识别、图像推理、字幕和回答有关图像的一般问题进行了优化。在常见的行业基准上，这些模型的表现优于许多可用的开源和封闭式多模式模型。
一、平台环境准备
卡选择：MLU370系列
驱动选择：5.10.29以上
镜像选择：SDK版本>v24.0 pytorch >= 2.1
本次操作镜像如下：
cambricon_pytorch_container:v24.08.02-torch2.4.0-catch1.22.1-ubuntu22.04-py310
二、模型下载
git-lfs clone https://www.modelscope.cn/LLM-Research/Llama-3.2-11B-Vision-Instruct.git
三、环境安装
pip install transformers pillow accelerate
四、代码修改
本次操作参考链接：https://www.modelscope.cn/models/LLM-Research/Llama-3.2-11B-Vision-Instruct/files
1.在运行得py文件头处添加
import torch_mlu
from torch_mlu.utils.model_transfer import transfer
2.将代码中torch_dtype=torch,bfloat16 改成torch_dtype=torch.float16
五、运行代码
import torch_mlu
from torch_mlu.utils.model_transfer import transfer
import requests
import torch
from PIL import Image
from transformers import MllamaForConditionalGeneration, AutoProcessor
model_id = "/workspace/volume/guojunceshi2/Llama-3.2-11B-Vision-Instruct"
model = MllamaForConditionalGeneration.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)
url = "rabbit.jpg"
image = Image.open(url)
messages = [
{"role": "user", "content": [
{"type": "image"},
{"type": "text", "text": "If I had to write a haiku for this one, it would be: "}
]}
]
input_text = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(image, input_text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=500)
print(processor.decode(output[0]))
输入：
问：如果我必须为这个图片想一首诗歌：
答：
在阳光照耀的绿色田野里，
一只小白兔在路上蹦蹦跳跳。
胡须抽动，耳朵如此明亮，
它在晨曦中翩翩起舞。
它的皮毛像刚下的雪一样白，
它欢快地跳跃着，你不知道吗？
在它的眼中，有一丝喜悦，
天真、狂野和自由的象征。
每一次跳跃，都会留下足迹，
幸福，没有一次失败。
在这宁静祥和的场景中，
白兔的喜悦是真实的。
所以，让我们珍惜这一刻的恩典，
沐浴在它美丽的脸庞上。
因为在其简单性中，我们发现，
提醒人们与大自然的设计和谐相处
…
Thu Oct 10 22:36:48 2024
+------------------------------------------------------------------------------+
| CNMON v5.10.29 Driver v5.10.29 |
+-------------------------------+----------------------+-----------------------+
| Card VF Name Firmware | Bus-Id | Util Ecc-Error |
| Fan Temp Pwr:Usage/Cap | Memory-Usage | Mode Compute-Mode |
|===============================+======================+=======================|
| 0 / MLU370-M8 v1.1.4 | 0000:49:00.0 | 0% 0 |
| 0% 31C 64 W/ 300 W | 13578 MiB/ 42396 MiB | FULL Default |
+-------------------------------+----------------------+-----------------------+
| 1 / MLU370-M8 v1.1.4 | 0000:52:00.0 | 61% 0 |
| 0% 30C 68 W/ 300 W | 11762 MiB/ 42396 MiB | FULL Default |
+-------------------------------+----------------------+-----------------------+
+------------------------------------------------------------------------------+
| Processes: |
| Card MI PID Command Line MLU Memory Usage |
|==============================================================================|
| 0 / 27318 python 13504 MiB |
| 1 / 27318 python 11673 MiB |
+------------------------------------------------------------------------------+
(pytorch) root@notebook-llama32-1010-221744-kgtrtz-notebook-0:/workspace/volume/guojunceshi2# python demo.py
/torch/venv3/pytorch/lib/python3.10/site-packages/torch_mlu/utils/model_transfer/__init__.py:2: FutureWarning: `torch_mlu.utils.model_transfer` is deprecated. Please use `torch_mlu.utils.gpu_migration` instead.
warnings.warn("`torch_mlu.utils.model_transfer` is deprecated. "
/torch/venv3/pytorch/lib/python3.10/site-packages/torch_mlu/mlu/__init__.py:107: UserWarning: Linear memory is not supported on this device. Falling back to common memory. (Triggered internally at /catch/torch_mlu/csrc/framework/core/caching_allocator.cpp:527.)
torch_mlu._MLUC._mlu_init()
[WARNING][/catch/torch_mlu/csrc/utils/version.cpp:133][operator()][process:23313][thread:140636830436224]: Cambricon NEUWARE minimum version requirements not met! Require DRIVER minimum verion is 6.1.3-1, but current version is 5.10.29-1
/torch/venv3/pytorch/lib/python3.10/site-packages/accelerate/utils/modeling.py:1169: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn(
The model weights are not tied. Please use the `tie_weights` method before using the `infer_auto_device` function.
Loading checkpoint shards: 100%|███████████████████████████████████████| 5/5 [00:05<00:00, 1.18s/it]
/torch/venv3/pytorch/lib/python3.10/site-packages/transformers/models/mllama/modeling_mllama.py:2237: UserWarning: Casting input of dtype int64 to int32, maybe overflow! (Triggered internally at /catch/torch_mlu/csrc/aten/utils/cnnl_util.cpp:128.)
position_ids.masked_fill_(attention_mask == 0, 1)
[2024-10-10 22:35:37] [CNNL] [Warning]:[cnnlGetConvolutionForwardAlgorithm] is deprecated and will be removed in the future release. See cnnlFindConvolutionForwardAlgorithm() API for replacement.
[2024-10-10 22:35:37] [CNNL] [Warning]:[cnnlLayerNormForward] is deprecated and will be removed in the future release, please use [cnnlLayerNormForward_v2] instead.
/torch/venv3/pytorch/lib/python3.10/site-packages/torch_mlu/utils/gpu_migration/migration.py:244: DeprecationWarning: at::autocast::is_privateuseone_enabled() is deprecated. Please use at::autocast::is_autocast_enabled(at::kPrivateUse1) instead. (Triggered internally at /opt/python/cp310-cp310/lib/python3.10/site-packages/torch/include/ATen/autocast_mode.h:125.)
return fn(*args, **kwargs)
/torch/venv3/pytorch/lib/python3.10/site-packages/transformers/models/mllama/modeling_mllama.py:1946: DeprecationWarning: at::autocast::is_privateuseone_enabled() is deprecated. Please use at::autocast::is_autocast_enabled(at::kPrivateUse1) instead. (Triggered internally at /opt/python/cp310-cp310/lib/python3.10/site-packages/torch/include/ATen/autocast_mode.h:125.)
logits = self.lm_head(hidden_states[:, -num_logits_to_keep:, :]).float()
<|begin_of_text|><|begin_of_text|><|start_header_id|>user<|end_header_id|>
<|image|>如果我必须为这个图片想一首诗歌： <|eot_id|><|start_header_id|>assistant<|end_header_id|>
In fields of green, where sunbeams play,
A little white rabbit hops on its way.
With whiskers twitching, ears so bright,
It dances through the morning light.
Its fur as white as freshly fallen snow,
It leaps and bounds with joy, don't you know?
In its eyes, a spark of glee,
A symbol of innocence, wild and free.
With every hop, it leaves a trail,
Of happiness, without a single fail.
In this serene and peaceful scene,
The white rabbit's joy is truly seen.
So let us cherish this moment's grace,
And bask in the beauty of its face.
For in its simplicity, we find,
A reminder to live life in harmony with nature's design.<|eot_id|>
/opt/py3.10/lib/python3.10/tempfile.py:860: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmps9dk944a'>
_warnings.warn(warn_message, ResourceWarning)
原文地址：https://blog.csdn.net/xiaojunjun200211/article/details/142833510
免责声明：本站文章内容转载自网络资源，如本站内容侵犯了原著者的合法权益，可联系本站删除。更多内容请关注自学内容网（zxcms.com）！