(Old)1.Mistral-7b + LangChain + ChromaDB
(Old)Mistral-7b + LangChain + ChromaDB 1 环境准备工作 1.1 安装Cuda和Cudnn 自行安装 1.2 配置Hugging环境变量 HF_HUB_OFFLINE - 1 HUGGINGFACEHUB_API_TOKEN - Token 2 实战开始 2.0 环境准备 新建一个conda环境并安装pytorch 1 2 conda create -n rag-langchain python=3.12 conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia 安装python环境以及相关的包 1 2 3 4 5 6 7 8 9 10 11 12 pip install sentencepiece pip install ipython pip install gradio pip install xformer pip install chromadb pip install langchain pip install langchain-community pip install accelerate pip install transformers pip install bitsandbytes pip install unstructured pip install sentence-transformers 导入相关包 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 import torch import gradio as gr from textwrap import fill from IPython.display import Markdown, display from langchain.prompts.chat import ( ChatPromptTemplate, HumanMessagePromptTemplate, SystemMessagePromptTemplate, ) from langchain import PromptTemplate from langchain import HuggingFacePipeline from langchain.vectorstores import Chroma from langchain.schema import AIMessage, HumanMessage from langchain.memory import ConversationBufferMemory from langchain.embeddings import HuggingFaceEmbeddings from langchain.text_splitter import RecursiveCharacterTextSplitter from langchain.document_loaders import UnstructuredMarkdownLoader, UnstructuredURLLoader from langchain.chains import LLMChain, SimpleSequentialChain, RetrievalQA, ConversationalRetrievalChain from transformers import BitsAndBytesConfig, AutoModelForCausalLM, AutoTokenizer, GenerationConfig, pipeline import warnings warnings.filterwarnings('ignore') 2.1 创建Mistral模型通道 导入相关包 1 2 3 import torch from langchain_huggingface import HuggingFacePipeline from transformers import BitsAndBytesConfig, AutoTokenizer, AutoModelForCausalLM, GenerationConfig, pipeline 使用Mistral-7b建立文本生成PipeLine 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 MODEL_NAME = "mistralai/Mistral-7B-Instruct-v0.3" #使用BitsAndBytesConfig定义量化配置。量化是一种用于降低深度学习模型的内存和计算需求的技术，通常通过使用更少的比特（在我们的情况下为4比特）来表示模型参数 quantization_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, ) #初始化Mistral-7b模型的tokenizer，允许您预处理输入到模型的文本数据 tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, use_fast=True) tokenizer.pad_token = tokenizer.eos_token #初始化了用于因果语言建模的预训练语言Mistral-7b模型。该模型配置有各种参数，包括先前设置的量化配置 model = AutoModelForCausalLM.from_pretrained( MODEL_NAME, torch_dtype=torch.float16, trust_remote_code=True, device_map="auto", quantization_config=quantization_config ) #模型创建一个生成配置，指定各种与生成相关的设置，如令牌的最大数量、采样温度、top-p采样和重复惩罚 generation_config = GenerationConfig.from_pretrained(MODEL_NAME) generation_config.max_new_tokens = 1024 generation_config.temperature = 0.0001 generation_config.top_p = 0.95 generation_config.do_sample = True generation_config.repetition_penalty = 1.15 #用pipeline函数创建一个文本生成管道。这个管道是为文本生成而设置的，它将预先训练的模型、标记器和生成配置作为输入。它被配置为返回全文输出 pipeline = pipeline( "text-generation", model=model, tokenizer=tokenizer, return_full_text=True, generation_config=generation_config, ) 创建模型通道实例 1 2 3 llm = HuggingFacePipeline( pipeline=pipeline, ) 训练前 - 问题测试 问一个正常的问题 ...