参考网址：https://github.com/ollama/ollama/blob/main/docs/linux.md
备注：CUDA drivers 和 ROCm 可以不用安装。
1、下载 ollama
下载地址：https://ollama.com/download/ollama-linux-amd64
2、安装 ollama
cp /home/software/ollama-linux-amd64 /usr/bin/
mv /usr/bin/ollama-linux-amd64 /usr/bin/ollama
sudo chmod +x /usr/bin/ollama
sudo useradd -r -s /bin/false -m -d /usr/share/ollama ollama
3、启动 ollama
1）添加 ollama 服务的配置文件：
vi /etc/systemd/system/ollama.service
ollama.service 文件内容：
[Unit]
Description=Ollama Service
After=network-online.target
[Service]
ExecStart=/usr/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
[Install]
WantedBy=default.target
2）启动 ollama 服务
sudo systemctl daemon-reload
sudo systemctl enable ollama
sudo systemctl start ollama
4、运行大模型
注：也可以先试用 ollama pull 命令拉取模型。
运行 8B 的 ollama 3：
# ollama run llama3:8b
pulling manifest
pulling 6a0746a1ec1a... 100% 4.7 GB
pulling 4fa551d4f938... 100% 12 KB
pulling 8ab4849b038c... 100% 254 B
pulling 577073ffcc6c... 100% 110 B
pulling 3f8eb4da87fa... 100% 485 B
verifying sha256 digest
writing manifest
removing any unused layers
success
>>> who are you?
I am LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational manner. I'm not a human, but a computer program designed to simulate
conversation and answer questions to the best of my knowledge based on my training data.
My primary goal is to assist and provide helpful information to users like you through text-based conversations. I can engage in discussions, provide definitions, answer trivia questions, and
even create short stories or poetry.
I'm constantly learning and improving my responses based on the interactions I have with users like you. My knowledge is based on a massive corpus of text data that I've been trained on,
which includes but is not limited to:
1. Web pages: Articles, blogs, and other online content.
2. Books: Fiction and non-fiction books from various genres and topics.
3. User-generated content: Social media posts, forums, and online discussions.
4. Wikipedia: A vast repository of knowledge covering a wide range of topics.
Feel free to ask me anything, and I'll do my best to provide a helpful and accurate response!
>>> what's your name?
I don't have a personal name, but I'm often referred to as LLaMA or Meta AI Assistant. My creators at Meta AI have given me this nickname because it's short for "Large Language Model
Application" – which is a fancy way of saying I'm a big language model designed to chat with humans!
You can call me whatever you like, though. Some people refer to me as "Assistant," "AI Buddy," or even just "LLaMA." The most important thing is that we have fun chatting and learning
together!
5、使用 curl 访问大模型
注：退出 ollama run 命令后，大模型仍然可以访问。
1）流式返回
请求：
curl -X POST http://localhost:11434/api/generate -d '{
"model": "llama3:8b",
"prompt":"介绍一下java,20字以内"
}'
响应：
{"model":"llama3:8b","created_at":"2024-07-24T15:11:49.397624848Z","response":"Java","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:49.665192525Z","response":"是一","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:49.907390485Z","response":"种","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:50.150166138Z","response":"高","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:50.392091571Z","response":"级","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:50.647981772Z","response":"编","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:50.900948302Z","response":"程","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:51.150578908Z","response":"语言","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:51.39226365Z","response":"，","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:51.634110906Z","response":"支持","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:51.901192859Z","response":"面","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:52.143209624Z","response":"向","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:52.384975259Z","response":"对象","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:52.626845715Z","response":"和","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:52.893597092Z","response":"面","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:53.137630036Z","response":"向","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:53.379856482Z","response":"过程","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:53.6221126Z","response":"编","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:53.883843282Z","response":"程","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:54.13612256Z","response":"。","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:54.378510465Z","response":"它","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:54.621467516Z","response":"是","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:54.867262797Z","response":"由","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:55.133948111Z","response":"Sun","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:55.376389584Z","response":" Microsystems","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:55.619272055Z","response":"开发","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:55.862090881Z","response":"的","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:56.137396964Z","response":"，","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:56.622803111Z","response":"旨","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:56.86543059Z","response":"在","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:57.132843204Z","response":"让","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:57.375357978Z","response":"不同的","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:57.617767992Z","response":"平台","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:57.860406125Z","response":"之间","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:58.114954992Z","response":"可以","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:58.372855455Z","response":"共","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:58.615168178Z","response":"享","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:58.857596571Z","response":"代码","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:59.100461941Z","response":"。","done":false}
{"model":"llama3:8b","created_at":"2024-07-24T15:11:59.367693094Z","response":"","done":true,"done_reason":"stop","context":[128006,882,128007,271,117814,111230,10248,11,508,19113,23897,32943,12800
9,128006,78191,128007,271,15391,107226,87502,45736,53434,31968,39607,120074,3922,109189,28190,70141,65529,34208,28190,70141,112696,31968,39607,1811,103282,21043,68171,31192,95761,110367,9554,3922,6
079,101,19000,102654,126644,106505,113778,74770,55999,72843,47200,1811],"total_duration":27071295214,"load_duration":16426393069,"prompt_eval_count":18,"prompt_eval_duration":659048000,"eval_count"
:41,"eval_duration":9970095000}
2）非流式返回：
请求：
curl -X POST http://localhost:11434/api/generate -d '{
"model": "llama3:8b",
"prompt":"介绍一下java,20字以内",
"stream":false
}'
响应：
{"model":"llama3:8b","created_at":"2024-07-24T15:15:06.871919164Z","response":"Java是一种面向对象的编程语言，运行于任何平台上，无需重新编译。","done":true,"done_reason":"stop","context":[128006,882
,128007,271,117814,111230,10248,11,508,19113,23897,32943,128009,128006,78191,128007,271,15391,107226,87502,28190,70141,65529,9554,31968,39607,120074,3922,117238,35304,109545,106505,17905,124293,594
62,71461,31968,115605,1811],"total_duration":5899766982,"load_duration":55767778,"prompt_eval_count":18,"prompt_eval_duration":250650000,"eval_count":23,"eval_duration":5552073000}
