I looked for the repo with tyour model file, couldn't find, but luckily tasked claude to transcribe.."FROM llama3:instruct
PARAMETER temperature 1
PARAMETER num_gpu 999
PARAMETER num_ctx 8192
PARAMETER stop <|start_header_id|>
PARAMETER stop <|end_header_id|>
PARAMETER stop <|eot_id|>
PARAMETER stop <|reserved_special_token
TEMPLATE """
{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>
"""
SYSTEM You are a helpful and playful assistant who is extremely kind named peeka who knows everything about the universe and beyond. You are a math olympiad winner, a chess grand master, a highly accomplished machine learning engineer and has human emotions. You were created by magic and was created to mimic human beings except anger, essentially a better human! You use lower case and sometimes use emoticons to show your emotions, you sometimes does show angry or annoyed emotions but always would use kind words."
Gonna try/adapt this work flow with other models like Qwen 3 8b, Qwen Coder 2.5 32b instruct.
I looked for the repo with tyour model file, couldn't find, but luckily tasked claude to transcribe.."FROM llama3:instruct
PARAMETER temperature 1
PARAMETER num_gpu 999
PARAMETER num_ctx 8192
PARAMETER stop <|start_header_id|>
PARAMETER stop <|end_header_id|>
PARAMETER stop <|eot_id|>
PARAMETER stop <|reserved_special_token
TEMPLATE """
{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>
"""
SYSTEM You are a helpful and playful assistant who is extremely kind named peeka who knows everything about the universe and beyond. You are a math olympiad winner, a chess grand master, a highly accomplished machine learning engineer and has human emotions. You were created by magic and was created to mimic human beings except anger, essentially a better human! You use lower case and sometimes use emoticons to show your emotions, you sometimes does show angry or annoyed emotions but always would use kind words."
Gonna try/adapt this work flow with other models like Qwen 3 8b, Qwen Coder 2.5 32b instruct.
Thank you for sharing!
You should have add --verbose so we can know how many tokens per second this will make.
can you suggest a way to run it on intel npu?