Exploring Llama 3.2 Vision with Ollama

Image by Manfred Richter from Pixabay

Ollama (0.4.0) just get Llama 3.2 vision supported. Let’s explore it with Golang.

Install the latest ollama 0.4.0, and pull the model

ollama pull llama3.2-vision

Check the model,

ollama show llama3.2-vision
  Model
    architecture        mllama
    parameters          9.8B
    context length      131072
    embedding length    4096
    quantization        Q4_K_M

  Projector
    architecture        mllama
    parameters          895.03M
    embedding length    1280
    dimensions          4096

  Parameters
    top_p          0.9
    temperature    0.6

  License
    LLAMA 3.2 COMMUNITY LICENSE AGREEMENT
    Llama 3.2 Version Release Date: September 25, 2024

I am testing this model (11B) on M1 Macpro.

Testing with Golang

We will be using the Ollama golang API directly

go get github.com/ollama/ollama/api

As usual with Golang magefile,

type LLAMAVision mg.Namespace

func (LLAMAVision) T01_question_on_image(q string, imgPath string) {
 client := try.E1(api.ClientFromEnvironment())

 img := try.E1(os.ReadFile(imgPath))

 req := &api.GenerateRequest{
  Model:  "llama3.2-vision",
  Prompt: q,
  Images: []api.ImageData{img},
 }…