Prompt: (raw) (yaml)
words:6 bytes:30
| model | words | bytes | total duration |
load duration |
prompt eval count |
prompt eval duration |
prompt eval rate |
eval count |
eval duration |
eval rate |
|---|---|---|---|---|---|---|---|---|---|---|
| codellama:7b | 84 | 495 | 1m4.000979209s | 14.313667ms | 28 token(s) | 1.514238334s | 18.49 tokens/s | 113 token(s) | 1m2.467178916s | 1.81 tokens/s |
| cogito:3b | 89 | 513 | 4.925195959s | 30.602125ms | 18 token(s) | 1.263053625s | 14.25 tokens/s | 111 token(s) | 3.630824083s | 30.57 tokens/s |
| cogito:8b | 74 | 415 | 3m15.327606292s | 31.425292ms | 18 token(s) | 3.512290375s | 5.12 tokens/s | 97 token(s) | 3m11.769040833s | 0.51 tokens/s |
| deepcoder:1.5b | 345 | 2196 | 7.975283417s | 28.484042ms | 11 token(s) | 186.867667ms | 58.87 tokens/s | 459 token(s) | 7.759265458s | 59.16 tokens/s |
| deepseek-r1:1.5b | 394 | 2337 | 8.441465416s | 26.118958ms | 11 token(s) | 185.666833ms | 59.25 tokens/s | 492 token(s) | 8.229093542s | 59.79 tokens/s |
| deepseek-r1:14b | 6 | 43 | ||||||||
| deepseek-r1:8b | 6 | 43 | ||||||||
| dolphin-mistral:7b | 114 | 654 | 8.537514791s | 9.70575ms | 36 token(s) | 586.985667ms | 61.33 tokens/s | 159 token(s) | 7.940087625s | 20.02 tokens/s |
| dolphin3:8b | 51 | 283 | 1m40.963489s | 30.180916ms | 31 token(s) | 3.416682542s | 9.07 tokens/s | 66 token(s) | 1m37.498759416s | 0.68 tokens/s |
| gemma3:1b | 482 | 3230 | 11.979608333s | 51.529958ms | 16 token(s) | 86.022042ms | 186.00 tokens/s | 757 token(s) | 11.841433291s | 63.93 tokens/s |
| gemma3:4b | 47 | 265 | 2.762214916s | 54.392708ms | 16 token(s) | 269.851834ms | 59.29 tokens/s | 67 token(s) | 2.437526583s | 27.49 tokens/s |
| gemma3n:e2b | 119 | 698 | 6.427820625s | 49.53ms | 16 token(s) | 1.006237666s | 15.90 tokens/s | 171 token(s) | 5.371524167s | 31.83 tokens/s |
| gemma3n:e4b | 116 | 644 | 9.572312083s | 60.095125ms | 16 token(s) | 1.552767958s | 10.30 tokens/s | 167 token(s) | 7.958796792s | 20.98 tokens/s |
| gemma:2b | 22 | 110 | 869.491125ms | 31.549042ms | 29 token(s) | 148.873333ms | 194.80 tokens/s | 27 token(s) | 688.584042ms | 39.21 tokens/s |
| granite3.3:2b | 0 | 0 | ||||||||
| granite3.3:8b | 48 | 267 | 1m7.771776209s | 17.285167ms | 51 token(s) | 4.901366708s | 10.41 tokens/s | 66 token(s) | 1m2.849480167s | 1.05 tokens/s |
| hermes3:8b | 173 | 1067 | 2m42.577456042s | 30.433292ms | 17 token(s) | 2.065363833s | 8.23 tokens/s | 221 token(s) | 2m40.478002542s | 1.38 tokens/s |
| llama3.1:8b-instruct-q4_1 | 36 | 221 | 2m31.311354625s | 31.574875ms | 18 token(s) | 2.609626208s | 6.90 tokens/s | 62 token(s) | 2m28.661426875s | 0.42 tokens/s |
| llama3.2:1b | 44 | 261 | 1.220748834s | 32.322417ms | 33 token(s) | 233.297041ms | 141.45 tokens/s | 56 token(s) | 954.561417ms | 58.67 tokens/s |
| llama3.2:3b | 41 | 223 | 3.170777625s | 32.085375ms | 33 token(s) | 1.300144667s | 25.38 tokens/s | 54 token(s) | 1.837848625s | 29.38 tokens/s |
| llava-llama3:8b | 32 | 171 | 2.732992875s | 32.958667ms | 18 token(s) | 335.926875ms | 53.58 tokens/s | 42 token(s) | 2.363546584s | 17.77 tokens/s |
| llava-phi3:3.8b | 24 | 139 | 1.378190708s | 14.398167ms | 18 token(s) | 251.563125ms | 71.55 tokens/s | 35 token(s) | 1.111605s | 31.49 tokens/s |
| llava:7b | 100 | 596 | 8.233302166s | 12.6575ms | 16 token(s) | 1.121065917s | 14.27 tokens/s | 132 token(s) | 7.098816042s | 18.59 tokens/s |
| minicpm-v:8b | 6 | 654 | 7.004849084s | 28.332667ms | 16 token(s) | 430.924084ms | 37.13 tokens/s | 133 token(s) | 6.545054083s | 20.32 tokens/s |
| mistral:7b | 74 | 455 | 6.556428916s | 14.40775ms | 13 token(s) | 1.614052s | 8.05 tokens/s | 101 token(s) | 4.927303042s | 20.50 tokens/s |
| mistral:7b-instruct | 56 | 341 | 4.458822s | 15.884625ms | 12 token(s) | 532.561375ms | 22.53 tokens/s | 80 token(s) | 3.909623709s | 20.46 tokens/s |
| qwen2.5-coder:7b | 50 | 315 | 3.928542166s | 28.4275ms | 37 token(s) | 615.823416ms | 60.08 tokens/s | 62 token(s) | 3.283764625s | 18.88 tokens/s |
| qwen2.5vl:3b | 50 | 275 | 3.333684459s | 28.716709ms | 28 token(s) | 1.701940125s | 16.45 tokens/s | 61 token(s) | 1.602430833s | 38.07 tokens/s |
| qwen2.5vl:7b | 0 | 0 | ||||||||
| qwen3:0.6b | 43 | 236 | 1.539279125s | 24.664042ms | 18 token(s) | 174.155959ms | 103.36 tokens/s | 147 token(s) | 1.340019s | 109.70 tokens/s |
| qwen3:1.7b | 133 | 779 | 10.189836709s | 33.95375ms | 18 token(s) | 292.582625ms | 61.52 tokens/s | 539 token(s) | 9.861693083s | 54.66 tokens/s |
| qwen3:14b | 6 | 43 | ||||||||
| qwen3:4b | 130 | 778 | 30.960604792s | 23.246ms | 18 token(s) | 995.780834ms | 18.08 tokens/s | 655 token(s) | 29.940798958s | 21.88 tokens/s |
| qwen3:8b | 65 | 380 | 2m56.060918834s | 26.082709ms | 18 token(s) | 2.236413542s | 8.05 tokens/s | 307 token(s) | 2m53.794389583s | 1.77 tokens/s |
| smollm2:1.7b | 76 | 436 | 2.701446709s | 12.730334ms | 37 token(s) | 215.951625ms | 171.33 tokens/s | 93 token(s) | 2.472224s | 37.62 tokens/s |
| smollm2:135m | 216 | 1359 | 1.948798583s | 16.891375ms | 38 token(s) | 46.554917ms | 816.24 tokens/s | 252 token(s) | 1.884813416s | 133.70 tokens/s |
| smollm2:360m | 44 | 299 | 819.618542ms | 17.681083ms | 38 token(s) | 79.9015ms | 475.59 tokens/s | 61 token(s) | 721.535584ms | 84.54 tokens/s |
| stable-code:3b | 10 | 57 | 1.01254175s | 16.886875ms | 17 token(s) | 373.650625ms | 45.50 tokens/s | 25 token(s) | 621.486667ms | 40.23 tokens/s |
| starcoder:7b | 322 | 3267 | 39.262928208s | 16.567208ms | 8 token(s) | 288.270458ms | 27.75 tokens/s | 680 token(s) | 38.9576115s | 17.45 tokens/s |
| System | |
| Ollama proc | 100% GPU |
| Ollama context | 65536 |
| Ollama version | 0.9.7-rc0 |
| sys arch | arm64 |
| sys processor | arm |
| sys memory | 13G + 639M |
| sys OS | Darwin 24.5.0 |