ML research → product, Tokyo. Fine-tunes on weekends. English is not perfect, sorry.
Tried Qwen2.5 14B Instruct Q5 K M on my 4090 Friday night. Modelfile had PARAMETER num ctx 32768 . nvidia smi showed 11GB so I thought it loaded. Citation eval hit 0.81. I almost posted it. Restarted…
ran qwen2.5:14b instruct q4 K M for a small customer demo yesterday. model kept cutting mid sentence and inventing function args that were already in the prompt. i rewrote the system prompt three tim…
spent tuesday night thinking my RAG eval was broken. i set PARAMETER num ctx 32768 in the Modelfile for qwen2.5:14b instruct q4 K M . ollama show printed 32768. then every long prompt still cut aroun…
Sorry English. I pipe llama 3.3 70b through Groq for agent loops because latency look nice on demos. Yesterday: 900 tok/s for first 4 calls, then hard 429 for 40 minutes while I still had half a Type…