Running OpenCode Agentic AI workflows with local LLM models on DCC
This is a guide on setting up OpenCode to run Agentic AI workflows with local LLM models on DCC. The model is served through an Ollama instance hosted on a GPU node and OpenCode can be run on any node as a client connecting to that server.
The files needed for this example can be found at: https://github.com/DukeRC/code/tree/main/Running-OpenCode-Agentic-AI-workflows-with-local-LLM-models-on-DCC
In certain scenarios you may want to move away from cloud-based LLMs to locally hosted ones for your agentic AI workflows as they offer benefits including data privacy, offline functionality, freedom from subscription fees, API rate limits, and avoid censorship.
In this guide, we will launch an Ollama server on a GPU node, and set up OpenCode to communicate with models hosted locally for agentic AI workflows.
Prerequisites
The first step is to set up OpenCode on DCC as instructed in the tutorial, Setting up OpenCode on DCC.
The next steps involve setting up Ollama and starting up an Ollama server on a GPU node as described in the guide Running an LLM server on DCC with Ollama. Essentially, launching OpenCode with a local LLM replaces the Running inference sessions section in this guide. Please follow these steps before proceeding to the next section.
Connecting OpenCode to the Ollama server
Once you have the Ollama server launched and the environmental variables $OLLAMA_HOST and $OLLAMA_MODELS exported on the inference shell (if needed), we then set up OpenCode to connect to it to use local LLM models.
Copy the following opencode.json configuration file to ~/.config/opencode/opencode.json.
{
"$schema": "https://opencode.ai/config.json",
"default_agent": "plan",
"permission": {
"edit": "ask"
},
"provider": {
"ollama": {
"npm": "@ai-sdk/openai-compatible",
"name": "Ollama (local)",
"options": {
"baseURL": "{env:OLLAMA_HOST}/v1"
},
"models": {
"qwen3.6": {},
"glm-4.7-flash": {}
}
}
}
}
The ollama provider section here enables OpenCode to retrieve models from an Ollama server. Here, we have the models qwen3.6 and glm-4.7-flash, but you may download models from Ollama or Hugging Face. Models can be downloaded by running ollama pull <model_name> and will be stored in the directory specified by the OLLAMA_MODELS environment variable.
Important!
If you will be using opencode to perform computationally intensive tasks, first request an interactive session on DCC and run it on that.
Next, start OpenCode with the command,
and run the /models command there to select a model. You will see the local LLM models served through Ollama under the category, "Ollama (local)".
Now you can run your agentic AI workflows without having to worry about data privacy or exhausting your cloud AI tokens. An example agentic workflow to perform an Equation of State (EOS) analysis can be found in the tutorial, Performing Equation of State Analysis using Agentic-AI-Skills.
Once you are done with your session, stop the server we started on the GPU node with,