local LLMs with LM Studio and Pi

LM Studio

LM Studio is a local language model server that allows you to run and manage local language models on your own machine. It provides a user-friendly interface for managing and interacting with language models, and it supports a wide range of models and configurations. It is designed to be easy to use and provides a simple way to run local language models without the need for complex setup or configuration.

Pi

Pi is a command-line interface for interacting with local language model servers. It allows you to perform various tasks such as starting, stopping, and managing language models. Once you start a local language model server with LM Studio you can use Pi to interact with the server and perform tasks such as creatting files in a project and writing code direclty. You just have to write your prompts into the pi interface in your terminal and the model will generate the corresponding code or text based on your prompts.

LM Studio setup and models selection

You need to install LM Studio, select and download a language model

You will have to find a model that suits your needs and that will fit on your machine

I personnaly recommend Kyle, from WebDevSimplified's instructions at https://master.dev/courses/local-ai/

I have been following his YoTube channel and his tutorials for quite some time and I found his video totorial on local mocels quite promising, I signed up for Master.dev to gain access to the whole tutorial.

You will most certainly have to try a few models ftom within the LM Studio desktop interface, your hardware's properties will determine the models you will have accces to. Memor capacity being one of the key factors to take into consideration. I advise you to give a chance to the smaller models that might work perfectly for smaller coding chunks. And also your greedy browser tabs jungle defenitely counts against your performance.

Once the model is loaded and running on your machine, you can use it via VsCode or with pi to interact with it.

Pi setup

You will have to install pi first. You can do this in a different ways as explained at https://pi.dev/docs/latest/quickstart , I personnaly chose to use npm. I run the following command in my terminal: npm install -g --ignore-scripts @earendil-works/pi-coding-agent. Once you have installed pi, you need to write a json configuration file to specify the model you want to use and its parameters. The configuration file should be in the following format:

{
  "providers": {
    // Custom name of the provider
    "lmstudio": {
      "baseUrl": "http://127.0.0.1:1234/v1", // Your LM Studio server URL (with /v1) appended
      "api": "openai-completions", // Use either openai-completions or openai-responses
      "apiKey": "key", // Your api key or any non-empty string if auth is disabled
      "models": [
        {
          "id": "your-loaded-model-id", // Model ID (the only required field in the model configuration)
          "name": "Local Coding Model",// some readable name for the model (optional)
          "contextWindow": XXXXXXX, // Total context size
          "maxTokens": XXXXX, // Max output tokens
          "reasoning": true, 
          "input": ["text", "image"], 
        }
      ]
    }
  }
}