Run the Mythos enhanced encoding model locally using llama.cpp and Pi files

# Entry

Qwythos-9B-Claude-Mythos-5-1M is a 9B inference and encoding model based on Qwen3.5, intended for local coding workflows, agentic programming, and long context tasks. What’s engaging is that it’s diminutive enough to run on consumer hardware while still being powerful enough to facilitate with practical coding tasks.

In this guide, I’ll show you how to run the Qwythos Enhanced Mythos model locally using call.cppand then connect it to Pi so you can apply it as a local encoding agent. I will be using an RTX 4070 Ti Super with 16GB of VRAM, which allows me to comfortably run Q6_K MTP quantization. If you have an 8GB GPU, I recommend starting with the Q4_K_M variant as it provides a better balance between quality, speed, and memory usage.

# Installing llama.cpp

First, install the official command line interface (CLI) llama.cpp. This will give you access to llama command, among others llama servewhich we will apply to run the model locally.

curl -LsSf https://llama.app/install.sh | sh

Then add the installation directory to the shell path so that the terminal can find the file llama order:

echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

To confirm that the installation worked, run:

If everything has been installed correctly, you should see the available llama.cpp commands and options.

# Setting the face cache directory

Settings Face Hugging Save somewhere in cache with enough free space. This is especially useful on cloud computers where the default home directory has constrained disk space.

export HF_HOME="/workspace/huggingface"
mkdir -p "$HF_HOME"

To persist it in modern terminal sessions, add it to your shell configuration:

echo 'export HF_HOME="/workspace/huggingface"' >> ~/.bashrc
source ~/.bashrc

# Running the Qwythos MTP model

Run the Q6_K MTP GGUF model with all available GPU layers:

llama serve 
  -hf "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:MTP-Q6_K" 
  --alias "qwythos-9b-mtp" 
  --host 0.0.0.0 
  --port 8910 
  --n-gpu-layers all 
  --ctx-size 100000 
  --parallel 1 
  --batch-size 1024 
  --ubatch-size 512 
  --flash-attn on 
  --cache-type-k q8_0 
  --cache-type-v q8_0 
  --spec-type draft-mtp 
  --spec-draft-n-max 6 
  --threads 12 
  --threads-batch 24 
  --temp 0.6 
  --top-p 0.95 
  --top-k 20 
  --repeat-penalty 1.05 
  --jinja 
  --perf

The first time you run this command, lama.cpp will download the model files from Hugging Face.

It will then load the model into GPU memory and start the local server.

After running the model, open the local interface in your browser:

This also gives you a local OpenAI compatible API endpoint at:

The Q6_K variant provides better print quality, but also uses more memory. If you encounter problems with VRAM or RAM, please reduce it --ctx-size First. If that’s still not enough, try the Q4_K_M variant instead.

Here are the most vital flags in the command:

On my RTX 4070 Ti Super I was getting about 81.74 tokens per second. In my test, the model was also able to rethink the task, invoke the required tools, and return the latest gold price when used in a local agent’s workflow.

# Installing Pi and llama.cpp integration

The Pi can connect to the local llama.cpp server and apply the model as an encoding agent. The pi-llama the plugin is designed to connect the Pi to a running local llama.cpp server, without the need for an external API key.

Install Pi:

curl -fsSL https://pi.dev/install.sh | sh

Install the llama.cpp plugin for Pi:

pi install git:github.com/huggingface/pi-llama

Create a diminutive test project:

mkdir new-project
cd new-project

By default, the plugin looks for the llama.cpp file on port 8080. In this guide, our local server is running on port 8910, so we need to set the correct local API address before starting the Pi:

export LLAMA_BASE_URL="http://127.0.0.1:8910/v1"

Now start Pi:

Inside Pi, type:

Select an alias for your local model:

You should see the model listed in Pi, and once selected you can start using it as a local encoding agent.

# Testing the Mythos enhanced coding model with Pi

It’s time to test the local model on real coding tasks inside the Pi. I used two elementary but practical tasks: a browser game and a diminutive Python CLI tool.

// Creating a elementary browser game

Start with a prompt asking your Pi to create a little browser game called “Beat the AI.”

Create a elementary browser game called “Beat the AI”.

The player has 30 seconds to answer miniature pattern recognition questions. Each correct answer increases your score by one. Show timer, score display, progress bar and final results screen.

Requirements:
– Operate only HTML, CSS and vanilla JavaScript.
– Generate at least three question types such as number patterns, quick math and verbal logic.
– Add a restart button when you finish the game.
– Make the interface fun and polished.
– Try to make all code understandable and avoid unnecessary files.

Please test the game in your browser before finishing.

In my test, the Pi created the full game in a single HTML file with embedded CSS and JavaScript, making the design elementary and effortless to check.

Pi then summarized what he had built, including a 30-second countdown, results tracker, progress bar, question types, and a restart button.

I then opened the game in a browser to see if it was working properly.

The result was a polished little game with:

• Countdown timer
• Displaying results
• Progress bar
• Many types of questions
• Restart after finishing the game

This is a good example of how a model can handle an entire front-end task locally, without having to apply an external API.

// Building a CSV to Excel CLI in Python

For the second test, ask Pi to build a diminutive Python CLI that converts a CSV file to an Excel file .xlsx file.

Build a elementary Python CLI that converts a CSV file to an Excel .xlsx file, accepts input and output file paths as arguments, preserves column headers, checks for missing files, and prints clear success or error messages. Before finishing, create a diminutive dummy CSV file with sample data, apply it to test the CLI, verify that the Excel file was created correctly, and then summarize the test result.

Pi created a Python script called csv2excel.pyI generated a sample CSV file, ran the test command, and then summarized the results.

python csv2excel.py sample_data.csv output.xlsx

In my execution, the summary showed that the script:

• Accepted input and output file paths.
• Column headings retained.
• Sample data converted correctly.
• Handled missing files with clear error messages.
• Missing output paths were handled correctly.

I also opened the generated Excel file to confirm that the output was correct.

The sample output correctly preserved the lines and headers, confirming that the CLI worked as expected.

# Final thoughts

I really like this little local coding model. It’s speedy, correct enough for everyday coding tasks, and doesn’t need a huge amount of VRAM to be useful. It can be used with Pi, Claude Koda, Open codeor any encoding configuration that supports OpenAI or Anthropic-compatible local endpoints.

For basic front-end applications, Python scripts, CLI tools, and quick prototypes, it works surprisingly well. You can create useful projects locally without having to apply external APIs and pay for each request.

For even better results, I highly recommend adding internet search skills, Context7and other useful Pi integrations. You can also check out my full guide on optimizing your Pi encoding agent setup here: How to Set Up Kimi K2.7 Code with Pi: The Ultimate AI Coding Environment.

Abid Ali Awan (@1abidaliawan) is a certified data science professional who loves building machine learning models. Currently, he focuses on creating content and writing technical blogs about machine learning and data science technologies. Abid holds a Master’s degree in Technology Management and a Bachelor’s degree in Telecommunications Engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.