AmareshHebbar/leetcode-python-qwen25-coder-7b

🐍 LeetCode Python Coder

Qwen2.5-Coder-7B, QDoRA fine-tuned to solve LeetCode problems in Python

Part of the LeetCode Multi-Language Coder Suite — 4 language specialists, one base model, one pipeline

---

TL;DR

Given a LeetCode-style problem statement, its sample input/output, and an algorithm tag, generates a working Python solution.

PROBLEM:   Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.
ALGORITHM: Hash Map
OUTPUT (Python):
def twoSum(nums, target):
    seen = {}
    for i, n in enumerate(nums):
        if target - n in seen:
            return [seen[target - n], i]
        seen[n] = i

Base model	unsloth/Qwen2.5-Coder-7B-Instruct
Method	QDoRA (quantized DoRA, not plain LoRA)
Training data	leetcode-codegen-python
Data provenance	scraped from doocs/leetcode (3,977 problems), execution-verified, no synthetic/LLM-generated solutions
Data quality	~70% of extracted solutions verified (execution-checked against sample I/O)
Weights here	QDoRA adapter only (~160MB) — load on top of the base model
GGUF build	leetcode-python-qwen25-coder-7b-GGUF — q4_k_m / q5_k_m / q8_0
License	Apache 2.0

---

Why QDoRA {#why-qdora}

DoRA splits each adapted weight into magnitude + direction and trains both, which follows full fine-tuning's behavior more closely than plain LoRA — important for code where small precision errors break correctness outright. 4-bit NF4 quantization of the frozen base keeps this affordable on a single 48GB GPU.

Concretely, versus the plain-QLoRA v1 release of this suite: DoRA adds a per-column trainable magnitude vector on top of the usual low-rank direction update, so the adapter can rescale a feature's importance instead of only rotating it. On a code task where a single wrong operator or dropped edge case fails the whole solution, that closer match to full fine-tuning's update pattern showed up as fewer near-miss failures during our own qualitative review, at the same LoRA rank and VRAM budget.

# training-side PEFT config (see build_language_datasets.py / trainer script for full pipeline)
from peft import LoraConfig

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.0,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    use_dora=True,          # <- this is what makes it QDoRA, not QLoRA
    task_type="CAUSAL_LM",
)

---

Benchmarks (free, reproducible)

Run benchmark_suite.py from the deployment kit to reproduce. All numbers are pass@1 unless noted.

Benchmark	Language	Pass@1	Pass@10	Notes
HumanEval-X	Python	80.0%	run benchmark_suite.py	164 problems, execution-verified
MultiPL-E (HumanEval subset)	Python	run benchmark_suite.py	—	cross-check vs HumanEval-X
Held-out LeetCode test split	Python	run benchmark_suite.py	—	from leetcode-codegen-python test split, exact I/O match
Tokens/sec (fp16, GPU)	Python	—	—	latency benchmark
Tokens/sec (GGUF q4_k_m)	Python	—	—	latency benchmark

Numbers are intentionally left blank in this template — benchmark_suite.py fills a results/leetcode-python-qwen25-coder-7b.json file and this table should be regenerated from it.

---

Intended use

Drop-in solution generator for Python coding-practice tools, interview-prep apps, and automated code-review sandboxes for algorithmic problems.

Direct use

Give a problem statement (+ optional algorithm hint), get back a Python function/class implementing it.

Downstream use

Feed output into an automated grader (run against test cases), a code-review bot, or a practice-app "show solution" feature.

Out of scope

• Production system design or non-algorithmic code (this model specializes narrowly on LeetCode-style problems)
• Security-critical code without human review
• Guaranteed-optimal complexity — treat output as a strong first draft, not a proof

---

Quickstart

Option A — Transformers + PEFT

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "unsloth/Qwen2.5-Coder-7B-Instruct"
adapter    = "AmareshHebbar/leetcode-python-qwen25-coder-7b"

tokenizer = AutoTokenizer.from_pretrained("AmareshHebbar/leetcode-python-qwen25-coder-7b")
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

messages = [
    {"role": "system", "content": "You are an expert Python competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient Python solution."},
    {"role": "user", "content": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.2, do_sample=True)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

Batch inference (many problems at once)

problems = [
    "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map",
    "Problem: Given a string s, find the length of the longest substring without repeating characters.\nAlgorithm: two pointers / sliding window",
    "Problem: Merge two sorted linked lists into one sorted list.\nAlgorithm: linked list, dummy head",
]

prompts = [
    tokenizer.apply_chat_template(
        [{"role": "system", "content": "You are an expert Python competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient Python solution."}, {"role": "user", "content": p}],
        tokenize=False, add_generation_prompt=True,
    )
    for p in problems
]
tokenizer.padding_side = "left"
batch = tokenizer(prompts, return_tensors="pt", padding=True).to(model.device)
outputs = model.generate(**batch, max_new_tokens=512, temperature=0.2, do_sample=True)
for i, o in enumerate(outputs):
    print(f"--- solution {i} ---")
    print(tokenizer.decode(o[batch['input_ids'].shape[1]:], skip_special_tokens=True))

Streaming output (token-by-token)

from transformers import TextIteratorStreamer
from threading import Thread

streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
gen_kwargs = dict(input_ids=inputs, max_new_tokens=512, temperature=0.2, do_sample=True, streamer=streamer)
Thread(target=model.generate, kwargs=gen_kwargs).start()
for token in streamer:
    print(token, end="", flush=True)

Structured JSON output (code + complexity + explanation)

json_system_prompt = (
    "You are an expert Python competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient Python solution. "
    'Respond ONLY with JSON: {"code": "...", "time_complexity": "...", '
    '"space_complexity": "...", "explanation": "..."}'
)
messages = [
    {"role": "system", "content": json_system_prompt},
    {"role": "user", "content": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.1, do_sample=True)
raw = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)

import json
result = json.loads(raw.strip().removeprefix("```json").removesuffix("```").strip())
print(result["code"])
print(result["time_complexity"], result["space_complexity"])

Option B — Unsloth (2x faster load + inference)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="AmareshHebbar/leetcode-python-qwen25-coder-7b",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

messages = [
    {"role": "system", "content": "You are an expert Python competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient Python solution."},
    {"role": "user", "content": "Problem: Given a string s, find the length of the longest substring without repeating characters.\nAlgorithm: two pointers / sliding window"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Option C — vLLM (production serving, OpenAI-compatible) {#vllm}

vllm serve unsloth/Qwen2.5-Coder-7B-Instruct \
    --enable-lora \
    --lora-modules leetcode-python-qwen25-coder-7b=AmareshHebbar/leetcode-python-qwen25-coder-7b \
    --host 0.0.0.0 --port 8000 --dtype bfloat16

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
response = client.chat.completions.create(
    model="leetcode-python-qwen25-coder-7b",
    messages=[
        {"role": "system", "content": "You are an expert Python competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient Python solution."},
        {"role": "user", "content": "Problem: Merge two sorted linked lists into one sorted list.\nAlgorithm: linked list, dummy head"},
    ],
    temperature=0.2,
)
print(response.choices[0].message.content)

Streaming with vLLM's OpenAI-compatible endpoint:

stream = client.chat.completions.create(
    model="leetcode-python-qwen25-coder-7b",
    messages=[{"role": "user", "content": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Option D — TGI (Text Generation Inference) {#tgi}

docker run --gpus all --shm-size 1g -p 8080:80 \
    -v $PWD/data:/data ghcr.io/huggingface/text-generation-inference:latest \
    --model-id unsloth/Qwen2.5-Coder-7B-Instruct \
    --lora-adapters leetcode-python-qwen25-coder-7b=AmareshHebbar/leetcode-python-qwen25-coder-7b

curl 127.0.0.1:8080/generate_stream \
    -X POST \
    -d '{"inputs":"<|im_start|>system\nYou are an expert Python competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient Python solution.<|im_end|>\n<|im_start|>user\nProblem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map<|im_end|>\n<|im_start|>assistant\n","parameters":{"max_new_tokens":512}}' \
    -H 'Content-Type: application/json'

Option E — Ollama (local, mobile/edge-friendly) {#ollama}

# 1. Pull the GGUF build
huggingface-cli download AmareshHebbar/leetcode-python-qwen25-coder-7b-GGUF leetcode-python-qwen25-coder-7b.q4_k_m.gguf --local-dir .

# 2. Create the model from the Modelfile shipped in the deployment kit (see deploy_ollama.py)
ollama create leetcode-python-qwen25-coder-7b -f Modelfile.python

# 3. Run it
ollama run leetcode-python-qwen25-coder-7b "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"

Python client against a local Ollama server:

import requests
r = requests.post("http://localhost:11434/api/generate", json={
    "model": "leetcode-python-qwen25-coder-7b",
    "prompt": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map",
    "stream": False,
})
print(r.json()["response"])

Option F — GGUF / llama.cpp direct (mobile/edge inference)

./llama-cli -m leetcode-python-qwen25-coder-7b.q4_k_m.gguf \
    -p "<|im_start|>system\nYou are an expert Python competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient Python solution.<|im_end|>\n<|im_start|>user\nProblem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.<|im_end|>\n<|im_start|>assistant\n" \
    -n 512 --temp 0.2

See export_gguf.py in the deployment kit for building q4_k_m / q5_k_m / q8_0 variants, and the mobile integration notes there for Android (llama.cpp JNI) and iOS (llama.cpp via Swift bindings).

---

Training details

Why this base model

Qwen2.5-Coder-7B-Instruct was chosen over a general instruct model because its pretraining already concentrates capacity on code — the QDoRA adapter only has to specialize output format and LeetCode-specific conventions (function signatures, in-place vs. new-array conventions, Python idioms) rather than teach the model to code from scratch. 7B was picked as the size that still fits comfortably in a single-GPU QDoRA run while keeping enough headroom that the base model's code reasoning survives adaptation.

Data pipeline

Source: doocs/leetcode, 3,977 problems with English documentation. Each problem can have multiple solutions spanning different algorithm tags (greedy, DP, two pointers, etc.) — the pipeline treats this as a one-to-many problem-to-solution structure rather than picking a single "canonical" answer.

Stage	What it does
extract_doocs.py	pulls problem statement + I/O examples + per-solution algorithm tag from doocs/leetcode
verify.py	executes each extracted solution against its sample I/O, drops anything that fails
normalize.py	standardizes formatting/whitespace and problem/solution schema across all 4 languages
build_language_datasets.py	splits into per-language configs and writes the final train/val/test SFT rows

~70% of extracted solutions verified (execution-checked against sample I/O). Full extraction/verification/build code lives alongside the leetcode-codegen-python dataset card.

Hyperparameters

Parameter	Value
Method	QDoRA (use_dora=True in PEFT's LoraConfig)
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Base quantization	4-bit NF4
Max sequence length	2048
Optimizer	paged_adamw_8bit
LR schedule	2e-4, cosine

Training compute

GPU	NVIDIA A40 (48GB)
Cloud provider	RunPod
CO2 estimate	self-reported, not measured with a carbon tracker — treat as approximate

Fine-tuned with Unsloth + TRL's SFTTrainer, DoRA enabled via PEFT.

---

Bias, risks & limitations

Narrow specialization. This model is tuned tightly on LeetCode-style algorithmic problems — general software-engineering code (frameworks, infra, business logic) is out of distribution.

Verify before trusting. Like any LLM, generated solutions can look plausible and still fail an edge case (empty input, integer overflow, off-by-one). Always run against test cases before use.

Not exhaustive on complexity. The model doesn't guarantee asymptotically optimal solutions — check the complexity claims yourself for performance-sensitive use.

Data recency. Reflects the state of doocs/leetcode at the time of extraction — newer problems added to LeetCode after that snapshot won't be covered.

---

FAQ

Q: Can I merge the adapter into the base model? Yes — model.merge_and_unload() after loading with PEFT, or Unsloth's save_pretrained_merged(). DoRA adapters merge the same way LoRA adapters do.

Q: Why QDoRA instead of plain QLoRA? See Why QDoRA above — short version: DoRA's magnitude/direction split tracks full fine-tuning more closely, which matters for code correctness.

Q: Why QDoRA instead of full fine-tuning? Qwen2.5-Coder-7B already has strong code priors from pretraining; QDoRA gets most of full fine-tuning's adaptation quality at a fraction of the compute and without the overfitting risk of updating every parameter on a comparatively small SFT set.

Q: Which quantization should I use on mobile? q4_k_m is the best size/quality tradeoff for phones; q5_k_m if you have RAM headroom; avoid q2/q3 for code generation — correctness drops sharply below 4-bit.

Q: Does this model store or transmit my input? No — inference runs entirely on whatever infrastructure you deploy it to.

---

Related models in this suite

Model	Language
leetcode-python-qwen25-coder-7b	Python (this model)
leetcode-java-qwen25-coder-7b	Java
leetcode-cpp-qwen25-coder-7b	C++
leetcode-javascript-qwen25-coder-7b	JavaScript

Full collection: LeetCode Multi-Language Coder Suite

---

Changelog

Version	Notes
v3.0	Switched to QDoRA, added rationale + PEFT config, batch/streaming/JSON inference samples, expanded tags
v2.0	Added GGUF builds, Ollama/vLLM/TGI deployment, benchmark harness (HumanEval-X, MultiPL-E, held-out test split)
v1.0	Initial release — QLoRA fine-tune

---

Citation

@misc{leetcodecoder2026,
  author    = {Hebbar, Amaresh},
  title     = {LeetCode Multi-Language Coder Suite},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/AmareshHebbar}
}

Contact