How to Run Qwen Locally — Alibaba's Powerful Multilingual Model

# Install Ollama if you haven't
curl -fsSL https://ollama.com/install.sh | sh

# Run Qwen 2.5 7B
ollama run qwen2.5:7b

# Or try other sizes
ollama run qwen2.5:3b    # For 4GB RAM
ollama run qwen2.5:14b   # For 16GB RAM

>>> Write a Python FastAPI endpoint that accepts a file upload,
    validates it's a PDF under 10MB, and stores it in S3.

```python
from fastapi import FastAPI, UploadFile, HTTPException
import boto3
from botocore.exceptions import ClientError

app = FastAPI()
s3 = boto3.client("s3")
MAX_SIZE = 10 * 1024 * 1024  # 10MB

@app.post("/upload")
async def upload_pdf(file: UploadFile):
    if not file.filename.endswith(".pdf"):
        raise HTTPException(400, "Only PDF files are accepted")

    content = await file.read()
    if len(content) > MAX_SIZE:
        raise HTTPException(400, "File must be under 10MB")

    try:
        s3.put_object(
            Bucket="my-uploads",
            Key=file.filename,
            Body=content,
            ContentType="application/pdf",
        )
    except ClientError:
        raise HTTPException(500, "Upload failed")

    return {"filename": file.filename, "size": len(content)}

### Multilingual

Qwen handles multiple languages well:

### General Reasoning

## Performance Benchmarks (Q4_K_M)

On an M2 MacBook Pro (16GB):

| Model | Tokens/sec | First token | RAM usage |
|-------|-----------|-------------|-----------|
| Qwen 2.5 3B | ~45 | ~0.3s | ~3 GB |
| Qwen 2.5 7B | ~28 | ~0.5s | ~6 GB |
| Qwen 2.5 14B | ~14 | ~1.0s | ~11 GB |

On an RTX 4090:

| Model | Tokens/sec | First token | VRAM usage |
|-------|-----------|-------------|-----------|
| Qwen 2.5 7B | ~90 | ~0.2s | ~5 GB |
| Qwen 2.5 14B | ~55 | ~0.3s | ~10 GB |
| Qwen 2.5 32B | ~25 | ~0.6s | ~20 GB |

## When to Use Qwen vs Other Models

| Need | Best Model |
|------|-----------|
| Coding tasks | Qwen 2.5 7B+ or DeepSeek R1 |
| Chinese language | Qwen 2.5 (best choice) |
| General chat | Llama 3.1 or Qwen 2.5 |
| Reasoning/math | DeepSeek R1 |
| Fast responses | Llama 3.2 3B |
| Low RAM (4GB) | Qwen 2.5 3B |

## Summary

Qwen 2.5 is an excellent choice for local AI, especially if you work with code or multiple languages. The 7B model fits on 8GB RAM devices and delivers strong performance for daily tasks.

## Next Steps

- [Best Models for 8GB RAM](/blog/models-for-8gb-ram) — compare Qwen with other models
- [How to Run Llama Locally](/blog/how-to-run-llama-locally) — compare with Meta's Llama
- [How to Run DeepSeek Locally](/blog/how-to-run-deepseek-locally) — another top model