Codeformer
Round avatar of the codeformer author: small illustrated portrait on a dark circular background sczhou / codeformer

Codeformer restores lost faces in old scans and AI portraits

Codeformer brings back eyes, skin texture and hair in faded prints, compressed selfies and generated images, while keeping the person recognisable.

Ready Public Open source L40S License Source Paper

Input

Soft, low-detail AI-generated portrait of a smiling blonde woman in a dark bikini top on a sunny beach, blurred sea behind her, square framing
* file

Portrait you want restored

number
(min 0, max 1)

Lower values favour a polished look; higher values stay truer to the original face.

Default: 0.5

boolean

Also clean up the surroundings using Real-ESRGAN

Default: true

boolean

Sharpen restored faces further for high-resolution generated images

Default: true

integer

Scale factor applied to the finished picture

Default: 2

{
  "image": ".../010.jpg",
  "codeformer_fidelity": 0.1,
  "background_enhance": true,
  "face_upsample": true,
  "upscale": 2
}

Call Codeformer from a Node.js script:

const out = await client.run(model, { input });

Call Codeformer from Python:

out = client.run(model, input=payload)

Send the same payload over plain HTTP:

curl -X POST "$API_URL/predictions" -d @input.json

Predict locally with Cog:

cog predict -i [email protected] -i upscale=2

Serve the container on your own GPU:

docker run --gpus=all -p 5000:5000 $IMAGE

Run Codeformer on your own photo

Upload a portrait and compare the result side by side. Want ideas first? Browse the samples below.

Upload a portrait

Opens the partner workspace
in a new step.

Output

Codeformer restored portrait of a smiling blonde woman on a beach with crisp eyes, teeth and hair strands, sea in soft focus Original blurry beach portrait of a smiling blonde woman before restoration Before After
{
  "status": "succeeded",
  "logs": "detect 1 faces",
  "output": ".../output.png"
}

Sample settings

Fidelity 0.1 · 2× upscale

detect 1 faces

This sample came from an earlier build of the model, sczhou/codeformer:7de2ea26.

Full-size restored beach portrait of a smiling blonde woman with sharp facial detail

Examples

More samples
Two smiling graduates in red and gold academic gowns and black caps holding a diploma in front of a blue backdrop Classic film-era portrait of a dark-haired woman in an off-shoulder white gown inside a stone hall, restored detail Close-up of a young brunette woman among green leaves holding an ornate gold fan, sharp eyes after restoration Movie still of a smiling young Asian woman in a white shirt looking at someone off frame, warm tones Smiling blonde woman in a navy bikini top on a sandy beach, crisp restored face Young woman with a yellow flower headband and bright floral shawl standing in a dark tropical jungle, moody cinematic light

Hardware and runtime

Codeformer is open source, so besides calling it in the cloud you can host it yourself with Docker. Processing time depends on image size, how many faces are detected and the upscale factor you choose.

In the cloud, this Codeformer build runs on Nvidia L40S GPU hardware. Single portraits are usually quick; group shots with background enhancement take longer.

Readme

Codeformer project logo: black curly brace and red pixel dots above the CODEFORMER wordmark on white

Codeformer paper: blind face restoration via a learned codebook (NeurIPS 2022) Paper | Project page | Video

Visitor counter badge

This demo is meant for research and testing. You can also run the Codeformer code on your own machine or inside a hosted notebook.


☕️ Codeformer is a sturdy face restorer for aged photographs and generated portraits. 🚀 Pair Codeformer with stable-diffusion output for cleaner, more natural faces!

Found Codeformer useful? Give the project a ⭐ on its source repository. Much appreciated!

Repository star count badge

📋 License — The Codeformer project is released under the S-Lab License 1.0. Redistribution and non-commercial use must respect its terms, and the hosted API version may not be used for commercial work.

📝 Citation — If the model supports your research, please reference it as follows:

@inproceedings{zhou2022codeformer,
    author = {Zhou, Shangchen and Chan, Kelvin C.K. and Li, Chongyi and Loy, Chen Change},
    title = {Towards Robust Blind Face Restoration with Codebook Lookup TransFormer},
    booktitle = {NeurIPS},
    year = {2022}
}

Model published September 2022

Latest from the blog

All articles →
Start now
Start now