๐ญ Affecto - Real MagicFace Emotion Transformation
Transform facial emotions using Action Units with the complete MagicFace diffusion model
โก Running on: CPU | โฑ๏ธ Estimated time: ~10-15 minutes per transformation on CPU
๐๏ธ Action Unit Parameters
20 100
โ ๏ธ Note: Transformation takes ~10-15 minutes on CPU. Please be patient!
๐จ Emotion Presets (click to use):
Examples
| AU Parameters (JSON) | Inference Steps (20=fast/lower quality, 50=balanced, 100=slow/high quality) | Seed (for reproducibility) |
|---|
๐ API Usage
Gradio API Endpoint
POST https://gauravvjhaa-affecto-inference.hf.space/api/predict
import requests
import base64
# Prepare image
with open("face.jpg", "rb") as f:
image_base64 = base64.b64encode(f.read()).decode()
# Call API
response = requests.post(
"https://gauravvjhaa-affecto-inference.hf.space/api/predict",
json={
"data": [
f"data:image/jpeg;base64,{image_base64}",
'{"AU6": 2.0, "AU12": 2.0}',
50, # steps
424 # seed
]
}
)
result = response.json()
# Result image is in result["data"][0]
๐ญ Available Action Units (AU):
๐ Happiness:
- AU6 (Cheek Raiser): 0-4 โ Raises cheeks, squints eyes
- AU12 (Lip Corner Puller): 0-4 โ Pulls lip corners up (smile)
๐ข Sadness:
- AU1 (Inner Brow Raiser): 0-4 โ Raises inner brows
- AU4 (Brow Lowerer): 0-4 โ Lowers brows (frown)
- AU15 (Lip Corner Depressor): 0-4 โ Pulls lip corners down
๐ Anger:
- AU4 (Brow Lowerer): 0-4
- AU5 (Upper Lid Raiser): 0-4
- AU7 (Lid Tightener): 0-4
- AU23 (Lip Tightener): 0-4
๐ฎ Surprise:
- AU1 (Inner Brow Raiser): 0-4
- AU2 (Outer Brow Raiser): 0-4
- AU5 (Upper Lid Raiser): 0-4
- AU26 (Jaw Drop): 0-4
๐จ Fear:
- AU1 (Inner Brow Raiser): 0-4
- AU2 (Outer Brow Raiser): 0-4
- AU4 (Brow Lowerer): 0-4
- AU5 (Upper Lid Raiser): 0-4
- AU20 (Lip Stretcher): 0-4
- AU25 (Lips Part): 0-4
๐ Example Combinations:
# Happy (smile)
{"AU6": 2.5, "AU12": 3.0}
# Sad
{"AU1": 2.0, "AU4": 2.5, "AU15": 2.5}
# Angry
{"AU4": 3.0, "AU5": 2.0, "AU7": 2.5, "AU23": 2.0}
# Surprised
{"AU1": 3.0, "AU2": 2.5, "AU5": 3.0, "AU26": 2.5}
# Fear
{"AU1": 2.5, "AU2": 2.5, "AU4": 2.0, "AU5": 3.0, "AU20": 2.0, "AU25": 2.0}
โ๏ธ Pipeline Details:
- Face Detection โ InsightFace detects face
- Cropping โ Crop to 512ร512 with alignment
- Background Extraction โ BiSeNet segments face from background
- Diffusion โ MagicFace transforms expression
- Output โ 512ร512 transformed face
โฑ๏ธ Performance:
- CPU: ~10-15 minutes per image
- GPU (T4): ~15-20 seconds per image
๐ Notes:
- Input images should contain at least one face
- If multiple faces, largest face is used
- AU values typically range 0-4 (higher = stronger)
- Seed ensures reproducibility
๐ญ Affecto - MagicFace Implementation
Architecture:
This is a complete implementation of the MagicFace diffusion model for facial emotion transformation.
Components:
- InsightFace: Face detection and landmark extraction
- BiSeNet: Face segmentation for background extraction
- Stable Diffusion 1.5: Base diffusion model
- Custom UNets: ID encoder + Denoising UNet from MagicFace
- Action Units: FACS-based emotion parameters
References:
- MagicFace Paper: arxiv.org/abs/2408.00623
- MagicFace Code: github.com/weimengting/MagicFace
- HuggingFace Model: huggingface.co/mengtingwei/magicface
Project Info:
- Project: Affecto - Facial Emotion Transformation System
- Implementation: Complete MagicFace pipeline with preprocessing
- Hardware: CPU (free tier) - upgrade to GPU for faster inference
Made by Gaurav Jha and Raj Shakya ๐
Built using Gradio, PyTorch, and Diffusers