If you want, you can manually remove unnecessary angles from src faceset after sort by yaw.
Optimized sample generators (CPU workers). Now they consume less amount of RAM and work faster.
added
4.2.other) data_src/dst util faceset pack.bat
Packs /aligned/ samples into one /aligned/samples.pak file.
After that, all faces will be deleted.
4.2.other) data_src/dst util faceset unpack.bat
unpacks faces from /aligned/samples.pak to /aligned/ dir.
After that, samples.pak will be deleted.
Packed faceset load and work faster.
4.2.other) data_src util faceset metadata save.bat
saves metadata of data_src\aligned\ faces into data_src\aligned\meta.dat
4.2.other) data_src util faceset metadata restore.bat
restore metadata from 'meta.dat' to images
if image size different from original, then it will be automatically resized
You can greatly enhance face details of src faceset by using Topaz Gigapixel software.
example https://i.imgur.com/Gwee99L.jpg
Example of workflow:
1) run 'data_src util faceset metadata save.bat'
2) launch Topaz Gigapixel
3) open 'data_src\aligned\' and select all images
4) set output folder to 'data_src\aligned_topaz' (create folder in save dialog)
5) set settings as on screenshot https://i.imgur.com/kAVWMQG.jpg
you can choose 2x, 4x, or 6x upscale rate
6) start process images and wait full process
7) rename folders:
data_src\aligned -> data_src\aligned_original
data_src\aligned_topaz -> data_src\aligned
8) copy 'data_src\aligned_original\meta.dat' to 'data_src\aligned\'
9) run 'data_src util faceset metadata restore.bat'
images will be downscaled back to original size (256x256) preserving details
metadata will be restored
10) now your new enhanced faceset is ready to use !
This is sort method by absolute per pixel difference between all faces.
options:
Sort by similar? ( y/n ?:help skip:y ) :
if you choose 'n', then most dissimilar faces will be placed first.
Synthesize new faces from existing ones by relighting them using DeepPortraitRelighter network.
With the relighted faces neural network will better reproduce face shadows.
Therefore you can synthsize shadowed faces from fully lit faceset.
https://i.imgur.com/wxcmQoi.jpg
as a result, better fakes on dark faces:
https://i.imgur.com/5xXIbz5.jpg
in OpenCL build Relighter runs on CPU,
install pytorch directly via pip install, look at requirements
fixed model sizes from previous update.
avoided bug in ML framework(keras) that forces to train the model on random noise.
Converter: added blur on the same keys as sharpness
Added new model 'TrueFace'. This is a GAN model ported from https://github.com/NVlabs/FUNIT
Model produces near zero morphing and high detail face.
Model has higher failure rate than other models.
Keep src and dst faceset in same lighting conditions.
Session is now saved to the model folder.
blur and erode ranges are increased to -400+400
hist-match-bw is now replaced with seamless2 mode.
Added 'ebs' color transfer mode (works only on Windows).
FANSEG model (used in FAN-x mask modes) is retrained with new model configuration
and now produces better precision and less jitter
With interactive converter you can change any parameter of any frame and see the result in real time.
Converter: added motion_blur_power param.
Motion blur is applied by precomputed motion vectors.
So the moving face will look more realistic.
RecycleGAN model is removed.
Added experimental AVATAR model. Minimum required VRAM is 6GB (NVIDIA), 12GB (AMD)
Usage:
1) place data_src.mp4 10-20min square resolution video of news reporter sitting at the table with static background,
other faces should not appear in frames.
2) process "extract images from video data_src.bat" with FULL fps
3) place data_dst.mp4 video of face who will control the src face
4) process "extract images from video data_dst FULL FPS.bat"
5) process "data_src mark faces S3FD best GPU.bat"
6) process "data_dst extract unaligned faces S3FD best GPU.bat"
7) train AVATAR.bat stage 1, tune batch size to maximum for your card (32 for 6GB), train to 50k+ iters.
8) train AVATAR.bat stage 2, tune batch size to maximum for your card (4 for 6GB), train to decent sharpness.
9) convert AVATAR.bat
10) converted to mp4.bat
updated versions of modules
Pretrain the model with large amount of various faces. This technique may help to train the fake with overly different face shapes and light conditions of src/dst data. Face will be look more like a morphed. To reduce the morph effect, some model files will be initialized but not be updated after pretrain: LIAE: inter_AB.h5 DF: both decoders.h5. The longer you pretrain the model the more morphed face will look. After that, save and run the training again.
Pixel loss may help to enhance fine details and stabilize face color. Use it only if quality does not improve over time.
SAE:
previous SAE model will not work with this update.
Greatly decreased chance of model collapse.
Increased model accuracy.
Residual blocks now default and this option has been removed.
Improved 'learn mask'.
Added masked preview (switch by space key)
Converter:
fixed rct/lct in seamless mode
added mask mode (6) learned*FAN-prd*FAN-dst
added mask editor, its created for refining dataset for FANSeg model, and not for production, but you can spend your time and test it in regular fakes with face obstructions
Fixed bug when SAE can be collapsed during a time.
SAE: removed CA weights and encoder/decoder dims.
added new options:
Encoder dims per channel (21-85 ?:help skip:%d)
More encoder dims help to recognize more facial features, but require more VRAM. You can fine-tune model size to fit your GPU.
Decoder dims per channel (11-85 ?:help skip:%d)
More decoder dims help to get better details, but require more VRAM. You can fine-tune model size to fit your GPU.
Add residual blocks to decoder? (y/n, ?:help skip:n) :
These blocks help to get better details, but require more computing time.
Remove gray border? (y/n, ?:help skip:n) :
Removes gray border of predicted face, but requires more computing resources.
added Intel's plaidML backend to use OpenCL engine. Check new requirements.
smart choosing of backend in device.py
env var 'force_plaidML' can be choosed to forced using plaidML
all tf functions transferred to pure keras
MTCNN transferred to pure keras, but it works slow on plaidML (forced to CPU in this case)
default batch size for all models and VRAMs now 4, feel free to adjust it on your own
SAE: default style options now ZERO, because there are no best values for all scenes, set them on your own.
SAE: return back option pixel_loss, feel free to enable it on your own.
SAE: added option multiscale_decoder default is true, but you can disable it to get 100% same as H,DF,LIAEF model behaviour.
fix converter output to .png
added linux fork reference to doc/doc_build_and_repository_info.md
Model files names will be prefixed with GPU index if GPU choosed explicitly on train/convert start.
if you leave GPU idx choice default, then best GPU idx will be choosed and model file names will not contain index prefix.
It gives you possibility to train same fake with various models or options on multiple GPUs.
H64 and H128: now you can choose 'Lighter autoencoder'. It is same as vram gb <= 4 before this update.
added archived_models.zip contains old experiments
RecycleGAN: archived
devicelib: if your system has no NVML installed (some old cards), then it will work with gpu_idx=0 as 'Generic GeForce GPU' with 2GB vram.
refactorings
SAE: new option 'lighter_encoder'.
Now model settings can be overrided by pressing enter in 2 seconds while model loading (works on Windows).
Removed all MultiGPU models, because keras multi_gpu in fact doesn't work.
removed AVATAR - useless model was just for demo
removed MIAEF128 - use UFM insted
removed LIAEF128YAW - use model option sort by yaw on start for any model
All models now ask some options on start.
Session options (such as target epoch, batch_size, write_preview_history etc) can be overrided by special command arg.
Converter now always ask options and no more support to define options via command line.
fix bug when ConverterMasked always used not predicted mask.
SampleGenerator now always generate samples with replicated border, exclude mask samples.
refactorings
* fix localization nullpointer exception
* fix devicelib error line:61,remove e
* support create docker from cpu dockerfile
* support preview or not when train(resolve cannot connect to X server)
