Synthesize new faces from existing ones by relighting them using DeepPortraitRelighter network.
With the relighted faces neural network will better reproduce face shadows.
Therefore you can synthsize shadowed faces from fully lit faceset.
https://i.imgur.com/wxcmQoi.jpg
as a result, better fakes on dark faces:
https://i.imgur.com/5xXIbz5.jpg
in OpenCL build Relighter runs on CPU,
install pytorch directly via pip install, look at requirements
Session is now saved to the model folder.
blur and erode ranges are increased to -400+400
hist-match-bw is now replaced with seamless2 mode.
Added 'ebs' color transfer mode (works only on Windows).
FANSEG model (used in FAN-x mask modes) is retrained with new model configuration
and now produces better precision and less jitter
With interactive converter you can change any parameter of any frame and see the result in real time.
Converter: added motion_blur_power param.
Motion blur is applied by precomputed motion vectors.
So the moving face will look more realistic.
RecycleGAN model is removed.
Added experimental AVATAR model. Minimum required VRAM is 6GB (NVIDIA), 12GB (AMD)
Usage:
1) place data_src.mp4 10-20min square resolution video of news reporter sitting at the table with static background,
other faces should not appear in frames.
2) process "extract images from video data_src.bat" with FULL fps
3) place data_dst.mp4 video of face who will control the src face
4) process "extract images from video data_dst FULL FPS.bat"
5) process "data_src mark faces S3FD best GPU.bat"
6) process "data_dst extract unaligned faces S3FD best GPU.bat"
7) train AVATAR.bat stage 1, tune batch size to maximum for your card (32 for 6GB), train to 50k+ iters.
8) train AVATAR.bat stage 2, tune batch size to maximum for your card (4 for 6GB), train to decent sharpness.
9) convert AVATAR.bat
10) converted to mp4.bat
updated versions of modules
Fixing an issue caused by attempting to sort Path objects. Directly using `<` is unsupported between these, so `sorted()` needs a key specified.
"PurePath" objects support `>` while normal paths do not, causing the confusion.
https://docs.python.org/3/library/pathlib.html
Linux does not guarantee filenames are returned in any specific order. This leads to exporting frames in random order, sorting them here makes the export run sequentially. Other portions of the program should remain unaffected, if not behave more consistently (E.G. get_first_file_by_stem).
This mostly helpful during exporting. Say you are expecting to not have faces for frames 1000-2000, during your export all the "no faces for..." messages will appear in random order. Since you are expecting to see this you ignore them. If you are also (unexpectedly) missing a face for frame 3000 you will not head the warning since it's mixed up in all the warnings that you are expecting. With this patch export runs in sequential order, you'll see the messages all in a row for frames 1000-2000, then again at 3000. The user is much more likely to see and head the warning this way.
This also allows you to force stop the export midway though and have a contiguous set of frames to encode and preview.
Enable autobackup? (y/n ?:help skip:%s) :
Autobackup model files with preview every hour for last 15 hours. Latest backup located in model/<>_autobackups/01
SAE: added option only for CUDA builds:
Enable gradient clipping? (y/n, ?:help skip:%s) :
Gradient clipping reduces chance of model collapse, sacrificing speed of training.
Pixel loss may help to enhance fine details and stabilize face color. Use it only if quality does not improve over time.
SAE:
previous SAE model will not work with this update.
Greatly decreased chance of model collapse.
Increased model accuracy.
Residual blocks now default and this option has been removed.
Improved 'learn mask'.
Added masked preview (switch by space key)
Converter:
fixed rct/lct in seamless mode
added mask mode (6) learned*FAN-prd*FAN-dst
added mask editor, its created for refining dataset for FANSeg model, and not for production, but you can spend your time and test it in regular fakes with face obstructions
added Intel's plaidML backend to use OpenCL engine. Check new requirements.
smart choosing of backend in device.py
env var 'force_plaidML' can be choosed to forced using plaidML
all tf functions transferred to pure keras
MTCNN transferred to pure keras, but it works slow on plaidML (forced to CPU in this case)
default batch size for all models and VRAMs now 4, feel free to adjust it on your own
SAE: default style options now ZERO, because there are no best values for all scenes, set them on your own.
SAE: return back option pixel_loss, feel free to enable it on your own.
SAE: added option multiscale_decoder default is true, but you can disable it to get 100% same as H,DF,LIAEF model behaviour.
fix converter output to .png
added linux fork reference to doc/doc_build_and_repository_info.md
SAE: new option 'lighter_encoder'.
Now model settings can be overrided by pressing enter in 2 seconds while model loading (works on Windows).
Removed all MultiGPU models, because keras multi_gpu in fact doesn't work.
removed AVATAR - useless model was just for demo
removed MIAEF128 - use UFM insted
removed LIAEF128YAW - use model option sort by yaw on start for any model
All models now ask some options on start.
Session options (such as target epoch, batch_size, write_preview_history etc) can be overrided by special command arg.
Converter now always ask options and no more support to define options via command line.
fix bug when ConverterMasked always used not predicted mask.
SampleGenerator now always generate samples with replicated border, exclude mask samples.
refactorings
We cannot use GPU for lab converter in converter multiprocesses, because almost all VRAM ate by model process, so even 300Mb free VRAM not enough for tensorflow lab converter.
Removed skimage dependency.
Refactorings.
Adds improvements to the manual selection process. You can now
click to lock the selection box, and can also go forward and backwards
to modify selections after they have been made. You can also now push
'Q' to skip all remaining images to quickly save the extractions you
have already made.
