The coaching half is simple. To coach the neural community together with your customized dataset, merely run the next command within the terminal:
python practice.py --net scene_flow_motion_field ^
--dataset custom_sequence --track_id customized ^
--log_time --epoch_batches 2000 --epoch 10 ^
--lr 1e-6 --html_logger --vali_batches 150 ^
--batch_size 1 --optim adam --vis_batches_vali 1 ^
--vis_every_vali 1 --vis_every_train 1 ^
--vis_batches_train 1 --vis_at_start --gpu 0 ^
--save_net 1 --workers 1 --one_way ^
--loss_type l1 --l1_mul 0 --acc_mul 1 ^
--disp_mul 1 --warm_sf 5 --scene_lr_mul 1000 ^
--repeat 1 --flow_mul 1 --sf_mag_div 100 ^
--time_dependent --gaps 1,2,4,6,8 --midas ^
--use_disp --logdir 'logdir/' ^
--suffix 'track_{track_id}' ^
--force_overwrite
After coaching the neural community for 10 epochs, I noticed that the loss started to saturate, and in consequence, I made a decision to not proceed coaching for added epochs. Under is the loss curve graph of my coaching:
All through the coaching, all checkpoints are saved within the listing ./logdir/nets/. Moreover, after each epoch, the coaching script generates check visualizations within the listing ./logdir/visualize. These visualizations may be notably useful in figuring out any potential points that may have occurred throughout coaching, along with monitoring the loss.
Utilizing the most recent checkpoint, we now generate the depth map of every body with check.py script. Merely run the next command within the terminal:
python check.py --net scene_flow_motion_field ^
--dataset custom_sequence --workers 1 ^
--output_dir .test_resultscustom_sequence ^
--epoch 10 --html_logger --batch_size 1 ^
--gpu 0 --track_id customized --suffix customized ^
--checkpoint_path .logdir
This can generate one .npz file for every body(a dictionary file consisting of RGB body, depth, digital camera pose, movement to subsequent picture, and so forth), and three depth renders (floor fact, MiDaS, and educated community’s estimation) for every body.
Within the final step, we load the batched .npz information frame-by-frame and create coloured level clouds by utilizing the depth and RGB data. I exploit the open3d library to create and render level clouds in Python. It’s a highly effective instrument with which you’ll be able to create digital cameras in 3D area and take captures of your level clouds with them. You too can edit/manipulate your level clouds; I utilized open3d’s built-in outlier removing capabilities to take away flickery and noisy factors.
Whereas I received’t delve into the precise particulars of my open3d utilization to maintain this weblog put up succinct, I’ve included the script, render_pointcloud_video.pywhich ought to be self-explanatory. If in case you have any questions or require additional clarification, please don’t hesitate to ask.
Here’s what the purpose cloud and depth map movies seem like for the video I processed.
The next-resolution model of this animation is uploaded to YouTube.
