incorrectly formatted yaml file for long file name #3119
Description
Is there an existing issue for this?
- I have searched the existing issues
Operating System
windows
DeepLabCut version
'3.0.0rc13'
What engine are you using?
pytorch
DeepLabCut mode
single animal
Device type
5090
Bug description 🐛
Hi,
The config YAML cannot handle a long file name with a space; whenever I call deeplabcut.convertcsv2h5, it attempts to format my config file and eventually make the YAML invalid. I think the root cause is that ruamel.yaml cannot reliably store the YAML setting. Here is a simple repex
from ruamel.yaml import YAML
ruamelFile = YAML()
# a dummpy config file
cfg = {
'video_sets': {
'G:/My Drive/lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllong_name.mp4': {
'crop': [0, 1280, 0, 720]
}
}
}
# save to yaml
with open("test.yaml", "w") as cf:
ruamelFile.dump(cfg, cf)
# Error: mapping values are not allowed here
with open("test.yaml", "r") as f:
cfg = ruamelFile.load(f)
Here is the resulting YAML:
video_sets:
G:/My
Drive/lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllong_name.mp4:
crop:
- 0
- 1280
- 0
- 720
It seems like ruamelFile.dump is not happy with the space in the video path. However, there is no way for me to avoid it since it is a Google Drive. using ruamelFile = YAML(typ="full") can solve the issue, but it will result in an unreadable YAML (I assume the motivation for using ruamel is readability? Otherwise yaml package can dump the data perfectly fine)
I found a similar issue in this repository: #3063
Steps To Reproduce
No response
Relevant log output
Anything else?
This is the actual config file deeplabcut.convertcsv2h5 will create
# Project definitions (do not edit)
Task: Sleap_Rat_test
scorer: T
date: Oct2
multianimalproject: false
identity:
# Project path (change when moving around)
project_path: G:\My
Drive\projects\Green_Tom\rat_pose\post_model\projects\rat_pose
# Default DeepLabCut engine to use for shuffle creation (either pytorch or tensorflow)
engine: pytorch
# Annotation data set configuration (and individual video cropping parameters)
video_sets:
G:/My
Drive/projects/Green_Tom/rat_pose/post_model/projects/rat_pose/videos/ai1.mp4:
crop: 0, 1280, 0, 720
G:/My
Drive/projects/Green_Tom/rat_pose/post_model/projects/rat_pose/videos/ai2.mp4:
crop: 0, 1280, 0, 720
G:/My
Drive/projects/Green_Tom/rat_pose/post_model/projects/rat_pose/videos/ai3.mp4:
crop: 0, 1280, 0, 720
G:/My
Drive/projects/Green_Tom/rat_pose/post_model/projects/rat_pose/videos/ai4.mp4:
crop: 0, 1280, 0, 720
G:/My
Drive/projects/Green_Tom/rat_pose/post_model/projects/rat_pose/videos/ai5.mp4:
crop: 0, 1280, 0, 720
G:/My
Drive/projects/Green_Tom/rat_pose/post_model/projects/rat_pose/videos/ai6.mp4:
crop: 0, 1280, 0, 720
G:/My
Drive/projects/Green_Tom/rat_pose/post_model/projects/rat_pose/videos/ai7.mp4:
crop: 0, 1280, 0, 720
G:/My
Drive/projects/Green_Tom/rat_pose/post_model/projects/rat_pose/videos/ai8.mp4:
crop: 0, 1280, 0, 720
G:\My Drive\projects\Green Tom\rat_pose\post_model\videos\Camera4_stitched.mp4:
crop: 0, 1280, 0, 720
G:\My Drive\projects\Green Tom\rat_pose\post_model\videos\RAT 11 FR1.mp4:
crop: 0, 1280, 0, 720
bodyparts:
- head
- nose
- spine1
- spine2
- spine3
- tailbase
- tail1
- tail2
- tail_tip
- L_hip
- L_backpaw
- R_backpaw
- L_shoulder
- R_frontpaw
- R_shoulder
- R_hip
- R_knee
- L_knee
- L_frontpaw
# Fraction of video to start/stop when extracting frames for labeling/refinement
start: 0
stop: 1
numframes2pick: 20
# Plotting configuration
skeleton:
- - head
- nose
- - head
- spine1
- - spine1
- spine2
- - spine2
- spine3
- - spine1
- L_shoulder
- - L_shoulder
- L_frontpaw
- - spine3
- L_hip
- - L_hip
- L_knee
- - L_knee
- L_backpaw
- - spine3
- tailbase
- - tailbase
- tail1
- - tail1
- tail2
- - tail2
- tail_tip
- - spine1
- R_shoulder
- - R_shoulder
- R_frontpaw
- - spine3
- R_hip
- - R_hip
- R_knee
- - R_knee
- R_backpaw
skeleton_color: black
pcutoff: 0.6
dotsize: 12
alphavalue: 0.7
colormap: rainbow
# Training,Evaluation and Analysis configuration
TrainingFraction:
- 0.95
iteration: 0
default_net_type: resnet_50
default_augmenter: default
snapshotindex: -1
detector_snapshotindex: -1
batch_size: 8
detector_batch_size: 1
# Cropping Parameters (for analysis and outlier frame detection)
cropping: false
#if cropping is true for analysis, then set the values here:
x1: 0
x2: 640
y1: 277
y2: 624
# Refinement configuration (parameters from annotation dataset configuration also relevant in this stage)
corner2move2:
- 50
- 50
move2corner: true
# Conversion tables to fine-tune SuperAnimal weights
SuperAnimalConversionTables:
It might be somehow related to the recent change in ruamel-yaml. Downgrading ruamel in the package requirement might help.
https://sourceforge.net/p/ruamel-yaml/tickets/546/
Code of Conduct
- I agree to follow this project's Code of Conduct