Training cannot proceed

[xueshuo123]

Your work is excellent, and I am very interested in it. However, when reproducing it, it has been stuck at 0 iterations and cannot proceed further. Do you know the reason? thank you

(SDS) b101-8-3090@8-3090:~/xueshuoopt/SDS/SDS$ python sds.py task=[bound,hop,pace,trot]
[2024-11-22 17:12:04,055][root][INFO] - Workspace: /home/b101-8-3090/xueshuoopt/SDS/SDS/outputs/sds/2024-11-22_17-12-04
[2024-11-22 17:12:04,055][root][INFO] - Project Root: /home/b101-8-3090/xueshuoopt/SDS/SDS
[2024-11-22 17:12:04,055][root][INFO] - Running for 5 iterations
[2024-11-22 17:12:04,055][root][INFO] - Training each RF for: 1000 iterations
[2024-11-22 17:12:04,055][root][INFO] - Generating 8 reward function samples per iteration
[2024-11-22 17:12:04,055][root][INFO] - Using LLM: gpt-4o
[2024-11-22 17:12:04,055][root][INFO] - Imitation Task: Quadruped Robot Trotting
[2024-11-22 17:12:04,055][root][INFO] - Demonstration Video: trot.mp4, Crop Option: None
[2024-11-22 17:12:16,769][agents][INFO] - Activated Agent SUSGenerator
[2024-11-22 17:12:16,769][agents][INFO] - Activated Agent TaskDescriptor
[2024-11-22 17:12:39,012][agents][INFO] - Activated Agent ContactSequenceAnalyser
[2024-11-22 17:13:21,584][agents][INFO] - Activated Agent GaitAnalyser
[2024-11-22 17:13:38,650][agents][INFO] - Activated Agent TaskRequirementAnalyser
[2024-11-22 17:13:58,834][root][INFO] - Iteration 0: Generating 8 samples with gpt-4o
[2024-11-22 17:14:58,631][root][INFO] - Iteration 0: Prompt Tokens: 3117, Completion Tokens: 5128, Total Tokens: 11362
[2024-11-22 17:14:58,631][root][INFO] - Iteration 0: Processing Code Run 0

[HaoGeDiao]

ML_LOGGER_USER is not set. This is required for online usage.
creating new logging client...
✓ created a new logging client
Dashboard: http://app.dash.ml/forward_locomotion_sds/2024-12-23/train/120146.038554
Log_directory: /home/wth/SDS/forward_locomotion_sds/runs
Importing module 'gym_38' (/home/wth/SDS/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/wth/SDS/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 2.4.0+cu121
Device count 1
/home/wth/SDS/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /root/.cache/torch_extensions/py38_cu121 as PyTorch extensions root...
Emitting ninja build file /root/.cache/torch_extensions/py38_cu121/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Training on 4000 environments
Running with headless and no recording, disabled graphics rendering
[Warning] [carb.gym.plugin] useGpuPipeline is set, forcing GPU PhysX
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Traceback (most recent call last):
File "/home/wth/SDS/SDS/../forward_locomotion_sds/scripts/train.py", line 120, in
train_mc(iterations=args.iterations, command_config=args.command_config, reward_config=args.reward_config, dr_config=args.dr_config, sds_target_velocity=args.sds_target_velocity,
File "/home/wth/SDS/SDS/../forward_locomotion_sds/scripts/train.py", line 61, in train_mc
env = VelocityTrackingEasyEnv(sim_device='cuda:0', headless=headless, cfg=Cfg)
File "/home/wth/SDS/forward_locomotion_sds/go1_gym/envs/mini_cheetah/velocity_tracking/velocity_tracking_easy_env.py", line 40, in init
super().init(cfg, sim_params, physics_engine, sim_device, headless, eval_cfg, initial_dynamics_dict)
File "/home/wth/SDS/forward_locomotion_sds/go1_gym/envs/base/legged_robot.py", line 54, in init
self._prepare_reward_function()
File "/home/wth/SDS/forward_locomotion_sds/go1_gym/envs/base/legged_robot.py", line 1171, in _prepare_reward_function
_, reward_components = self.reward_container.compute_reward()
File "/home/wth/SDS/forward_locomotion_sds/go1_gym/rewards/sds_reward.py", line 23, in compute_reward
velocity_reward = torch.exp(-torch.square(self.base_lin_vel[:, 0] - desired_velocity))
AttributeError: 'SDSReward' object has no attribute 'base_lin_vel'

[JeliPenguin]

@xueshuo123 After a deep investigation, it seems to be due to a GPU incompatibility issue with running headless simulations which throws seg faults but wasn't handled appropriately. We used NVIDIA 4090 and had no issues though we did run into this with A4000. Would you share the GPU that you used?

[JeliPenguin]

@HaoGeDiao This is the expected behaviour as not all reward functions generated within a batch will be valid functions. You should let the training proceed and see what happens.