无法复现SSv2准确率

[ykx3]

您好，我在尝试训练VideoMamba-tiny时发现训练结果与您论文中报告的准确率相差很大，只获得了60.97的top1准确率，下面是我的python环境和训练的详细日志，您能看看是不是哪里出了问题吗？顺便，我使用了4张2080ti，没有使用amp和bf16（另一个问题是一旦启用amp那么很快会出现NaN），我认为这不是GPU的问题，因为在另一台服务器上我使用2张3090也没能复现，两个模型的准确率曲线几乎一致。

log.txt

fulllog.zip

python_env.txt

[ykx3]

您好，我翻阅了您的论文，但是没有找到训练SSv2时使用的批次，我查找了issue，发现您提到使用8张A100训练 #6，于是我使用梯度累积重新训练了一个更大批次（1024）的tiny模型，获得了63.2的准确率，这仍与论文不符，于是我又尝试使用256（base_lr）批次训练了tiny，但是只得到了62.8的准确率。另外，当我尝试验证您提供的检查点videomamba_t16_ssv2_f8_res224.pth时，我发现您在github仓库中报告的测试方式（3x4）和论文中报告的（3x2）不一致，或者我理解错您在论文中报告的FLOPS的含义，但我使用两个都进行了测试，其中3x4的方式获得了64.98，比较接近您报告的65.1，而3x2的方式只获得了64.65，更进一步地，我测试了small版本的检查点videomamba_s16_ssv2_f8_res224.pth，3x4和3x2分别获得了66.25和66.02的准确率，都与论文报告的66.6有所差距，值得一提的是， #47 中也显示了使用small模型在SSv2上进行的测试，结果与我使用3x2进行测试时颇为相似，所以我现在怀疑您提供的代码中包含某种隐藏的漏洞，您能否提供您训练时使用的代码以便于更仔细的研究呢？

[ykx3]

我发现在您的附录中报告了Ti模型没有使用layer-wise lr decay，但是在您提供的脚本中没有关闭这一选项（默认值是0.75），这可能导致问题，但我尝试关闭之后得到的结果与之前的情况类似。

srun -p $PARTITION \
        --job-name=${JOB_NAME} \
        --gres=gpu:${GPUS_PER_NODE} \
        --ntasks=${GPUS} \
        --ntasks-per-node=${GPUS_PER_NODE} \
        --cpus-per-task=${CPUS_PER_TASK} \
        --kill-on-bad-exit=1 \
        python run_class_finetuning.py \
        --model videomamba_tiny \
        --data_path ${DATA_PATH} \
        --prefix ${PREFIX} \
        --data_set 'SSV2' \
        --nb_classes 174 \
        --log_dir ${OUTPUT_DIR} \
        --output_dir ${OUTPUT_DIR} \
        --batch_size 64 \
        --num_sample 2 \
        --input_size 224 \
        --short_side_size 224 \
        --save_ckpt_freq 100 \
        --num_frames 8 \
        --num_workers 12 \
        --warmup_epochs 5 \
        --tubelet_size 1 \
        --epochs 35 \
        --lr 4e-4 \
        --drop_path 0.1 \
        --aa rand-m5-n2-mstd0.25-inc1 \
        --opt adamw \
        --opt_betas 0.9 0.999 \
        --weight_decay 0.1 \
        --test_num_segment 2 \
        --test_num_crop 3 \
        --dist_eval \
        --test_best \
        --bf16

[ykx3]

另外，根据日志显示，在使用学习率层衰减时，几乎所有的层都被归类为最后一层，经过检查，我发现您的代码optim_factory.py中有几行可能被错误地注释了，导致了这个问题：

def get_num_layer_for_vit(var_name, num_max_layer):
    if var_name in ("cls_token", "mask_token", "pos_embed"):
        return 0
    elif var_name.startswith("patch_embed"):
        return 0
    elif var_name.startswith("rel_pos_bias"):
        return num_max_layer - 1
    elif var_name.startswith("blocks"):
        layer_id = int(var_name.split('.')[1])
        return layer_id + 1
    elif var_name.startswith("transformer.resblocks"):
        layer_id = int(var_name.split('.')[2])
        return layer_id + 1
    # elif var_name.startswith("layers"): # for VideoMamba
    #     layer_id = int(var_name.split('.')[1])
    #     return layer_id + 1
    elif var_name in ("class_embedding", "positional_embedding", "temporal_positional_embedding"):
        return 0
    elif var_name.startswith("conv1"):
        return 0
    else:
        return num_max_layer - 1

日志：

Param groups = {
  "layer_0_no_decay": {
    "weight_decay": 0.0,
    "params": [
      "cls_token",
      "pos_embed",
      "patch_embed.proj.bias"
    ],
    "lr_scale": 0.0007525434581650003
  },
  "layer_25_no_decay": {
    "weight_decay": 0.0,
    "params": [
      "temporal_pos_embedding",
      "head.bias",
      "layers.0.mixer.D",
      "layers.0.mixer.D_b",
      "layers.0.mixer.conv1d.bias",
      "layers.0.mixer.dt_proj.bias",
      "layers.0.mixer.conv1d_b.bias",
      "layers.0.mixer.dt_proj_b.bias",
      "layers.0.norm.weight",
      "layers.1.mixer.D",
      "layers.1.mixer.D_b",
      "layers.1.mixer.conv1d.bias",
      "layers.1.mixer.dt_proj.bias",
      "layers.1.mixer.conv1d_b.bias",
      "layers.1.mixer.dt_proj_b.bias",
      "layers.1.norm.weight",
      "layers.2.mixer.D",
      "layers.2.mixer.D_b",
      "layers.2.mixer.conv1d.bias",
      "layers.2.mixer.dt_proj.bias",
      "layers.2.mixer.conv1d_b.bias",
      "layers.2.mixer.dt_proj_b.bias",
      "layers.2.norm.weight",
      "layers.3.mixer.D",
      "layers.3.mixer.D_b",
      "layers.3.mixer.conv1d.bias",
      "layers.3.mixer.dt_proj.bias",
      "layers.3.mixer.conv1d_b.bias",
      "layers.3.mixer.dt_proj_b.bias",
      "layers.3.norm.weight",
      "layers.4.mixer.D",
      "layers.4.mixer.D_b",
      "layers.4.mixer.conv1d.bias",
      "layers.4.mixer.dt_proj.bias",
      "layers.4.mixer.conv1d_b.bias",
      "layers.4.mixer.dt_proj_b.bias",
      "layers.4.norm.weight",
      "layers.5.mixer.D",
      "layers.5.mixer.D_b",
      "layers.5.mixer.conv1d.bias",
      "layers.5.mixer.dt_proj.bias",
      "layers.5.mixer.conv1d_b.bias",
      "layers.5.mixer.dt_proj_b.bias",
      "layers.5.norm.weight",
      "layers.6.mixer.D",
      "layers.6.mixer.D_b",
      "layers.6.mixer.conv1d.bias",
      "layers.6.mixer.dt_proj.bias",
      "layers.6.mixer.conv1d_b.bias",
      "layers.6.mixer.dt_proj_b.bias",
      "layers.6.norm.weight",
      "layers.7.mixer.D",
      "layers.7.mixer.D_b",
      "layers.7.mixer.conv1d.bias",
      "layers.7.mixer.dt_proj.bias",
      "layers.7.mixer.conv1d_b.bias",
      "layers.7.mixer.dt_proj_b.bias",
      "layers.7.norm.weight",
      "layers.8.mixer.D",
      "layers.8.mixer.D_b",
      "layers.8.mixer.conv1d.bias",
      "layers.8.mixer.dt_proj.bias",
      "layers.8.mixer.conv1d_b.bias",
      "layers.8.mixer.dt_proj_b.bias",
      "layers.8.norm.weight",
      "layers.9.mixer.D",
      "layers.9.mixer.D_b",
      "layers.9.mixer.conv1d.bias",
      "layers.9.mixer.dt_proj.bias",
      "layers.9.mixer.conv1d_b.bias",
      "layers.9.mixer.dt_proj_b.bias",
      "layers.9.norm.weight",
      "layers.10.mixer.D",
      "layers.10.mixer.D_b",
      "layers.10.mixer.conv1d.bias",
      "layers.10.mixer.dt_proj.bias",
      "layers.10.mixer.conv1d_b.bias",
      "layers.10.mixer.dt_proj_b.bias",
      "layers.10.norm.weight",
      "layers.11.mixer.D",
      "layers.11.mixer.D_b",
      "layers.11.mixer.conv1d.bias",
      "layers.11.mixer.dt_proj.bias",
      "layers.11.mixer.conv1d_b.bias",
      "layers.11.mixer.dt_proj_b.bias",
      "layers.11.norm.weight",
      "layers.12.mixer.D",
      "layers.12.mixer.D_b",
      "layers.12.mixer.conv1d.bias",
      "layers.12.mixer.dt_proj.bias",
      "layers.12.mixer.conv1d_b.bias",
      "layers.12.mixer.dt_proj_b.bias",
      "layers.12.norm.weight",
      "layers.13.mixer.D",
      "layers.13.mixer.D_b",
      "layers.13.mixer.conv1d.bias",
      "layers.13.mixer.dt_proj.bias",
      "layers.13.mixer.conv1d_b.bias",
      "layers.13.mixer.dt_proj_b.bias",
      "layers.13.norm.weight",
      "layers.14.mixer.D",
      "layers.14.mixer.D_b",
      "layers.14.mixer.conv1d.bias",
      "layers.14.mixer.dt_proj.bias",
      "layers.14.mixer.conv1d_b.bias",
      "layers.14.mixer.dt_proj_b.bias",
      "layers.14.norm.weight",
      "layers.15.mixer.D",
      "layers.15.mixer.D_b",
      "layers.15.mixer.conv1d.bias",
      "layers.15.mixer.dt_proj.bias",
      "layers.15.mixer.conv1d_b.bias",
      "layers.15.mixer.dt_proj_b.bias",
      "layers.15.norm.weight",
      "layers.16.mixer.D",
      "layers.16.mixer.D_b",
      "layers.16.mixer.conv1d.bias",
      "layers.16.mixer.dt_proj.bias",
      "layers.16.mixer.conv1d_b.bias",
      "layers.16.mixer.dt_proj_b.bias",
      "layers.16.norm.weight",
      "layers.17.mixer.D",
      "layers.17.mixer.D_b",
      "layers.17.mixer.conv1d.bias",
      "layers.17.mixer.dt_proj.bias",
      "layers.17.mixer.conv1d_b.bias",
      "layers.17.mixer.dt_proj_b.bias",
      "layers.17.norm.weight",
      "layers.18.mixer.D",
      "layers.18.mixer.D_b",
      "layers.18.mixer.conv1d.bias",
      "layers.18.mixer.dt_proj.bias",
      "layers.18.mixer.conv1d_b.bias",
      "layers.18.mixer.dt_proj_b.bias",
      "layers.18.norm.weight",
      "layers.19.mixer.D",
      "layers.19.mixer.D_b",
      "layers.19.mixer.conv1d.bias",
      "layers.19.mixer.dt_proj.bias",
      "layers.19.mixer.conv1d_b.bias",
      "layers.19.mixer.dt_proj_b.bias",
      "layers.19.norm.weight",
      "layers.20.mixer.D",
      "layers.20.mixer.D_b",
      "layers.20.mixer.conv1d.bias",
      "layers.20.mixer.dt_proj.bias",
      "layers.20.mixer.conv1d_b.bias",
      "layers.20.mixer.dt_proj_b.bias",
      "layers.20.norm.weight",
      "layers.21.mixer.D",
      "layers.21.mixer.D_b",
      "layers.21.mixer.conv1d.bias",
      "layers.21.mixer.dt_proj.bias",
      "layers.21.mixer.conv1d_b.bias",
      "layers.21.mixer.dt_proj_b.bias",
      "layers.21.norm.weight",
      "layers.22.mixer.D",
      "layers.22.mixer.D_b",
      "layers.22.mixer.conv1d.bias",
      "layers.22.mixer.dt_proj.bias",
      "layers.22.mixer.conv1d_b.bias",
      "layers.22.mixer.dt_proj_b.bias",
      "layers.22.norm.weight",
      "layers.23.mixer.D",
      "layers.23.mixer.D_b",
      "layers.23.mixer.conv1d.bias",
      "layers.23.mixer.dt_proj.bias",
      "layers.23.mixer.conv1d_b.bias",
      "layers.23.mixer.dt_proj_b.bias",
      "layers.23.norm.weight",
      "norm_f.weight"
    ],
    "lr_scale": 1.0
  },
  "layer_0_decay": {
    "weight_decay": 0.1,
    "params": [
      "patch_embed.proj.weight"
    ],
    "lr_scale": 0.0007525434581650003
  },
  "layer_25_decay": {
    "weight_decay": 0.1,
    "params": [
      "head.weight",
      "layers.0.mixer.A_log",
      "layers.0.mixer.A_b_log",
      "layers.0.mixer.in_proj.weight",
      "layers.0.mixer.conv1d.weight",
      "layers.0.mixer.x_proj.weight",
      "layers.0.mixer.dt_proj.weight",
      "layers.0.mixer.conv1d_b.weight",
      "layers.0.mixer.x_proj_b.weight",
      "layers.0.mixer.dt_proj_b.weight",
      "layers.0.mixer.out_proj.weight",
      "layers.1.mixer.A_log",
      "layers.1.mixer.A_b_log",
      "layers.1.mixer.in_proj.weight",
      "layers.1.mixer.conv1d.weight",
      "layers.1.mixer.x_proj.weight",
      "layers.1.mixer.dt_proj.weight",
      "layers.1.mixer.conv1d_b.weight",
      "layers.1.mixer.x_proj_b.weight",
      "layers.1.mixer.dt_proj_b.weight",
      "layers.1.mixer.out_proj.weight",
      "layers.2.mixer.A_log",
      "layers.2.mixer.A_b_log",
      "layers.2.mixer.in_proj.weight",
      "layers.2.mixer.conv1d.weight",
      "layers.2.mixer.x_proj.weight",
      "layers.2.mixer.dt_proj.weight",
      "layers.2.mixer.conv1d_b.weight",
      "layers.2.mixer.x_proj_b.weight",
      "layers.2.mixer.dt_proj_b.weight",
      "layers.2.mixer.out_proj.weight",
      "layers.3.mixer.A_log",
      "layers.3.mixer.A_b_log",
      "layers.3.mixer.in_proj.weight",
      "layers.3.mixer.conv1d.weight",
      "layers.3.mixer.x_proj.weight",
      "layers.3.mixer.dt_proj.weight",
      "layers.3.mixer.conv1d_b.weight",
      "layers.3.mixer.x_proj_b.weight",
      "layers.3.mixer.dt_proj_b.weight",
      "layers.3.mixer.out_proj.weight",
      "layers.4.mixer.A_log",
      "layers.4.mixer.A_b_log",
      "layers.4.mixer.in_proj.weight",
      "layers.4.mixer.conv1d.weight",
      "layers.4.mixer.x_proj.weight",
      "layers.4.mixer.dt_proj.weight",
      "layers.4.mixer.conv1d_b.weight",
      "layers.4.mixer.x_proj_b.weight",
      "layers.4.mixer.dt_proj_b.weight",
      "layers.4.mixer.out_proj.weight",
      "layers.5.mixer.A_log",
      "layers.5.mixer.A_b_log",
      "layers.5.mixer.in_proj.weight",
      "layers.5.mixer.conv1d.weight",
      "layers.5.mixer.x_proj.weight",
      "layers.5.mixer.dt_proj.weight",
      "layers.5.mixer.conv1d_b.weight",
      "layers.5.mixer.x_proj_b.weight",
      "layers.5.mixer.dt_proj_b.weight",
      "layers.5.mixer.out_proj.weight",
      "layers.6.mixer.A_log",
      "layers.6.mixer.A_b_log",
      "layers.6.mixer.in_proj.weight",
      "layers.6.mixer.conv1d.weight",
      "layers.6.mixer.x_proj.weight",
      "layers.6.mixer.dt_proj.weight",
      "layers.6.mixer.conv1d_b.weight",
      "layers.6.mixer.x_proj_b.weight",
      "layers.6.mixer.dt_proj_b.weight",
      "layers.6.mixer.out_proj.weight",
      "layers.7.mixer.A_log",
      "layers.7.mixer.A_b_log",
      "layers.7.mixer.in_proj.weight",
      "layers.7.mixer.conv1d.weight",
      "layers.7.mixer.x_proj.weight",
      "layers.7.mixer.dt_proj.weight",
      "layers.7.mixer.conv1d_b.weight",
      "layers.7.mixer.x_proj_b.weight",
      "layers.7.mixer.dt_proj_b.weight",
      "layers.7.mixer.out_proj.weight",
      "layers.8.mixer.A_log",
      "layers.8.mixer.A_b_log",
      "layers.8.mixer.in_proj.weight",
      "layers.8.mixer.conv1d.weight",
      "layers.8.mixer.x_proj.weight",
      "layers.8.mixer.dt_proj.weight",
      "layers.8.mixer.conv1d_b.weight",
      "layers.8.mixer.x_proj_b.weight",
      "layers.8.mixer.dt_proj_b.weight",
      "layers.8.mixer.out_proj.weight",
      "layers.9.mixer.A_log",
      "layers.9.mixer.A_b_log",
      "layers.9.mixer.in_proj.weight",
      "layers.9.mixer.conv1d.weight",
      "layers.9.mixer.x_proj.weight",
      "layers.9.mixer.dt_proj.weight",
      "layers.9.mixer.conv1d_b.weight",
      "layers.9.mixer.x_proj_b.weight",
      "layers.9.mixer.dt_proj_b.weight",
      "layers.9.mixer.out_proj.weight",
      "layers.10.mixer.A_log",
      "layers.10.mixer.A_b_log",
      "layers.10.mixer.in_proj.weight",
      "layers.10.mixer.conv1d.weight",
      "layers.10.mixer.x_proj.weight",
      "layers.10.mixer.dt_proj.weight",
      "layers.10.mixer.conv1d_b.weight",
      "layers.10.mixer.x_proj_b.weight",
      "layers.10.mixer.dt_proj_b.weight",
      "layers.10.mixer.out_proj.weight",
      "layers.11.mixer.A_log",
      "layers.11.mixer.A_b_log",
      "layers.11.mixer.in_proj.weight",
      "layers.11.mixer.conv1d.weight",
      "layers.11.mixer.x_proj.weight",
      "layers.11.mixer.dt_proj.weight",
      "layers.11.mixer.conv1d_b.weight",
      "layers.11.mixer.x_proj_b.weight",
      "layers.11.mixer.dt_proj_b.weight",
      "layers.11.mixer.out_proj.weight",
      "layers.12.mixer.A_log",
      "layers.12.mixer.A_b_log",
      "layers.12.mixer.in_proj.weight",
      "layers.12.mixer.conv1d.weight",
      "layers.12.mixer.x_proj.weight",
      "layers.12.mixer.dt_proj.weight",
      "layers.12.mixer.conv1d_b.weight",
      "layers.12.mixer.x_proj_b.weight",
      "layers.12.mixer.dt_proj_b.weight",
      "layers.12.mixer.out_proj.weight",
      "layers.13.mixer.A_log",
      "layers.13.mixer.A_b_log",
      "layers.13.mixer.in_proj.weight",
      "layers.13.mixer.conv1d.weight",
      "layers.13.mixer.x_proj.weight",
      "layers.13.mixer.dt_proj.weight",
      "layers.13.mixer.conv1d_b.weight",
      "layers.13.mixer.x_proj_b.weight",
      "layers.13.mixer.dt_proj_b.weight",
      "layers.13.mixer.out_proj.weight",
      "layers.14.mixer.A_log",
      "layers.14.mixer.A_b_log",
      "layers.14.mixer.in_proj.weight",
      "layers.14.mixer.conv1d.weight",
      "layers.14.mixer.x_proj.weight",
      "layers.14.mixer.dt_proj.weight",
      "layers.14.mixer.conv1d_b.weight",
      "layers.14.mixer.x_proj_b.weight",
      "layers.14.mixer.dt_proj_b.weight",
      "layers.14.mixer.out_proj.weight",
      "layers.15.mixer.A_log",
      "layers.15.mixer.A_b_log",
      "layers.15.mixer.in_proj.weight",
      "layers.15.mixer.conv1d.weight",
      "layers.15.mixer.x_proj.weight",
      "layers.15.mixer.dt_proj.weight",
      "layers.15.mixer.conv1d_b.weight",
      "layers.15.mixer.x_proj_b.weight",
      "layers.15.mixer.dt_proj_b.weight",
      "layers.15.mixer.out_proj.weight",
      "layers.16.mixer.A_log",
      "layers.16.mixer.A_b_log",
      "layers.16.mixer.in_proj.weight",
      "layers.16.mixer.conv1d.weight",
      "layers.16.mixer.x_proj.weight",
      "layers.16.mixer.dt_proj.weight",
      "layers.16.mixer.conv1d_b.weight",
      "layers.16.mixer.x_proj_b.weight",
      "layers.16.mixer.dt_proj_b.weight",
      "layers.16.mixer.out_proj.weight",
      "layers.17.mixer.A_log",
      "layers.17.mixer.A_b_log",
      "layers.17.mixer.in_proj.weight",
      "layers.17.mixer.conv1d.weight",
      "layers.17.mixer.x_proj.weight",
      "layers.17.mixer.dt_proj.weight",
      "layers.17.mixer.conv1d_b.weight",
      "layers.17.mixer.x_proj_b.weight",
      "layers.17.mixer.dt_proj_b.weight",
      "layers.17.mixer.out_proj.weight",
      "layers.18.mixer.A_log",
      "layers.18.mixer.A_b_log",
      "layers.18.mixer.in_proj.weight",
      "layers.18.mixer.conv1d.weight",
      "layers.18.mixer.x_proj.weight",
      "layers.18.mixer.dt_proj.weight",
      "layers.18.mixer.conv1d_b.weight",
      "layers.18.mixer.x_proj_b.weight",
      "layers.18.mixer.dt_proj_b.weight",
      "layers.18.mixer.out_proj.weight",
      "layers.19.mixer.A_log",
      "layers.19.mixer.A_b_log",
      "layers.19.mixer.in_proj.weight",
      "layers.19.mixer.conv1d.weight",
      "layers.19.mixer.x_proj.weight",
      "layers.19.mixer.dt_proj.weight",
      "layers.19.mixer.conv1d_b.weight",
      "layers.19.mixer.x_proj_b.weight",
      "layers.19.mixer.dt_proj_b.weight",
      "layers.19.mixer.out_proj.weight",
      "layers.20.mixer.A_log",
      "layers.20.mixer.A_b_log",
      "layers.20.mixer.in_proj.weight",
      "layers.20.mixer.conv1d.weight",
      "layers.20.mixer.x_proj.weight",
      "layers.20.mixer.dt_proj.weight",
      "layers.20.mixer.conv1d_b.weight",
      "layers.20.mixer.x_proj_b.weight",
      "layers.20.mixer.dt_proj_b.weight",
      "layers.20.mixer.out_proj.weight",
      "layers.21.mixer.A_log",
      "layers.21.mixer.A_b_log",
      "layers.21.mixer.in_proj.weight",
      "layers.21.mixer.conv1d.weight",
      "layers.21.mixer.x_proj.weight",
      "layers.21.mixer.dt_proj.weight",
      "layers.21.mixer.conv1d_b.weight",
      "layers.21.mixer.x_proj_b.weight",
      "layers.21.mixer.dt_proj_b.weight",
      "layers.21.mixer.out_proj.weight",
      "layers.22.mixer.A_log",
      "layers.22.mixer.A_b_log",
      "layers.22.mixer.in_proj.weight",
      "layers.22.mixer.conv1d.weight",
      "layers.22.mixer.x_proj.weight",
      "layers.22.mixer.dt_proj.weight",
      "layers.22.mixer.conv1d_b.weight",
      "layers.22.mixer.x_proj_b.weight",
      "layers.22.mixer.dt_proj_b.weight",
      "layers.22.mixer.out_proj.weight",
      "layers.23.mixer.A_log",
      "layers.23.mixer.A_b_log",
      "layers.23.mixer.in_proj.weight",
      "layers.23.mixer.conv1d.weight",
      "layers.23.mixer.x_proj.weight",
      "layers.23.mixer.dt_proj.weight",
      "layers.23.mixer.conv1d_b.weight",
      "layers.23.mixer.x_proj_b.weight",
      "layers.23.mixer.dt_proj_b.weight",
      "layers.23.mixer.out_proj.weight"
    ],
    "lr_scale": 1.0
  }
}

[wwz-vi]

大佬能教教怎么复现吗？

[ykx3]

大佬能教教怎么复现吗？

请问是什么地方？数据集？环境？脚本？抱歉时间有点久了，可能不是很记得，然后也只是学生，我尽量帮忙吧

[wwz-vi]

我是看这个模型对视频的处理可能有用，所以想用在我的方向上。环境配过了，数据集下载了k400，我在复现时，不知道要怎么运行这个代码，是运行.sh脚本吗？还是videomamba.py?如果是.sh脚本的话，那么里面的配置怎么修改呀。感谢大佬的回复，今年研一很多代码逻辑都搞不懂，有点无从下手

[ykx3]

主要的可执行文件应该是run_class_finetuning.py，你可以找个ai解读一下，然后你可以参考他提供的.sh文件来写脚本，把里面的路径等信息改成你自己的，然后没有集群环境的话就直接用torchrun启动，我习惯使用环境变量，你可以参考我这个脚本以及命令CUDA_VISIBLE_DEVICES=6 NC=1 B=64 NP=8 DATA_PATH=.. ./train.bash --model videomamba_tiny --nb_classes 400 --no_amp

#!/bin/bash
export MASTER_PORT=$((12000 + $RANDOM % 20000))
export OMP_NUM_THREADS=1

JOB_NAME='videomamba_tiny_f8_res224'
# OUTPUT_DIR="$(dirname $0)/$JOB_NAME"
LOG_DIR="./logs/${JOB_NAME}"
# PREFIX='your_k400_path'
# DATA_PATH='your_k400_metadata_path'

PARTITION='video5'
GPUS=8
GPUS_PER_NODE=8
CPUS_PER_TASK=16

# RE=32
# NC=1
# NP=8
# OUTPUT_DIR=runs/$RE
# DATA_PATH=/data3/VRR/data/k40
# B=128

# srun -p $PARTITION \
#         --job-name=${JOB_NAME} \
#         --gres=gpu:${GPUS_PER_NODE} \
#         --ntasks=${GPUS} \
#         --ntasks-per-node=${GPUS_PER_NODE} \
#         --cpus-per-task=${CPUS_PER_TASK} \
#         --kill-on-bad-exit=1 \
python -m torch.distributed.launch --nproc_per_node=$NC --master-port $MASTER_PORT --use_env run_class_finetuning.py \
--model videomamba_tiny \
--data_path ${DATA_PATH} \
--prefix ${DATA_PATH} \
--data_set 'Kinetics_sparse' \
--split ' ' \
--nb_classes 40 \
--log_dir ${OUTPUT_DIR} \
--output_dir ${OUTPUT_DIR} \
--batch_size $B \
--num_sample 2 \
--input_size 224 \
--short_side_size 224 \
--save_ckpt_freq 100 \
--num_frames 8 \
--num_workers $NP \
--warmup_epochs 5 \
--tubelet_size 1 \
--epochs 70 \
--lr 2e-4 \
--drop_path 0.1 \
--aa rand-m5-n2-mstd0.25-inc1 \
--opt adamw \
--opt_betas 0.9 0.999 \
--weight_decay 0.1 \
--test_num_segment 4 \
--test_num_crop 3 \
--dist_eval \
--test_best \
$*
# --bf16
echo $*

[wwz-vi]

谢谢大佬的教程，我用这个进行微调在我的领域没效果，现在正在找其他的处理视频创新点