environment setup

git clone https://github.com/es2ilver/hide-reproduce.git
conda create -n hide python=3.10 -y
conda activate hide
cd hide-reproduce
pip install -e .
pip install -e ".[train]"
pip install flash-attn==2.6.3 --no-build-isolation
pip install numpy==1.26.4

prepare base model

• downlaod base model LLaVA-v1.5 from https://huggingface.co/liuhaotian/llava-v1.5-7b
• replcae llava-v1.5-7b/config.json with this configs.json.

config - update checkpoint paths

• please update the checkpoint paths in model_configs/LLaVA/UCIT/train/task1.json ~ task6.json to the actual model path.

dataset

prepare images

• 1. Downlaod images from google drive.
• 2. Unzip the image archive using the unzip_data.sh script. (The default path is set to /local_datasets/UCIT.)

bash unzip_data.sh

unzip annotation file

• unzip annotations.zip which located in /UCIT/annotations.zip

start training

• start training w/ sbatch -> sbatch hide_train_all_sbatch.sh

bash hide_train_all_sbatch.sh