Hi,
I am trying to reproduce some of the results from your paper using the provided fine-tuned model. I followed the instructions to download the weight diff from https://huggingface.co/epfml/landmark-attention-llama7b-wdiff, and then I recovered the fine-tuned weights.
However, when I ran the run_test.py script in llama_legacy, I couldn't reproduce your results. The base model achieves the correct accuracy, while the fine-tuned model has an accuracy close to zero. I suspect there might be an issue with the uploaded weight diff.
Could you please check the weight diff?
Thank you!
Hi,
I am trying to reproduce some of the results from your paper using the provided fine-tuned model. I followed the instructions to download the weight diff from https://huggingface.co/epfml/landmark-attention-llama7b-wdiff, and then I recovered the fine-tuned weights.
However, when I ran the run_test.py script in llama_legacy, I couldn't reproduce your results. The base model achieves the correct accuracy, while the fine-tuned model has an accuracy close to zero. I suspect there might be an issue with the uploaded weight diff.
Could you please check the weight diff?
Thank you!