Positional encodingFreeYou extended a RoPE model pretrained on 4,096 tokens to 32K with position interpolation () and about 1,000 fine-tuning steps on long documents.
Positional encodingFreePositional encodingFreePositional encodingFreeYou extended a RoPE model pretrained on 4,096 tokens to 32K with position interpolation () and about 1,000 fine-tuning steps on long documents.