I am interested to know some details about replicating the GenPPL and Auto-BLEU results from the paper. As stated on page 3, we provide the SLM with a short speech prompt and generate speech tokens ...

Thanks for the great work! I'm having a little problem reproducing the PPL results in the paper. I used the code snippet from the gptq repo for measuring ppl and was able to reproduce the fp16 ...