feat: respect prepend_bos and add return_input_tokens flag#1439

MdSadiqMd wants to merge 3 commits into

TransformerLensOrg:devTransformerLensOrg/TransformerLens:devfrom

MdSadiqMd:sadiq/record-prepend_bosMdSadiqMd/TransformerLens:sadiq/record-prepend_bosCopy head branch name to clipboard