Matías Carrasco, Franz Mayr, Sergio Yovine, Johny Kidd, Martín Iturbide, Juan Pedro da Silva, Alejo Garat
{"title":"Analyzing constrained LLM through PDFA-learning","authors":"Matías Carrasco, Franz Mayr, Sergio Yovine, Johny Kidd, Martín Iturbide, Juan Pedro da Silva, Alejo Garat","doi":"arxiv-2406.08269","DOIUrl":null,"url":null,"abstract":"We define a congruence that copes with null next-symbol probabilities that\narise when the output of a language model is constrained by some means during\ntext generation. We develop an algorithm for efficiently learning the quotient\nwith respect to this congruence and evaluate it on case studies for analyzing\nstatistical properties of LLM.","PeriodicalId":501124,"journal":{"name":"arXiv - CS - Formal Languages and Automata Theory","volume":"42 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Formal Languages and Automata Theory","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2406.08269","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
We define a congruence that copes with null next-symbol probabilities that
arise when the output of a language model is constrained by some means during
text generation. We develop an algorithm for efficiently learning the quotient
with respect to this congruence and evaluate it on case studies for analyzing
statistical properties of LLM.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
