通过 PDFA 学习分析受限 LLM

arXiv - CS - Formal Languages and Automata Theory Pub Date : 2024-06-12 DOI:arxiv-2406.08269

Matías Carrasco, Franz Mayr, Sergio Yovine, Johny Kidd, Martín Iturbide, Juan Pedro da Silva, Alejo Garat

{"title":"通过 PDFA 学习分析受限 LLM","authors":"Matías Carrasco, Franz Mayr, Sergio Yovine, Johny Kidd, Martín Iturbide, Juan Pedro da Silva, Alejo Garat","doi":"arxiv-2406.08269","DOIUrl":null,"url":null,"abstract":"We define a congruence that copes with null next-symbol probabilities that\narise when the output of a language model is constrained by some means during\ntext generation. We develop an algorithm for efficiently learning the quotient\nwith respect to this congruence and evaluate it on case studies for analyzing\nstatistical properties of LLM.","PeriodicalId":501124,"journal":{"name":"arXiv - CS - Formal Languages and Automata Theory","volume":"42 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Analyzing constrained LLM through PDFA-learning\",\"authors\":\"Matías Carrasco, Franz Mayr, Sergio Yovine, Johny Kidd, Martín Iturbide, Juan Pedro da Silva, Alejo Garat\",\"doi\":\"arxiv-2406.08269\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"We define a congruence that copes with null next-symbol probabilities that\\narise when the output of a language model is constrained by some means during\\ntext generation. We develop an algorithm for efficiently learning the quotient\\nwith respect to this congruence and evaluate it on case studies for analyzing\\nstatistical properties of LLM.\",\"PeriodicalId\":501124,\"journal\":{\"name\":\"arXiv - CS - Formal Languages and Automata Theory\",\"volume\":\"42 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-06-12\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Formal Languages and Automata Theory\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2406.08269\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Formal Languages and Automata Theory","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2406.08269","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

摘要

当语言模型的输出在文本生成过程中受到某种限制时，就会出现下一符号概率为空的情况。我们开发了一种算法，用于高效地学习与该一致性相关的商，并在分析 LLM 统计属性的案例研究中对其进行了评估。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Analyzing constrained LLM through PDFA-learning

We define a congruence that copes with null next-symbol probabilities that arise when the output of a language model is constrained by some means during text generation. We develop an algorithm for efficiently learning the quotient with respect to this congruence and evaluate it on case studies for analyzing statistical properties of LLM.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

arXiv - CS - Formal Languages and Automata Theory

自引率

0.00%

发文量