{"title":"从复杂符号序列训练的递归神经网络中提取随机机器","authors":"P. Tiňo, V. Vojtek","doi":"10.1109/KES.1997.619435","DOIUrl":null,"url":null,"abstract":"We train a recurrent neural network on a single, long, complex symbolic sequence with positive entropy. The training process is monitored through information theory based performance measures. We show that although the sequence is unpredictable, the network is able to code the sequence's topological and statistical structure in recurrent neuron activation scenarios. Such scenarios can be compactly represented through stochastic machines extracted from the trained network. Generative models, i.e. trained recurrent networks and extracted stochastic machines, are compared using entropy spectra of generated sequences. In addition, entropy spectra computed directly from the machines capture generalization abilities of extracted machines and are related to a machines' long term behavior.","PeriodicalId":166931,"journal":{"name":"Proceedings of 1st International Conference on Conventional and Knowledge Based Intelligent Electronic Systems. KES '97","volume":"164 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"1997-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"13","resultStr":"{\"title\":\"Extracting stochastic machines from recurrent neural networks trained on complex symbolic sequences\",\"authors\":\"P. Tiňo, V. Vojtek\",\"doi\":\"10.1109/KES.1997.619435\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"We train a recurrent neural network on a single, long, complex symbolic sequence with positive entropy. The training process is monitored through information theory based performance measures. We show that although the sequence is unpredictable, the network is able to code the sequence's topological and statistical structure in recurrent neuron activation scenarios. Such scenarios can be compactly represented through stochastic machines extracted from the trained network. Generative models, i.e. trained recurrent networks and extracted stochastic machines, are compared using entropy spectra of generated sequences. In addition, entropy spectra computed directly from the machines capture generalization abilities of extracted machines and are related to a machines' long term behavior.\",\"PeriodicalId\":166931,\"journal\":{\"name\":\"Proceedings of 1st International Conference on Conventional and Knowledge Based Intelligent Electronic Systems. KES '97\",\"volume\":\"164 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"1997-05-27\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"13\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of 1st International Conference on Conventional and Knowledge Based Intelligent Electronic Systems. KES '97\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/KES.1997.619435\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of 1st International Conference on Conventional and Knowledge Based Intelligent Electronic Systems. KES '97","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/KES.1997.619435","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Extracting stochastic machines from recurrent neural networks trained on complex symbolic sequences
We train a recurrent neural network on a single, long, complex symbolic sequence with positive entropy. The training process is monitored through information theory based performance measures. We show that although the sequence is unpredictable, the network is able to code the sequence's topological and statistical structure in recurrent neuron activation scenarios. Such scenarios can be compactly represented through stochastic machines extracted from the trained network. Generative models, i.e. trained recurrent networks and extracted stochastic machines, are compared using entropy spectra of generated sequences. In addition, entropy spectra computed directly from the machines capture generalization abilities of extracted machines and are related to a machines' long term behavior.