{"title":"PRC Inc.: description of the Paktus system used for MUC-4","authors":"Bruce Loatman","doi":"10.3115/1072064.1072101","DOIUrl":"https://doi.org/10.3115/1072064.1072101","url":null,"abstract":"The PRC Adaptive Knowledge-based Text Understanding System (PAKTUS) has been under development as an Independent Research and Development project at PRC since 1984. It includes a core English lexicon and grammar, a concept network, processes for applying these to lexical, syntactic, semantic, and discourse analysis, and tools that support the adaptation of the generic core to new domains, primarily by acquiring sublanguage and domain-specific lexicon and conceptual topic patterns of interest. The lexical, syntactic, and semantic analysis components were completed before MUC-4 and required little adaptation. The discourse analysis component is new and was completed in the course of applying the system to MUC-4, although it is generic. The overall system is described in [1]. The present description concentrates on discourse analysis.","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130432662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Overview of the Fourth Message Understanding Evaluation and Conference","authors":"B. Sundheim","doi":"10.3115/1072064.1072066","DOIUrl":"https://doi.org/10.3115/1072064.1072066","url":null,"abstract":"The Fourth Message Understanding Conference (MUC-4) is the latest in a series of conferences that concern the evaluation of natural language processing (NLP) systems. These conferences have reported on progress being made both in the development of systems capable of analyzing relatively short English texts and in the definition of a rigorous performance evaluation methodology. MUC-4 was preceded by a period of intensive system development by each of the participating organizations and blind testing using materials prepared by NRaD and SAIC that are described in this paper, other papers in this volume, and the MUC-3 proceedings [1].","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130443766","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jerry R. Hobbs, D. Appelt, M. Tyson, J. Bear, David J. Israel
{"title":"SRI International: Description of the FASTUS System Used for MUC-4","authors":"Jerry R. Hobbs, D. Appelt, M. Tyson, J. Bear, David J. Israel","doi":"10.3115/1072064.1072103","DOIUrl":"https://doi.org/10.3115/1072064.1072103","url":null,"abstract":"FASTUS is a (slightly permuted) acronym for Finite State Automaton Text Understanding System. It is a system for extracting information from free text in English, and potentially other languages as well, for entry into a database, and potentially for other applications. It works essentially as a cascaded, nondeterministic finite state automaton.","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"43 8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130471994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Statistical Significance of the MUC-5 Results","authors":"Nancy A. Chinchor","doi":"10.3115/1072017.1072027","DOIUrl":"https://doi.org/10.3115/1072017.1072027","url":null,"abstract":"The MUC-4 scores of recall, precision, and the F-measures are used to measure the performance of the participating systems. The differences in the scores between any two systems may be due to chance or may be due to a significant difference between the two systems. To rule out the possibility that the difference is due to chance, statistical hypothesis testing is used. The method of hypothesis testing used is a computationally-intensive method known as approximate randomization. The method and the statistical significance of the results for the two MUC-4 test sets, TST3 and TST4, will be discussed in this paper.","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"111 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116206544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chinatsu Aone, Sharon Flank, Douglas McKee, P. Krause
{"title":"SRA: description of the SOLOMON system as used for MUC-5","authors":"Chinatsu Aone, Sharon Flank, Douglas McKee, P. Krause","doi":"10.3115/1072017.1072038","DOIUrl":"https://doi.org/10.3115/1072017.1072038","url":null,"abstract":"SRA's knowledge-based natural language processing system SOLOMON has been developed for text understanding since 1986. In addition to being a domain-independent NLP system, starting in the fall of 1990, SOLOMON has been extended as part of the MURASAKI project to become a multi-lingual text understanding system. It currently understands Spanish and Japanese as well as English texts. In order to achieve domain- and language-independence, SOLOMON separates data from processing modules. The processing modules do not assume any domain- or language-dependent facts; rather they are designed so that they work off separate data, i.e. lexicons, grammars, patterns, and knowledge bases, which vary according to the domain or language. To facilitate data acquisition, SRA has developed 2 tools: LEXTool for the development of lexicons and KBTool for the development of knowledge bases.","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124014916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"GE adjunct test report: object-oriented design and scoring for MUC-4","authors":"George B. Krupka, L. Rau","doi":"10.3115/1072064.1072071","DOIUrl":"https://doi.org/10.3115/1072064.1072071","url":null,"abstract":"This paper reports on the results of the adjunct test performed by GE for the MUC-4 evaluation of text processing systems. In this test, we evaluated the effect of an object-oriented template design and associated matching conditions on the scores. The results indicate that the current MUC-4 \"flat\" templade design with cross-references closely approximates a true object-oriented design. However the object-oriented design allows for additional performance data to be calculated, facilitating diagnosis.","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129036882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An adjunct test for discourse processing in MUC-4","authors":"L. Hirschman","doi":"10.3115/1072064.1072070","DOIUrl":"https://doi.org/10.3115/1072064.1072070","url":null,"abstract":"The motivation for this adjunct test came from an exploratory study done by Beth Sundheim during MUC-3. This study showed a degradation in correctness of message processing as the information distribution in the message became more complex, that is, as slot fills were drawn from larger portions of the message and required more discourse processing to extract the information and reassemble it correctly in the required template(s). The study also suggested that systems did worse on messages requiring multiple templates than on single-template messages. These observations led us define the MUC-4 adjunct test to examine two hypotheses related to discourse complexity and expected system performance:• The Source Complexity HypothesisThe more complex the distribution of the source information for filling a given slot or template (the more sentences, and the more widely separated the sentences), the more difficult it will be to process the message correctly.•The Output Complexity HypothesisThe more complex the output (in terms of number of templates), the harder it will be to process the message correctly.","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131982333","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Cowie, Louise Guthrie, Y. Wilks, J. Pustejovsky, Scott Waterman
{"title":"CRL/NMSU and Brandeis: description of the MucBruce system as used for MUC-4","authors":"J. Cowie, Louise Guthrie, Y. Wilks, J. Pustejovsky, Scott Waterman","doi":"10.3115/1072064.1072098","DOIUrl":"https://doi.org/10.3115/1072064.1072098","url":null,"abstract":"Through their involvement in the Tipster project the Computing Research Laboratory at New Mexico State University and the Computer Science Department at Brandeis University are developing a method for identifying articles of interest and extracting and storing specific kinds of information from large volumes of Japanese and English texts. We intend that the method be general and extensible. The techniques involved are not explicitly tied to these two languages nor to a particular subject area. Development for Tipster has been going on since September, 1992.","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126678010","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
L. Rau, George B. Krupka, P. Jacobs, I. Sider, L. Childs
{"title":"GE NLToolset: MUC-4 test results and analysis","authors":"L. Rau, George B. Krupka, P. Jacobs, I. Sider, L. Childs","doi":"10.3115/1072064.1072074","DOIUrl":"https://doi.org/10.3115/1072064.1072074","url":null,"abstract":"This paper reports on the GE NLTOOLSET customization effort for MUC-4, and analyzes the results of the TST3 and TST4 runs.","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121640377","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Moldovan, Seungho Cha, Minhwa Chung, K. J. Hendrickson, Jun-Tae Kim, Stephen V. Kowalski
{"title":"USC: description of the SNAP system used for MUC-4","authors":"D. Moldovan, Seungho Cha, Minhwa Chung, K. J. Hendrickson, Jun-Tae Kim, Stephen V. Kowalski","doi":"10.3115/1072064.1072107","DOIUrl":"https://doi.org/10.3115/1072064.1072107","url":null,"abstract":"The main goal of the SNAP project is to build a massively parallel computer capable of fast and accurate natural language processing [3]. Under NSF funding, a parallel computer was built in the Parallel Knowledge Processing Laboratory at USC and software was developed to operate the machine [2]. The approach in designing SNAP was to find a knowledge representation and a reasoning paradigm useful for natural language processing which exibits massive parallelism. We have selected marker-passing on semantic networks as a way to represent and process linguistic knowledge.","PeriodicalId":424990,"journal":{"name":"Message Understanding Conference","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1992-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115247503","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}