{"title":"The importance of convexity in learning with squared loss","authors":"Wee Sun Lee, P. Bartlett, R. C. Williamson","doi":"10.1145/238061.238082","DOIUrl":"https://doi.org/10.1145/238061.238082","url":null,"abstract":"We show that if the closure of a function class under the metric induced by some probability distribution is not convex, then the sample complexity for agnostically learning with squared loss (using only hypotheses in )i s where is the probability of success and is the required accuracy. In comparison, if the class is convex and has finite pseudodimension, then the sample complexity is . If a nonconvex class has finite pseudodimension, then the sample complexity for agnostically learning the closure of the convex hull of ,i s . Hence, for agnostic learning, learning the convex hull provides better approximation capabilities with little sample complexity penalty.","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"53 1","pages":"1974-1980"},"PeriodicalIF":0.0,"publicationDate":"1998-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88115377","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Rate Distortion and Detection in Quantum Communication","authors":"C. Bendjaballah, J. Leroy, A. Vourdas","doi":"10.1109/18.681346","DOIUrl":"https://doi.org/10.1109/18.681346","url":null,"abstract":"Several relations between the rate distortion function and the probability of error in detection are studied in the context of quantum communication theory. Both of these criteria are associated with the same fidelity function.","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"221 1","pages":"1658-1665"},"PeriodicalIF":0.0,"publicationDate":"1998-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79881053","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Rate-Distortion Theorem for Arbitrary Discrete Sources","authors":"Po-Ning Chen, F. Alajaji","doi":"10.1109/18.681347","DOIUrl":"https://doi.org/10.1109/18.681347","url":null,"abstract":"A rate-distortion theorem for arbitrary (not necessarily stationary or ergodic) discrete-time finite-alphabet sources is given. This result, which provides the expression of the minimum /spl epsiv/-achievable fixed-length coding rate subject to a fidelity criterion, extends a recent data compression theorem by Steinberg and Verdu (see ibid., vol.42, p.63-86 (Jan. 1996).","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"53 1","pages":"1666-1668"},"PeriodicalIF":0.0,"publicationDate":"1998-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87477668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Real-Complex Normal Distribution","authors":"A. Bos","doi":"10.1109/18.681349","DOIUrl":"https://doi.org/10.1109/18.681349","url":null,"abstract":"An expression is derived for the distribution of a mixture of real and complex normal variates. The asymptotic distribution of the resulting real-complex maximum-likelihood estimates is the real-complex normal distribution derived. The covariance matrix of this distribution is particularly important. It is the asymptotic covariance matrix for maximum-likelihood estimates and the Cramer-Rao lower bound on the variance of the real-complex estimates in general. From this covariance matrix, the variance of the reconstructed complex-valued exit wave then follows using the pertinent propagation formulas. The resulting expressions show the dependence of the variance on the free microscope parameters used for experimental design.","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"76 1","pages":"1670-1672"},"PeriodicalIF":0.0,"publicationDate":"1998-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77397395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Pattern Distributions of Legendre Sequences","authors":"C. Ding","doi":"10.1109/18.681353","DOIUrl":"https://doi.org/10.1109/18.681353","url":null,"abstract":"Legendre sequences have a number of interesting randomness properties and are closely related with quadratic residue codes. We give lower and upper bounds on the number of patterns distributed in a cycle of the Legendre sequences and establish the relationship between the weight distribution of quadratic residue codes and the pattern distribution of Legendre sequences. Our result shows that Legendre sequences have an ideal distribution of patterns of length s, when s is not large compared with log/sub 2/N, where N is the prime used to define the sequence.","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"37 1","pages":"1693-1698"},"PeriodicalIF":0.0,"publicationDate":"1998-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74334385","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Zero-Crossing Rates of Mixtures and Products of Gaussian Processes","authors":"J. Barnett, B. Kedem","doi":"10.1109/18.681350","DOIUrl":"https://doi.org/10.1109/18.681350","url":null,"abstract":"Formulas for the expected zero-crossing rate of non-Gaussian mixtures and products of Gaussian processes are obtained. The approach we take is to first derive the expected zero-crossing rate in discrete time and then obtain the rate in continuous time by an appropriate limiting argument. The processes considered, which are non-Gaussian but derived from Gaussian processes, serve to illustrate the variability of the zero-crossing rate in terms of the normalized autocorrelation function p(t) of the process. For Gaussian processes, Rice's formula gives the expected zero-crossing rate in continuous time as 1//spl pi//spl radic/(-/spl rho/\"(0)). We show processes exist with expected zero-crossing rates given by /spl kappa///spl pi//spl radic/(-/spl rho/\"(0)) with either /spl kappa//spl Gt/1 or /spl kappa//spl Lt/1. Consequently, such processes can have an arbitrarily large or small zero-crossing rate as compared to a Gaussian process with the same autocorrelation function.","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"36 1","pages":"1672-1677"},"PeriodicalIF":0.0,"publicationDate":"1998-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89548194","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Error Control Coding for the N-User mod-2 Multiple-Access Channel","authors":"V. Telang, M. Herro","doi":"10.1109/18.681342","DOIUrl":"https://doi.org/10.1109/18.681342","url":null,"abstract":"The efficient use of transmission facilities often requires the sharing of resources by a number of users. Commonly used methods of shared media access are time division multiple access (TDMA), frequency-division multiple access (FDMA), code-division multiple access (CDMA), carrier-sense multiple access (CSMA), etc. We propose a different approach to designing an N-user multiple access system for the binary input binary adder channel. When all users are sending data, the total rate of information transfer is equal to the time-share rate. However, when fewer than N users are active, the unused channel capacity is used to achieve error control. The extent of error control is inversely proportional to T, the number of active users. We present in detail two coding schemes; the first based on cyclic block codes, particularly BCH codes and Reed-Solomon codes, and the second based on convolutional codes.","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"12 1","pages":"1632-1642"},"PeriodicalIF":0.0,"publicationDate":"1998-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82512706","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Lower Bounds on the Code Rate for a Model of Data Transmission with Side Information","authors":"V. Balakirsky","doi":"10.1109/18.681343","DOIUrl":"https://doi.org/10.1109/18.681343","url":null,"abstract":"Asymptotic lower bounds on the code rate when the encoder and decoder have some partial knowledge about the positions where errors may occur during transmission of a codeword are derived.","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"88 1","pages":"1642-1648"},"PeriodicalIF":0.0,"publicationDate":"1998-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87115390","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Frequency and Symbol Rate Offset Compensating Algorithms for Simultaneous Estimation of Echo and Channel Responses","authors":"Weiping Li, Xixian Chen, Yi Wang, N. Miki","doi":"10.1109/18.681355","DOIUrl":"https://doi.org/10.1109/18.681355","url":null,"abstract":"This article addresses two critical problems in designing new full-duplex fast training algorithms for simultaneously estimating echo and channel responses. Although algorithms of this type have been discussed and evaluated in previous work, the proposed algorithms concentrate on coping with the following two problems that were not solved in the previous approaches: (1) the symbol rate difference between the local transmitter and the remote transmitter, and (2) the frequency offsets in both far echo and far signal that are caused by the analog carrier network. The performance of the new methods is analyzed in terms of the mean square error. Simulation results are presented to confirm the analysis.","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"35 1","pages":"1702-1708"},"PeriodicalIF":0.0,"publicationDate":"1998-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81597951","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Blind On-Line Testing for Equalization Errors in Digital Communication Systems","authors":"K. Doğançay, V. Krishnamurthy","doi":"10.1109/18.681351","DOIUrl":"https://doi.org/10.1109/18.681351","url":null,"abstract":"We present an on-line test for blind detection of equalization errors in digital communication systems. The test is based on the observation that for linear time-invariant channels the relationship between the transmitted symbol estimates generated by the equalizer and the noisy channel output can be represented by an underlying linear time-invariant model if and only if the equalizer output sequence is not in error. The presence of equalization errors renders this relationship time-varying, whose occurrence is detected by the proposed on-line test. The test is obtained from a previously proposed off-line least squares test by essentially replacing the least squares algorithm with its recursive version.","PeriodicalId":13250,"journal":{"name":"IEEE Trans. Inf. Theory","volume":"16 1","pages":"1677-1686"},"PeriodicalIF":0.0,"publicationDate":"1998-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82333422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}