{"title":"Eye-Beam: A mmWave 5G-Compliant Platform for Integrated Communications and Sensing Enabling AI-Based Object Recognition","authors":"Arun Paidimarri;Asaf Tzadok;Sara Garcia Sanchez;Atsutse Kludze;Alexandra Gallyas-Sanhueza;Alberto Valdes-Garcia","doi":"10.1109/JSAC.2024.3413978","DOIUrl":null,"url":null,"abstract":"We present Eye-Beam, a programmable platform for integrated communication and sensing. Eye-Beam leverages the hardware and processing required for standard millimeter-wave (mmWave) 5G directional communications to enable sensing functions. Specifically, our platform (1) receives and synchronizes to the data frame of broadcast 5G signals, (2) extracts directional communication features, creating a tensor of spatial information, and (3) utilizes this data as input to a DNN that infers the presence of specific objects in the propagation environment. Eye-Beam includes a programmable 28 GHz 64-element phased array, an SDR, and custom FPGA-based firmware. Eye-Beam’s key capabilities and metrics include (i) synchronization of I/Q data (up to 200 MSPS) with beam steering (among 9,601 beams) with 10 ns accuracy; (ii) a signal processing pipeline that extracts communication features such as the SNR and channel response from received 5G waveforms; and (iii) system orchestration that synchronizes the receiver (RX) to the 5G frame structure of the base station (gNodeB) and maintains it within a worst-case OFDM cyclic prefix of \n<inline-formula> <tex-math>$0.29~\\mu $ </tex-math></inline-formula>\ns. Eye-Beam is also able to emulate gNodeB transmissions. We demonstrate Eye-Beam’s performance by showcasing its communication capability (decoding up to 64-QAM), as well as its performance as a channel sounder (extracting detailed directional 5G features in 2,401 beam directions within just 20 ms). We then, for the first time, demonstrate AI-based object classification only using the directional communication features derived by Eye-Beam from ambient mmWave 5G signals transmitted by a gNodeB. Six object classes, including 4 distinct objects concealed in a backpack, are classified with 98% accuracy in an indoor environment.","PeriodicalId":73294,"journal":{"name":"IEEE journal on selected areas in communications : a publication of the IEEE Communications Society","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE journal on selected areas in communications : a publication of the IEEE Communications Society","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10556723/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
We present Eye-Beam, a programmable platform for integrated communication and sensing. Eye-Beam leverages the hardware and processing required for standard millimeter-wave (mmWave) 5G directional communications to enable sensing functions. Specifically, our platform (1) receives and synchronizes to the data frame of broadcast 5G signals, (2) extracts directional communication features, creating a tensor of spatial information, and (3) utilizes this data as input to a DNN that infers the presence of specific objects in the propagation environment. Eye-Beam includes a programmable 28 GHz 64-element phased array, an SDR, and custom FPGA-based firmware. Eye-Beam’s key capabilities and metrics include (i) synchronization of I/Q data (up to 200 MSPS) with beam steering (among 9,601 beams) with 10 ns accuracy; (ii) a signal processing pipeline that extracts communication features such as the SNR and channel response from received 5G waveforms; and (iii) system orchestration that synchronizes the receiver (RX) to the 5G frame structure of the base station (gNodeB) and maintains it within a worst-case OFDM cyclic prefix of
$0.29~\mu $
s. Eye-Beam is also able to emulate gNodeB transmissions. We demonstrate Eye-Beam’s performance by showcasing its communication capability (decoding up to 64-QAM), as well as its performance as a channel sounder (extracting detailed directional 5G features in 2,401 beam directions within just 20 ms). We then, for the first time, demonstrate AI-based object classification only using the directional communication features derived by Eye-Beam from ambient mmWave 5G signals transmitted by a gNodeB. Six object classes, including 4 distinct objects concealed in a backpack, are classified with 98% accuracy in an indoor environment.