2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)最新文献

{"title":"A Novel Programmatic Hybrid 2D / 3D Image Segmentation and Labeling Methodology","authors":"Mike Busch, Andrew Funk, M. McIlvride, Trevor Thomas, Brian Fuller, Dan Blair, John Chapman","doi":"10.1109/AIPR57179.2022.10092218","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092218","url":null,"abstract":"Three-dimensional image segmentation is used in an increasing number of applications, from big-ticket Hollywood films to automated manufacturing to autonomous vehicles. Automated image segmentation relies upon a properly trained deep learning model, which in turn depends upon accurately labelled input. However, the labelling process is very time-intensive and does not lend itself to automation. We set out to automate the labelling process for our specific image content, using Microsoft Kinect RGBD video as input. We created a 3D image segmentation and labelling process that starts with 2D object detection and segmentation. This process is suitable for batch pipeline processing and real-time streaming video. Software demonstrating streaming video processing is shown.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"527 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116199379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Relationship Between Tracking Performance and Video Quality","authors":"Sofia Hirschmann, James Tanis, Nazario Irizarry, B. Martin, Michelle Brennan, J. Irvine","doi":"10.1109/AIPR57179.2022.10092225","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092225","url":null,"abstract":"The push to automate processes is ever expanding, including the process of video imagery object tracking. The Motion Imagery Standards Board (MISB) has published the Video National Imagery Interpretability Rating Scale (VNIIRS) which provides a framework for video data quality. One purpose of VNIIRS is to assist analysts in searching motion imagery quickly and efficiently. This paper explores the problems that arise when tracking tasks are performed by tracking algorithms rather than by human analysts. The discussed obstacles emphasize a need for a new task-based scale that supplies a rating for the observability of video data, within the context of automated exploitation. The first section of this paper presents our development of classifiers and quantifications of critical motion imagery characteristics, extracted from various video data features. The second portion exhibits our findings on the relationships between these characteristics and object tracker performance. Conducting this evaluation and assessing the interpretability loss associated with the various video data characteristics enables an automated method of predicting tracker performance. These findings can be leveraged in the execution of tailored processing and analysis of incoming data streams as well as in the development of an updated video quality standard to be applied in data collection management.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"331 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116232643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Detecting Software Code Vulnerabilities Using 2D Convolutional Neural Networks with Program Slicing Feature Maps","authors":"Anne Watson, Ekincan Ufuktepe, K. Palaniappan","doi":"10.1109/AIPR57179.2022.10092211","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092211","url":null,"abstract":"Throughout the post-PC world, IoT devices and real-time embedded systems have integrated themselves into every aspect of our lives. However, as technology integrates itself, the risk of security vulnerabilities and their impact also grows. Different languages have different susceptibilities to cyberattacks. Programs based on C and C++ are particularly vulnerable to cyberattacks through buffer overflow errors due to their lack of memory management. While security vulnerabilities are well-studied areas, with the increasing popularity of deep learning applications, the attack scenarios have become more sophis-ticated, and traditional techniques may no longer be enough to detect attacks or potential vulnerabilities. Therefore, the application of using deep learning to detect such vulnerabilities has become a trend. The appeal of using deep learning in software security is usually based on graphical models and data, which encourages researchers to use graphical-based deep learning methods like Graph Neural Networks (GNN). In this study, we use a Convolutional Neural Network (CNN) to detect security vulnerabilities due to its scalability and lower computation need. This paper proposes scanning vulnerable C/C++ functions for vulnerabilities using deep learning-based image classification. Our experiments are performed on the NIST SARD dataset. Inspired by existing high-level language vulnerability detection, we propose converting the function into an image through three steps: sentence embedding, a statically created program dependency graph, and social network analysis.We employ sentence embedding to convert each line of code into an equally sized representative vector. Following this, we produce a program dependency graph (PDG) for each function. Each line of code is then analyzed using three metrics: Katz, closeness, and destructibility. We compute the destructibility metric by employing forward-slicing on the PDG to determine the reachability of each line. The combination of the equally sized vectors and the analysis metrics are used to create a representative image of the source code. Finally, the image is classified through a Convolutional Neural Network.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130484625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Maritime Object Detection with Event-Based Cameras","authors":"Elan Sharghi, J. Rodriguez, Justin Mauger, Martin Jaszewski, S. Parameswaran","doi":"10.1109/AIPR57179.2022.10092227","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092227","url":null,"abstract":"Event cameras differ from conventional frame-based cameras in that each individual pixel responds independently and asynchronously to brightness changes. Instead of absolute brightness measurements reported as entire frames at regular time intervals, data from event cameras come as a stream of spatially and temporally sparse brightness change events. Event cameras have several characteristics that are favorable to maritime computer vision tasks, including high dynamic range and high temporal resolution. In this work, we apply Asynet, a sparse convolutional neural network based object detection model, to maritime event data sets we collected in the field. To address the limited size of our data set, we propose fine-tuning from pretrained weights learned from the neuromorphic Caltech101 (N-Caltech101) data set and then using a combination of augmentation techniques drawn from traditional image-based computer vision as well as event-specific augmentations. Empirical findings show that using simple image-based augmentation strategies are enough to significantly boost the performance of the Asynet model.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132883610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"AI Augmentation to Remote Sensing Imagery in Forestry Conservation & Restoration for Increased Responsive Capabilities","authors":"Dhanuj Gandikota, Taissa Gladkova, Kha-Ai Tran, Sanika Bapat, Jennifer A. Richkus, J. Arnold","doi":"10.1109/AIPR57179.2022.10092215","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092215","url":null,"abstract":"Responsive forestry management is critical to carbon management and climate change mitigation. The United Nation’s Intergovernmental Panel on Climate Change (IPCC) Special Report stated that mitigation measures in forests from 2010-2019 have delivered approximately 80% of carbon mitigation from Land Use sectors. Advancements in Remote Sensing (RS) and Artificial Intelligence (AI) technologies hold significant potential to increase the necessary coverage and speed of forestry management; however, there is a delay in the current capability integration due to technical, cost, and human-factor constraints that prevent adoption and deployment. This research details the formation of the ART3MIS-AI (Augmented Real-Time 3D Mapping with Intelligent Sensing AI) system framework, an AI-augmentation framework design that minimizes these constraints. It is tailored for responsive deployment in forestry agencies and provides long-term robustness and extensibility for all four functions of forestry management. Our research analyzes AI capabilities in forestry RS from 75 research papers and assesses the potential integration pathways of these capabilities into the post-processing infrastructures of RS within forestry management for augmented-data transformation and critical metric extraction. The ART3MIS-AI system framework optimizes these observed capabilities for additional high levels of interpretability and extensibility given the technical and sensing platform constraints of forestry agencies. We generated a set of objectives and guidelines to facilitate the responsive deployment and integration of AI-augmentation frameworks within forestry agencies. The guidelines are based on established work in Human Machine Teaming, AI-Assurance, and prior research, in addition to consultation from our forestry professional partners in United States forestry agencies. The proposed ART3MIS-AI system framework is a detailed four-phase, automated and pipelined system implemented in Python and C++. It transforms RS imagery into \"Smart\" Point Clouds with augmented hyperspectral information, vegetation classification, and structural mission metrics to an individual tree scale (canopy height/width, timber volume/fuel loading, species identification, etc.). The ART3MIS-AI system framework accelerates structured tactical data delivery to forestry professionals and provides capabilities in tailored virtual planning. The system framework design, integration analysis, and additional guidelines within this paper provide the foundations for the large-scale and methodological integration of AI-augmentation into RS forestry conservation and restoration capabilities.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"65 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133626433","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploring the Effect of Adversarial Attacks on Deep Learning Architectures for X-Ray Data","authors":"Ilyas Bankole-Hameed, Arav Parikh, Josh Harguess","doi":"10.1109/AIPR57179.2022.10092220","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092220","url":null,"abstract":"As artificial intelligent models continue to grow in their capacity and sophistication, they are often trusted with very sensitive information. In the sub-field of adversarial machine learning, developments are geared solely towards finding reliable methods to systematically erode the ability of artificial intelligent systems to perform as intended. These techniques can cause serious breaches of security, interruptions to major systems, and irreversible damage to consumers. Our research evaluates the effects of various white box adversarial machine learning attacks on popular computer vision deep learning models leveraging a public X-ray dataset from the National Institutes of Health (NIH). We make use of several experiments to gauge the feasibility of developing deep learning models that are robust to adversarial machine learning attacks by taking into account different defense strategies, such as adversarial training, to observe how adversarial attacks evolve over time. Our research details how a variety white box attacks effect different components of InceptionNet, DenseNet, and ResNeXt and suggest how the models can effectively defend against these attacks.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131986937","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Adversarial Barrel! An Evaluation of 3D Physical Adversarial Attacks","authors":"Mohammad Zarei, C. Ward, Josh Harguess, Marshal Aiken","doi":"10.1109/AIPR57179.2022.10092199","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092199","url":null,"abstract":"Computer vision models based on Deep Neural Networks (DNNs) are vulnerable to adversarial attacks. It has also been demonstrated that physical adversarial attacks can affect computer vision models though printed medium or physical 3D objects. However, the efficacy of physical adversarial attacks, is highly variable under real-world conditions. In this research, we leverage a synthetic validation environment to evaluate 2D and 3D physical adversarial attacks on state-of the-art object detection models (Faster-RCNN, RetinaNet, YOLOv3, YOLOv4). Using the Unreal Engine, we create synthetic environments to evaluate the limitations of physical adversarial attacks. We evaluate 2D adversarial patches under varying lighting conditions and poses. We optimize the same adversarial attacks for 3D shapes including a pyramid, a cube, and a barrel (cylinder), and evaluate the robustness of the 3D physical attacks against the 2D attack baseline. We test our attacks against object-detection models trained on MSCOCO, VIRAT, VISDRONE, and synthetic datasets. By advancing physical adversarial attacks and validation-methodology, we improve our ability to red-team computer vision models with a goal toward defending and assuring AI systems used in fields like Transportation, and Security.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128727583","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Down-to-Earth Approach for Camera to World Map Georeferencing Using SfM","authors":"Timothy Krock, Jaired Collins, Joshua Fraser, H. Akbarpour, K. Palaniappan","doi":"10.1109/AIPR57179.2022.10092204","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092204","url":null,"abstract":"This paper explores georegistration of drone imagery by mapping the collection of images to a satellite-centric view, such as that provided by Google Earth. Typically, Structure-from-Motion (SfM) algorithms produce camera poses that can be expressed in a coordinate system related by a similarity transformation in relation to a global coordinate system. We developed Down-to-Earth (D2E) a georegistration method to remap the collection of camera views from relative to world coordinates using estimated transformations back into latitude, longitude, and altitude. We circumvent the difficulty of 2D-to-3D feature matching by projecting the original drone images onto an estimated local ground plane. Matching between oblique drone images and the satellite map with significant perspective viewpoint differences is a challenge for traditional handcrafted image features. Deep learning based features performed better and were used to successfully estimate a five degree-of-freedom similarity transform that provides a better alignment of drone imagery with the satellite world map. Accuracy of the similarity transform can be improved by using the (reconstructed) 3D point cloud for orthorectification of the feature points. The similarity transform is applied to the set of corrected drone camera poses to reenter world space. We evaluate D2E georegistration accuracy using a set of manually selected ground-truth points and show improvement compared to bundle adjusted cameras, even with global geodesic alignment of the cameras.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125335570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Detecting Physical Adversarial Patch Attacks with Object Detectors","authors":"Melanie Jutras, Ethan Liang, Sara Leary, C. Ward, Keith Manville","doi":"10.1109/AIPR57179.2022.10092200","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092200","url":null,"abstract":"Machine learning models are vulnerable to adversarial attacks which can cause integrity violations in real- world systems with machine learning components. Alarmingly, these attacks can also manifest in the physical world where an adversary can disrupt systems without gaining digital access. These attacks are becoming more concerning as safety-critical infrastructure such as healthcare and transportation increasingly rely on machine learning.This work is motivated by the need for safeguarding vision- based systems against physical adversarial pattern attacks—an important domain for autonomous vehicles. We propose the use of a separate detection module that can identify inputs that contain physical adversarial patterns. This approach allows for independent development of the defensive mechanism which can be updated without affecting the performance of the protected model. This methodology allows the model developers to focus on performance and leave security to a separate team. It is a practical approach that can provide security in cases where a model is acquired from a third party and cannot be re-trained.We perform experimentation demonstrating that we can detect unknown adversarial patterns with high accuracy using standard object detectors trained on datasets containing adversarial patches. A single detector is capable of detecting a variety of adversarial patterns trained from models with different datasets and tasks. Additionally, we introduce a new class of visually distinct adversarial patch attack we call GAN patches. Our experimentation shows that once observed the detection module can be updated to identify additional classes of patch attacks. Finally, we experiment with detectors trained trained on innocuous patches and examine how they can generalize to detecting a variety of known patch attacks.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123938695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Achieving Adversarial Robustness in Deep Learning-Based Overhead Imaging","authors":"Dagen Braun, Matthew D. Reisman, L. Dewell, Andrzej Banburski-Fahey, Arturo Deza, T. Poggio","doi":"10.1109/AIPR57179.2022.10092213","DOIUrl":"https://doi.org/10.1109/AIPR57179.2022.10092213","url":null,"abstract":"The Intelligence, Surveillance, and Reconnaissance (ISR) community relies heavily on the use of overhead imagery for object detection and classification. In these applications, machine learning frameworks have been increasingly used to assist analysts in distinguishing high value targets from mundane objects quickly and effectively. In recent years, the robustness of these frameworks has come under question due to the possibility for disruption using image-based adversarial attacks, and as such, it is necessary to harden existing models against these threats. In this work, we survey a collection of three techniques to address these concerns at various stages of the image processing pipeline: external validation using Activity Based Intelligence, internal validation using Latent Space Analysis, and adversarial prevention using biologically inspired techniques. We found that biologically-inspired techniques were most effective and generalizable for mitigating adversarial attacks on overhead imagery in machine learning frameworks, with improvements as much as 34.6% over traditional augmentations, and 80.4% over a model without any augmentation-based defense.","PeriodicalId":301162,"journal":{"name":"2022 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130263684","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}