没有保存的 Kaleidosope:具有 Pythonic 语法的 100% Jitted 神经网络编码语言

Augusto Seben da Rosa, Marlon Daniel Angeli, Jorge Aikes Junior, Alef Iury Ferreira, Lucas Rafael Gris, Anderson da Silva Soares, Arnaldo Candido Junior, Frederico Santos de Oliveira, Gabriel Trevisan Damke, Rafael Teixeira Sousa
{"title":"没有保存的 Kaleidosope:具有 Pythonic 语法的 100% Jitted 神经网络编码语言","authors":"Augusto Seben da Rosa, Marlon Daniel Angeli, Jorge Aikes Junior, Alef Iury Ferreira, Lucas Rafael Gris, Anderson da Silva Soares, Arnaldo Candido Junior, Frederico Santos de Oliveira, Gabriel Trevisan Damke, Rafael Teixeira Sousa","doi":"arxiv-2409.11600","DOIUrl":null,"url":null,"abstract":"We developed a jitted compiler for training Artificial Neural Networks using\nC++, LLVM and Cuda. It features object-oriented characteristics, strong typing,\nparallel workers for data pre-processing, pythonic syntax for expressions,\nPyTorch like model declaration and Automatic Differentiation. We implement the\nmechanisms of cache and pooling in order to manage VRAM, cuBLAS for high\nperformance matrix multiplication and cuDNN for convolutional layers. Our\nexperiments with Residual Convolutional Neural Networks on ImageNet, we reach\nsimilar speed but degraded performance. Also, the GRU network experiments show\nsimilar accuracy, but our compiler have degraded speed in that task. However,\nour compiler demonstrates promising results at the CIFAR-10 benchmark, in which\nwe reach the same performance and about the same speed as PyTorch. We make the\ncode publicly available at: https://github.com/NoSavedDATA/NoSavedKaleidoscope","PeriodicalId":501197,"journal":{"name":"arXiv - CS - Programming Languages","volume":"26 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"No Saved Kaleidosope: an 100% Jitted Neural Network Coding Language with Pythonic Syntax\",\"authors\":\"Augusto Seben da Rosa, Marlon Daniel Angeli, Jorge Aikes Junior, Alef Iury Ferreira, Lucas Rafael Gris, Anderson da Silva Soares, Arnaldo Candido Junior, Frederico Santos de Oliveira, Gabriel Trevisan Damke, Rafael Teixeira Sousa\",\"doi\":\"arxiv-2409.11600\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"We developed a jitted compiler for training Artificial Neural Networks using\\nC++, LLVM and Cuda. It features object-oriented characteristics, strong typing,\\nparallel workers for data pre-processing, pythonic syntax for expressions,\\nPyTorch like model declaration and Automatic Differentiation. We implement the\\nmechanisms of cache and pooling in order to manage VRAM, cuBLAS for high\\nperformance matrix multiplication and cuDNN for convolutional layers. Our\\nexperiments with Residual Convolutional Neural Networks on ImageNet, we reach\\nsimilar speed but degraded performance. Also, the GRU network experiments show\\nsimilar accuracy, but our compiler have degraded speed in that task. However,\\nour compiler demonstrates promising results at the CIFAR-10 benchmark, in which\\nwe reach the same performance and about the same speed as PyTorch. We make the\\ncode publicly available at: https://github.com/NoSavedDATA/NoSavedKaleidoscope\",\"PeriodicalId\":501197,\"journal\":{\"name\":\"arXiv - CS - Programming Languages\",\"volume\":\"26 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Programming Languages\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.11600\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Programming Languages","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.11600","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

我们使用 C++、LLVM 和 Cuda 开发了一个用于训练人工神经网络的 jitted 编译器。它具有面向对象的特点、强类型化、用于数据预处理的并行工作者、表达式的 pythonic 语法、类似 PyTorch 的模型声明和自动微分功能。我们实现了缓存和池化主题机制,以管理 VRAM、用于高性能矩阵乘法的 cuBLAS 和用于卷积层的 cuDNN。通过在 ImageNet 上使用残差卷积神经网络进行实验,我们发现速度与之相近,但性能有所下降。此外,GRU 网络实验也显示了相似的准确性,但我们的编译器在该任务中的速度有所下降。不过,我们的编译器在 CIFAR-10 基准测试中取得了不错的成绩,性能和速度与 PyTorch 差不多。我们公开了代码,网址是:https://github.com/NoSavedDATA/NoSavedKaleidoscope。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
No Saved Kaleidosope: an 100% Jitted Neural Network Coding Language with Pythonic Syntax
We developed a jitted compiler for training Artificial Neural Networks using C++, LLVM and Cuda. It features object-oriented characteristics, strong typing, parallel workers for data pre-processing, pythonic syntax for expressions, PyTorch like model declaration and Automatic Differentiation. We implement the mechanisms of cache and pooling in order to manage VRAM, cuBLAS for high performance matrix multiplication and cuDNN for convolutional layers. Our experiments with Residual Convolutional Neural Networks on ImageNet, we reach similar speed but degraded performance. Also, the GRU network experiments show similar accuracy, but our compiler have degraded speed in that task. However, our compiler demonstrates promising results at the CIFAR-10 benchmark, in which we reach the same performance and about the same speed as PyTorch. We make the code publicly available at: https://github.com/NoSavedDATA/NoSavedKaleidoscope
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信