《基于开发标准OpenCL的深度学习研究与探索.pdf》由会员分享，可在线阅读，更多相关《基于开发标准OpenCL的深度学习研究与探索.pdf（20页珍藏版）》请在三一文库上搜索。
1、基于开放标准OpenCL的深度 学习研究和探索 谷俊丽 AMD Research Collaborated with product team Junli.GuAMD.com Outline 深度学习及其发展状况深度学习及其发展状况 深度学习对系统实现的挑战深度学习对系统实现的挑战 基于OpenCL的深度学习探索 | IMPLEMENTING A LEADING LOADS PERFORMANCE PREDICTOR ON COMMODITY PROCESSORS | JUNE 19, 2014 3 DNN 模型 What is a Deep Neural Network (DNN)? 324
2、 hidden layers, millions to billions of parameters DNN + Big Data is leading recent direction in machine learning Rich Varieties of DNN Structures MLP (Multi-level Perceptron)/ AutoEncoder CNN (Convolutional Neural Network) DBN (Deep belief network)/RBM (Restricted Boltzmann Machine) Deep Learning o
3、n DNN model Random initialized parameters Trained to converge by feeding large scale of data (Big Data) Starting to get really hot after winning 2012 ILSVRC competition neurons weighted connection Input Output hidden1 hidden2 hidden3 | IMPLEMENTING A LEADING LOADS PERFORMANCE PREDICTOR ON COMMODITY
4、PROCESSORS | JUNE 19, 2014 4 深度学习过程 (DEEP LEARNING) Deep Learning ! Voice, ! Text ! Image ! DNN for Speech! 10k hours of voice data! 10b training samples! Months on a GPU cluster! Results Deep Learning: DNN model + Big Data Actually human defined features no longer work well for Big Data scenarios w
5、ith noise. All features learnt by training data, without human interference. DNN model | AMD DNN 5 深度学习为何强大？ HIERARCHICAL FEATURE EXTRACTION Extract features layer by layer from input data, to form hierarchical representation that is beyond humans definition Features have semantic meanings | PRESENT
6、ATION TITLE | DECEMBER 19, 2014 | CONFIDENTIAL 6 深度学习正在引领潮流深度学习正在引领潮流 Why internet companies purse DNN these days? Original human defined algorithms dont work well for Big Data Competing in machine learning to understand Big Data DNN (deep neural networks) is breaking through Hopefully HSAs features
7、 will enable more effective solution Task and device synchronization We designed synchronization protocols using context, command queues and events | DNN PROJECT 17 Forward_gpuBackward_gpu MaxPoolForwardfloat AvePoolFowardfloatAvePoolBackwardfloat MaxPoolBackwardfloat Forward_gpuBackward_gpu Im2col_
8、gpu Caffe_gpu_gemmCaffe_gpu_gemv col2im_gpu Forward_gpuBackward_gpu ReLUForwardfloatReLUBackwardfloat Forward_gpuBackward_gpu Caffe_gpu_gemmCaffe_gpu_gemv Forward_gpu Kernel_get_maxfloatCaffe_gpu_gemmKernel_softmax_divfloatCaffe_gpu_gemv Im2col_gpuCol2im_gpu Caffe_gpu_gemmCaffe_gpu_gemvCaffe_gpu_axp
9、y Caffe_gpu_axpby Caffe_gpu_scal Caffe_gpu_dotCaffe_gpu_asumCaffe_gpu_scale Caffe_gpu_axpy OPENCL DNN HIERARCHY DESIGN Layer1: C+ interfaces (for domain experts) Layer2: OpenCL wrapper hides hardware details (for systems) Layer3: Underlying GPU kernels (for deep optimizations) GPU kernels Hand coded
10、 kernels OpenCL APIs Layer 3: GPU kernels Layer 2:OpenCL wrappers Layer 1: C+ machine learning interfaces | DNN PROJECT 18 搭建深度学习的大数据应用场景 Classification and recognition based on MLP model Optical Character Recognition (OCR) Driver license plate recognition Voice recognition with industry scale of da
11、ta Image/object classification based on CNN Small images are done, next scaling to industry size images Content based image retrieval (CBIR) Retrieve images that are similar in content to the query image from a database Model used: Autoencoder + RBM Large scale object recognition Using our kernels y
12、our application is able to run on CPU/GPU/APU/accelerators etc. | DNN PROJECT 19 AMD之深度学习解决方案 H/W solutions: Parallel implementation on systems and system level evaluation CPU + GPUs cluster APU server S/W solutions: OpenCL solution of deep learning applications Applicable to general heterogeneous p
13、latforms Set up real world application scenarios with external companys involvement and apply AMD solutions to industry Note: Collaboration from both academia and industry is welcomed | DNN PROJECT 20 人工智能与系统相结合：机遇与挑战并存人工智能与系统相结合：机遇与挑战并存 人工智能的新浪潮将引领未来20年的技术和 系统革命，这个浪潮首先在互联网公司掀起， 正在如火如荼的进行研究。 光光有算法是解
14、决不了最终问题的有算法是解决不了最终问题的，硬硬件件系统系统 是是大数大数据据+算算法的法的enabler。硬件领域也需要 抓住此时机，回答硬件系统如何设计具有人 工智能的本领，这是系统研究人员面临的机 遇。 IBM的沃森处理器是一个好的研究成果，并 且已经投入使用解决一些大数据的金融分析、 实时语音翻译等应用。 现有的分布式系统上的实现方法，节点间需 要传输大量数据和参数，通信代价太高，当 节点数目超过一定数量时，不能获得持续的 加速比。多个节点间训练不同数据时如何协 调和同步，可能需要从算法角度重新设计。 分分布式系统如何设计，需要布式系统如何设计，需要DNN算法专家和算法专家和 系统专家共同协同解决系统专家共同协同解决，解决的方法可能既 要修改算法使之跟底层硬件架构匹配，又要 求系统专家设计计算能力强大的单机器，又 要设计高密度整合、高效通信的服务器。 机遇机遇 挑战挑战