[go: up one dir, main page]

WO2015178977A3 - In situ neural network co-processing - Google Patents

In situ neural network co-processing Download PDF

Info

Publication number
WO2015178977A3
WO2015178977A3 PCT/US2015/015917 US2015015917W WO2015178977A3 WO 2015178977 A3 WO2015178977 A3 WO 2015178977A3 US 2015015917 W US2015015917 W US 2015015917W WO 2015178977 A3 WO2015178977 A3 WO 2015178977A3
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
processing
situ
processing node
executing
Prior art date
Application number
PCT/US2015/015917
Other languages
French (fr)
Other versions
WO2015178977A2 (en
Inventor
Michael Campos
Anthony Lewis
Naveen Gandham Rao
Original Assignee
Qualcomm Incorporated
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Incorporated filed Critical Qualcomm Incorporated
Priority to EP15759970.5A priority Critical patent/EP3108414A2/en
Priority to CN201580009326.3A priority patent/CN106030622B/en
Priority to JP2016553381A priority patent/JP2017509982A/en
Publication of WO2015178977A2 publication Critical patent/WO2015178977A2/en
Publication of WO2015178977A3 publication Critical patent/WO2015178977A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Neurology (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Image Analysis (AREA)
  • Feedback Control In General (AREA)
  • Advance Control (AREA)
  • Image Processing (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

A method of executing co-processing in a neural network comprises swapping a portion of the neural network to a first processing node for a period of time. The method also includes executing the portion of the neural network with the first processing node. Additionally, the method includes returning the portion of the neural network to a second processing node after the period of time. Further, the method includes executing the portion of the neural network with the second processing node.
PCT/US2015/015917 2014-02-21 2015-02-13 In situ neural network co-processing WO2015178977A2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP15759970.5A EP3108414A2 (en) 2014-02-21 2015-02-13 In situ neural network co-processing
CN201580009326.3A CN106030622B (en) 2014-02-21 2015-02-13 Neural network collaboration processing in situ
JP2016553381A JP2017509982A (en) 2014-02-21 2015-02-13 In-situ neural network coprocessing

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201461943155P 2014-02-21 2014-02-21
US61/943,155 2014-02-21
US14/273,214 2014-05-08
US14/273,214 US20150242741A1 (en) 2014-02-21 2014-05-08 In situ neural network co-processing

Publications (2)

Publication Number Publication Date
WO2015178977A2 WO2015178977A2 (en) 2015-11-26
WO2015178977A3 true WO2015178977A3 (en) 2016-01-28

Family

ID=53882555

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2015/015917 WO2015178977A2 (en) 2014-02-21 2015-02-13 In situ neural network co-processing

Country Status (5)

Country Link
US (1) US20150242741A1 (en)
EP (1) EP3108414A2 (en)
JP (1) JP2017509982A (en)
CN (1) CN106030622B (en)
WO (1) WO2015178977A2 (en)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3185184A1 (en) 2015-12-21 2017-06-28 Aiton Caldwell SA The method for analyzing a set of billing data in neural networks
US11922313B2 (en) 2016-02-11 2024-03-05 William Marsh Rice University Partitioned machine learning architecture
CN106897768B (en) * 2017-01-25 2020-04-21 清华大学 Neural network information sending method and system
WO2018149217A1 (en) * 2017-02-17 2018-08-23 清华大学 Information processing method and system for neural network computation core, and computer device
EP3563307B1 (en) 2017-02-23 2023-04-12 Cerebras Systems Inc. Accelerated deep learning
EP3586277B1 (en) * 2017-02-24 2024-04-03 Google LLC Training policy neural networks using path consistency learning
EP3607504B1 (en) 2017-04-17 2022-09-28 Cerebras Systems Inc. Neuron smearing for accelerated deep learning
CA3108089C (en) 2017-04-17 2024-04-02 Cerebras Systems Inc. Task activating for accelerated deep learning
US11488004B2 (en) 2017-04-17 2022-11-01 Cerebras Systems Inc. Neuron smearing for accelerated deep learning
US12017241B2 (en) 2017-07-21 2024-06-25 The Regents Of The University Of California Acoustic wave atomizer
GB2566702B (en) 2017-09-20 2021-11-03 Imagination Tech Ltd Hardware implementation of a deep neural network with variable output data format
KR102569086B1 (en) * 2017-11-20 2023-08-22 상하이 캠브리콘 인포메이션 테크놀로지 컴퍼니 리미티드 Task parallel processing method, device, system, storage medium and computer device
US10846621B2 (en) * 2017-12-12 2020-11-24 Amazon Technologies, Inc. Fast context switching for computational networks
US10803379B2 (en) 2017-12-12 2020-10-13 Amazon Technologies, Inc. Multi-memory on-chip computational network
US11429862B2 (en) 2018-03-20 2022-08-30 Sri International Dynamic adaptation of deep neural networks
EP3543917B1 (en) * 2018-03-19 2024-01-03 SRI International Inc. Dynamic adaptation of deep neural networks
WO2020044152A1 (en) 2018-08-28 2020-03-05 Cerebras Systems Inc. Scaled compute fabric for accelerated deep learning
WO2020044208A1 (en) 2018-08-29 2020-03-05 Cerebras Systems Inc. Isa enhancements for accelerated deep learning
WO2020044238A1 (en) 2018-08-29 2020-03-05 Cerebras Systems Inc. Processor element redundancy for accelerated deep learning
TW202018596A (en) * 2018-11-09 2020-05-16 財團法人資訊工業策進會 Distributed network computing system, distributed network computing method and non-transitory computer readable storage medium
CN109901878B (en) 2019-02-25 2021-07-23 北京灵汐科技有限公司 Brain-like computing chip and computing equipment
CN112418389A (en) * 2019-08-23 2021-02-26 北京希姆计算科技有限公司 Data processing method and device, electronic equipment and computer readable storage medium
WO2021074795A1 (en) 2019-10-16 2021-04-22 Cerebras Systems Inc. Dynamic routing for accelerated deep learning
WO2021074865A1 (en) 2019-10-16 2021-04-22 Cerebras Systems Inc. Basic wavelet filtering for accelerated deep learning
US12197926B2 (en) * 2020-07-03 2025-01-14 Mediatek Inc. Dynamic loading neural network inference at DRAM/on-bus SRAM/serial flash for power optimization
US20240005162A1 (en) * 2020-11-20 2024-01-04 University Of Zurich Error-triggered learning of multi-layer memristive spiking neural networks

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6418423B1 (en) * 1999-01-29 2002-07-09 International Business Machines Corporation Method and apparatus for executing neural network applications on a network of embedded devices

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6804632B2 (en) * 2001-12-06 2004-10-12 Intel Corporation Distribution of processing activity across processing hardware based on power consumption considerations
US7219085B2 (en) * 2003-12-09 2007-05-15 Microsoft Corporation System and method for accelerating and optimizing the processing of machine learning techniques using a graphics processing unit
US7533071B2 (en) * 2005-06-28 2009-05-12 Neurosciences Research Foundation, Inc. Neural modeling and brain-based devices using special purpose processor
US9665822B2 (en) * 2010-06-30 2017-05-30 International Business Machines Corporation Canonical spiking neuron network for spatiotemporal associative memory
US9111224B2 (en) * 2011-10-19 2015-08-18 Qualcomm Incorporated Method and apparatus for neural learning of natural multi-spike trains in spiking neural networks
US8819489B2 (en) * 2011-12-14 2014-08-26 Ati Technologies Ulc Accelerated processing unit debugging using a graphics processing unit centric debug core

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6418423B1 (en) * 1999-01-29 2002-07-09 International Business Machines Corporation Method and apparatus for executing neural network applications on a network of embedded devices

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
C. ROSSANT ET AL: "Playdoh: a lightweight Python library for distributed computing and optimisation", JOURNAL OF COMPUTATIONAL SCIENCE, vol. 4, no. 5, 6 July 2011 (2011-07-06), pages 352 - 359, XP055183262, DOI: 10.1016/j.jocs.2011.06.002 *
D. CALVERT ET AL: "Distributed artificial neural network architectures", PROCEEDINGS OF THE 19TH INTERNATIONAL SYMPOSIUM ON HIGH PERFORMANCE COMPUTING SYSTEMS AND APPLICATIONS (HPCS'05), 15 May 2005 (2005-05-15), pages 2 - 10, XP010800324, DOI: 10.1109/HPCS.2005.24 *
E. SCHIKUTA, E. MANN: "N2Sky - neural networks as services in the clouds", PROCEEDINGS OF THE 2013 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN'13), 4 August 2013 (2013-08-04), XP032542139, DOI: 10.1109/IJCNN.2013.6707113 *
F. GALLUPPI ET AL: "A hierachical configuration system for a massively parallel neural hardware platform", PROCEEDINGS OF THE 9TH CONFERENCE ON COMPUTING FRONTIERS (CF'12), 15 May 2012 (2012-05-15), pages 183 - 192, XP055179253, DOI: 10.1145/2212908.2212934 *
K. MINKOVICH ET AL: "HRLSim: a high performance spiking neural network simulator for GPGPU clusters", IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, vol. 25, no. 2, 15 August 2013 (2013-08-15), pages 316 - 331, XP011536922, DOI: 10.1109/TNNLS.2013.2276056 *
X. CHEN ET AL: "Pipelined back-propagation for context-dependent deep neural networks", PROCEEDINGS OF THE 13TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH'12), 9 September 2012 (2012-09-09), pages 26 - 29, XP055113689, ISSN: 1990-9770 *

Also Published As

Publication number Publication date
CN106030622B (en) 2019-09-20
CN106030622A (en) 2016-10-12
US20150242741A1 (en) 2015-08-27
EP3108414A2 (en) 2016-12-28
WO2015178977A2 (en) 2015-11-26
JP2017509982A (en) 2017-04-06

Similar Documents

Publication Publication Date Title
WO2015178977A3 (en) In situ neural network co-processing
IL261574A (en) Modified hyaluronic acid, method for making same and uses thereof
EP3118752A4 (en) Search method, system, and device
PL3189174T3 (en) Flat steel product with an al coating, method for producing the same, and method for producing a hot-formed steel component
EP3093361A4 (en) Bearing component, steel for bearing component, and production method for same
ZA201801830B (en) Matting agents and method for the production thereof
EP3206304A4 (en) Echo cancellation device, echo cancellation program, and echo cancellation method
EP3239741A4 (en) State calculating apparatus, state calculating method and state calculating program
PL3194681T3 (en) Honeycomb, in particular deformable honeycomb, for lightweight component and a block for production of the honeycomb and corresponding production method.
MX373312B (en) Process for the preparation of 4-alkoxy-3-hydroxypicolinic acids
EP3183241A4 (en) Catalyst and process for producing 2,5-furandicarboxylic acid from hydromethylfurfural in water
EP3248704A4 (en) Flanging method
PL3139770T3 (en) Fatty acid composition, method for the production thereof and use thereof
EP3109245A4 (en) Method for producting thiolane skeleton-type glycoconjugate, and thiolane skeleton-type glycoconjugate
EP3113087A4 (en) Personnel expense simulation system, personnel expense simulation method, and personnel expense simulation program
EP3205414A4 (en) Method for producing metal plate with protruding ridge, metal plate with protruding ridge, and structural component
EP3169827B8 (en) Steel piston ring, and method for producing the same
PL3018497T3 (en) Method for detecting earthquakes and locating epicentres by means of a network of lights.
EP3178803A4 (en) Method for preparing 2,3,3,3-tetrafluoropropene
EP3170794A4 (en) Composition including silicotitanate having sitinakite structure, and production method for same
PL3240887T3 (en) Oleaginous yeast variant, method for obtaining thereof and use thereof for lipid production
EP3193285A4 (en) Simulation system, simulation method, and simulation program
EP3113046A4 (en) Product search device, system, method, and program
SG11201709899QA (en) Microorganism producing o-acetyl-homoserine, and method for producing o-acetyl-homoserine by using same
EP3550034A4 (en) Block structure, container and method for constructing block structure

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15759970

Country of ref document: EP

Kind code of ref document: A2

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
REEP Request for entry into the european phase

Ref document number: 2015759970

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2015759970

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2016553381

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE