Email Login
News | Research | DeepLearning | Seminar | Q&A

Google Details Tensor Chip Powers..!!!

GreatMind 2017.04.14 11:31 조회 수 : 101

Google Details Tensor Chip Powers

By David Schneider

Posted 6 Apr 2017 | 17:00 GMT


Google's Tensor Processing Unit boardGoogleGoogle's Tensor Processing Unit is a printed-circuit card, which inserts into existing servers and acts as a co-processor, one tailored for neural-network calculations.


In January’s special Top Tech 2017 issue, I wrote about various efforts to produce custom hardware tailored for performing deep-learning calculations. Prime among those is Google’s Tensor Processing Unit, or TPU, which Google has deployed in its data centers since early in 2015.


In that article, I speculated that the TPU was likely designed for performing what are called  “inference” calculations. That is, it’s designed to quickly and efficiently calculate whatever it is that the neural-network it’s running was created to do. But that neural network would also have to be “trained,” meaning that its many parameters would be tuned to carry out the desired task. Training a neural network normally takes a different set of computational skills: In particular, training often requires the use of higher-precision arithmetic than does inference.


Yesterday, Google released a fairly detailed description of the TPU and its performance relative to CPUs and GPUs. I was happy to see that the surmise I had made in January was correct: The TPU is built for doing inference, having hardware that operates on 8-bit integers rather than higher-precision floating-point numbers.


Yesterday afternoon, David Patterson, an emeritus professor of computer science at the University of California, Berkeley and one of the co-authors of the report, presented these findings at a regional seminar of the National Academy of Engineering, held at the Computer History Museum in Menlo Park, Calif. The abstract for his talk summed up the main point nicely. It reads in part: “The TPU is an order of magnitude faster than contemporary CPUs and GPUs and its relative performance per watt is even larger.”


Google’s blog post about the release of the report shows how much of a difference in relative performance there can be, particularly in regard to energy efficiency. For example, compared with a contemporary GPU, the TPU is said to offer 83 times the performance per watt.  That might be something of an exaggeration, because the report itself claims only that there’s a range of between 41 times and 83 times. And that’s for a quantity the authors call incremental performance. The range of improvement for total performance is less: from 14 to 16 times better for the TPU compared with that of a GPU.


The benchmark tests used to reach these conclusions are based on a half dozen of the actual kinds of neural-network programs that people are running at Google data centers. So it’s unlikely that anyone would critique these results on the basis of the tests not reflecting real-world circumstances. But it struck me that a different critique might well be in order.


The problem is this: These researchers are comparing their 8-bit TPU with higher-precision GPUs and CPUs, which are just not well suited to inference calculations. The GPU exemplar Google used in its report is Nvidia’s K80 board, which performs both single-precision (32-bit) and double-precision (64-bit) calculations. While they’re often important for training neural networks, such levels of precision aren’t typically needed for inference.


In my January story, I noted that Nvidia’s newer Pascal family of GPUs can perform “half-precision” (16-bit) operations and speculated that the company may soon produce units fully capable of 8-bit operations, in which case they might be much more efficient when carrying out inference calculations for neural-network programs.


The report’s authors anticipated such a criticism in the final section of their paper; there they considered the assertion (which they label a fallacy) that “CPU and GPU results would be comparable to the TPU if we used them more efficiently or compared to newer versions.” In discussing this point, they say they had tested only one CPU that could support 8-bit calculations, and the TPU was 3.5 times better. But they don’t really address the question of how GPU’s tailored for 8-bit calculations would fare—an important question if such GPUs soon became widely available.


Should that come to pass, I hope that these Googlers will re-run their benchmarks and let us know how TPUs and 8-bit-capable GPUs compare.



번호 제목 글쓴이 날짜 조회 수
공지 2019 IVPL 송년회 [1] yjchoi 2019.12.20 263
공지 [한국과총]글로벌 현장연수 참가자 모집 GreatMind 2018.04.04 292
공지 Inform Your Students: Global Student Challenge. Over US$2,000 in Prizes! GreatMind 2017.01.12 4343
공지 nVidia Parallel Computing Software Engineer 채용 GreatMind 2016.11.16 717
67 [Notice] Mr. Hong's paper has been accepted in Displays Journal (Elsevier)....!!!! GreatMind 2017.07.13 58
66 [Notice] Dr. Tae-Jung Kim's paper has been accepted for publication in Personal and Ubiquitous Computing (Springer)...!!! GreatMind 2017.07.06 62
65 JCR 2016 Update...!!! GreatMind 2017.06.27 97
64 [Notice] Ms. Choi's paper has been accepted in Journal of Multimedia and Information System...!!! GreatMind 2017.06.27 69
63 [Notice] MR. Lee's paper has been accepted for publication in Mobile Networks and Applications (ACM/Springer)...!! GreatMind 2017.06.27 67
62 [Notice] Mr. Avishek's paper has been accepted for publication in Personal and Ubiquitous Computing (Springer)....! GreatMind 2017.06.26 101
61 Reuters Top 75 : Asia’s Most Innovative Universities ...!!!! GreatMind 2017.06.08 77
60 [Notice] Da-mi Jung and Ha-Yoon Jung's paper has been published in Journal of Digital Contents Society (KCI indexed)...! GreatMind 2017.05.31 192
59 Russian Team Takes World Champion Title in ACM ICPC Programming Contest..!!! GreatMind 2017.05.26 170
58 CLOUD VIDEO INTELLIGENCE api by Google. file GreatMind 2017.05.04 105
57 Nvidia wants AI to Get Out of the Cloud and Into a Camera, Drone, or Other Gadget Near You GreatMind 2017.04.14 120
» Google Details Tensor Chip Powers..!!! GreatMind 2017.04.14 101
55 Technique makes more efficient, independent holograms..!! GreatMind 2017.04.13 207
54 How Is Mastering Autonomous Driving with Deep Learning..!! GreatMind 2017.03.16 100
53 Human OS Alert GreatMind 2017.02.23 172
52 China’s Artificial-Intelligence Boom GreatMind 2017.02.20 97
51 [News] Project aims for wearable systems in people centred smart cities GreatMind 2017.01.14 51
50 Inform Your Students: Global Student Challenge. Over US$2,000 in Prizes! GreatMind 2017.01.12 4343
49 Scopus Newsletter: CiteScore metrics Special Edition..!!! GreatMind 2016.12.16 526
48 [Notice] Mr. Baek has vistited to our research group...!!! file GreatMind 2016.12.05 132