09 - GPUs TPUs


  • Description:: How does storage work in a large-scale datacenter


  • from ‘10 to ‘20 GPU usage has increased
  • CPUs vs GPUs:
    • the CPUs have cores and a lot of caches
    • the latter ones have much more cores, at the cost of simpler/smaller caches and control units
    • program few threads CPUs, many threads GPUs
  • array of streaming multiprocessors
  • each of them: multiple cores with shared control logic and instruction cache
    • they can also have other modules (e.g. raytracing)
    • shared global memory

  • they are built on the classic von Neumann architecture, that it is not performance-oriented (other models are better)

Programming a GPU

Several options can be used:

  • C++
  • directive-based languages (openMP)
  • frameworks that abstract the hardware away (Kokkos)
  • native libraries (OpenCL)
  • native code (CUDA)


  • is the most used and most optimized
  • threads are divided in thread blocks, multiple thread blocks from a grid
  • instructions used to be copied back and forth central memory to GPU, but now they have a common shared area
  • one big challenge is how to connect multiple GPUs each other


  • similar tp GPUs but they use tensors, a multidimensional array structure
  • Key feature is the Matrix Multiplication Unit
  • best way to connect them: torus
  • performance high, cost low if compared with classic GPUs