05 - Is RDMA enough?


  • Description:: is it?

  • User-space networking : Let the process manage its networking resources

  • Kernel Bypass: Access hardware/NIC resources directly from the user-space

  • shuffle acceleration in spark

    • mapreduce: pick data and divide it in chunkies and shuffle them
  • gradient aggregation in distributed deep learning

    • every deep learning queue is aggregated and merged into another giant one
  • key-value store acceleration

    • redis style
  • distributed file systems storage policies

    • a combination of RDMA and SPIN