moonshotai

mixture-of-experts ,  long-context ,  moba ,  moonshot-ai ,  moonshotai ,  block-attention ,  sparse-attention ,  mixture-of-block-attention

cuda ,  pytorch ,  linear-attention ,  kimi ,  cutlass ,  gpu-kernel ,  moonshotai ,  flash-attention ,  kimi-delta-attention ,  flashkda

transformer ,  paper ,  attention ,  architecture ,  kimi ,  attention-residuals ,  residual-connection ,  moonshotai ,  scaling