Embedded Files
Adaptive Mixture-of-Heads: Routing Attention Heads in GPT-2 with Fixed and Dynamic Sparsity
Rebuilt GPT-2 with David Stutz using dynamic attention head routing, improving efficiency by 50%+ in FLOPs while maintaining overall model performance.
[Poster]
Adaptive Mixture-of-Heads.pdfUsed probabilistic models and recursive simulations to see in which situations teams should foul.
Determining If Teams Should Foul Late in a Basketball Game PDF.pdfWorked with Daniel Rashes to build Markov matrices and analyze which sports leagues allow bad teams to improve the quickest.
Page updated
Google Sites
Report abuse