[Question] Pytorch 2.0 with colossalAI #3150
Unanswered
skyshine102
asked this question in
Community | Q&A
Replies: 1 comment 1 reply
-
|
Hi @skyshine102 Thanks for your suggestion! Yes, we are working on it : ) |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Looking forward to it! New backend in torch 2.0 can reduce the burden of compiling various fused/ special kernel (i.e. internal flash attention) |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Today, pytorch 2.0 has been released and it seems that the backend is quite efficient. I am wondering if colossalai will support pytorch 2.0 in the near future?
Or someone who has been successfully built the environment with torch 2.0 + colossalai can share experiences?
Beta Was this translation helpful? Give feedback.
All reactions