I found Deep Learning Curriculum(DLC) very helpful and challenging. It is a good way to get hands-on experience in topics related with technical AI Safety. Here are some points I want to mention about self-studying it:

  1. Difficulty : As mentioned in the DLC, it is indeed hard to solve some topics alone. I am not a newbie in the field as I had some experience with DL through Kaggle and work, but I still find implementing something from scratch can be intimidating and often lost direction in the middle of the exercise. There is no standard solution available for this curriculum, which made it hard to know whether the solution is correct or not. For example, in the Alignment topic, I couldn’t get the RLHF model perform better than the pretrained model but I doubt there is anything wrong with the RL algorithm but most likely from the model size and data size… It probably works like a real research environment, we need to find evidence to prove our solution is right or not instead of relying on existing solution, which doesn’t exist in real world.
  2. Worth or not: Yes, definitely! I learnt a lot through the process and enriched my resume with relevant project and thoughts. This can also act as an proof for our determination to technical AI safety road or AI in general.
  3. Helpful resources:
    1. ARENA is quite helpful since it provides more instructions on related topics.
    2. Google Colab is quite helpful. I don’t have personal GPU to use and I find Colab Pro is enough for most of the exercise. Colab Pro+ can offer non-stop training when we need to train something longer (like the exercise in RL topic). The downside is that: It is tricky to set up multi-GPU on Colab and that’s part of the reason I couldn’t finish “Topic 3 Training at Scale” because I am not sure how to get multi-GPU environment using MPI, even though it is possible to fake multi-GPU using NCCL (relevant solution here).
    3. The existing solution(s) is quite helpful to refer to when I get stuck. Kudos to the existing solution from ckkissane! This is also part of the reason I want to share my solution and experience about this curriculum.
  4. How much time for each exercise: As we can see in the end, I spent ~20 hours on average on each topic but it varies a lot, some topics take <= 10 hours while some takes >=30. It depends on the exercise difficulty and also personal interests – whether or not we want to dive deeper into the topics.
  5. Skip or not skip some topics: I skipped Topic 4 & 5 according to my own familiarity, interest and time. I think this should be encouraged since one curriculum wouldn’t fit perfectly for everyone. We’d better refrain ourselves from finishing mode and instead focus on those we really need and even spend more time on the topics that interest us. I may come back to them when I find them relevant in the future.
  6. Study buddy: Unfortunately, I didn’t find a study buddy to talk with and from hindsight, it would be nice to have that for motivation and learning reason. It would be nice to have a Discord channel or something to form a study group. But…. doing the exercise seems to mimic a real research experience, which is quite valuable in some sense :)

Below is how I spent my time on various topics in DLC and relevant solutions to each topic and hope it can work as a reference for someone also interested in self-studying this material. I arranged the topics by the sequence (from first to last) I did them.

I finished topic 1, 8, 2, 6, 7, 9, 3 (half) and skipped 4 and 5. Total Hours: ~134 hours.

  1. Topic 1 Transformer (~20 hours): post here and Colab here.
  2. Topic 8 Interpretability(~8 hours): post here and Colab here. The exercise takes me around 3 hours, in addition to 5 hours learning through ARENA.
  3. Topic 2 Scaling Laws (~15 hours): post here and Colab here.
  4. Topic 6 RL(~36 hours): post hereand Colab here. The exercise takes around 25 hours, in addition to 11 hours learning through ARENA.
  5. Topic 7 Alignment (~30 hours): GitHub repository here
  6. Topic 9 Adversarial Training (~10 hours): post here and Colab here
  7. Topic 3 Training at Scale (~15 hours): I spent around 10 hours learning from ARENA and 5 hours to do the exercise. Unfortunately, due to computation limitation, I didn’t successfully finish the exercise. But I got the main idea from ARENA and I felt satisfied at this moment.
  8. Skipped Topic 4 Optimization and Topic 5 Modeling Objectives due to time limitation and personal interests.