Over the weekend, I participated in the Eval Hackathon, where we are asked to design new ways to audit, monitor, red-team, and evaluate language models and we need to submit an report in the end to showcase our design and result.

I am glad that I have participated and put a lot efforts into it. I felt proud what we have came up through the 36 hours and here is our report: Can Large Language Models Solve Security Challenges? and our code is open sourced here. In this report, we test large language models’ ability to interact and break out of shell environments using the OverTheWire Wargames environment, showing the models’ surprising ability to do action-oriented cyberexploits in shell environments.

Through the Hackathon, I learnt the following things:

  1. Teaming up with people with similar goals is really valuable: I found myself having higher motivation working as a team. Collaborating with teammates who share similar goal is also a great productivity booster. Comparing with working alone, we can also learn more and have more output from this process.
  2. Doing some prep work would be great: from hindsight, I think it could be better if we had more discussion in the beginning about which direction to go and prepare some code and paper template in the beginning, especially for this kind of Hackathon, where we are asked to submit a report in the end.
  3. Asking for help, don’t just get stuck: Asking help from teammates, forum, group chat and etc. Time is limited and we should jump out of local stuck zone as quickly as possible and move on.
  4. Fast Prototyping: This is one thing I am still learning. Testing different ideas, feasibility and impact as quickly as possible. I think this skill can grow as we become more experienced and have better research taste. But it is also a skill can be learnt. I generally find myself should read more relevant work before diving into the coding part. I still need to practice the skill of grasping the gist from a long context.

Hope you enjoy this blog post and feel free to reach out if you have any question or want to have more discussion!