Reflections from CDDC

[daniellimws]

Hi, here are some notes @Enigmatrix and I made at the end of CDDC. Mainly for all of us to know what to not do for our CTF. Please add on if you got any more ideas/observations.

Challenges

1. Challenges were guessy
   a. If you know, you know
   b. Sometimes need to guess esoteric commands

2. Improper scoring for the challenges
   a. Static scoring
   b. Solution: Use dynamic scoring to organically let the challenges define their worth

3. Weird setup for pwn challenge
   a. Wrong libc provided
   b. SSH to pwn, instead of over nc

4. Improper timing for challenge releases
   a. There are multiple teams who solved all available challenges, and are just waiting for the final (known) release at 10am to quickly solve the rest of the newly released challenges. (Solved in ~15 minutes. Effectively, the prizes boil down to the last 15 minutes amongst top teams. Everything before that did not matter.)

   b. Solutions: Create sufficient challenges, with sufficient difficulty for the duration of the CTF. When/If challenges are solved too fast, release the rest of the challenges accordingly.

Infrastructure

1. [Most Obvious] CTF was postponed by 1 day, but connection issues were not fixed
   a. CTF site cannot handle traffic

2. Improper permissions
   a. Users could break the challenge for others by deleting or overwriting files, and changing passwords
   b. Can just read flag.txt from base directory
   c. Solution: Have a unique challenge instance for every team, like TAMUctf

3. Shared infrastructure for multiple challenges
   a. Solve 1 challenge on same server = solve everything
   b. People were using wall to spam the Linux server
   c. Solution Proper dockerization between challenges

4. Challenge files/endpoints should be unenumerable
   a. Solution: Use uuids, and disable directory listing

5. Flag format is not maintained!

6. Admins fighting fire instead of preemptively checking known issues in challenges
   a. Solution: Reset the challenge every XX minutes, and have monitoring scripts, IP ban, rate limit etc.

7. Can see scoreboard after it's frozen.

Moderating

1. No ticketing system
   a. When the competition started, there was no support channel for tech help/it was poorly managed. Meme messages were spread across the cddc-s channel along with legitimate questions about the challenge

2. We don't know the lines of communication
   a. "Can I dm an admin"

3. Information is not shared properly
   a. Announcements were not tagged to @everyone, so some people did not get any notifications.
   b. Uptime and breaking changes are not announced at all, or not tagged to @everyone, so people do not know if they made a mistake or the challenge was down

4. No proper moderation
   a. People were swearing and spamming, and even disgusting ascii arts.
   b. Some people were discussing/disclosing sensitive challenge information, but support team doesn't bother.

5. Support team does not know the challenges well
   a. Half the time, after asking the admins about the challenges, they don't seem to know anything useful to give any helpful response.

6. No FAQ for common problems encounted by users