The project began during a conversation with Nicholas Carlini, Bo Li, and Tom Goldstein at a PI meeting in San Jose. We were discussing a deceptively simple question: if an image carries an invisible watermark, how much confidence should that signal really earn?
Laboratory evaluations can make a watermark look dependable, yet the real world is not a clean benchmark. Images are resized, cropped, compressed, edited, and passed between platforms. False positives can wrongly label authentic work as synthetic, while open detection interfaces can help an attacker reverse-engineer the very signal they are meant to protect.
From research question to competition
We wanted a test that resembled the black-box conditions a deployed watermark would face. That led to Erasing the Invisible, a NeurIPS competition asking participants to remove watermarks while preserving image quality and meaning.
Getting the scientific design accepted was only the beginning. Students were balancing internships, paper deadlines, and the job market. We recruited additional help, built a starter kit and submission process, and began negotiating the practical details of running a public competition.
Choosing a platform under pressure
Kaggle offered reach and a mature competition environment, but the research track brought fundraising and legal requirements that did not fit our timeline. A community track reduced some barriers, yet technical support constraints remained. Near launch, we pivoted to Codabench so we could control the evaluation pipeline and keep the challenge on schedule.
The move required the team to divide front-end, back-end, evaluation, and resource work while the ICLR cycle was already underway. It was exhausting, but it also made the competition a genuine systems project rather than a static benchmark release.
Academic service creates the platforms on which ideas can be tested, challenged, and exchanged.
What stayed with me
The experience changed how I value service. We readily celebrate papers and intellectual breakthroughs, but well-run workshops, benchmarks, and competitions demand their own rigor, judgment, and sustained coordination. That labor is often invisible precisely when the final event works.
I am grateful to the full organizing team, especially Mucong Ding, Tahseen Rabbani, Bang An, Chenghao Deng, and Tom Goldstein. Their persistence turned an informal discussion into a community resource for evaluating whether image watermarks survive outside idealized settings.