FAQ
Does my model have to reconstruct images in full resolution or can it be cropped?
The decoder has to produce PNG images where each image has the same resolution as the corresponding image in the validation or test set.
How is PSNR calculated?
We compute a single MSE value by averaging across all RGB channels of all pixels of the whole dataset, and from that calculate a PSNR value.
How is MS-SSIM calculated?
First, a single MS-SSIM value is calculated for each PNG image. We then compute a weighted average where each PNG is weighted by the number of its pixels.
Why are you computing PSNR and MS-SSIM?
We chose two popular metrics to be computed for all methods. These gives some idea to participants on how they might stack up, but provides no guarantees on correlation with the final evaluation.
How do I get a CMT Paper ID?
Please register a paper on our CMT site. You will need to provide a team name, a paper title, and (optionally) an abstract.
The evaluation server gives "ERROR: Missing image IMG_20170114_210112.png". What am I doing wrong?
The error means that the decoder failed and did not produce all required files. Make sure that the names of the files match the target files' names exactly.
How can I see the decoding progress?
Make sure that your decoder outputs this information to the standard output. This information should be visibile to you once you've submitted a decoder.
In which directory should the decoder save images?
The decoder can save images in the current working directory . or in any arbitrary subfolder
such as images.
Do we need to encode the Y, U, and V channels of the video data separately?
You may encode the data however you want. For example, you may encode the Y, U, and V channels of one frame into a single file. However, your decoder needs to output separate PNGs for each channel.
Does the model count towards the target bit-rate?
No. This year we don't do this.
What are the specifications of the server the decoder is run on?
Each submission is provided 2 CPUs, 12GB RAM and P100 GPU with 16GB memory. There is also a time limit for each decoder which is set to 5 hours for the validation set (if the test set turns out to be larger, the time limit will be proportionally larger).
Can a single institution (company, university) participate with more than one team?
As long as there are no common team members, and the methods are sufficiently different (i.e., the participants of the various teams didn't work together in the past), yes. One notable exception to this rule is when a common advisor (or research group's leader) is shared. That is explicitly allowed.
An insufficiently different pair of methods is, for example, the same neural network with different parameters (i.e., learning rate, loss weights, etc.). A sufficiently different pair of methods should be such that if one were to envision submitting them at the same conference/workshop, they would be both accepted without any concerns of plagiarism.
Will the images/videos my method generates be made public?
Yes. In the spirit of being able to compare across the various submitted methods at any point in the future, we plan to make all participants' outputs from the test set publicly available. These will follow the licensing terms from Unsplash for images, and Pexels for videos.
If I participate as part of a company's team, am I eligible for prize money?
No. You're eligible for an award and an award certificate, but we will be distributing the money to the next eligible student team. We're doing this to enable more students to participate in person (if the conference is held in person).
Please email us at clic-organizers+questions@googlegroups.com. This should hopefully provide you with a faster reply than contacting a single individual organizer.
How can I request to remove an image from the dataset(s)?
Please email us at clic-organizers+request@googlegroups.com.
Can I request a custom docker image?
Before requesting one, please check whether one of the already avialable images would work for you. If that does not work, please email us at clic-list@googlegroups.com.
Can I store any data in my Docker image?
The only acceptable usage for Docker images is dependencies for your code. No data may be stored within these images. In the submission page you're required to provide two files: (1) decoder binary/code; and (2) data. Everything needed to decode the data must be present in the first part (1). We explicitly separate the decoder from data so that you can freeze the decoder after validation. That decoder will be used to decode the data for the test set which will be secret until the freeze happens. One more time: no data may be stored in custom Docker images and if you need a large file that's common for validation/testing, please include it in the decoder zip file.
