The container was killed as it exceeded its memory limit

At two points in time, Sept. 16, 2024, 8:37 a.m. and Sept. 16, 2024, 6:55 a.m., I submitted two different containers. In the first category, both containers were submitted and working fine during the preliminary testing phase, but now it prompts The container was killed as it exceeded its memory limit. i have tested locally using a long video that the peak memory footprint is only 7~8GB, and i have selected in test.sh and when creating the image 32G RAM. Please let me know why this error is prompted.

best,

The commit with mirror ID 0277ccff-23ae-4343-bde6-efce111ed255 also fails with its message The algorithm failed on one or more cases.Please provide me the error message if you can, thanks.

All of your failed submissions are running out of time for the final testing phase cat 1. There is a 60 mins time limit per video processing so please make sure your algorithm isn't taking too long to process each frame.

FYI, the max video length in the testing set is ~50 mins. The compute available has 8 CPU, 32 GB RAM and 1 T4 GPU.

I have examined my submitted image in detail and tested a 3600 frame video (simulated at 1fps, 1 hour) in the Try Out interface, and its only taken 20 minutes to complete the inference. Also, this time it took just under half an hour from submission to error reporting, not even reaching the time limit for a video. But it still says The container was killed as it exceeded its memory limit. as you can see in the image I uploaded, tested in the same environment of the Try Out interface, the peak memory usage is only 3%. So I don't think the error is due to a runtime timeout or memory exhaustion, and I would like to get further information about the error.

Dear Salen,

Could you share an example of your JSON output file? The evaluation container exceeds its memory limits because your JSON output file is bigger than usual.

Just so you can compare, other team's evaluations for bboxes are happening in 1-2 minutes, while yours took 9 minutes before it crashed. Plots here: https://ibb.co/vJ5xL2q

Thank you, SurgVU Organizing Team