-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix iou precision error #1592
base: main
Are you sure you want to change the base?
fix iou precision error #1592
Conversation
currently, when cal iou, yolox use float16 (in float16 mode), but float16 only has 10 bits for valid part this may lead iou precision error, for example torch.tensor(841, dtype=float16) - torch.tensor(5.2, dtype=float16) ==torch.tensor(836., dtype=torch.float16). this can lead iou to be larger than 1 and in cls loss, yolox multiplies target with iou, so that the bce loss can be negtive! change iou calculation to float32 seems to fix that
fix line length 103 > 100
Thanks for pointing it out. Another way to solve this issue is using clip to ensure the iou value. Why not using it? @h-bo |
That's also a solution, I'm not sure which one is better. I just tried float32 in my dataset and it improves 2%~ |
@h-bo Thanks! We will try it on COCO and see what happens. |
@h-bo Hi. We have tested this PR on COCO. Here are our reproduced results: Seems this PR does not affect the result on COCO. We recommend using ''value clip'' to tackle your reported issue. What do you think? |
@Joker316701882 It will be fine. Maybe the negtive cls loss problem is more severe in my dataset, if have time, I will dive into the reason. |
@h-bo |
currently, when cal iou, yolox use float16 (in float16 mode), but float16 only has 10 bits for valid part this may lead iou precision error:
for example torch.tensor(841, dtype=float16) - torch.tensor(5.2, dtype=float16) ==torch.tensor(836., dtype=torch.float16).
this can lead iou to be larger than 1 and in cls loss, yolox multiplies target with iou, so that the bce loss can be negtive!
change iou calculation to float32 seems to fix that