V2 Performance Signal Detected by TorchBench CI on '2.6.0.dev20240928+cu124'

[xuzhao9]

TorchBench CI has detected a performance signal.

Base PyTorch version: 2.4.0.dev20240606+cu121

Base PyTorch commit: ffaea656b5d8ff6518669494cc8f664b94f8e8b1

Affected PyTorch version: 2.6.0.dev20240928+cu124

Affected PyTorch commit: f42e88fea5915012ddb4cd9530f367524bf57019

Affected Tests:

• test_train[Super_SloMo-cuda-eager]: +14.62201%
• test_eval[Super_SloMo-cuda-eager]: +11.07397%
• test_train[Super_SloMo-cuda-jit]: +14.74666%
• test_eval[Super_SloMo-cuda-jit]: +11.19280%
• test_eval[alexnet-cuda-eager]: +10.39477%
• test_eval[alexnet-cuda-jit]: +13.61220%
• test_train[dcgan-cuda-eager]: +63.66396%
• test_train[dcgan-cuda-jit]: +63.59831%
• test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]: -20.34751%
• test_eval[pytorch_stargan-cpu-eager]: -14.58000%
• test_eval[pytorch_stargan-cpu-jit]: -11.78831%
• test_eval[shufflenet_v2_x1_0-cpu-eager]: +11.27917%
• test_eval[shufflenet_v2_x1_0-cuda-jit]: -27.55347%
• test_eval[timm_regnet-cuda-eager]: +170.62257%
• test_eval[timm_regnet-cuda-jit]: +356.95663%
• test_train[tts_angular-cpu-eager]: +7.23293%
• test_train[yolov3-cuda-eager]: -9.82416%

cc @xuzhao9

Result json:

{
  "start": "ffaea656b5d8ff6518669494cc8f664b94f8e8b1",
  "end": "f42e88fea5915012ddb4cd9530f367524bf57019",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "734e8f6ad7e",
      "commit1_time": "2024-06-11 00:40:31 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.24733077911660076
      },
      "commit2": "3b555ba4771",
      "commit2_time": "2024-06-11 01:32:45 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.29744484294205903
      }
    },
    {
      "commit1": "841d87177a9",
      "commit1_time": "2024-06-11 02:26:06 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.3014581101946533
      },
      "commit2": "d1d9bc7aa65",
      "commit2_time": "2024-06-11 02:37:04 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2667920907959342
      }
    },
    {
      "commit1": "665e568381e",
      "commit1_time": "2024-06-11 03:02:51 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.24831760479137302
      },
      "commit2": "ca45649eb58",
      "commit2_time": "2024-06-11 03:02:51 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2672467709518969
      }
    },
    {
      "commit1": "d3b458e6032",
      "commit1_time": "2024-08-15 17:37:45 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.29927923288196323
      },
      "commit2": "cfec69e2a1f",
      "commit2_time": "2024-08-15 17:55:07 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2704269388690591
      }
    },
    {
      "commit1": "cfec69e2a1f",
      "commit1_time": "2024-08-15 17:55:07 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2704269388690591
      },
      "commit2": "5ec9c0bc4a9",
      "commit2_time": "2024-08-15 17:55:36 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.24651952590793372
      }
    },
    {
      "commit1": "27538671ae6",
      "commit1_time": "2024-08-15 18:52:08 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.24935734514147043
      },
      "commit2": "dff388491bb",
      "commit2_time": "2024-08-15 18:56:55 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.27220484204590323
      }
    },
    {
      "commit1": "73604eed0ce",
      "commit1_time": "2024-08-26 17:43:52 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.29835418863222
      },
      "commit2": "0f5b052dbae",
      "commit2_time": "2024-08-26 17:43:58 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2612370106391609
      }
    },
    {
      "commit1": "bc21689136f",
      "commit1_time": "2024-09-27 21:37:34 +0000",
      "commit1_digest": {
        "test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]": 49.8167310513556
      },
      "commit2": "adbcaee950a",
      "commit2_time": "2024-09-27 22:22:37 +0000",
      "commit2_digest": {
        "test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]": 40.27286036200822
      }
    },
    {
      "commit1": "ca45649eb58",
      "commit1_time": "2024-06-11 03:02:51 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2672467709518969
      },
      "commit2": "7afffdf48b5",
      "commit2_time": "2024-06-11 03:12:11 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2934564116410911
      }
    },
    {
      "commit1": "cdb9c7d2283",
      "commit1_time": "2024-08-22 06:17:21 +0000",
      "commit1_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.24883492691442372
      },
      "commit2": "b7baa062fc0",
      "commit2_time": "2024-08-22 06:27:03 +0000",
      "commit2_digest": {
        "test_eval[pytorch_stargan-cpu-jit]": 0.2984951770864427
      }
    }
  ]
}

Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/11086313345