onnxruntime/orttraining
Peishen Yan 2cdc05f189
Move Gelu and LayerNorm fusion to L1 optimization (#21332)
According to https://github.com/microsoft/onnxruntime/issues/20915, we
move the Gelu and LayerNorm fusion to L1 with a condition on the ONNX
opset the model imports (LayerNorm requires opset 16+ and Gelu requires
opset 20+.) If the opset version doesn't meet the requirements, the
fusion is delayed to L2 optimization since the internal contrib op
doesn't have a requirement for any specific ONNX opset.

---------

Co-authored-by: Scott McKay <Scott.McKay@microsoft.com>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
2024-09-09 13:27:52 +10:00
..
orttraining Move Gelu and LayerNorm fusion to L1 optimization (#21332) 2024-09-09 13:27:52 +10:00
tools Update ruff and clang-format versions (#21479) 2024-07-24 11:50:11 -07:00