Add tests for reduction with sum of squares (RSS), compute of RNSNorm, and RNSNorm fused with Matmul #593
Conversation
| } | ||
| } | ||
|
|
||
| private static void finalSum(KernelContext context, FloatArray reduce, int size, float eps) { |
| float expected = outputSeqLogits.get(i); // Expected value from the sequential output | ||
| float actual = outputLogits.get(i); // Actual value from the RNS output | ||
|
|
||
| // assertEquals("Mismatch at index " + i, expected, actual, 1f); // Allow some tolerance |
There was a problem hiding this comment.
I guess the test should assert the expected and actual values, right?
|
Include the new test in the test-suite: |
|
The new test enters in an infinite loop when running with the SPIR-V backend: tornado-test -V uk.ac.manchester.tornado.unittests.compute.LLMFusedKernelsTest
/home/juan/tornadovm/TornadoVM/bin/sdk/bin/tornado --jvm "-Xmx6g -Dtornado.recover.bailout=False -Dtornado.unittests.verbose=True " -m tornado.unittests/uk.ac.manchester.tornado.unittests.tools.TornadoTestRunner --params "uk.ac.manchester.tornado.unittests.compute.LLMFusedKernelsTest"The PTX and OpenCL backends run fine. |
There was a problem hiding this comment.
I think we should add the new LLMFusedKernelsTest class in the tornado-test in order to be run when Jenkins runs the unit-tests. In my setup, the tests pass for PTX. But, the tests in the LLMFusedKernelsTest class are not finishing when running with SPIR-V. I guess, that they are not supported for SPIR-V?
| * </code> | ||
| */ | ||
|
|
||
| public class LLMFusedKernelsTest extends TornadoTestBase { |
There was a problem hiding this comment.
To keep consistency with other test classes, I would suggest to move the "Test" at the beginning of the name of the class.
| public static void normalizeAndScale(KernelContext context, | ||
| FloatArray out, FloatArray input, FloatArray weight, FloatArray scalingFactorBuffer, | ||
| int size, float eps) { | ||
|
|
||
| int globalIdx = context.globalIdx; | ||
|
|
||
| if (globalIdx < size) { | ||
| float scaledValue = weight.get(globalIdx) * (scalingFactorBuffer.get(0) * input.get(globalIdx)); | ||
| out.set(globalIdx, scaledValue); | ||
| } | ||
| } |
|
|
||
| @Test | ||
| public void testRNSNorm() throws TornadoExecutionPlanException { | ||
| final int size = 2048; |
There was a problem hiding this comment.
I think we should add the following, unless it is supported:
assertNotBackend(TornadoVMBackendType.SPIRV);|
|
||
| @Test | ||
| public void testRNSNormFusedWithMatMul() throws TornadoExecutionPlanException { | ||
| final int size = 2048; |
There was a problem hiding this comment.
I think we should add the following, unless it is supported:
assertNotBackend(TornadoVMBackendType.SPIRV);| float expected = outputSeqLogits.get(i); // Expected value from the sequential output | ||
| float actual = outputLogits.get(i); // Actual value from the RNS output | ||
|
|
||
| // assertEquals("Mismatch at index " + i, expected, actual, 1f); // Allow some tolerance |
There was a problem hiding this comment.
I guess the test should assert the expected and actual values, right?
|
@mikepapadim , is this ready? |
Description
This PR adds tests to test the code gen and and vailidity of the results for some common operation required in the LLM architecture.
Backend/s tested
Mark the backends affected by this PR.
OS tested
Mark the OS where this PR is tested.
Did you check on FPGAs?
If it is applicable, check your changes on FPGAs.
How to test the new patch?