Sub-project cannot display output from junit tests: OutOfMemoryError: Java heap space

[Charlweed]

I have a testcase which spews exceptions, and that crashes the gradle plugin, and I must restart NetBeans. I have tried setting the heap to 8 gigs(!) of ram or more, everywhere I can. I tried in netbeans.conf, settings,gradle, build.gradle, and in .nb-gradle-properties with the UI.
In fact, the console shows that gradle starts with -Xmx10g
But the NetBeans test UI never displays anything, even though I think they finish. Instead I get the error dialog with the exception below.
When I run gradlew from the command-line, the tests complete as expected, showing the exceptions. I also see the exceptions on the console when I run BuildShip in Eclipse.
The wrapper version of gradle is 4.10.
NetBeans 9
Windows 7 amd64

The project is [email protected]:RPTools/maptool.git, and the sub-project is maptool, but the testcase with the hundreds of exceptions is my own.

java.lang.OutOfMemoryError: Java heap space
  	at com.sun.org.apache.xerces.internal.util.XMLStringBuffer.append(XMLStringBuffer.java:208)
  	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.scanData(XMLEntityScanner.java:1515)
  	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanCDATASection(XMLDocumentFragmentScannerImpl.java:1654)
  	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3014)
  	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
  	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
  	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
  	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
  	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
  	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
  	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
  	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327)
  	at javax.xml.parsers.SAXParser.parse(SAXParser.java:328)
  	at org.netbeans.gradle.project.java.test.TestXmlDisplayer.displayTestSuite(TestXmlDisplayer.java:131)
  	at org.netbeans.gradle.project.java.test.TestXmlDisplayer.displayTestSession(TestXmlDisplayer.java:162)
  	at org.netbeans.gradle.project.java.test.TestXmlDisplayer.displayReport(TestXmlDisplayer.java:179)
  	at org.netbeans.gradle.project.java.test.TestXmlDisplayer.displayReport(TestXmlDisplayer.java:196)
  	at org.netbeans.gradle.project.java.tasks.GradleJavaBuiltInCommands.displayTestReports(GradleJavaBuiltInCommands.java:504)
  	at org.netbeans.gradle.project.java.tasks.GradleJavaBuiltInCommands.lambda$displayTestResults$6(GradleJavaBuiltInCommands.java:481)
  	at org.netbeans.gradle.project.java.tasks.GradleJavaBuiltInCommands$$Lambda$481/1206921533.onComplete(Unknown Source)
  	at org.netbeans.gradle.project.tasks.AsyncGradleTask.doGradleTasksWithProgressIgnoreTaskDefCancel(AsyncGradleTask.java:612)
  	at org.netbeans.gradle.project.tasks.AsyncGradleTask.doGradleTasksWithProgressIgnoreTaskDefCancel(AsyncGradleTask.java:473)
  	at org.netbeans.gradle.project.tasks.AsyncGradleTask.doGradleTasksWithProgress(AsyncGradleTask.java:464)
  	at org.netbeans.gradle.project.tasks.AsyncGradleTask.access$300(AsyncGradleTask.java:99)
  	at org.netbeans.gradle.project.tasks.AsyncGradleTask$BuildExecutionItem.lambda$new$0(AsyncGradleTask.java:886)
  	at org.netbeans.gradle.project.tasks.AsyncGradleTask$BuildExecutionItem$$Lambda$482/727654745.run(Unknown Source)
  	at org.netbeans.gradle.project.tasks.GradleDaemonManager.runNonBlockingGradleTask(GradleDaemonManager.java:32)
  	at org.netbeans.gradle.project.tasks.GradleDaemonManager.lambda$submitGradleTask$1(GradleDaemonManager.java:117)
  	at org.netbeans.gradle.project.tasks.GradleDaemonManager$$Lambda$214/818562924.execute(Unknown Source)
  	at org.jtrim2.executor.TaskExecutor.lambda$execute$0(TaskExecutor.java:94)
  	at org.jtrim2.executor.TaskExecutor$$Lambda$44/1972093789.execute(Unknown Source)
  	at org.jtrim2.executor.CancelableTasks.complete(CancelableTasks.java:133)
  

[kelemen]

I'm almost certain this is because the plugin tries to read the stdout produced by the tests and it is too big. Would it be all right, if I put a global limit on the maximum length of stdout (and stderr) read?

[Charlweed]

I think that sounds OK, because of course, I don't really want to read all the exceptions. It is possible to display a warning if the plugin knows that it has truncated stdout?
And it would also be nice if there was a set-able property for the size of whatever heap is being exhausted.

[kelemen]

It is possible but I would prefer to have the first line start with something like [stdout/stderr was truncated]. And of course, I would like to make the size configurable (I just hate messing with the UI and worrying if resizing and whatnot didn't get broken).