Adding constrained sensing capabilities to the SPSL post processor. Added a test_SPSL_constrained xml for testing. #2346

niharika2999 · 2024-07-29T19:39:35Z

Pull Request Description

What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)

Closes #2347

What are the significant changes in functionality due to this change request?

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

1. Review all computer code.
2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
8. If an analytic test is changed/added is the the analytic documentation updated/added?
9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

joshua-cogliati-inl · 2024-07-29T21:18:50Z

I believe this uses features in sparsesensing that are not yet in a released version, so besides any other work that needs to be done, this should not be merged until the sparsesensing release comes out.

…9/raven into SPSL_add_constraints Need for testing

…whitespace changes to Sparsesensing.py

moosebuild · 2024-08-01T16:14:28Z

Job Test mac on d2dc13b : invalidated by @joshua-cogliati-inl

failed in fetch

joshua-cogliati-inl · 2024-08-01T16:15:08Z

Job Test mac on d2dc13b : invalidated by @joshua-cogliati-inl

failed in fetch

Opps, actually failed in set python environment.

joshua-cogliati-inl

Here are some comments for your consideration.

joshua-cogliati-inl · 2024-08-01T21:14:45Z

doc/workshop/SparseSensing/exercises/testSPSLOptiTwist.xml

      <Output class="DataObjects" type="DataSet">outPP</Output>
    </PostProcess>
    <IOStep name="print">
-      <Input class="DataObjects" type="DataSet">outPP</Input>
+      <Input class="DataObjects" type="DataSet">outPP</Input>cd 


Is cd a typo?

Yes! Thank you for catching that. I have removed cd.

joshua-cogliati-inl · 2024-08-01T21:17:19Z

ravenframework/Models/PostProcessors/SparseSensing.py

+    goal.addSub(xValue)
+    yValue = InputData.parameterInputFactory("yValue", contentType=InputTypes.StringListType,
+                                                printPriority=108,
+                                                descr=r"""Variable plotted on the Y-axis of the data model""")  ##Do we want another Z-value


Hm, do we want a zValue?

Yes! In case we have 3D data! Thank you for catching that!

joshua-cogliati-inl · 2024-08-01T21:20:53Z

ravenframework/Models/PostProcessors/SparseSensing.py

+                                                           printPriority=108,
+                                                           descr=r"""X coordinate of the focus of the parabola""")
+    goal.addSub(a)
+    xy_coords = InputData.parameterInputFactory("xy_coords", contentType=InputTypes.IntegerListType,   ##What is the type for an array?


Hm, some of the comments like "What is the type for an array?" look like they may be out of date.

joshua-cogliati-inl · 2024-08-01T21:34:54Z

ravenframework/Models/PostProcessors/SparseSensing.py

    self.sampleTag = 'RAVEN_sample_ID'                       # The sample tag
+    self.center_x = None                                     # The x co-ordinate of center of circle,ellipse


Hm, it looks like this only supports one Constrained Region? I think PySensors supports multiple constrained regions?

joshua-cogliati-inl · 2024-08-01T21:35:39Z

ravenframework/Models/PostProcessors/SparseSensing.py

+      ## TODO: Add GQR for constrained optimization
+      elif self.optimizer.lower() == 'gqr':
+        optimizer = ps.optimizers.GQR()
+      ## TODO: Add CCQR for cost constrained optimization


Do any of these TODO's need to be fixed?

joshua-cogliati-inl · 2024-08-01T21:45:55Z

tests/framework/PostProcessors/SparseSensing/testSPSLOptiTwistConstraints.xml

@@ -0,0 +1,136 @@
+<?xml version="1.0" ?>


This file does not seem to be used in the tests yet.

It looks like new tests have been added, so maybe this file can be removed?

…'cd' in the workshop test

Jimmy-INL · 2024-08-04T00:37:54Z

tests/framework/PostProcessors/SparseSensing/testSPSLOptiTwistConstraints.xml

+        <constraintOption>"exact_n"</constraintOption>
+        <ConstrainedRegions type = "circle"> 
+          <center_x></center_x>  #center coordinates instead (x,y) (r, \theta)
+          <center_y></center_y>   


Comments in xml are done using 
center_x and center_y seems not to be recognized

Jimmy-INL · 2024-08-04T00:39:37Z

tests/framework/PostProcessors/SparseSensing/tests

+    image = 'classificationOptiTwist/mySensorPlot_scatter-scatter.png classificationOptiTwist/mySensorPlot3D_scatter.png'
+    required_libraries = 'imageio'
+    rel_err = 0.01
+  [../]


This will only review the image with a relative error of 0.01 which is too small for image comparison. We need to test the csv and the image. rel_err for image can be 0.1

joshua-cogliati-inl · 2024-08-30T14:08:45Z

...s/framework/PostProcessors/SparseSensing/reconstructionConstraintsPolygonOptiTwist/outPP.csv

Should this be a gold file? (Also for the other outPP.csv files)

joshua-cogliati-inl · 2024-09-11T19:26:30Z

ravenframework/Models/PostProcessors/SparseSensing.py

@@ -52,10 +54,26 @@ class cls.
                                                printPriority=108,
                                                descr=r"""Features/inputs of the data model""")
    goal.addSub(features)
-    target = InputData.parameterInputFactory("target", contentType=InputTypes.StringType,
+    xValue = InputData.parameterInputFactory("xValue", contentType=InputTypes.StringListType,


Note that changing the input usually requires an automated script be created and placed in scripts/conversionScripts/

joshua-cogliati-inl · 2024-09-11T19:31:29Z

ravenframework/Models/PostProcessors/SparseSensing.py

+                                                printPriority=108,
+                                                descr=r"""Variable plotted on the Y-axis of the data model""")
+    goal.addSub(yValue)
+    zValue = InputData.parameterInputFactory("yValue", contentType=InputTypes.StringListType,


Why is this "zValue" = ... "yValue"?

joshua-cogliati-inl · 2024-09-11T21:29:40Z

ravenframework/Models/PostProcessors/SparseSensing.py

+          idxConstrained, rank = parabola.get_constraint_indices(all_sensors = allSensors, info=dataframe)
+        elif self._ConstrainedRegionsType == 'Polygon':
+          xyTuple = [(self.xCoords[i], self.yCoords[i]) for i in range(0, len(self.xCoords))]
+          print(xyTuple)


print probably should be commented out, or switched to a self.raiseADebug

moosebuild · 2024-09-16T22:49:20Z

Job Test Ubuntu 18 PIP on c06a2bb : invalidated by @joshua-cogliati-inl

Cleaned up files on test machine.

…9/raven into SPSL_add_constraints

moosebuild · 2024-09-16T23:47:32Z

Job Mingw Test on 254895a : invalidated by @niharika2999

moosebuild · 2024-09-17T00:03:15Z

Job Mingw Test on 254895a : invalidated by @niharika2999

…straintsCircleOptiTwist/.DS_Store Deleting unwanted files

…straintsCircleOptiTwist/mySensorPlot_scatter-scatter.png Deleting unwanted files

…straintsCircleOptiTwist/mySensorPlot3D_scatter.png Deleting unwanted files

…straintsCircleOptiTwist/outPP.csv Deleting unwanted files

…straintsCircleOptiTwist/.ravenStatus Deleting unwanted files

…straintsEllipseOptiTwist/.ravenStatus Deleting unwanted files

…straintsEllipseOptiTwist/mySensorPlot3D_scatter.png Deleting unwanted files

…straintsEllipseOptiTwist/mySensorPlot_scatter-scatter.png Deleting unwanted files

…straintsEllipseOptiTwist/outPP.csv Deleting unwanted files

Jimmy-INL · 2024-09-17T05:22:45Z

tests/framework/PostProcessors/SparseSensing/testSPSLOptiTwistConstraints.xml

+
+  <RunInfo>
+    <WorkingDir>reconstructionConstraintsOptiTwist</WorkingDir>
+    <Sequence>LoadCSV, mySPpp, print</Sequence>


@niharika2999 , the reason tests are failing is that the path is inconsistent. For instance, the working directory here is reconstructionConstraintsOptiTwistCircle but in tests it is reconstructionOptiTwistConstraintsCircle. Hope this helps

…9/raven into SPSL_add_constraints

…straintsCircleOptiTwist directory Deleting wrongly named folders

…straintsEllipseOptiTwist directory Deleting wrongly named folders

…straintsLineOptiTwist directory Deleting wrongly named folders

…straintsParabolaOptiTwist directory Deleting wrongly named folders

…straintsPolygonOptiTwist directory Deleting wrongly named folders

moosebuild · 2024-09-17T18:13:50Z

Job Mingw Test on e3b6a09 : invalidated by @niharika2999

Jimmy-INL and others added 7 commits November 5, 2023 10:06

starting classification

5055f61

adding classification data

32f6a5c

hard coding the labels for now till we decide on naming

8a1b5d5

adding the state variable and remove the hard coding

66b4adf

adding test and golded files

226b5a8

Adding modified SPSL with constraints

53c72e3

Adding test file for constraints SPSL

3c19e83

joshua-cogliati-inl added the Do Not Merge label Jul 29, 2024

niharika2999 requested a review from Jimmy-INL July 29, 2024 21:23

Niharika Karnik and others added 11 commits July 29, 2024 16:00

Merge branch 'SPSL_add_constraints' of https://github.com/niharika299…

9c435e0

…9/raven into SPSL_add_constraints Need for testing

Adding path the pysensors branch for constraints in dependencies and …

a092359

…whitespace changes to Sparsesensing.py

deleting trailing whitespaces in the file SparseSensing.py

047fdf1

Removing git+ from dependencies as the library handler adds git+

357a16a

Fixing constrainedRegion being None error

bd72312

Changing variable name target to measuredState in the workshop SPSL test

4dbfa69

Fixing constrainedRegions Typo

4a71208

Fixing missing else statement for ConstrainedRegions

5a55ae4

Fixing error by including _ConstrainedRegionsType= None

207b0cf

Adding the test from workshop with target changed to measuredState

b59e4d0

Changing target to measuredState in the workshop inputs folder

d2dc13b

joshua-cogliati-inl reviewed Aug 1, 2024

View reviewed changes

niharika2999 added 3 commits August 2, 2024 11:15

Updating documentation for the sparse sensing postprocessor

a288140

Fixing \_n issues for max_n and exact_n in the documentation

fb43bec

Adding \_ to the documentation inside xml node and removing the typo …

26284ee

…'cd' in the workshop test

Jimmy-INL reviewed Aug 4, 2024

View reviewed changes

Changing xy_coords in the documentation

3f7b970

joshua-cogliati-inl reviewed Aug 30, 2024

View reviewed changes

joshua-cogliati-inl reviewed Sep 11, 2024

View reviewed changes

Niharika Karnik and others added 2 commits September 16, 2024 14:37

adding circle, polygon, ellipse to gold in tests

8155b90

Merge branch 'devel' into SPSL_add_constraints

c06a2bb

Niharika Karnik added 2 commits September 16, 2024 16:41

Trying to fix errors for circle and ellipse constraints tests

9eaf38b

Merge branch 'SPSL_add_constraints' of https://github.com/niharika299…

254895a

…9/raven into SPSL_add_constraints

niharika2999 added 9 commits September 16, 2024 17:23

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

bf98b39

…straintsCircleOptiTwist/.DS_Store Deleting unwanted files

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

03428a5

…straintsCircleOptiTwist/mySensorPlot_scatter-scatter.png Deleting unwanted files

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

4975ad8

…straintsCircleOptiTwist/mySensorPlot3D_scatter.png Deleting unwanted files

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

7e12525

…straintsCircleOptiTwist/outPP.csv Deleting unwanted files

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

ad67ac2

…straintsCircleOptiTwist/.ravenStatus Deleting unwanted files

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

995c889

…straintsEllipseOptiTwist/.ravenStatus Deleting unwanted files

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

f143170

…straintsEllipseOptiTwist/mySensorPlot3D_scatter.png Deleting unwanted files

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

d66eae9

…straintsEllipseOptiTwist/mySensorPlot_scatter-scatter.png Deleting unwanted files

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

21d686b

…straintsEllipseOptiTwist/outPP.csv Deleting unwanted files

Jimmy-INL reviewed Sep 17, 2024

View reviewed changes

Niharika Karnik and others added 8 commits September 17, 2024 10:03

Changing working directory for the tests

9e5835e

Merge branch 'SPSL_add_constraints' of https://github.com/niharika299…

0b1b272

…9/raven into SPSL_add_constraints

Renaming folders for tests

307dabe

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

6bcca52

…straintsCircleOptiTwist directory Deleting wrongly named folders

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

a5b9be5

…straintsEllipseOptiTwist directory Deleting wrongly named folders

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

676fd20

…straintsLineOptiTwist directory Deleting wrongly named folders

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

99066b5

…straintsParabolaOptiTwist directory Deleting wrongly named folders

Delete tests/framework/PostProcessors/SparseSensing/reconstructionCon…

e3b6a09

…straintsPolygonOptiTwist directory Deleting wrongly named folders

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding constrained sensing capabilities to the SPSL post processor. Added a test_SPSL_constrained xml for testing. #2346

Adding constrained sensing capabilities to the SPSL post processor. Added a test_SPSL_constrained xml for testing. #2346

niharika2999 commented Jul 29, 2024 •

edited

Loading

joshua-cogliati-inl commented Jul 29, 2024

moosebuild commented Aug 1, 2024

joshua-cogliati-inl commented Aug 1, 2024

joshua-cogliati-inl left a comment

joshua-cogliati-inl Aug 1, 2024

niharika2999 Aug 2, 2024

joshua-cogliati-inl Aug 1, 2024

niharika2999 Aug 1, 2024

joshua-cogliati-inl Aug 1, 2024

joshua-cogliati-inl Aug 1, 2024

joshua-cogliati-inl Aug 1, 2024

joshua-cogliati-inl Aug 1, 2024

joshua-cogliati-inl Aug 30, 2024

Jimmy-INL Aug 4, 2024

Jimmy-INL Aug 4, 2024

joshua-cogliati-inl Aug 30, 2024

joshua-cogliati-inl Sep 11, 2024

joshua-cogliati-inl Sep 11, 2024

joshua-cogliati-inl Sep 11, 2024

moosebuild commented Sep 16, 2024

moosebuild commented Sep 16, 2024

moosebuild commented Sep 17, 2024

Jimmy-INL Sep 17, 2024

moosebuild commented Sep 17, 2024

		self.sampleTag = 'RAVEN_sample_ID' # The sample tag
		self.center_x = None # The x co-ordinate of center of circle,ellipse

Adding constrained sensing capabilities to the SPSL post processor. Added a test_SPSL_constrained xml for testing. #2346

Are you sure you want to change the base?

Adding constrained sensing capabilities to the SPSL post processor. Added a test_SPSL_constrained xml for testing. #2346

Conversation

niharika2999 commented Jul 29, 2024 • edited Loading

Pull Request Description

What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)

What are the significant changes in functionality due to this change request?

For Change Control Board: Change Request Review

joshua-cogliati-inl commented Jul 29, 2024

moosebuild commented Aug 1, 2024

joshua-cogliati-inl commented Aug 1, 2024

joshua-cogliati-inl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moosebuild commented Sep 16, 2024

moosebuild commented Sep 16, 2024

moosebuild commented Sep 17, 2024

Choose a reason for hiding this comment

moosebuild commented Sep 17, 2024

niharika2999 commented Jul 29, 2024 •

edited

Loading