Running SofaRegressionProgram on SOFA /examples directory #94
Description
Using the legacy mode (see PR #93 ) running it on the examples folder of SOFA returns a lot of errors due to state filename missmatches.
To reproduce (branch needed: https://github.com/sofa-framework/Regression/tree/dev_regression)
export SOFA_ROOT=/c/projects/sofa-build/
export REGRESSION_DIR=/c/projects/sofa-plugins/Regression/
python /c/projects/sofa-plugins/Regression/SofaRegressionProgram.py --input examples/ --legacy-regression
Full log for now:
Compare all scenes from: examples/RegressionStateScenes.regression-tests: 100%|████████████████████████████████████████████████████| 67/67 [01:18<00:00, 1.17s/it]
Compare all scenes from: examples/RegressionTopologyScenes.regression-tests: : 0it [00:00, ?it/s] | 1/2 [01:18<01:18, 78.43s/it]
Compare All sets: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [01:18<00:00, 39.21s/it]
### Number of sets Done: 2
### Number of scenes Done: 67
### Number of scenes failed: 47
### Failed: examples\Demos/TriangleSurfaceCutting.scn
### Total Error per MechanicalObject: [0.001411652895172418]
### Error by Dofs: [3.4497871338524387e-07]
### Failed: No frames were tested for examples\Demos/chainAll.scn
### Failed: No frames were tested for examples\Demos/chainHybrid.scn
### Failed: No frames were tested for examples\Demos/chainHybridNoGroup.scn
### Failed: examples\Demos/collisionMultiple.scn
### Total Error per MechanicalObject: [0.000372312758441868, 0.0015472831183819295, 0.0007623599311863419, 0.0028865856526393606, 0.0007250478446112322, 0.00203
992118274441]
### Error by Dofs: [2.068404213565933e-06, 6.447012993258039e-07, 2.8235553006901553e-06, 8.08567409702902e-07, 2.685362387449008e-06, 1.1564179040501191e-06]
### Success: examples\Demos/liver.scn | Number of key frames compared without error: 2
### Success: examples\Demos/simpleBoundaryConditions.scn | Number of key frames compared without error: 2
### Failed: examples\Demos/fallingBeamLagrangianCollision.scn
### Total Error per MechanicalObject: [1.2660520042993556e-05, 1.2660520042993556e-05]
### Error by Dofs: [6.75227735626323e-09, 6.75227735626323e-09]
### Failed: examples\Demos/fallingSOFA.scn
### Total Error per MechanicalObject: [8.864889117006331e-06, 8.864889117006331e-06]
### Error by Dofs: [1.0944307551859667e-08, 1.0944307551859667e-08]
### Failed: examples\Demos/SofaScene.scn
### Total Error per MechanicalObject: [0.0013379029059500456, 9.415874492864297e-05, 3.326468363146977e-06, 7.550511054434961e-06, 5.0259805529949355e-06, 0.000
9196014317737517, 0.0009196014317737517, 3.767619750694152e-06, 0.00032337211637917934, 0.0012090819685236594, 2.403695150876618e-05, 6.397953791482483e-05, 0.00116
85661150702335, 0.0006839340947877948]
### Error by Dofs: [5.498984405877704e-07, 6.2772496619095315e-06, 1.108822787715659e-06, 2.1572888726957033e-07, 8.376634254991559e-07, 7.26383437420025e-07, 7
.26383437420025e-07, 1.255873250231384e-06, 3.3684595456164516e-06, 8.292743268337855e-07, 3.4338502155380254e-06, 4.2653025276549886e-06, 9.431526352463548e-07, 1.
6520147217096492e-06]
### Failed: No frames were tested for examples\Component/Collision/Response/RuleBasedContactManager.scn
### Failed: No frames were tested for examples\Component/Collision/Response/FrictionContact.scn
### Failed: No frames were tested for examples\Component/Constraint/Lagrangian/BilateralLagrangianConstraint_NNCG.scn
### Failed: No frames were tested for examples\Component/Constraint/Lagrangian/BilateralLagrangianConstraint_PGS.scn
### Failed: No frames were tested for examples\Component/Constraint/Lagrangian/BilateralLagrangianConstraint_UGS.scn
### Failed: No frames were tested for examples\Component/Constraint/Lagrangian/FrictionContact_LCP_with_friction.scn
### Failed: No frames were tested for examples\Component/Constraint/Lagrangian/FrictionContact_LCP_without_friction.scn
### Failed: No frames were tested for examples\Component/Constraint/Projective/FixedProjectiveConstraint.scn
### Failed: No frames were tested for examples\Component/Constraint/Projective/LinearVelocityProjectiveConstraint.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Direct/FEMBAR_EigenSimplicialLDLT.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Direct/FEMBAR_EigenSimplicialLLT.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Direct/FEMBAR_EigenSparseLU.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Direct/FEMBAR_EigenSparseQR.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Direct/FEMBAR_PrecomputedLinearSolver.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Direct/FEMBAR_SparseLDLSolver.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Iterative/FEMBAR_MinResLinearSolver.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Iterative/FEMBAR_CGLinearSolver.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Iterative/FEMBAR_CGLinearSolver_assembled.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Preconditioner/FEMBAR_PCG_BlockJacobiPreconditioner.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Preconditioner/FEMBAR_PCG_JacobiPreconditioner.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Preconditioner/FEMBAR_PCG_NoPreconditioner.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Preconditioner/FEMBAR_PCG_PrecomputedWarpPreconditioner.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Preconditioner/FEMBAR_PCG_SSORPreconditioner.scn
### Failed: No frames were tested for examples\Component/LinearSolver/Preconditioner/FEMBAR_PCG_WarpPreconditioner.scn
### Failed: No frames were tested for examples\Component/Mapping/NonLinear/DistanceMapping.scn
### Failed: No frames were tested for examples\Component/Mapping/NonLinear/DistanceMultiMapping.scn
### Failed: No frames were tested for examples\Component/Mapping/NonLinear/SquareDistanceMapping.scn
### Failed: No frames were tested for examples\Component/Mass/UniformMass.scn
### Failed: No frames were tested for examples\Component/Mass/DiagonalMass.scn
### Failed: No frames were tested for examples\Component/Mass/MeshMatrixMass.scn
### Failed: examples\Component/MechanicalLoad/InteractionEllipsoidForceField.scn
### Total Error per MechanicalObject: [2.0319024492720644e-07, 0.0005856154197453525]
### Error by Dofs: [2.9027177846743777e-08, 4.880128497877937e-07]
### Failed: examples\Component/ODESolver/Forward/EulerExplicitSolver.scn
### Total Error per MechanicalObject: [0.003902076824270398]
### Error by Dofs: [7.390297015663633e-06]
### Success: examples\Component/ODESolver/Forward/EulerExplicitSolver_diagonal.scn | Number of key frames compared without error: 2
### Failed: examples\Component/ODESolver/Forward/RungeKutta4Solver.scn
### Total Error per MechanicalObject: [0.0009849005736630844]
### Error by Dofs: [5.113710143629722e-07]
### Failed: No frames were tested for examples\Component/ODESolver/Backward/EulerImplicitSolver.scn
### Failed: No frames were tested for examples\Component/SolidMechanics/FEM/BeamFEMForceField.scn
### Failed: No frames were tested for examples\Component/SolidMechanics/FEM/HexahedronFEMForceField.scn
### Failed: examples\Component/SolidMechanics/FEM/HexahedronFEMForceFieldAndMass.scn
### Total Error per MechanicalObject: [0.000358375358865198]
### Error by Dofs: [3.7330766548458125e-07]
### Failed: examples\Component/SolidMechanics/FEM/QuadBendingFEMForceField.scn
### Total Error per MechanicalObject: [0.0012572259442622173]
### Error by Dofs: [1.3431901113912576e-06]
### Failed: No frames were tested for examples\Component/SolidMechanics/FEM/TetrahedralCorotationalFEMForceField.scn
### Failed: No frames were tested for examples\Component/SolidMechanics/FEM/TetrahedronFEMForceField.scn
### Failed: No frames were tested for examples\Component/SolidMechanics/FEM/TetrahedronFEMForceField_assemble.scn
### Failed: No frames were tested for examples\Component/SolidMechanics/FEM/TetrahedronFEMForceField_plasticity.scn
### Failed: No frames were tested for examples\Component/SolidMechanics/FEM/TetrahedronHyperelasticityFEMForceField.scn
### Failed: examples\Component/SolidMechanics/FEM/TriangularFEMForceFieldOptim.scn
### Total Error per MechanicalObject: [0.003295310990097832]
### Error by Dofs: [1.419169246381495e-06]
### Success: examples\Component/SolidMechanics/Spring/FastTriangularBendingSprings.scn | Number of key frames compared without error: 2
### Failed: No frames were tested for examples\Component/SolidMechanics/Spring/MeshSpringForceField.scn
### Failed: examples\Component/Topology/Mapping/Mesh2PointTopologicalMapping.scn
### Total Error per MechanicalObject: [0.0, 0.00031219228753090884]
### Error by Dofs: [0.0, 9.503570396679112e-08]
### Failed: No frames were tested for examples\Component/Topology/Container/Grid/RegularGridTopology.scn
### Failed: No frames were tested for examples\Component/Topology/Container/Grid/SparseGridTopology.scn
### Failed: No frames were tested for examples\Component/Topology/Container/Grid/SparseGridRamificationTopology.scn
### Failed: examples\Benchmark/Performance/MatrixAssembly/MatrixAssembly_assembledCG.scn
### Total Error per MechanicalObject: [0.0009846309091148771]
### Error by Dofs: [5.112310016172778e-07]
### Failed: examples\Benchmark/Performance/MatrixAssembly/MatrixAssembly_assembledCG_blocs.scn
### Total Error per MechanicalObject: [0.000984630910775951]
### Error by Dofs: [5.112310024797253e-07]
### Failed: examples\Benchmark/Performance/MatrixAssembly/MatrixAssembly_direct.scn
### Total Error per MechanicalObject: [0.0009905241571585928]
### Error by Dofs: [5.14290839646206e-07]
### Failed: examples\Benchmark/Performance/MatrixAssembly/MatrixAssembly_direct_blocs.scn
### Total Error per MechanicalObject: [0.0009905241571576148]
### Error by Dofs: [5.142908396456982e-07]
### Failed: examples\Benchmark/Performance/MatrixAssembly/MatrixAssembly_matrixfreeCG.scn
### Total Error per MechanicalObject: [0.000984630908577535]
### Error by Dofs: [5.11231001338284e-07]
### Success: examples\Benchmark/Analysis/Pendulum.scn | Number of key frames compared without error: 2