ASK/TELL DEVELOP #1307

jlnav · 2024-05-09T17:27:15Z

codecov · 2024-05-16T20:14:01Z

Codecov Report

❌ Patch coverage is 92.09184% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.95%. Comparing base (ce582d2) to head (6b54991).
⚠️ Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
libensemble/generators.py	85.05%	7 Missing and 6 partials ⚠️
libensemble/gen_classes/aposmm.py	88.40%	1 Missing and 7 partials ⚠️
libensemble/utils/runners.py	93.33%	3 Missing and 3 partials ⚠️
libensemble/utils/misc.py	96.87%	1 Missing and 2 partials ⚠️
libensemble/libE.py	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #1307       +/-   ##
============================================
+ Coverage    78.23%   89.95%   +11.72%     
============================================
  Files           76       79        +3     
  Lines         7561     7915      +354     
  Branches      1116     1176       +60     
============================================
+ Hits          5915     7120     +1205     
+ Misses        1447      615      -832     
+ Partials       199      180       -19

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

shuds13 · 2024-05-21T18:03:50Z

libensemble/tests/unit_tests/test_persistent_aposmm.py

+    }
+
+    my_APOSMM = APOSMM(gen_specs)
+    my_APOSMM.setup()


Does this need to be separate to constructor?

Perhaps you've seen already, but setup() sets attributes that can't be pickled, but need to be done anyway. So I believe a separate generator.setup() needs to exist.

But perhaps for outside-of-libE purposes, to save a line, this could resemble: my_APOSMM = APOSMM(gen_specs, setup=True).

shuds13 · 2024-05-30T18:40:58Z

jlnav · 2024-05-30T20:44:43Z

@jlnav

Currently this has two existing user functions refactored to ask/tell, and it is a breaking change for codes using those gens. I don't think this is what we want. So, for now, we could supply these as duplicates (keeping the original), or remove them. Alternative would be to refactor all appropriate gens to ask/tell, but that would hold this up, so I think it may be better to supply these two as a duplicate for now.

Sounds good. Maybe we could raise DeprecationWarnings?

Also, would it be better for these ask/tell gens (rand sample and gpCAM) to use AskTellGenRunner and not use the wrapper gen_f?

I prefer the AskTellGenRunner myself (I did develop it), but if all the gens move to my runner, then what would you want to do with your wrapper?

shuds13 · 2024-05-30T20:59:20Z

I think there is also some opportunity for inheritence with the gpCAM class. And gp_cam_simple is more complicated than gp_cam_asktell. So, if anything the latter would be the base class, and renamed to avoid confusion. See #1316

shuds13 · 2024-06-01T20:08:29Z

libensemble/libE.py

@@ -441,6 +441,24 @@ def libE_mpi_worker(libE_comm, sim_specs, gen_specs, libE_specs):
 # ==================== Local version ===============================


+def _retrieve_generator(gen_specs):


I don't know why this is needed. If gen is on a thread, should not need to be pickled.

I've removed on my branch for now, so don't change here

jlnav · 2024-06-05T17:49:55Z

Checking random sample when running without Optimas:

>>> import numpy as np
>>> normal = np.load("persistent_aposmm_nlopt_history_length=2005_evals=2000_workers=4.npy")
>>> asktell = np.load("persistent_aposmm_nlopt_asktell_history_length=2002_evals=2000_workers=3.npy")
>>> all([i in asktell["x"][:100] for i in normal["x"][:100]])
True

jlnav · 2024-06-05T17:52:17Z

With Optimas, currently looks like random sample doesn't match, even when trying to account for seed

jlnav · 2024-06-06T16:06:44Z

Precision fixed in the Optimas example: optimas-org/optimas@08a835b

shuds13 · 2024-07-17T21:19:28Z

We may need to update gpCAM gens for latest gpCAM release. Make sure any changes made in gen_f is reflected here before pulling in.

Check gpCAM updates

First experiment with creating an ask/tell gen that creates the agreed-upon data-type

…ble.utils.misc

…y add code-samples

…nner, into subclasses depending on the type of ask/tell gen being interacted with

…r better creation of subsubclasses

…into develop

shuds13 · 2024-07-31T16:11:03Z

libensemble/gen_classes/gpCAM.py

+            self.all_y = np.vstack((self.all_y, self.y_new))
+
+            if self.my_gp is None:
+                self.my_gp = GP(self.all_x, self.all_y, noise_variances=self.noise * np.ones(len(self.all_y)))


I need to update for new gpCAM interface

shuds13 · 2024-07-31T16:11:29Z

libensemble/gen_classes/sampling.py

+    """
+
+    def __init__(self, _, persis_info, gen_specs, libE_info=None) -> list:
+        # self.H = H


Remove I guess.

shuds13 · 2024-07-31T16:21:01Z

libensemble/gen_classes/gpCAM.py

+        self.all_y = np.empty((0, 1))
+        np.random.seed(0)
+
+    def __init__(self, H, persis_info, gen_specs, libE_info=None):


I will put above _initialize_gpcAM

and make _initialize_gpcAM _initialize_gpCAM

…d instead

…vious states, except they use VOCS instead of variables/objectives. Quick fix to the end of the manager routine to make num_gens_started=0; otherwise we run into the *technically already understood and dealt with problem* where the gen doesn't start again upon an additional libE()

…y VOCS

…all of VOCS.variables down to single "x"

…the gen is tested in a regression test?

…m classes to accept VOCS

…ging len(np.unique(H["gen_ended_time"])) != num_batches; it exceeds it

…ers .ask(15) instead of the default 4, re-add the gpcam gen

…'ve imported an external generator

…random seed

Experiments with VOCS in our gen classes

jlnav marked this pull request as ready for review May 16, 2024 20:06

shuds13 reviewed May 21, 2024

View reviewed changes

shuds13 reviewed Jun 1, 2024

View reviewed changes

jlnav and others added 15 commits July 18, 2024 09:22

Merge pull request #1328 from Libensemble/feature/asktell_from_standard

bc1bf6a

First experiment with creating an ask/tell gen that creates the agreed-upon data-type

Merge branch 'develop' into experimental/jlnav_plus_shuds_asktell

55011d7

rename LibensembleGenInterfacer to LibensembleGenThreadInterfacer

3363e4a

move numpy-> list-of-dicts converters out of generators into libensem…

25b4d4d

…ble.utils.misc

small docstrings for current classes in generators.py, should probabl…

1ca123e

…y add code-samples

refactor + more comments throughout AskTellGenRunner

838057b

necessary, but currently unfunctional refactoring of the AskTellGenRu…

2638d33

…nner, into subclasses depending on the type of ask/tell gen being interacted with

replacing __new__ magic in Runner superclass with factory function fo…

c3c19e1

…r better creation of subsubclasses

typo

4d48ab9

fix Runners unit test

9f7d485

Merge branch 'main' into develop

30bf2d2

Merge branch 'develop' of https://github.com/Libensemble/libensemble …

0c3307b

…into develop

Merge branch 'develop' of https://github.com/Libensemble/libensemble …

009f324

…into develop

Merge branch 'develop' into experimental/jlnav_plus_shuds_asktell

68c9b7c

type/bug fixes

06995d0

jlnav requested a review from shuds13 July 30, 2024 22:00

shuds13 reviewed Jul 31, 2024

View reviewed changes

jlnav and others added 30 commits July 16, 2025 16:36

dunno how this validator change made it through

80c09d7

fix isinstance check for gen_f when gen_specs["generator"] is provide…

7ca34b1

…d instead

this spec-checker was lost to time somehow

f6341a4

Merge branch 'develop' into experimental/jlnav_plus_shuds_asktell

fef5833

Merge branch 'experimental/jlnav_plus_shuds_asktell' into asktell/VOCS

673c3eb

various experiments with updating samplers and the test to accept onl…

1ed05e5

…y VOCS

add parameter for UniformSample, instructing the generator to squash …

9b1195b

…all of VOCS.variables down to single "x"

making variables_mapping an explicit kwarg for a libensemble generator

24df60f

bump pydantic - this should've been done in a previous PR

715773b

fix UniformSample LibensembleGenerator to use variables_mapping

08ead4a

just commenting out, but maybe we don't need this unit test now that …

22b69b4

…the gen is tested in a regression test?

removing unnecessary GPCAM_Standard class; refactoring the other gpca…

f664e37

…m classes to accept VOCS

update regression test to (presumably) use classes that now use VOCS

99bc450

... why was this test excluded?

423f0d6

lists instead of tuples...

56e59aa

fix default value of test_points_file kwarg to be None

145e09d

somehow this tell_numpy got missed

9a6f299

various fixes to building variables_mapping and extra.ci, still debug…

9c0e258

…ging len(np.unique(H["gen_ended_time"])) != num_batches; it exceeds it

temporary debugging print?

4a52e0b

fix objective name

73bbf69

batch_size fix to runners.py, add batch_size to gen_specs so the runn…

bf0d79e

…ers .ask(15) instead of the default 4, re-add the gpcam gen

we still want to map {"f": ["energy"]), obviously

b3ce513

move StandardSample to test_asktell_sampling.py, to test as-though we…

2428152

…'ve imported an external generator

various fixes from PR suggestions

0bcd7c9

move gpcam superclass init to top of init, make UniformSample accept …

bf9ed05

…random seed

gpcam asserts objectives, then a better description of variables_mapping

acc4811

move _validate_vocs call to superclass

2a67724

Merge pull request #1572 from Libensemble/asktell/VOCS

8c9e313

Experiments with VOCS in our gen classes

Merge branch 'develop' into experimental/jlnav_plus_shuds_asktell

6b54991

		@@ -441,6 +441,24 @@ def libE_mpi_worker(libE_comm, sim_specs, gen_specs, libE_specs):
		# ==================== Local version ===============================


		def _retrieve_generator(gen_specs):

ASK/TELL DEVELOP #1307

Are you sure you want to change the base?

ASK/TELL DEVELOP #1307

Uh oh!

Conversation

jlnav commented May 9, 2024 • edited by shuds13 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented May 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

shuds13 May 21, 2024

Choose a reason for hiding this comment

Uh oh!

jlnav May 21, 2024

Choose a reason for hiding this comment

Uh oh!

shuds13 commented May 30, 2024 • edited by jlnav Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jlnav commented May 30, 2024

Uh oh!

shuds13 commented May 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shuds13 Jun 1, 2024

Choose a reason for hiding this comment

Uh oh!

shuds13 Jun 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jlnav commented Jun 5, 2024

Uh oh!

jlnav commented Jun 5, 2024

Uh oh!

jlnav commented Jun 6, 2024

Uh oh!

shuds13 commented Jul 17, 2024

Uh oh!

shuds13 Jul 31, 2024

Choose a reason for hiding this comment

Uh oh!

shuds13 Jul 31, 2024

Choose a reason for hiding this comment

Uh oh!

shuds13 Jul 31, 2024

Choose a reason for hiding this comment

Uh oh!

shuds13 Jul 31, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jlnav commented May 9, 2024 •

edited by shuds13

Loading

codecov bot commented May 16, 2024 •

edited

Loading

shuds13 commented May 30, 2024 •

edited by jlnav

Loading

shuds13 commented May 30, 2024 •

edited

Loading

shuds13 Jun 1, 2024 •

edited

Loading