batch saving genomes by Xiangs18 · Pull Request #212 · kbaseapps/GenomeFileUtil

Xiangs18 · 2024-08-15T00:15:54Z

No description provided.

codecov · 2024-08-16T01:53:52Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.07%. Comparing base (4819598) to head (2eb63c1).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #212      +/-   ##
==========================================
+ Coverage   80.88%   81.07%   +0.18%     
==========================================
  Files          11       11              
  Lines        2998     3017      +19     
==========================================
+ Hits         2425     2446      +21     
+ Misses        573      571       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Xiangs18 · 2024-08-16T01:58:30Z

self note:

a2fabf4 verifies that the refactored code works with the save_one_genome function.
330d6c2 verifies that the refactored code works with the save_genome_mass function.

MrCreosote

Haven't looked at tests yet, but this is already a lot of comments

EDIT: All comments addressed

RELEASE_NOTES.md

lib/GenomeFileUtil/core/GenbankToGenome.py

lib/GenomeFileUtil/core/GenomeInterface.py

…me_mass loop; 3. make the note much more explicit

MrCreosote

Note to self: haven't finished looking at the tests yet, this is enough comments for now and will probably require some re-investigation due to the amount of time this PR has been sitting

test/problematic_tests/save_genome_test.py

lib/GenomeFileUtil/core/GenomeInterface.py

MrCreosote · 2025-06-18T21:43:07Z

lib/GenomeFileUtil/core/GenomeInterface.py

-        if "AnnotatedMetagenomeAssembly" in ws_datatype:
-            if params.get('upgrade') or 'feature_counts' not in data:
-                data = self._update_metagenome(data)


I really wish we had documented why this was ok to delete because I don't remember any more:

#212 (comment)

More discussion here: #212 (comment)

Codedov says there is a test where the type is AMA: https://app.codecov.io/gh/kbaseapps/GenomeFileUtil/blob/master/lib%2FGenomeFileUtil%2Fcore%2FGenomeInterface.py#L142

Sijie - do you remember anything about why we thought this was safe to delete?

Especially since it seems to be exercised by tests per Codecov

Whoops, looks like there were some tests for this that were deleted also, those should be undeleted as well: #212 (comment)

Restored test and updated logic to check all object types: Assembly, Genome, and Metagenome.

Metagenome coverage issue is now fixed

In this case although now the _update_metagenome function call is covered, there's no actual effect from the function call - it's a no op, and so if call were deleted no tests would fail. There needs to be a test that would fail if the function call were removed. A unit test here would be fine - I think there just needs to be a tests that calls save_genomes_mass and save_one_genome with no "molecule_type" field in the genome object

If there's already a test that saves a genome I'd think it'd be pretty easy to add a case for this

lib/GenomeFileUtil/core/GenbankToGenome.py

lib/GenomeFileUtil/core/GenomeInterface.py

…est_genomes_with_upgrade

…ched context

MrCreosote · 2025-08-27T21:46:29Z

test/utility/genome_size_tests.py

+        cls.wsClient.create_workspace({'workspace': cls.wsName})
+        cls.wsID = cls.dfuClient.ws_name_to_id(cls.wsName)


Suggested change

cls.wsClient.create_workspace({'workspace': cls.wsName})

cls.wsID = cls.dfuClient.ws_name_to_id(cls.wsName)

cls.wsID = cls.wsClient.create_workspace({'workspace': cls.wsName})[0]

pretty sure that's the right syntax, but you might need to fix it if not. create-workspace def returns the wsid

add save_genomes function

6208a9a

Xiangs18 requested review from Tianhao-Gu, jkbaumohl and jsfillman as code owners August 15, 2024 00:15

fix positional arg #1 is the wrong type bug

a2fabf4

Xiangs18 added 3 commits August 15, 2024 19:58

use batch genome save in GenbankToGenome.py

330d6c2

make save_genome_mass function internal

27158ec

add tests for save_genome_mass function

04aa203

Xiangs18 changed the title ~~[WIP] add save_genomes~~ add save_genomes Aug 17, 2024

Xiangs18 changed the title ~~add save_genomes~~ batch saving genomes Aug 17, 2024

fix bug

8d672d1

Xiangs18 requested a review from MrCreosote August 17, 2024 06:22

MrCreosote reviewed Aug 19, 2024

View reviewed changes

Xiangs18 added 10 commits August 26, 2024 14:42

update release notes && make the dicts in the loop

d1dd768

remove logging && add NOTE for workspace_datatype

e79c297

move set_up_single_params && validate_mass_params into GenomeUtils

35030a0

remove redundant tests

4b1b88b

add test to cover the missing line

b56d77f

fix params name typo

788c7ee

add metagenome json file && cover the missing line

65467ea

rm gff_handle_ref

66af0b7

add features_handle_ref && protein_handle_ref before upload

fcbb510

add boolean flag for validate_genome

32f96a0

Xiangs18 requested a review from MrCreosote September 10, 2024 23:50

Xiangs18 added 4 commits September 10, 2024 17:10

test validate_genome boolean flag

41eed0b

1. add pydoc for save_genme_mass; 2. make the dicts in the _save_geno…

71670bc

…me_mass loop; 3. make the note much more explicit

update release notes && remove tiny files

0cbe155

add more info and warnings checks in test

52e280d

use g2g_mass for testing

7022f35

MrCreosote reviewed Jun 18, 2025

View reviewed changes

update docString in save_genome_mass

be96213

Xiangs18 requested a review from MrCreosote July 16, 2025 19:19

Xiangs18 added 24 commits July 22, 2025 20:43

link genbank file instead of JSON file

f624b11

check prov diff

7fb8355

update prov

79b44fd

update subactions dict in prov

6f64bbd

revert changes on run_tests_within_container.sh

a3eaec2

simplify check_hidden function

f22ca6a

pass in object_id

19ccfb5

restore deleted AnnotatedMetagenomeAssembly code

c306fc6

1.fix _update_metagenome bug; 2.update prepare_data function; 3.add t…

948e3b1

…est_genomes_with_upgrade

display genome info

b626756

refactor _get_object_type logic

5378631

add missing param

ef95de7

display data info

fafc886

fix missing key bug

e7d0e04

add expected_genome_data

0ae6e9d

pop features_handle_ref and protein_handle_ref

6f0f0f7

add missing is_metagenome param in _check_data

6abafd6

pass in correct expected_data

ab8b829

finish updating metagenome test

f5f6aee

fix params bug

d360fda

refactor current genome_size_tests script and setup for mass saves test

e30e0e1

run all tests

8dc7b37

Uses a flag (g2g_mass) to determine which method to call inside a pat…

1571b43

…ched context

add mass saves tests in genome_size_tests.py script

ed57557

MrCreosote reviewed Aug 27, 2025

View reviewed changes

simplify getting wsID

2eb63c1

		cls.wsClient.create_workspace({'workspace': cls.wsName})
		cls.wsID = cls.dfuClient.ws_name_to_id(cls.wsName)

	cls.wsClient.create_workspace({'workspace': cls.wsName})
	cls.wsID = cls.dfuClient.ws_name_to_id(cls.wsName)
	cls.wsID = cls.wsClient.create_workspace({'workspace': cls.wsName})[0]

Conversation

Xiangs18 commented Aug 15, 2024

Uh oh!

codecov bot commented Aug 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Xiangs18 commented Aug 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MrCreosote left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MrCreosote left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MrCreosote Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Aug 16, 2024 •

edited

Loading

Xiangs18 commented Aug 16, 2024 •

edited

Loading

MrCreosote left a comment •

edited

Loading

MrCreosote left a comment •

edited

Loading

MrCreosote Aug 27, 2025 •

edited

Loading