Skip to content

LINE and TIR detection failed while running EDTA on a tapeworm genome (1.20Gb), but the pipeline finished #619

Description

@hliu45749-ops

Hello developers, I'm sorry to bother you. I'm encountering issues when running EDTA on a tapeworm genome and would greatly appreciate your help.

Description:
I'm using EDTA v2.2.0 to annotate transposable elements in a tapeworm genome (1.20Gb) to generate a softmasked genome for downstream gene annotation. The pipeline completed but with critical failures in LINE and TIR detection steps.

Environment:

  • EDTA version: 2.2.0 (via Docker container)
  • Image: quay.io/biocontainers/edta:2.2.0--hdfd78af_1
  • Command: EDTA.pl --genome genome.fa --anno 1 --evaluate 1 --sensitive 1 --force 1 -t 64
  • Genome size: 1.20Gb, FASTA format
  • Species: Tapeworm (platyhelminth)

Key Errors Observed:

  1. LINE detection failed:
    cp: cannot stat 'genome.fa.mod.RM2.raw.fa': No such file or directory
    Warning: The LINE result file has 0 bp!
    
  2. TIR detection failed with TensorFlow/Keras error:
    AttributeError: as_proto
    
    This appears to be a model loading issue in TIR-Learner component. Multiple downstream file errors followed.

Run Timeline:

  • LTR detection: Completed successfully (~6.5 hours)
  • SINE detection: Completed successfully (~6.5 hours)
  • LINE detection: Failed after 3 days with file not found error
  • TIR detection: Failed with TensorFlow/Keras error
  • Helitron detection: Completed successfully (~8.5 hours)
  • Final processing: Completed despite earlier failures

Generated Output Files:

  • genome.fa.mod.EDTA.TElib.fa (final TE library, 2.5MB)
  • genome.fa.mod.EDTA.TEanno.gff3 (whole-genome TE annotation, 306MB)
  • genome.fa.mod.MAKER.masked (masked file, 1.4GB, showing 18.42% masked)

Concerns and Questions:

  1. Data quality concerns:
    • The final TE annotation shows 12875.90% coverage (clearly a display issue)
    • Only 18.42% masking seems low for a eukaryotic genome
    • Missing LINE and TIR elements could significantly affect completeness
  2. Technical questions:
    • Why did the pipeline complete despite these critical failures?
    • Is the current genome.fa.mod.MAKER.masked file reliable for gene annotation?
    • How can we fix the LINE detection issue (missing RM2.raw.fa)?
    • How to resolve the TensorFlow/Keras as_proto error in TIR-Learner?
  3. Potential solutions needed:
    • Are there workarounds for these specific errors?
    • Should I try a different EDTA version or installation method?
    • Are there parameter adjustments recommended for low-complexity genomes like tapeworms?
    • Is there a way to run LINE and TIR detection separately to debug?

Additional Context:

  • Complete run log is attached
  • I generated a separate genome.fa.softmasked file using other methods but want to use EDTA's proper softmasking
  • The goal is accurate TE masking for gene annotation pipelines like BRAKER/Maker

Specific Command and Output Context:

docker run -it --rm -v /home/EDTA:/in -w /in quay.io/biocontainers/edta:2.2.0--hdfd78af_1 /bin/bash 
EDTA.pl --genome genome.fa --anno 1 --evaluate 1 --sensitive 1 --force 1 -t 64
[Full output provided in the attached log]

Main Request:
I would be very grateful for your expert guidance on properly running EDTA to achieve complete transposable element annotation and reliable softmasking for my tapeworm genome. Specifically:

  1. How to fix the LINE and TIR detection failures
  2. Whether the current output is usable or should be rerun after fixes
  3. Best practices for tapeworm/low-complexity genomes with EDTA

This is the specific operation process:
docker run -it --rm -v /home/EDTA:/in -w /in quay.io/biocontainers/edta:2.2.0--hdfd78af_1 /bin/bash
root@189c04abca30:/in# EDTA.pl --genome genome.fa --anno 1 --evaluate 1 --sensitive 1 --force 1 -t 64

#########################################################

Extensive de-novo TE Annotator (EDTA) v2.2.0
Shujun Ou (shujun.ou.1@gmail.com)

#########################################################

Parameters: --genome genome.fa --anno 1 --evaluate 1 --sensitive 1 --force 1 -t 64

Wed Jan 14 09:25:44 UTC 2026 Dependency checking:
All passed!

Wed Jan 14 09:28:01 UTC 2026 Obtain raw TE libraries using various structure-based programs:
Wed Jan 14 09:28:01 UTC 2026 EDTA_raw: Check dependencies, prepare working directories.

Wed Jan 14 09:29:08 UTC 2026 Start to find LTR candidates.

Wed Jan 14 09:29:08 UTC 2026 Identify LTR retrotransposon candidates from scratch.

Wed Jan 14 15:48:43 UTC 2026 Finish finding LTR candidates.

Wed Jan 14 15:48:43 UTC 2026 Start to find SINE candidates.

Wed Jan 14 22:20:04 UTC 2026 Finish finding SINE candidates.

Wed Jan 14 22:20:04 UTC 2026 Start to find LINE candidates.

Wed Jan 14 22:20:04 UTC 2026 Identify LINE retrotransposon candidates from scratch.

cp: cannot stat 'genome.fa.mod.RM2.raw.fa': No such file or directory
Sat Jan 17 13:06:24 UTC 2026 Warning: The LINE result file has 0 bp!

Sat Jan 17 13:06:24 UTC 2026 Start to find TIR candidates.

Sat Jan 17 13:06:24 UTC 2026 Identify TIR candidates from scratch.

Species: others
/usr/local/lib/python3.10/site-packages/dask/dataframe/_pyarrow_compat.py:17: FutureWarning: Minimal version of pyarrow will soon be increased to 14.0.1. You are using 13.0.0. Please consider upgrading.
warnings.warn(
find: ./TIR-Learner-+-TIRvish.gff3: No such file or directory
Traceback (most recent call last):
File "/usr/local/share/EDTA/bin/TIR-Learner3.0/TIR-Learner3.0.py", line 80, in
TIRLearner_instance = TIRLearner(genome_file, genome_name, species, TIR_length,
File "/usr/local/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 72, in init
self.execute()
File "/usr/local/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 110, in execute
self.execute_M4()
File "/usr/local/share/EDTA/bin/TIR-Learner3.0/bin/main.py", line 634, in execute_M4
self["base"] = CNN_predict.execute(self)
File "/usr/local/share/EDTA/bin/TIR-Learner3.0/bin/CNN_predict.py", line 108, in execute
df = predict(df, TIRLearner_instance.genome_file_path,
File "/usr/local/share/EDTA/bin/TIR-Learner3.0/bin/CNN_predict.py", line 59, in predict
model = load_model(path_to_model)
File "/usr/local/lib/python3.10/site-packages/keras/src/saving/saving_api.py", line 262, in load_model
return legacy_sm_saving_lib.load_model(
File "/usr/local/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/usr/local/lib/python3.10/site-packages/tensorflow/python/framework/function_def_to_graph.py", line 278, in function_def_to_graph_def
input_shape = input_shape.as_proto()
AttributeError: as_proto
Can't open ./TIR-Learner-Result/TIR-Learner_FinalAnn.fa: No such file or directory at /usr/local/share/EDTA/util/rename_tirlearner.pl line 19.
Warning: LOC list genome.fa.mod.TIR.ext30.list is empty.

Error: Error while loading sequence
Filter sequence based on TEsorter classifications. Unclassified sequences will also be output to the clean file.
Usage: perl cleanup_misclas.pl sequence.fa.rexdb.cls.tsv
Author: Shujun Ou (shujun.ou.1@gmail.com) 10/11/2019

mv: cannot stat 'genome.fa.mod.TIR.ext30.fa.pass.fa.dusted.cln.cln': No such file or directory
cp: cannot stat 'genome.fa.mod.TIR.ext30.fa.pass.fa.dusted.cln.cln.list': No such file or directory
cp: cannot stat 'genome.fa.mod.TIR.intact.raw.fa.anno.list': No such file or directory
Can't open ./TIR-Learner-Result/TIR-Learner_FinalAnn.gff3: No such file or directory.
ERROR: No such file or directory at /usr/local/share/EDTA/util/output_by_list.pl line 39.
Warning: The TIR result file has 0 bp!

Sat Jan 17 21:58:11 UTC 2026 Start to find Helitron candidates.

Sat Jan 17 21:58:11 UTC 2026 Identify Helitron candidates from scratch.

Sun Jan 18 06:25:47 UTC 2026 Finish finding Helitron candidates.

Sun Jan 18 06:25:47 UTC 2026 Execution of EDTA_raw.pl is finished!

Sun Jan 18 06:25:51 UTC 2026 Obtain raw TE libraries finished.
All intact TEs found by EDTA:
genome.fa.mod.EDTA.intact.raw.fa
genome.fa.mod.EDTA.intact.raw.gff3

Sun Jan 18 06:25:51 UTC 2026 Perform EDTA advance filtering for raw TE candidates and generate the stage 1 library:

Sun Jan 18 06:29:00 UTC 2026 EDTA advance filtering finished.

Sun Jan 18 06:29:00 UTC 2026 Perform EDTA final steps to generate a non-redundant comprehensive TE library.

                            Skipping the RepeatModeler results (--sensitive 0).
                            Run EDTA.pl --step final --sensitive 1 if you want to add RepeatModeler results.

                            Skipping the CDS cleaning step (--cds [File]) since no CDS file is provided or it's empty.

Sun Jan 18 06:31:32 UTC 2026 EDTA final stage finished! You may check out:
The final EDTA TE library: genome.fa.mod.EDTA.TElib.fa
Sun Jan 18 06:31:32 UTC 2026 Perform post-EDTA analysis for whole-genome annotation:

Sun Jan 18 06:31:32 UTC 2026 Homology-based annotation of TEs using genome.fa.mod.EDTA.TElib.fa from scratch.

Sun Jan 18 12:07:44 UTC 2026 TE annotation using the EDTA library has finished! Check out:
Whole-genome TE annotation (total TE: 12875.90%): genome.fa.mod.EDTA.TEanno.gff3
Whole-genome TE annotation summary: genome.fa.mod.EDTA.TEanno.sum
Low-threshold TE masking for MAKER gene annotation (masked: 18.42%): genome.fa.mod.MAKER.masked

Sun Jan 18 12:18:56 UTC 2026 Evaluate the level of inconsistency for whole-genome TE annotation:

Sun Jan 18 12:39:11 UTC 2026 Evaluation of TE annotation finished! Check out these files:

                            Overall: genome.fa.mod.EDTA.TE.fa.stat.all.sum
                            Nested: genome.fa.mod.EDTA.TE.fa.stat.nested.sum
                            Non-nested: genome.fa.mod.EDTA.TE.fa.stat.redun.sum

                            If you want to learn more about the formatting and information of these files, please visit:
                                    https://github.com/oushujun/EDTA/wiki/Making-sense-of-EDTA-usage-and-outputs---Q&A

Metadata

Metadata

Assignees

No one assigned

    Labels

    TIR-LearnerTIR-Learner related issues

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions