You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've noticed some pretty nasty performance issues related to the use of the Text::Hyphen package. When passed in really long words it can take 5+ minutes to 'visualise'. Given there is no memoization of the Text::Hyphen dictionary (and it's caches) then calling each of the various methods that use your syllable_count method can end up taking hours to calculate what is otherwise a pretty simple piece of text (albeit with a few errant 'long' words).
Suggested solution would be to memoize the dictionary. Given that each of the methods in the TextStat class are class methods it does complicate things a little. An option would be to move each of the the class methods into instance methods then create wrappers on the class to create an instance (to keep backward compatibility).
ie
def self.sentence_count(text)
new.sentence_count text
end
def sentence_count(text)
...
end
Of course could also use some meta programming to simplify the various class method calls too.
This would allow the dictionary to be memoized and minimise its cache being flushed, reducing runtime from hours to minutes.
Happy to put together a PR
The text was updated successfully, but these errors were encountered:
I've noticed some pretty nasty performance issues related to the use of the Text::Hyphen package. When passed in really long words it can take 5+ minutes to 'visualise'. Given there is no memoization of the Text::Hyphen dictionary (and it's caches) then calling each of the various methods that use your
syllable_count
method can end up taking hours to calculate what is otherwise a pretty simple piece of text (albeit with a few errant 'long' words).Suggested solution would be to memoize the dictionary. Given that each of the methods in the
TextStat
class are class methods it does complicate things a little. An option would be to move each of the the class methods into instance methods then create wrappers on the class to create an instance (to keep backward compatibility).ie
Of course could also use some meta programming to simplify the various class method calls too.
This would allow the dictionary to be memoized and minimise its cache being flushed, reducing runtime from hours to minutes.
Happy to put together a PR
The text was updated successfully, but these errors were encountered: