Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

difference in the confidence values after stop word removal #831

Open
cahuja1992 opened this issue Jul 18, 2019 · 2 comments
Open

difference in the confidence values after stop word removal #831

cahuja1992 opened this issue Jul 18, 2019 · 2 comments

Comments

@cahuja1992
Copy link

Even though the stop word removal is enabled, the confidence of the utterance with and without stop words are different.

Example:
"turn on the wifi" vs "turn on wifi", here the is the stop word.
After looking into the code, I realized that it is actually the confidence value calculation that might be taking the number of tokens into account also.

@adrienball
Copy link
Contributor

@cahuja1992

Even though the stop word removal is enabled

Are you referring to the ignore_stop_words parameter of the LookupIntentParserConfig ?

This parser tries to find an exact match with one of the training samples, and it can do so by ignoring stop words. If it does not find a match, then the probabilistic intent parser is used and this one will not ignore stop words.

In your case, is one of the two formulations ("turn on the wifi" or "turn on wifi") in your training data ?

@cahuja1992
Copy link
Author

cahuja1992 commented Jul 18, 2019

@adrienball
Okay got it. Then what I need is while predicting also, can we remove the stop words from the input utterance?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants