Slot/Entity match probabilities

[AdolfVonKleist]

Is it possible to retrieve a probability, confidence or even raw score for each filled slot in a given intent response?

Sometimes an utterance will produce multiple candidate resolutions for a given slot/entity within an intent. It would be useful to be able to sometimes differentiate these, especially if the application only expects one resolution for a give slot type.

[adrienball]

Yes, we could indeed add a confidence score along each extracted slot. It is not trivial though, as the probabilities provided by the CRF are very poorly calibrated, but there are probably some things to explore.

However, this won't provide you multiple candidates for each slot but only enrich the extracted slot with additional information.

[AdolfVonKleist]

Right now system will often return more than one candidate filler per slot, in a situation where there should only be candidate. "I want to buy a blue widget under twenty dollars pretty please." Maybe it will identify 'blue widget' and 'pretty please' as candidates for 'PRODUCT'.

Right now I receive nothing in the response except the candidate texts, so I have no way to do any kind of further processing or filtering other than by explicit keyword or similar. Access even to the raw, uncalibrated scores would provide an entry point. Maybe that would also be an easier starting point as well.

It would be equally useful to simply be able to specify that a target should only ever produce a single filler value.