[Discussion] Claudes "success" Investigation

[coffeebe4code]

I would like to investigate or discuss Claude's ability to have more green cases.

These seem like an outlier, or out of band detection of bullshit.

1. Is there detection at the api level: meaning they catch on to your api keys sooner?

• could we use the 100 api keys through 100 residential proxies?

2. Is there a phrasing or patterns in the questions the models have been specifically trained on? meaning: Anthropic developers could have the first set of questions, and a few of their own, and train the models to give the rejected or pushback responses for those patterns.

3. Any other potential success reasons?

[stilkin-pxl]

If I recall correctly, Anthropic invests more research, time and effort in alignment and safety than their competitors.
This may include trying to harden their models against manipulation and misdirection, which could definitely include similar prompts and situations comparable to the ones used in this benchmark.

However, since this repository has only been up since February 24th of this year, it does not seem plausible (or even possible) for any of the models to have been "benchmaxed" on this specific benchmark...