Fix speculative decoding example

[Framartin]

What does this PR do?

Bug fix: ?
Fix #211

Fix several bugs related to the speculative decoding example.

Overview:

• add support to freeze the base model (this was recommended, but not actually implemented, contrary to what the README previously stated) in main.py, launch.sh and README
• fix data format to support the Daring-Anteater dataset
• use --chat flag in the generate_server.py calls
• add system prompt to fastchat prompt
• use client.chat.completions.create() instead client.completions.create() and fix vllm-specific args
• expose --gradient_accumulation_steps
• replace assert to handle the exception
• add --train_bs in README to ease adaptation to multiple GPUs while keeping the effective batch-size constant

Testing

I've run the modified scripts.

Before your PR is "Ready for review"

• Make sure you read and follow Contributor guidelines
• Is this change backward compatible?: Yes
• Did you write any new necessary tests?: No
• Did you add or update any necessary documentation?: Yes
• Did you update Changelog?: No

Additional Information

[Framartin]

@yeyu-nvidia In addition to fixing your comment above, I've added additional commits to fix other issues:

• add support to freeze the base model. This was recommended, but not actually implemented, contrary to what the README previously stated
• support the data format of server_generate.py in medusa_utils.py and eagle_utils.py
• expose --train_bs (to hint how to keep the effective batch-size constant when increasing the number of GPUs)
• expose --gradient_accumulation_steps
• replace an assert to handle the exception
• update the README accordingly

This is good on my side. Please let me know if there are any issue.

[yeyu-nvidia]

This is not needed. We have freeze_base_model in forward and it is default set to True https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/modelopt/torch/speculative/plugins/transformers.py#L102

[yeyu-nvidia]

This is not needed.

[yeyu-nvidia]

Again, this part is not needed.

[Tala-mahhmmoodi]

Hey @Framartin, while reviewing your PR, I'd suggest the following code changes:

👉 Code Suggestion for #214

#214

You can also review and apply these suggestions locally on your machine.

Learn more about GitKraken Code Suggest

Code Suggest liberates your code reviews from GitHub's restrictive, comment-only feedback style. As simple as suggesting changes in a Google-doc, provide real code suggestions from where you code, e.g. your IDE, and on anything in your project — not just on the lines of code changed in the PR.

Join your team on GitKraken to speed up PR review.

[Tala-mahhmmoodi]

Hey @Framartin, while reviewing your PR, I'd suggest the following code changes:

👉 Code Suggestion for #214

#214

You can also review and apply these suggestions locally on your machine.

Learn more about GitKraken Code Suggest

Code Suggest liberates your code reviews from GitHub's restrictive, comment-only feedback style. As simple as suggesting changes in a Google-doc, provide real code suggestions from where you code, e.g. your IDE, and on anything in your project — not just on the lines of code changed in the PR.

Join your team on GitKraken to speed up PR review.

[kevalmorabia97]

Hi @Framartin can you please address feedback and update your PR?