[Feature Request] Need for ability to save partial results

[thameem-abbas]

As it stands, when run with multiple values to the rate parameter, there is nothing written to disk until the end of all the rates. This causes a potential for loss of all results when the higher concurrencies might potentially run into some error. Some checkpointing to the disk would greatly improve the usability of the tool when running into potential errors.

The following is another scenario that we regularly encounter

1. We run a list of concurrencies.
2. We observe KV Cache depletion on the server side.
3. The latter(higher) concurrencies are of no real use. The user cannot choose to end the run. The runs have to complete to get any usable results.

[markurtz]

One note on this is that the error rate and over saturation detection implementations should fix the latter scenario