I210 dev [do not merge]

[AboudyKreidieh]

This PR is meant to help us keep track of the difference between i210_dev and master and help us share things that need to be modified in here.

I've resolved a good deal of the pep8 and pydocstyle issues, but there are still some that I cannot easily address. Let's try to resolve these:

pep8

• ./flow/algorithms/centralized_PPO.py:231:9: E722 do not use bare 'except'

pydocstyle

• ./flow/algorithms/custom_ppo.py:31 in public class PPOLoss: D101: Missing docstring in public class
• ./flow/algorithms/custom_ppo.py:129 in public function ppo_surrogate_loss: D103: Missing docstring in public function
• ./flow/algorithms/custom_ppo.py:165 in public function kl_and_loss_stats: D103: Missing docstring in public function
• ./flow/algorithms/custom_ppo.py:218 in public function clip_gradients: D103: Missing docstring in public function
• ./flow/algorithms/custom_ppo.py:231 in public class ValueNetworkMixin: D101: Missing docstring in public class
• ./flow/algorithms/custom_ppo.py:257 in public function setup_config: D103: Missing docstring in public function
• ./flow/algorithms/custom_ppo.py:262 in public function setup_mixins: D103: Missing docstring in public function
• ./flow/algorithms/custom_ppo.py:270 in public class KLCoeffMixin: D101: Missing docstring in public class
• ./flow/algorithms/custom_ppo.py:282 in public method update_kl: D102: Missing docstring in public method
• ./flow/algorithms/custom_ppo.py:302 in public function validate_config: D103: Missing docstring in public function
• ./flow/algorithms/centralized_PPO.py:1 at module level: D100: Missing docstring in public module
• ./flow/algorithms/centralized_PPO.py:67 in public method forward: D102: Missing docstring in public method
• ./flow/algorithms/centralized_PPO.py:70 in public method central_value_function: D102: Missing docstring in public method
• ./flow/algorithms/centralized_PPO.py:75 in public method value_function: D102: Missing docstring in public method
• ./flow/algorithms/centralized_PPO.py:147 in public method forward_rnn: D102: Missing docstring in public method
• ./flow/algorithms/centralized_PPO.py:153 in public method get_initial_state: D102: Missing docstring in public method
• ./flow/algorithms/centralized_PPO.py:159 in public method central_value_function: D102: Missing docstring in public method
• ./flow/algorithms/centralized_PPO.py:164 in public method value_function: D102: Missing docstring in public method
• ./flow/algorithms/centralized_PPO.py:177 in public method compute_central_vf: D102: Missing docstring in public method
• ./flow/algorithms/centralized_PPO.py:186 in public function centralized_critic_postprocessing: D103: Missing docstring in public function
• ./flow/algorithms/centralized_PPO.py:288 in public function fill_missing: D103: Missing docstring in public function
• ./flow/algorithms/centralized_PPO.py:380 in public function loss_with_central_critic: D103: Missing docstring in public function
• ./flow/algorithms/centralized_PPO.py:410 in public class PPOLoss: D101: Missing docstring in public class
• ./flow/algorithms/centralized_PPO.py:509 in public function new_ppo_surrogate_loss: D103: Missing docstring in public function
• ./flow/algorithms/centralized_PPO.py:514 in public class KLCoeffMixin: D101: Missing docstring in public class
• ./flow/algorithms/centralized_PPO.py:526 in public method update_kl: D102: Missing docstring in public method
• ./flow/algorithms/centralized_PPO.py:530 in public function setup_mixins: D103: Missing docstring in public function
• ./flow/algorithms/centralized_PPO.py:542 in public function central_vf_stats: D103: Missing docstring in public function
• ./flow/algorithms/centralized_PPO.py:551 in public function kl_and_loss_stats: D103: Missing docstring in public function