ray/rllib/tuned_examples/appo/cartpole-appo-vtrace.yaml

cartpole-appo-vtrace:
    env: CartPole-v0
    run: APPO
    stop:
        episode_reward_mean: 180
        timesteps_total: 200000
    config:
        # Works for both torch and tf.
        framework: tf
        num_envs_per_worker: 5
        num_workers: 4
        num_gpus: 0
        observation_filter: MeanStdFilter
        num_sgd_iter: 1
        vf_loss_coeff: 0.01
        vtrace: true
        vtrace_drop_last_ts: false
        model:
            fcnet_hiddens: [32]
            fcnet_activation: linear
            vf_share_layers: true
[RLlib] Benchmark and regression test yaml cleanup and restructuring. (#8414) 2020-05-26 11:10:27 +02:00			`cartpole-appo-vtrace:`
Appo (#3779) * Deleted old fork, updated new ray and moved PPO-impala to APPO in ppo folder * Deleted unneccesary vtrace.py file * Update pong-impala.yaml * Cleaned PPO Code * Update pong-impala.yaml * Update pong-impala.yaml * wip * new ifle * refactor * add vtrace off option * revert * support any space * docs * fix comment * remove kl * Update cartpole-appo-vtrace.yaml 2019-01-18 13:40:26 -08:00			`env: CartPole-v0`
			`run: APPO`
			`stop:`
[RLlib] Minor fixes/cleanups; chop_into_sequences now handles nested data. (#19408) 2021-11-05 14:39:28 +01:00			`episode_reward_mean: 180`
[RLlib] PyTorch version of APPO. (#8120) - Translate all vtrace functionality to torch and added torch to the framework_iterator-loop in all existing vtrace test cases. - Add learning test cases for APPO torch (both w/ and w/o v-trace). - Add quick compilation tests for APPO (tf and torch, v-trace and no v-trace). 2020-04-23 09:11:12 +02:00			`timesteps_total: 200000`
Appo (#3779) * Deleted old fork, updated new ray and moved PPO-impala to APPO in ppo folder * Deleted unneccesary vtrace.py file * Update pong-impala.yaml * Cleaned PPO Code * Update pong-impala.yaml * Update pong-impala.yaml * wip * new ifle * refactor * add vtrace off option * revert * support any space * docs * fix comment * remove kl * Update cartpole-appo-vtrace.yaml 2019-01-18 13:40:26 -08:00			`config:`
[RLlib] Benchmark and regression test yaml cleanup and restructuring. (#8414) 2020-05-26 11:10:27 +02:00			`# Works for both torch and tf.`
[RLlib] Auto-framework, retire `use_pytorch` in favor of `framework=...` (#8520) 2020-05-27 16:19:13 +02:00			`framework: tf`
Appo (#3779) * Deleted old fork, updated new ray and moved PPO-impala to APPO in ppo folder * Deleted unneccesary vtrace.py file * Update pong-impala.yaml * Cleaned PPO Code * Update pong-impala.yaml * Update pong-impala.yaml * wip * new ifle * refactor * add vtrace off option * revert * support any space * docs * fix comment * remove kl * Update cartpole-appo-vtrace.yaml 2019-01-18 13:40:26 -08:00			`num_envs_per_worker: 5`
[RLlib] Minor fixes/cleanups; chop_into_sequences now handles nested data. (#19408) 2021-11-05 14:39:28 +01:00			`num_workers: 4`
Appo (#3779) * Deleted old fork, updated new ray and moved PPO-impala to APPO in ppo folder * Deleted unneccesary vtrace.py file * Update pong-impala.yaml * Cleaned PPO Code * Update pong-impala.yaml * Update pong-impala.yaml * wip * new ifle * refactor * add vtrace off option * revert * support any space * docs * fix comment * remove kl * Update cartpole-appo-vtrace.yaml 2019-01-18 13:40:26 -08:00			`num_gpus: 0`
[RLlib] Refactor: All tf static graph code should reside inside Policy class. (#17169) 2021-07-20 14:58:13 -04:00			`observation_filter: MeanStdFilter`
[RLlib] Minor fixes/cleanups; chop_into_sequences now handles nested data. (#19408) 2021-11-05 14:39:28 +01:00			`num_sgd_iter: 1`
[RLlib] Refactor: All tf static graph code should reside inside Policy class. (#17169) 2021-07-20 14:58:13 -04:00			`vf_loss_coeff: 0.01`
Appo (#3779) * Deleted old fork, updated new ray and moved PPO-impala to APPO in ppo folder * Deleted unneccesary vtrace.py file * Update pong-impala.yaml * Cleaned PPO Code * Update pong-impala.yaml * Update pong-impala.yaml * wip * new ifle * refactor * add vtrace off option * revert * support any space * docs * fix comment * remove kl * Update cartpole-appo-vtrace.yaml 2019-01-18 13:40:26 -08:00			`vtrace: true`
[RLlib] Optionally don't drop last ts in v-trace calculations (APPO and IMPALA). (#19601) 2021-11-03 10:01:34 +01:00			`vtrace_drop_last_ts: false`
[RLlib] Refactor: All tf static graph code should reside inside Policy class. (#17169) 2021-07-20 14:58:13 -04:00			`model:`
			`fcnet_hiddens: [32]`
			`fcnet_activation: linear`
			`vf_share_layers: true`