Why use a positive learning rate in finetuning?

[QinwenLuo]

In the file train_finetune, this code schedule_fn = optax.cosine_decay_schedule(-actor_lr, max_steps) seems to use a positive learning rate?why?

[lchamp87x]

I think this must be a mistake, since if you just choose to not have a cosine decay schedule, the learning rate for actor is just actor_lr. There are other implementations of IQL available now (e.g., TorchRL) and none of them set a "positive" learning rate (i.e., -actor_lr). My theory is that there was some confusion on how to handle equation (7) in the paper. Despite being a "loss" function, it sounds like (7) should be maximized. Since most gradient-based optimizers implemented in machine learning libraries assume that we are doing descent (i.e., minimization), they automatically take a negative step. So maybe by setting -actor_lr, they could get the optimizer to take a positive step instead. However, this can all be avoided by just taking the negative of equation (7). If you look, they took the negative of equation (7) and did -actor_lr for the cosine decay schedule when they should have only did one of these things.