The ultimate guide to the meta robots tag by Yoast

https://yoast.com/robots-meta-tags/