How to Stay Safe When Exploring New Environments