I haven't gone out looking for geocaches with a GPS in hand, but I've stumbled across one before, while out walking in the Fells. There was something slightly unnatural about the rocks covering a hole in a tree. I somehow intuitively knew it would be OK to take one of the items in the box (I got a glitter toy) if I replaced it with something at least as cool (I left a color-changing LED key fob) and put it back where I found it.

I love that they left extra wrapping material for you to re-wrap the box with.