As far as plastic bottle caps go, leave your caps on. Small recyclables jam up the process and are often tossed aside for the trash if left off. Leave them on and the probability of them getting recycled is much higher. This isn’t universal - some recycling facilities will tell you to take your caps off, but if we’re talking about a single stream recycling system, you should be able to leave the caps on to be properly recycled. If you want a definite answer for where you live, call your waste management facility and have them direct you to the appropriate recycling facility.
As far as metal bottle caps for (beer or soda bottle tops), these should not be placed in the recycling bin if they are loose. They will jam up the gears and end up in the trash. Instead, save your metal tops and either find a way to artistically reuse them, or wait until you have a lot and place them in a recyclable aluminum can. Pinch the top of the can closed and when it goes through the recycling system, the caps will be able to go through the system as well.