Bioshock Infinite has less of a focus on exploration than previous games in the series, but that doesn't mean there's none involved at all. One of the earliest examples is when you find an item marked "key to chest" and must find out where the key belongs.
Where to Find the Chest Key
Technically it's possible to reach the chest before you get to the key itself, but it's far more likely that you'll do it the other way around. The mysterious key is resting on the altar at the Fraternal Order of the Raven, the hooded corvid-obsessed racists who have a statue of John Wilkes Booth in the lobby. You'll see their main church area from the balcony first, numerous enemies in worship to another who stands on the stage, the key behind him. Kill them all and get the key, which is placed next to an Infusion bottle. You'll actually get a prompt at this point telling you to go back and explore areas you've already been to.
Where to Find the Chest
The chest itself is a long way back the way you came, back out the building, through the house with the frightened couple and the printing presses, and back out across the Skyhooks to the rooftop with the peaked skylight windows, where you fought the Columbian forces earlier. You can crash down through them those windows or go through the balcony on the side of the building, but either way you'll end up in a bedroom of the Lansdowne Residence. Walk downstairs into a smaller bedroom on the right, and you'll see a blue chest: that's the one you have the key for. Open it up to get salt, health and another infusion.
Now that you've done that, you should take a look at some of our other Bioshock guides. We've got some helpful info on how to find and deactivate the golden guns here, or follow this guide to see the strange Cage/Bird choice explained at last.