Voltage-gated sodium, potassium, and calcium channels consist of four voltage-sensing domains (VSDs) that surround a central pore domain and transition from a down state to an up state in response to membrane depolarization. While many types of drugs bind pore domains, the number of organic molecules known to bind VSDs is limited. The Hv1 voltage-gated proton channel is made of two VSDs and does not contain a pore domain, providing a simplified model for studying how small ligands interact with VSDs. Here, we describe a ligand, named HIF, that interacts with the Hv1 VSD in the up and down states. We find that HIF rapidly inhibits proton conduction in the up state by blocking the open channel, as previously described for 2-guanidinobenzimidazole and its derivatives. HIF, however, interacts with a site slowly accessible in the down state. Functional studies and MD simulations suggest that this interaction traps the compound in a narrow pocket lined with charged residues within the VSD intracellular vestibule, which results in slow recovery from inhibition. Our findings point to a “wrench in gears” mechanism whereby side chains within the binding pocket trap the compound as the teeth of interlocking gears. We propose that the use of screening strategies designed to target binding sites with slow accessibility, similar to the one identified here, could lead to the discovery of new ligands capable of interacting with VSDs of other voltage-gated ion channels in the down state.