Proteins controlling cellular networks have evolved distinct mechanisms to ensure specificity in protein-protein interactions. Raf kinase inhibitor protein (RKIP) is a multifaceted kinase modulator, but it is not well understood how this small protein (21 kDa) can coordinate its diverse signaling functions. Raf1 and G protein-coupled receptor kinase (GRK) 2 are direct interaction partners of RKIP and thus provide the possibility to untangle the mechanism of its target specificity. Here, we identify RKIP dimer formation as an important mechanistic feature in the target switch from Raf1 to GRK2. Co-immunoprecipitation and cross-linking experiments revealed RKIP dimerization upon phosphorylation of RKIP at serine 153 utilizing purified proteins as well as in cells overexpressing RKIP. A functional phosphomimetic RKIP mutant had a high propensity for dimerization and reproduced the switch from Raf1 to GRK2. RKIP dimerization and GRK2 binding, but not Raf1 interaction, were prevented by a peptide comprising amino acids 127-146 of RKIP, which suggests that this region is critical for dimer formation. Furthermore, a dimeric RKIP mutant displayed a higher affinity to GRK2, but a lower affinity to Raf1. Functional analyses of phosphomimetic as well as dimeric RKIP demonstrated that enhanced dimerization of RKIP translates into decreased Raf1 and increased GRK2 inhibition. The detection of RKIP dimers in a complex with GRK2 in murine hearts implies their physiological relevance. These findings represent a novel mechanistic feature how RKIP can discriminate between its different interaction partners and thus advances our understanding how specific inhibition of kinases can be achieved.