Endosymbionts are very common in nature, offering multiple occasions to recapitulate events that have led to the generation of mitochondria and plastids. However, both these organelles are unique because they are thought to derive from two individual events that gave rise to all eukaryotes and the plastids in algae and plants (excluding Paulinella chromatophora), respectively. This review focuses on the differences and similarities existing between extant endosymbionts and the two major endosymbiont derived organelles: the mitochondria and plastids. Emphasis is put on recent developments that point to the major role of intracellular pathogens in the establishment of these organelles. We argue that metabolic integration of bacterial endosymbionts into mitochondria and plastids required an unusually high degree of preadaptation not shared by most extant endosymbionts. We propose that this was achieved by either recruiting intracellular bacterial pathogens as "helper genomes" providing needed gene products, or by selecting endosymbionts destined to become organelles directly from such obligate intracellular bacteria.