We have decided to discontinue the publication of preprints on our preprint server as of 1 March 2024. The publication culture within mathematics has changed so much due to the rise of repositories such as ArXiV (www.arxiv.org) that we are encouraging all institute members to make their preprints available there. An institute's repository in its previous form is, therefore, unnecessary. The preprints published to date will remain available here, but we will not add any new preprints here.
MiS Preprint
28/2016
The Equidistance Index of Population Structure
Yaron Granot, Omri Tal, Saharon Rosset and Karl Skorecki
Abstract
Measures of population differentiation, such as $F_{ST}$, are traditionally derived from a partition of heterozygosity within and between populations. However, the emergence of population clusters from multilocus analysis is a function of genetic structure (departures from panmixia) rather than of diversity. If the populations are close to panmixia, slight differences between the mean pairwise distance within and between populations (low $F_{ST}$) can manifest as strong separation between the populations, thus population clusters are often evident even when the vast majority of diversity is partitioned within populations rather than between them. Moreover, because $F_{ST}$ is also a function of internal diversity, it does not directly reflect the strength of separation between population clusters. For any given $F_{ST}$ value, clusters can be tighter (more panmictic) or looser (more stratified), and in this respect higher $F_{ST}$ does not always imply stronger differentiation. Finally, $F_{ST}$ as a measure of structure or population distance is a ‘supervised’ measure, in the sense that target populations have to be predefined (samples labeled). In this study we propose a measure for the partition of structure, denoted $E_{ST}$, which is more consistent with results from clustering schemes. Crucially, our measure is based on a statistic of the data that is a good measure of internal structure, mimicking the information extracted by unsupervised clustering or dimensionality reduction schemes. To assess the utility of our metric, we ranked various human (HGDP) population pairs based on $F_{ST}$ and $E_{ST}$ and found substantial differences in ranking order. In some cases examined, most notably among isolated Amazonian tribes, $E_{ST}$ ranking seems more consistent with demographic, phylogeographic and linguistic measures of classification compared to $F_{ST}$. Thus, $E_{ST}$ may at times outperform $F_{ST}$ in identifying evolutionary significant differentiation.
