Plants vary widely in the extent to which seeds are produced via self-fertilization vs. outcrossing, and evolutionary change in the mating system is thought to be accompanied by genetic differentiation in a syndrome of floral traits. We quantified the pattern of variation and covariation in floral traits and the proportion of seeds outcrossed (t) to better understand the evolutionary processes involved in mating system differentiation among and within populations of the short-lived Pacific coastal dune endemic Camissoniopsis cheiranthifolia across its geographic range in western North America.

We quantified corolla width and herkogamy, two traits expected to influence the mating system, for 48 populations sampled in the field and for a sub-sample of 29 populations grown from seed in a glasshouse. We also measured several other floral traits for 9–19 populations, estimated t for 16 populations using seven allozyme polymorphisms, and measured the strength of self-incompatibility for nine populations.

Floral morphology and self-incompatibility varied widely but non-randomly, such that populations could be assigned to three phenotypically and geographically divergent groups. Populations spanned the full range of outcrossing (t = 0·001–0·992), which covaried with corolla width, herkogamy and floral life span. Outcrossing also correlated with floral morphology within two populations that exhibited exceptional floral variation.

Populations of C. cheiranthifolia seem to have differentiated into three modal mating systems: (1) predominant outcrossing associated with self-incompatibility and large flowers; (2) moderate selfing associated with large but self-compatible flowers; and (3) higher but not complete selfing associated with small, autogamous, self-compatible flowers. The transition to complete selfing has not occurred even though the species appears to possess the required genetic capacity. We hypothesize that outcrossing populations in this species have evolved to different stable states of mixed mating.

Hybrid proline-rich proteins (HyPRPs) represent a large family of putative cell-wall proteins characterized by the presence of a variable N-terminal domain and a conserved C-terminal domain that is related to non-specific lipid transfer proteins. The function of HyPRPs remains unclear, but their widespread occurrence and abundant expression patterns indicate that they may be involved in a basic cellular process.

To elucidate the cellular function of HyPRPs, we modulated the expression of three HyPRP genes in tobacco (Nicotiana tabacum) BY-2 cell lines and in potato (Solanum tuberosum) plants.

In BY-2 lines, over-expression of the three HyPRP genes with different types of N-terminal domains resulted in similar phenotypic changes, namely increased cell elongation, both in suspension culture and on solid media where the over-expression resulted in enhanced calli size. The over-expressing cells showed increased plasmolysis in a hypertonic mannitol solution and accelerated rate of protoplast release, suggesting loosening of the cell walls. In contrast to BY-2 lines, no phenotypic changes were observed in potato plants over-expressing the same or analogous HyPRP genes, presumably due to more complex compensatory mechanisms in planta.

Based on the results from BY-2 lines, we propose that HyPRPs, more specifically their C-terminal domains, represent a novel group of proteins involved in cell expansion.

Heterostyly is a floral polymorphism characterized by the reciprocal position of stamens and stigmas in different flower morphs in a population. This reciprocal herkogamy is usually associated with an incompatibility system that prevents selfing and intra-morph fertilization, termed a heteromorphic incompatibility system. In different evolutionary models explaining heterostyly, it has been alternately argued that heteromorphic incompatibility either preceded or followed the evolution of reciprocal herkogamy. In some models, reciprocal herkogamy and incompatibility have been hypothesized to be linked together during the evolution of the heterostylous system.

We examine the incompatibility systems in species with different stylar polymorphisms from the genera Lithodora and Glandora (Boraginaceae). We then test whether evolution towards reciprocal herkogamy is associated with the acquisition of incompatibility. To this end, a phylogeny of these genera and related species is reconstructed and the morphological and reproductive changes that occurred during the course of evolution are assessed.

Both self-compatibility and self-incompatibility are found within the studied genera, along with different degrees of intra-morph compatibility. We report for the first time extensive variability among members of the genus Glandora and related species in terms of the presence or absence of intraspecies polymorphism and heteromorphic incompatibility. Overall, our results do not support a tight link between floral polymorphism and incompatibility systems.

The independent evolution of stylar polymorphism and incompatibility appears to have occurred in this group of plants. This refutes the canonical view that there is strong linkage between these reproductive traits.

The trailing edges of species ranges are becoming a subject of increasing interest as the environment changes due to global warming. Trailing edge populations are likely to face extinction because of a decline in numbers and an inability to evolve new adaptations with sufficient speed. Discussions of character change in the trailing edge have focused on physiological, exomorphic and phenological traits. The mating pattern within populations has not been part of the discourse, in spite of the fact that the mating pattern may affect the ability of populations to respond to environmental change and to maintain their sizes. In this paper, the case is made that a substantial increase in self-fertilization rates may occur via plastic responses to stress.

Small populations on the trailing edge are especially vulnerable to environmental change because of inadequate levels of cross-fertilization. Evidence is presented that a deficiency of cross-seed production is due to inadequate pollinator services and a paucity of self-incompatibility alleles within populations. Evidence also is presented that if plants are self-compatible, self-fertilization may compensate in part for this deficiency through a stress-induced increase in levels of self-compatibility and stress-induced alterations in floral morphology that elevate self-pollination. Whereas increased self-fertility may afford populations the time to adapt to their changing environments, it can be concluded that increased selfing is not a panacea for the ills of environmental change, because it will lead to substantial reductions in genetic diversity, which may render adaptation unlikely.

How and why plants evolve to become selfing is a long-standing evolutionary puzzle. The transition from outcrossing to highly selfing is less well understood in self-compatible (SC) mixed-mating (MM) species where potentially subtle interactions between floral phenotypes and the environment are at play. We examined floral morphological and developmental traits across an entire SC MM genus, Collinsia, to determine which, if any, predict potential autonomous selfing ability when pollinators are absent (AS) and actual selfing rates in the wild, s_m, and to best define the selfing syndrome for this clade.

Using polymorphic microsatellite markers, we obtained 30 population-level estimates of s_m across 19 Collinsia taxa. Species grand means for the timing of herkogamy (stigma–anther contact) and dichogamy (stigmatic receptivity, SR), AS, floral size, longevity and their genetic correlations were quantified for 22 taxa.

Species fell into discrete selfing and outcrossing groups based on floral traits. Loss of dichogamy defines Collinsia's selfing syndrome. Floral size, longevity and herkogamy also differ significantly between these groups. Most taxa have high AS rates (>80 %), but AS is uncorrelated with any measured trait. In contrast, s_m is significantly correlated only with SR. High variance in s_m was observed in the two groups.

Collinsia species exhibit clear morphological and developmental traits diagnostic of ‘selfing’ or ‘outcrossing’ groups. However, many species in both the ‘selfing’ and the ‘outcrossing’ groups were MM, pointing to the critical influence of the pollination environment, the timing of AS and outcross pollen prepotency on s_m. Flower size is a poor predictor of Collinsia species' field selfing rates and this result may apply to many SC species. Assessment of the variation in the pollination environment, which can increase selfing rates in more ‘outcrossing’ species but can also decrease selfing rates in more ‘selfing’ species, is critical to understanding mating system evolution of SC MM taxa.