Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology.

Although variation in effect sizes and predicted values among studies of similar phenomena is inevitable, such variation far exceeds what might be produced by sampling error alone. One possible explanation for variation among results is differences among researchers in the decisions they make regarding statistical analyses. A growing array of studies has explored this analytical variability in different fields and has found substantial variability among results despite analysts having the same data and research question.

Publication bias impacts on effect size, statistical power, and magnitude (Type M) and sign (Type S) errors in ecology and evolutionary biology.

Collaborative efforts to directly replicate empirical studies in the medical and social sciences have revealed alarmingly low rates of replicability, a phenomenon dubbed the 'replication crisis'. Poor replicability has spurred cultural changes targeted at improving reliability in these disciplines. Given the absence of equivalent replication projects in ecology and evolutionary biology, two inter-related indicators offer the opportunity to retrospectively assess replicability: publication bias and statistical power.

Predicting reliability through structured expert elicitation with the repliCATS (Collaborative Assessments for Trustworthy Science) process.

As replications of individual studies are resource intensive, techniques for predicting the replicability are required. We introduce the repliCATS (Collaborative Assessments for Trustworthy Science) process, a new method for eliciting expert predictions about the replicability of research. This process is a structured expert elicitation approach based on a modified Delphi technique applied to the evaluation of research claims in social and behavioural sciences.

Preferred reporting items for systematic reviews and meta-analyses in ecology and evolutionary biology: a PRISMA extension.

Since the early 1990s, ecologists and evolutionary biologists have aggregated primary research using meta-analytic methods to understand ecological and evolutionary phenomena. Meta-analyses can resolve long-standing disputes, dispel spurious claims, and generate new research questions. At their worst, however, meta-analysis publications are wolves in sheep's clothing: subjective with biased conclusions, hidden under coats of objective authority.

Towards open, reliable, and transparent ecology and evolutionary biology.

Unreliable research programmes waste funds, time, and even the lives of the organisms we seek to help and understand. Reducing this waste and increasing the value of scientific evidence require changing the actions of both individual researchers and the institutions they depend on for employment and promotion. While ecologists and evolutionary biologists have somewhat improved research transparency over the past decade (e.

The role of replication studies in ecology.

Recent large-scale projects in other disciplines have shown that results often fail to replicate when studies are repeated. The conditions contributing to this problem are also present in ecology, but there have not been any equivalent replication projects. Here, we survey ecologists' understanding of and opinions about replication studies.

Empowering peer reviewers with a checklist to improve transparency.

Peer review is widely considered fundamental to maintaining the rigour of science, but it often fails to ensure transparency and reduce bias in published papers, and this systematically weakens the quality of published inferences. In part, this is because many reviewers are unaware of important questions to ask with respect to the soundness of the design and analyses, and the presentation of the methods and results; also some reviewers may expect others to be responsible for these tasks. We therefore present a reviewers' checklist of ten questions that address these critical components.

Nest size is predicted by female identity and the local environment in the blue tit (), but is not related to the nest size of the genetic or foster mother.

The potential for animals to respond to changing climates has sparked interest in intraspecific variation in avian nest structure since this may influence nest microclimate and protect eggs and offspring from inclement weather. However, there have been relatively few large-scale attempts to examine variation in nests or the determinates of individual variation in nest structure within populations. Using a set of mostly pre-registered analyses, we studied potential predictors of variation in the size of a large sample (803) of blue tit () nests across three breeding seasons at Wytham Woods, UK.

Detecting and avoiding likely false-positive findings - a practical guide.

Recently there has been a growing concern that many published research findings do not hold up in attempts to replicate them. We argue that this problem may originate from a culture of 'you can publish if you found a significant effect'. This culture creates a systematic bias against the null hypothesis which renders meta-analyses questionable and may even lead to a situation where hypotheses become difficult to falsify.

Transparency in Ecology and Evolution: Real Problems, Real Solutions.

To make progress scientists need to know what other researchers have found and how they found it. However, transparency is often insufficient across much of ecology and evolution. Researchers often fail to report results and methods in detail sufficient to permit interpretation and meta-analysis, and many results go entirely unreported.

Replicating research in ecology and evolution: feasibility, incentives, and the cost-benefit conundrum.

We believe that replicating studies in ecology and evolution is extremely valuable, but replication within species and systems is troublingly rare, and even 'quasi-replications' in different systems are often insufficient. We make a case for supporting multiple types of replications and point out that the current incentive structure needs to change if ecologists and evolutionary biologist are to value scientific replication sufficiently.

What do we really know about the signalling role of plumage colour in blue tits? A case study of impediments to progress in evolutionary biology.

Evolutionary biologists seek to explain the origin and maintenance of phenotypes, and a substantial portion of this research is accomplished by thorough study of individual species. For instance, many researchers study individual species to understand evolution of ornamental traits which appear to be products of sexual selection. I explored our understanding of sexual ornaments in a well-studied vertebrate species that may serve as a case study for research programs in evolutionary biology.

Genetic benefits of mate choice separated from differential maternal investment in red junglefowl (Gallus gallus).

Females may choose more attractive mates to obtain better viability or attractiveness genes for their offspring. A number of studies have demonstrated a positive relationship between paternal attractiveness and offspring quality. However, this pattern could be due to inheritance of paternal genes and/or it could be due to increased maternal investment in the offspring of more attractive males.