New Science’s Report on the NSF

By Matt Faherty
October 2026

Contents

Executive Summary

Introduction

In April 2022, New Science published my 33,000-word report on the National Institutes of Health (NIH) based on interviews with 41 NIH-funded scientists, current and former NIH employees, and science policy analysts, as well as extensive online research. This work was funded by an Emergent Ventures grant provided by Tyler Cowen at George Mason University.

In March 2025, I received a second grant from Emergent Ventures to do a similar project but focused on the National Science Foundation. While the NSF is significantly smaller than the NIH, with a sub-$10 billion budget compared to the NIH’s almost $50 billion budget, the NSF is the premier federal basic science funder in the U.S. and plays key roles across a wide range of scientific domains, from engineering to the social sciences. Whereas my 2022 NIH piece was focused almost exclusively on the inner workings of the NIH’s operations and what NIH employees and grantees thought about them, this essay goes beyond the operations of the NSF to reform proposals and how they can be enacted.

About two months into my NSF work, the second Trump administration released its 2026 budget proposal, which included slashing the NSF’s funding from about $9 billion to $3.9 billion, which would be by far the biggest cut in the agency’s history in both absolute and relative terms. When combined with a host of other reforms floated by the administration, these cuts threatened to fundamentally reshape the NSF. Eventually, the proposed Trump budget was rejected and the NSF only suffered a modest cut down to about $8.75 billion. Then, in April 2026, the Trump administration released its 2027 budget proposal, which offered a similarly severe cut to the NSF budget, down to $4 billion.

The context of these cuts shifted the tenor of my conversations with NSF-associated scientists. Back in 2021 and 2022, my NIH-associated interviewees expressed plenty of praise for the abstract institution of the NIH, but were often highly negative on the specific mechanisms by which the NIH funded projects, ran its study sections, and impacted bioscience research. In 2026, my NSF interviewees tended to be far more defensive of the NSF, and more consciously couched their criticisms of the agency within their praise of the institution and wishes to see it protected from Trump’s cuts. Numerous interviewees understandably were reticent to talk to me at all, and, in my estimation, expressed implicit concerns that my published criticisms of the NSF could be used to justify its budget cuts. However, they were also open about the weaknesses of the NSF and eager to explore reform possibilities to make it a stronger organization.

In the same vein, when comparing my NIH interviews to my NSF interviews, I get the strong sense that scientists have become significantly more open to exploring reforms to the way science is currently funded, particularly by using modern metascience techniques. Researchers I spoke to from fields as diverse as oceanography to neuroscience expressed keen awareness of discussions around the Replication Crisis, the scientific response to the COVID-19 pandemic, and metascience theories about how to tweak funding mechanisms to produce higher-quality work.

These interview trends were reinforced by my ultimate high-level takeaway from my two projects: federal science funding agencies like the NSF and NIH need to do more metascience. Metascience is the application of scientific methods to the process of science.

I have had dozens of conversations with scientists about how to restructure the NSF’s funding, whether the grant review process should put more power in the hands of peer review or Program Officers, how to prioritize basic vs. applied science funding, the proper role of federal science funding in a wider science funding ecosystem, whether the NSF should implement policies to improve replication rates, and many other interesting science policy questions that are worth studying and debating. But my conclusion to nearly all of these issues is that we simply don’t have good answers for them at this time. We have lots of theory and conjecture, but little hard evidence for what works. We need metascience studies and experimentation at large science funding bodies like the NSF and NIH to figure out which science policies work best. We need to put more funding and energy into applying the scientific method to science funding.

The NSF has been operating for 76 years and the NIH for nearly 100 years, and throughout these generations, both organizations have changed little, especially over the last 30+ years. Both agencies devote most of their resources to project-based extramural grants to researchers based in universities; both primarily use a grant review process based on peer review and Program Officers; both are run and operated by a mixture of long-term federal employees and researchers who take temporary positions; etc. Neither organization has engaged in (what I and others would consider) substantial experimentation with its practices to see what works and what doesn’t, and subsequently implemented reforms based on empirical data.

Whether the NSF is cut substantially by the Trump administration or not, the agency is undergoing a period of transformation. It has lost over 20% of its staff, its Directorates and Divisions face consolidations and restructurings, and the NSF will likely have a new leader in the coming months. While these reforms are understandably threatening to NSF staff and researchers, they also present an opportunity to point the NSF in the right direction.

Based on my interviews and research, I would like to see the NSF, NIH, and other federal science funding agencies develop metascience programs, drawing on models like the United Kingdom’s Metascience Unit. In September 2026, late into editing this report, the NSF announced its new Office of Metascience. The questions now concern how this office should operate, what resources and authority it should have, and whether similar efforts should extend across federal science funding agencies.

The Metascience Unit’s budget is only £49 million over three years; the NSF’s new office could support a larger and more ambitious effort, and could reshape American science funding policy to a vastly more optimized form. Experiments in metascience over the next five, ten, or twenty years could improve science dramatically over the following centuries. A proper NSF or federal metascience program could do to scientific discovery what actuarial science did to life insurance, or what Sabermetrics did to professional baseball. Fortunately, Michael Kratsios’s “Science: A New Golden Age” report indicates that metascience is now a priority of the Trump administration, and hopefully my report will buttress many of Kratsios’s recommendations, as well as offer recommendations on how to apply them to the National Science Foundation.

In Part 1 - The National Science Foundation’s Past and Present, I describe the history of the development of the NSF, its foundational philosophy, its impact on the scientific ecosystem today, the best and worst parts of its modern operations according to the testimonies of my interviewees, its largely favorable comparisons to the NIH, and how the second Trump administration has reformed the NSF thus far.

In Part 2 - Why the National Science Foundation Needs Metascience and Experimentation, I describe what metascience is, why it matters, the central metascience question of how to study the quality of research, and why effective reform is extremely difficult at the NSF or any other science funding institution without tackling this question and metascience as a whole head-on.

In Part 3 - Potential NSF Reforms and Experiments, I describe the dozens of NSF and science funding reform ideas offered by my interviewees, their arguments, their feasibility, and their tradeoffs, and I attempt to present a menu of reforms that a proper metascience unit could use as the basis for its experiments at the NSF or elsewhere.

In Part 4 - How the NSF Can Implement Metascience, I discuss the development of the NSF’s new Office of Metascience, the attitudes of scientists toward metascience, examples of metascience experimentation at other institutions and governments, and the political support needed to sustain and expand these efforts.

In the Conclusion, I will briefly summarize Michael Kratsios’s “Science: A New Golden Age,” and where my report agrees and disagrees with it.


Part 1 – The National Science Foundation’s Past and Present

History1, 2, 3, 4

The National Science Foundation was founded in 1950 as the brainchild of Vannevar Bush, an electrical engineer who had a distinguished career in academia and private labs before joining the federal government’s National Advisory Committee for Aeronautics (the precursor to NASA) in 1938.

From this post, Bush was able to act on his concern that the United States would soon be at war and that there was a lack of productive communication between private industry and the military concerning technological developments with military applications. He conceived of a new government body called the National Defense Research Committee that would coordinate private research across the country to support military priorities. Bush initially sought approval from Congress to form the Committee and drafted a proposal for consideration, but by mid-1940, Nazi Germany had conquered Poland and the Low Countries, and would soon control France, so Bush expedited the process by pitching the idea directly to President Franklin Roosevelt; the President approved the plan within 15 minutes and made Bush the Committee’s first chairman.

Though the Committee had broad oversight powers, it was initially funded solely by the limited discretionary budget of the executive. In June 1941, Roosevelt created the Office of Scientific Research and Development as a new entity that subsumed the Committee and would be funded by Congressional appropriations. Bush was appointed the Office’s director and he oversaw the coordination between military needs and private industry throughout the war; most notably, the Office formed the S-1 Section devoted to nuclear research, which was eventually transferred to the Army and grew into the Manhattan Project.

In 1944, President Roosevelt asked Bush to write a report on his recommended science policies for after the war based on the Office of Scientific Research and Development’s operations, especially since the Office was expected to be phased out (it was closed in 1947). In 1945, Bush published his report, entitled Science—The Endless Frontier, which would become not just the philosophical bedrock of the National Science Foundation, but of arguably all federal scientific funding.

In The Endless Frontier, Bush conceptualized and presented his cutting-edge model of government science policy. He asserted that the invention of new technologies was downstream from “basic research,” or the scientific exploration of fundamental knowledge and principles, often far removed from technological or commercial concerns. In this framework, scientific progress is achieved through the development of “basic” and “applied” research in tandem. For instance, Bush pointed out that the death rate from disease in the Army declined from 14.1/1,000 in WWI to 0.6/1,000 in WWII; he attributed this trend to applied research breakthroughs after WWI that produced vaccines, prophylaxes, and treatments for the major diseases of the day, but these breakthroughs were built on basic research breakthroughs produced over prior decades in medical clinics, hospitals, and universities.

In his words:

“Basic research is performed without thought of practical ends. It results in general knowledge and an understanding of nature and its laws. This general knowledge provides the means of answering a large number of important practical problems, though it may not give a complete specific answer to any one of them. The function of applied research is to provide such complete answers. The scientist doing basic research may not be at all interested in the practical applications of his work, yet the further progress of industrial development would eventually stagnate if basic scientific research were long neglected.”

By Bush’s account, there were three producers of research: the government, private for-profit industry, and philanthropy. When it came to applied research, he was confident that the federal government’s needs and industry’s profit motives would produce sufficient quantities. He observed that private industry was superior to the government in producing effective applied research for most technological development but recommended more federal investment in government labs to produce applied research that was more directly aligned with state interests. He also recommended changes to the tax code and intellectual property system to encourage more commercial research.

To Bush, stimulating basic research was a harder problem. Though Bush didn’t use too much economic language in The Endless Frontier, he outlined the standard economic reasoning for why “basic research” is structurally underfunded by private for-profit enterprises. Basic research, by its nature, has broad, unpredictable results with no direct commercial applications; it can only produce results that can be converted into commercial uses by later applied research. Therefore, for-profit companies lack sufficient incentive to invest money in research projects that are unlikely to produce profitable ends, and may even produce results that will enrich competitors. In modern economic terms, basic research is a “public good” that produces positive externalities that aren’t captured by the producer, and therefore basic research will be underfunded by the standards of optimizing general welfare.

Or in Bush’s words:

“Industry is generally inhibited by preconceived goals, by its own clearly defined standards, and by the constant pressure of commercial necessity. Satisfactory progress in basic science seldom occurs under conditions prevailing in the normal industrial laboratory.”

As for philanthropic support for basic research, Bush recognized that the incentives are well-aligned, but, at least at the time, philanthropy simply lacked the funding and scale to provide sufficient basic research output for the American economy.

Thus, to strengthen the United States and continue its technological ascendency, Bush believed it was imperative that the federal government take the lead in funding basic research, partially through government labs, but mostly by financing academia and private research institutions. To this end, the report concluded with Bush encouraging President Roosevelt and Congress to create a “National Research Foundation” consisting of five Divisions devoted to “Medical Research,” “Natural Sciences,” “National Defense,” “Scientific Personnel and Education,” and “Publications and Scientific Collaboration.”

With the death of President Roosevelt in 1945, the realization of this vision fell to President Harry Truman and his Congresses. The process of forming the National Science Foundation took five years, with numerous bills shot down over structural disputes and one bill vetoed by Truman for lack of financial accountability. The National Science Foundation Act of 1950 created an institution based on Bush’s model, but with a few modifications, including the five Divisions being consolidated into four - “Mathematical, Physical and Engineering Sciences,” “Biological Sciences,” “Medical Research,” and “Scientific Personnel and Education.”

Another key change was a move away from Bush’s intended block grant structure, which funded institutions that then distributed the money to their researchers. The 1950 NSF Act didn’t prescribe a grant form, but the agency soon adopted the NIH’s grant structure, which gave funding to individual researchers at institutions based on their proposed projects. Indirect costs, or funds paid to the institution that houses the researcher, were initially negotiated on a case-by-case basis before beginning to be standardized in the late 1950s. Arguably, this structure resembled a contract with the researcher and institution more than a grant.

In May 1950, the National Science Foundation was established, and in 1951, it was given an initial appropriation of $225,000. In February 1952, the NSF issued its first grant, worth $10,300, to the Institute for Cancer Research, and in that year, it gave out 97 grants in total.

Throughout the NSF’s 76 years of operations, it has largely retained the same structure, albeit with some new funding mechanisms and Division restructuring. Starting in the late 1950s, the NSF began funding major scientific projects, such as the National Radio Astronomy Observatory and the American South Pole station in 1957, and the National Center for Atmospheric Research in 1959. In 1968, President Lyndon Johnson passed a bill amending the NSF to rename its “Divisions” to “Directorates” and mandating that social sciences be part of its mission, a task that would have been strongly opposed by the STEM-minded Vannevar Bush. By 2022, after numerous directorate additions, subtractions, and mergers, the NSF had eight Directorates:

Since its founding, the NSF’s budget has steadily increased in absolute terms, reaching approximately $9.9 billion in total funding in FY2023, including supplemental funding. Nevertheless, the agency has experienced occasional budget cuts. The largest cut in absolute terms to NSF’s base appropriation was $479 million in FY2024. In percentage terms, some earlier reductions were considerably larger; for example, NSF funding fell by roughly 19 percent in 1969.

National Science Foundation budget, 1950–2025

When these numbers are inflation-adjusted, the NSF still had a fast growth period in its first 20 years, followed by a decade of slight decline in the 1970s, then mostly fast growth from 1980 to 2023, after which the NSF has been in a short but sharp contraction.

National Science Foundation budget, inflation-adjusted to approximately 2025 US dollars

Throughout its 76-year history, the NSF’s funded research has played some role in the development or discovery of:

Role of the NSF in U.S. Scientific Research

To get a sense of what the National Science Foundation looks like today, I’ll give some figures from its operations in 2023 and 2024, prior to the uncertainties and reforms of the second Trump administration.19

In 2023, spending on research and development (R&D) in the United States from businesses, government, universities, and nonprofits totaled about $937 billion: $138 billion for basic research, $174 billion for applied research, and $625 billion for experimental development. Basic research therefore accounted for about 15% of total R&D spending. Of the $138 billion spent on basic research, about $56 billion was funded by the federal government and $47 billion by businesses. For comparison, the NSF received approximately $9.9 billion in total funding in fiscal year 2023, including supplemental funding, and approximately $9.1 billion in FY2024.20

In 2024, the NSF had a budget of $9 billion. Of this sum, 79% was devoted to traditional research activity, 13% to STEM Education (one of the Directorates), 3% to major research equipment and facility construction, and about 5% to general staff overhead. The NSF has a well-earned reputation for being a lean government agency.

In 2024, the NSF evaluated 40,000 proposals, conducted 187,829 proposal reviews, and awarded about 11,000 grants to researchers in 1,850 universities and other institutions. 76% of grant dollar obligations went to academic institutions, 14% went to private institutions (combining nonprofit and for-profit), 4% went to major research projects, and the rest went mostly to local governments. The major research projects funded by the NSF include the Laser Interferometer Gravitational-Wave Observatory (LIGO), the National Center for Atmospheric Research (NCAR), the National Ecological Observatory Network (NEON), and the AI Materials Institute at Cornell University.

Counting new grant recipients, existing grant recipients, staff at NSF-funded facilities, and all staff and students otherwise connected to NSF funding, including graduate students and postdocs working in labs funded by the NSF, the NSF estimated that it “supported” 357,600 individuals in 2024.

In 2024, 26% of NSF grant proposals were approved,21 ranging from 36% at the Geosciences Directorate to 19% at the Biological Sciences and Social, Behavioral and Economic Sciences Directorates. For contrast, the NIH’s grant approval rate that year was 19%, with the funding rate for standard R01 grants at only 17%.

What the National Science Foundation Does and Doesn’t Do Well

I interviewed 31 individuals who are either currently or were previously funded by the NSF, worked at the NSF, or collaborated with the NSF in some capacity. In this section, I will describe their assessments of many aspects of the agency, particularly in comparison to the assessments of the National Institutes of Health I received from my interviewees in my previous report.

My big picture takeaway is that the NSF seems like a better-run organization than the NIH. In general, my NSF interviewees had more good things to say about the NSF and fewer criticisms, while my NIH interviews were more often filled with complaints about NIH policy and the general scientific research system built around it.

Or to put it another way, I heard many of the same criticisms about both organizations – a lack of risk-taking, excessive bureaucracy, problems with peer review, etc. – but the criticisms tended to be far more intense with the NIH.

For all of these assessments of both the NSF and NIH, I need to caveat that both agencies are relatively siloed in their operations. Just as the National Cancer Institute has a different culture and some different structures than the National Eye Institute, so too does the NSF’s Directorate of Biological Sciences differ from the Directorate of Geosciences. Furthermore, my interviewees interacted with the NSF and NIH at different points in time, some going as far back as the 1970s, and some as recently as late 2025. I did my best to aggregate responses with these factors in mind.

Grant Review

Many of the contrasting viewpoints between the NSF and NIH derive from the grant review structures described in my interviews, with the NSF often being called the “gold standard of peer review.” In these accounts, the NSF gave greater weight to the judgment of Program Officers, while the NIH processes my interviewees experienced depended more heavily on peer review scores. Both approaches have tradeoffs, but my interviewees generally preferred the NSF’s system. My NIH interviews were conducted in 2021 and 2022, so this comparison reflects experiences that preceded substantial NIH funding reforms instituted in January 2026.

The NSF grant review process was streamlined in December 2025, but to summarize its broad outline in most cases:

Standard NSF grant proposals go through what the NSF calls “Merit Review.” Proposals are submitted to a specific NSF program within a specific NSF Directorate (for instance, to the Division of Environmental Biology core program of the Directorate of Biological Sciences) and then assigned to a Program Officer(s), who is either a long-term employee of the NSF or a “rotator” from outside the agency working at the NSF on a short-term contract.

The proposals then go through a peer review process overseen by the Program Officer. The form of the peer review varies by program, but typically it consists of either an in-person panel discussion group, an external ad hoc group of reviewers who do not meet, or a combination of the two. The reviewers in all cases are chosen by the Program Officer, and are usually researchers in the relevant field.

After the review processes are done, the Program Officer goes through the reviews and makes recommendations on which proposals to fund based on two criteria: “intellectual merit” and “broader impacts.” The first summarizes the straightforward scientific value of the proposal and its potential to advance knowledge, while the second considers the societal benefits of the proposal, and is understandably more divisive. A few of my sources told me that Program Officers tend to pay a lot more attention to broader impacts than the reviewers, and that the smart grant applicants eventually learn how to pad any proposal with enough broader impacts content to check the box.

After selecting grants, the Program Officer makes funding recommendations to the Division Director, who must approve the grants, before making further recommendations to the NSF’s Division of Grants and Agreements or Division of Acquisition and Cooperative Support for business, financial, policy, and risk review before grants are finally approved. Interviewees suggested that the extent to which a Division Director’s decisions diverge from those of the Program Officers below him/her generally depends on the degree to which the Program Officer disagreed with their review panels and the costs of the proposed projects.

When this process is cleared, the grant applicants are notified of the decisions and given anonymous feedback from the reviewers. Declined proposals can be resubmitted after revisions or appealed through a separate internal NSF process.

The NIH study section system (prior to January 2026 reforms) had a similar overall structure, except in most cases funding decisions were weighted more heavily from the aggregated and averaged scores assigned by individual peer reviewers on large review panels (called Study Sections). This not only gave a faceless quality to the NIH system, but led to a host of bad patterns and incentives. NIH interviewees complained about:

In contrast, the NSF avoids or at least ameliorates most of these issues by empowering the Program Officers to make funding decisions while keeping the peer review panels in an advisory capacity. Program Officers can still be subject to groupthink or pedantry, but their greater authority incentivizes more independent thinking, and as one interviewee put it, a focus on “scientific taste.”

Plenty of NSF interviewees expressed that they had been frustrated by decisions made by Program Officers, but there was a far higher level of respect for this process and the decisionmakers behind it than at the NIH, where funding was determined, as one interviewee put it, “algorithmically.” Or to paraphrase a different interviewee, at the NIH, any single reviewer could shoot down a proposal, while at the NSF, it’s the opposite. Or from the NSF’s own description: “A proposal doesn’t need to receive all ‘excellent’ scores to be funded. Conversely, receiving all ‘excellent’ scores is no guarantee of funding.”22

This contrast was never absolute, and the NIH’s funding rules have since changed. According to the NIH, only around half of its Institutes, Centers, and Offices previously used paylines—cutoffs based on peer review scores or percentiles—and even those allowed exceptions.

Nevertheless, the NIH’s January 2026 reforms have moved the agency away from the more “algorithmic” processes. The reforms prohibit using paylines to determine which applications receive funding. Peer review scores and critiques remain important, but they must be considered alongside scientific priorities, available funds, and the balance of the research portfolio. Institute, Center, and Office directors retain final funding authority. Whether the change substantially alters funding decisions or improves research outcomes will require further evaluation.23

Professionalism

The potential weakness of an NSF-style grant review structure that empowers Program Officers is that the outcomes of grant reviews are heavily based on the quality of individual Program Officers. Bad Program Officers can make worse funding decisions than committee-style NIH study sections.

Fortunately, I heard nearly universal praise for the professionalism of the NSF, both in terms of its staff and the conduct of review panels. Given the political climate, a few interviewees emphasized the apolitical nature of the NSF staff and a strong awareness that it was acting as a steward for taxpayer funds. Biases will always exist, but my interviewees always commended NSF personnel for doing their best to evaluate every grant application on its own merits, in contrast to the constant complaints I heard about NIH study section conduct. The difference likely comes down to a matter of the culture of the two organizations as much as their rules or structures.

Likewise, I heard almost universal praise for the higher-level NSF staff and their dealings with NSF-funded projects and collaboration with outside institutions.

Rotators

A special aspect of the NSF that should be highlighted regarding both its Merit Review process and professionalism is its use of “rotators,” or staff sourced from outside the NSF. NSF staff working through the Intergovernmental Personnel Act remain employees of their institutions (usually universities), but work for the NSF for 1-4 years; NSF staff working through its Visiting Scientist, Engineer, and Educator program are federal employees for 1-2 years. Staff from both groups are casually referred to as rotators and usually work as Program Officers or in other roles at the NSF. While at the NSF, these researchers cannot apply for new NSF grants, but can apply to have existing grants extended. In 2020, about 175 of the NSF’s 2,000 personnel were rotators.24

One of the biggest concerns at the NSF, NIH, or any science funding agency is avoiding insularity and stagnation; it is easy for an institution, especially one based within the federal bureaucracy, to become too inwardly facing or lose touch with the wider scientific community and its new innovations. According to many of my interviewees, the NSF’s rotator program is a strong antidote to this concern, as it consistently refreshes the blood in the NSF’s system, which both brings in new ideas and stops cliques from entrenching. Additionally, rotators gain valuable experience and perspective on their fields, which they can then take back to their outside institutions.

I spoke to several former NSF rotators, and all spoke extremely highly of the rotator system and their experiences at the NSF. One called it “the best job I ever had.”

The NIH has some temporary-assignment mechanisms that resemble the NSF’s rotator system, but its Scientific Review Officers—who perform the peer-review-management portion of an NSF Program Officer’s role—are generally long-term NIH staff. My NIH interviewees often complained about cliquishness and insularity in the study sections, while my NSF interviewees frequently brought up the rotator programs without prompting as one of the agency’s best features.

Research Risk Tolerance

In my interviews, the NSF was consistently praised for being more supportive of high-risk, potentially transformative, “blue sky” research than the NIH. This is likely mostly a product of the grant review processes, with the NSF’s Merit Review favoring Program Officers who can make riskier grant selections, while the NIH’s study section consensus model favors more conservative research. I think the difference is also partially a result of the differing cultures at the two institutions, with the NSF being more broadly risk-tolerant and less bureaucratic. These impressions concern the systems my interviewees had experienced; whether the NIH’s 2026 funding changes alter its tolerance for research risk will require evaluation.

However, numerous interviewees still complained that the NSF was too risk-averse for their preferences, and noted the same incentive structure and biases found at the NIH: grant proposals are more likely to be funded if they have defined goals, target incremental advancements, and contain lots of preliminary data, while higher-risk, potentially transformative projects are less likely to be funded.

My NIH interviewees often praised programs like the Pioneer Award and the Transformative Research Award, which are explicitly designed to fund high-risk, high-reward research proposals, often by circumventing the traditional study section structure, but my NIH interviewees often lamented that these programs received very small portions of the overall NIH budget.

Likewise, my NSF interviewees praised the EAGER (high-risk research in any field), RAPID (fast turnaround grants that respond to recent emergencies), and RAISE (novel, interdisciplinary projects) programs at the NSF, which serve the same functions of encouraging high-risk, high-reward research. And just like the NIH interviewees, my NSF interviewees wished these programs were larger and more well-funded.

Bureaucracy

My NSF interviewees consistently portrayed the agency as being less bureaucratic than the NIH in terms of paperwork, hoops to jump through, and general hassle of trying to get things done.

The NSF is often praised, both through my interviews and elsewhere, for being a “lean” organization, with only 5% of costs going to administrative overhead.25 While this is an admirable figure, some interviewees suggested that the NSF was a bit too lean, with staffers being overworked and stretched thin, especially after the wave of layoffs and resignations during the Trump administration. Somewhat surprisingly compared to its reputation, the NIH actually has an almost identical level of administrative overhead at 5%.26 DARPA, an agency that has an even greater reputation than the NSF for leanness, spends 3-8% of its budget on overhead costs (depending on how you categorize certain items).27 It seems that in general, federal government research funding agencies tend to have admirably low levels of overhead costs.

(Note that the overhead costs in these figures do not reflect the “indirect costs” paid to research institutions that host NSF/NIH-funded researchers.)

One potential model for reducing the administrative burden of federal science funding is Stuart Buck’s proposal for “Minimum Viable Bureaucracy.”28 Rather than simply standardizing grant requirements across agencies, Buck argues that the federal government should compare agencies performing similar functions, identify which accomplishes them with the least administrative burden, and copy them elsewhere (with due diligence). For example, Buck contrasts the NIH’s F31 graduate fellowship application, which he estimated could require 50–100 hours of work and substantially more documentation, with the far simpler NSF Graduate Research Fellowship application. The relevant question is not merely whether the NIH process is more burdensome, but whether its additional requirements actually improve grant selection or research outcomes. If they do not, the NSF’s simpler process could serve as a benchmark for reform.

Another way to think about red tape bureaucracy is to evaluate an agency’s nimbleness, or how quickly it can respond to emergencies. A potential case study is how the NSF and NIH responded to the COVID-19 pandemic; two interviewees with whom I spoke expressed that the NSF’s response was better and faster. In April 2020, the NSF sent out a Dear Colleague letter29 soliciting COVID research and offering numerous pathways for expedited funding, including its RAPID awards, which are reviewed and approved within two to six weeks by a small internal NSF team and a dedicated Program Officer.

In contrast, the NIH was perceived to be sluggish in its response by critics despite its domain expertise. For instance, in a May 2022 STAT piece, the NIH’s study of COVID was likened to “A slow-moving glacier,”30 a June 2022 article from The Atlantic called the NIH “strangely quiet” “amid the biggest health crisis in 100 years,” and in the following year, STAT reported that the Congressionally mandated $1.6 billion spent on studying Long COVID at the NIH produced “underwhelming” results31 due to “bungled” experimental designs.32

Grant Burdens

A typical NSF grant application has a 15-page research description followed by 15 to 30 pages of budget justifications, data management plans, and administrative details. The average NSF grant takes about 120 hours to write.33 The average principal investigator submits 2.3 grant proposals for every one that gets approved. 36% of NSF grantees receive an award on their first attempt.34

Various sources and surveys estimate that a typical scientific researcher spends around 40%35 of their research-related work time on grant applications, administration, and maintenance, rather than actual scientific research, though that figure varies considerably by field and institution. This was one of the most common complaints I heard from NIH grantees, and I heard similar complaints from NSF grantees, though some suggested anecdotally that NSF grants are a little less burdensome. For some slight and potentially outdated empirical support, the Federal Demonstration Partnership’s 2012 Faculty Workload Survey reported that NSF-funded researchers spent 13.4% of their time on grant preparation, compared to 16.3% for NIH-funded researchers.36

From both my NIH and NSF interviews, it is clear that most researchers believe they spend too much time applying for grants and maintaining them through periodic grant reports. Many interviewees wished that someone – NSF administrators, Congress, or anyone else – would impose a top-down simplification of NSF/NIH grant applications and especially cut down on their reporting requirements. At the same time, most of these interviewees recognized the inherent complexity of federal grants financed by taxpayer funds. Much of the grant application and maintenance burden comes from forms meant to increase researcher accountability, and they seem reasonable in isolation, but when stacked up, they become excessively burdensome. Regardless, researchers and an army of administrators at universities and research institutes waste an enormous amount of time, money, and energy on federal grant application processes.

The Grant Treadmill, Hyper-Competition, and the Science Funding Complex

The most common high-level complaint I heard about the NSF, NIH, and entire federal science funding ecosystem is that it has led to “hyper-competition” between researchers who are trapped in a “Grant Treadmill,” both of which contribute to a demoralization of scientists and possibly a diminishment of the quality of scientific output.

In about a dozen interviews, Bell Labs was brought up as an ideal research model, and though the reality of the institution is complicated, the abstract concept of Bell Labs expressed by many of my researcher interviewees is an environment where scientists are given sufficient resources, freedom, and time to follow their curiosity and explore interesting research pathways, no matter where they may lead. This environment is allegedly conducive to both major research breakthroughs and the sort of passionate work that inspires scientists to embark on their careers.

In the real world, my NSF and NIH interviewees reported widespread dissatisfaction with the status quo system that puts immense pressure on researchers to constantly scrounge for grant money and pump out research, often with the threat of severe negative career consequences if they ever slow down in this process. Far from the theoretical Bell Labs environment, many researchers feel that they are cogs in a machine where their scientific passions are subordinated to the interests of their employers (usually universities) who profit off of them (through indirect grants) and their own hyper-competitive careers. Rather than pursue natural curiosities, researchers are incentivized to run a “Grant Treadmill” in which short-term (3-5 year) grants oriented toward conservative, incremental goals must consistently be generated to justify their positions at their research institutions, and with generally falling grant proposal acceptance rates at the NSF and other federal agencies, only some researchers can even survive this cycle, let alone explore their preferred projects.

The NIH has a hegemonic position in bioscience research and deserves a large amount of the blame for establishing this atmosphere in its domain. The NSF is smaller and plays a less important role in its many scientific domains, but it deserves some blame for perpetuating this system. At least NSF grants are generally easier to get, with a 2024 funding rate of 26% compared to an NIH R01 funding rate of under 19%. However, the standard NIH grant (R01) runs for 3-5 years (more often 4-5 years), whereas a typical NSF grant (called a Standard Research Grant) tends to run for three years, so in some sense, the NSF’s Grant Treadmill demands its researchers run faster than the NIH’s Grant Treadmill.

There is no easy fix for the high-pressure environment and Grant Treadmill perpetuated by the NSF, but my interviewees consistently expressed support for NSF reforms or even overhauls to the entire federal science funding structure to ameliorate the pressure and cultivate an atmosphere better suited for passionate scientific discovery.

Basic vs. Applied Science

At its founding, the NSF was established to fund “basic science,” or research on fundamental aspects of a given field, sometimes with no clear goals, and often with no accessible commercial applicability.

According to the NSF’s National Center for Science and Engineering Statistics, basic research accounted for about 83% of the NSF’s R&D obligations in 2024 ($5.8 billion), while applied research accounted for 17% ($1.2 billion).37

Opinions on whether the NSF is or should be devoted to basic science varied wildly among my interviewees. A few said that the NSF should return to its core mission and refocus more on basic science, while others said that funding a mixture of basic and applied science creates better synergies than pure basic science. Other interviewees dismissed the dichotomy entirely, claiming the borders between basic and applied science are too blurry or the distinction matters less than simply trying to fund the best science proposals possible.

I heard a similar range of opinions from my NIH interviewees, but there was stronger support for reorienting the NIH back toward a basic science focus. I think this is likely due to the NIH’s proximity to the extremely large applied bioscience research ecosystem maintained by private industry (primarily pharmaceutical companies), so the NIH interviewees see the NIH as filling a more natural role as a basic science funder. In contrast, the NSF is spread across numerous fields with widely varying levels of applied science funding outside the federal domain.

Social Sciences and STEM Education

Similarly, NSF interviewee opinions varied on the NSF’s Social, Behavioral and Economic Sciences Directorate (SBE, which oversees the NSF’s social science research) and the STEM Education Directorate.

Social sciences were not part of the NSF’s mission until 1968, and the SBE wasn’t created until 1991. I get the sense that the SBE is still considered vestigial to the core purpose of the NSF by many researchers in the hard sciences, and in light of the cuts proposed by the Trump administration in 2026, the SBE may be eliminated entirely by 2027.38 Two interviewees, neither of whom worked directly with the SBE, were adamant in their support for it due to the general importance of social sciences.

The STEM Education Directorate had even less support among my interviewees. Though the Directorate also funds basic research, much of its spending is dedicated to other goals like workforce training and curriculum development, with even the NSF’s budget reports counting it separately from the other Directorates at 13% of the budget in 2024.39 Admittedly, I did not talk to anyone who worked in or with this Directorate, but my impression from those outside of it is that the Directorate is seen to deviate from the NSF’s core functions. A few interviewees preferred shutting down the STEM Education Directorate entirely or at least spinning it off as an independent agency outside of the NSF’s purview.

In the cases of both the SBE and STEM Education Directorates, evaluations are complicated by political controversies. Both Directorates have been accused of being used to promote political goals rather than useful scientific research, with the SBE funding university social sciences that tend to lean significantly to the political left, and the STEM Education Directorate being used to promote DEI by targeting racial minority students and communities.

Talent Cultivation

A possibly underrated aspect of the NSF brought up by numerous interviewees is its role as a researcher cultivator. Through direct employment, labs, grants, etc., the NSF’s operations provided funding to over 350,000 researchers, research administrators, students, teachers, etc. in 2024, all of whom contribute in some way to the American scientific ecosystem that stands as the envy of the world.

While the NIH is far larger and plays a far more dominant role in biomedical sciences than the NSF does across its scientific domains, many interviewees pointed out this NSF function and were grateful for the way it supported their careers.

Second Trump Administration Reforms and Their Opposition

From the start of the second Trump administration in 2025, the National Science Foundation has faced realized and potential changes like never before in its history. Writing this essay in the spring and summer of 2026, it is difficult to summarize these reforms because some are still in their proposal stages, others have been officially enacted but their enforcement is unclear, and some have been shot down entirely, but I will do my best to explain what has happened and what might happen to the NSF.

Most interviewees were extremely opposed to the Trump administration’s science agenda. All interviewees were unfailingly kind to me, but they did occasionally get heated when discussing the Trump administration and its proposed policy reforms. The terms “disaster” and “apocalyptic” were used more than once. A few interviewees described these reforms as being maliciously targeted at scientists for being generally opposed to the culture, style, and policy goals of the Trump administration. Others said the reforms were merely misguided attempts to shake up the status quo.

However, while my interviewees were strongly opposed to the funding cuts, staff cuts, and the generally haphazard nature of the administration’s science policy, they tended to be neutral or even positive on the more substantive policy changes, such as those to the Merit Review process or indirect cost policies. A minority of interviewees were cautiously optimistic about the Trump reforms, less so for their specific policy outlines, and more so for creating an opportunity to make more drastic changes than normal to the NSF to improve its functioning.

Budget

The most significant change proposed by the Trump administration to the NSF is its budget. Even after the proposed 2026 budget cuts were rejected, the administration is proposing a similarly severe cut for 2027, from a little under $9 billion to $4 billion. If enacted, this would take the NSF’s nominal budget back to its level in the late 1990s, or even the mid-1980s with inflation-adjustment.

While the extent and intent of the proposed cut were universally derided by my interviewees, those who speculated on its outcome were optimistic that it would not pass. Congress has historically been highly supportive of federal science funding, particularly over the last 20 years, and given the relatively small total budget of the NSF, they were confident that the cuts would mostly fail to materialize.

Research Funding Disruptions and Canceled Grants

The NSF’s short-term funding has been even rockier than its long-term funding during the second Trump administration.

In late January 2025, the incoming Trump administration began reviewing federal grants for consistency with a series of new executive orders, particularly those targeting DEI-related programs. On January 27, the Office of Management and Budget issued a memorandum directing agencies to pause obligations and disbursements for financial assistance programs potentially implicated by those orders. The next day, the NSF paused review panels, new awards, and payments under existing awards. Courts quickly blocked the freeze as applied to existing awards, and the OMB rescinded the memorandum on January 29. On February 2, NSF funding was resumed for most existing grants, but all existing and new grants were henceforth put under scrutiny for potential violations of the administration’s directives.40, 41

In April, 402 NSF grants were canceled for violating the White House directive, and the NSF announced, “it will no longer fund grants that involve engaging with people based on characteristics that are protected from discrimination under various laws, such as race, color, religion, sex, and national origin.”42

According to a report from Nature, the NSF canceled or suspended a total of 1,996 grants in 2025 (though some were later reinstated), while the NIH canceled or suspended 5,844 grants. That year, the NSF issued over 8,000 grants, a drop of about 25% from almost 12,000 in 2024.43

Personnel

The NSF has seen unprecedented staff turnover since early 2025, driven by a combination of layoffs and resignations. The agency’s staff fell from over 1,700 in 2024 (including about 200 rotators) to about 1,300 by February 2026.44

Staff cuts started in February 2025 when then-NSF Director Sethuraman Panchanathan ordered the layoffs of 170 workers,45 or about 10% of the agency’s staff, in response to mandates from the Department of Government Efficiency (DOGE).46 In March, 84 of these employees were reinstated in response to a court decision.47 In July, the cuts hit the higher ranks of the NSF, with Senior Executive Service rolls being reduced from 143 to 59. These positions, some of which were already vacant, are for the agency’s highest-level bureaucrats, many of whom are just below political appointees.48

A few of my interviewees were NSF rotators or longer-term personnel who left the agency in 2025. They, and other interviewees, told me about many colleagues at the NSF who were demoralized or disgruntled and left due to the Trump cuts and reforms. When I asked, “how can the NSF be improved?,” a common response was to hire back lost workers or otherwise replace the diminished ranks. One interviewee said, “The people who are still there deserve medals.”

Though the firings and layoffs of NSF personnel leveled out in early 2026, in April, the administration terminated the entire National Science Board, which oversees the NSF on behalf of Congress and the President, presumably to assemble a new board that is more compliant with the administration.

In the NSF’s 2027 budget request, the agency is asking for funding for 1,227 full-time employees, including rotators.49

Rotators

The NSF’s use of “rotators,” or scientists who take temporary full-time positions at the NSF, was one of the most praised aspects of the agency in my interviews. Many interviewees expressed concern that the Trump administration had diminished the role of rotators, and a few even believed that they had been phased out entirely.

As of writing this in September 2026, the rotator programs continue to function at the NSF, and may be getting a boost next year, but rotators have been diminished in number and importance.

At the start of 2025, there were 368 rotators at the NSF. In May, as part of the administration-initiated cuts, the rotator cohort was planned to be cut down to 70 by June. These remaining rotators would then be assigned to newly assembled research “clusters” (a replacement for Divisions) based on the administration’s preferred domains: artificial intelligence, quantum information science, biotechnology, nuclear energy, and translational science.50

As part of this restructuring, rotators were moved out of higher-level Senior Executive Service roles and were prohibited from having direct supervisory power over federal employees.51 This supervisory restriction provides some justification for the rotator cuts as it addressed a compliance issue that predated the second Trump administration. In October 2024, the NSF’s Office of Inspector General flagged the agency’s use of non-federal rotators in supervisory positions as inconsistent with Office of Personnel Management guidance.52

Regardless of the administrative logic, these reforms are bad news for supporters of the NSF rotator program, but I want to stress that it’s not as bad as many people seem to think. Quite a few of my interviewees believed that the rotator program had ended or was being wound down, but it’s more accurate to say that it has been shrunk and modified to play a less important role at the NSF.

For its part, in the official 2027 NSF budget request, the agency is asking to bring the number of rotators back up to 150.53

Leadership

In April 2025, NSF Director Sethuraman Panchanathan resigned after almost five years on the job, and as of September 2026, he has not been replaced, though Jim O’Neill has been nominated as the new director and is awaiting confirmation by the Senate. O’Neill has served in federal administration roles in the Department of Health and Human Services during the second Trump and Bush II administrations, but also has a background in venture capital. He has supported libertarian policies, including deregulation in healthcare.

I asked a few interviewees who had served in senior staff positions at the NSF or NIH what they thought of O’Neill, and most declined to comment, but some offered mild optimism that he would be more open to reforming the status quo of the NSF than most directors.

I spoke to one policy analyst who had met with O’Neill a few times in casual settings and was willing to divulge a bit more insight into his intentions. According to this individual, O’Neill’s priorities included:

On the whole, these priorities align with my recommendations in this report and at least directionally align with the reform priorities of most scientists whom I interviewed.

More Top-Down Research Prioritization

A perennial topic of discussion at federal science funding agencies is the degree to which the government should take the lead in directing research priorities or leave it more to the scientists.

The Trump administration has taken a stronger lead in directing federal research, with the NSF’s new explicitly stated priorities being “artificial intelligence, quantum information science, biotechnology and translational science,”54 all of which are allegedly crucial to the future of the American economy and its national defense. A few of my interviewees suggested these priorities were pushed to keep the U.S. research system ahead of China.

I also spoke with two interviewees who still have many colleagues at the NSF about how these new research priorities have shifted NSF operations. They both claim that the effects on normal operations have been minor and ambiguous; mostly the result has been easier funding for proposals that touch on these topics. However, the NSF has committed to funding these domains with large-scale projects outside of normal grantmaking, such as the $100 million initiative to launch new National Artificial Intelligence Research Institutes.55 Also, the NSF states that it spends $700 million annually on research related to AI, which is a large portion of its budget that cuts across all of its Directorates and normal grantmaking operations.56

On the other hand, the NSF now explicitly does not fund any research associated with DEI or misinformation/disinformation.57 According to interviewees, many articles, and even X threads, this new policy is stringently enforced. According to a report from the Urban Institute, 90% of the 1,500 canceled NSF grants contained DEI language or were related to DEI.58 According to an admittedly anonymous X user, his or her 5-year NSF physics grant was canceled because it contained the word “polarization” regarding light, which set off DEI triggers because of the word’s usage in sociological contexts, which has DEI connotations.59

On May 29, 2026, after I had conducted my interviews and was in the process of writing and editing this report, the Office of Management and Budget (OMB) and dozens of federal grantmaking agencies (including the NSF) released a proposed rule that economist Alex Tabarrok described as “the nationalization of American science.”60

The proposal would rewrite the Regulation for Federal Financial Assistance to give far greater power to the executive to manage and steer federal research funding toward the administration’s priority goals. For instance, federal research agencies like the NSF would have to house a political appointee tasked with reviewing and potentially vetoing discretionary spending, and all federal research grants could be terminated at any time if they were deemed to no longer serve the “national interests.” All federal agencies would be legally bound by this rule and be subordinated to the OMB’s enforcement.61

The purported benefits of this policy would be to streamline the federal research apparatus, cut out DEI and other political interference in nonpartisan research, and direct the might of the federal research ecosystem toward more practical ends. To be very optimistic, centralizing authority in this manner might permit the federal government more latitude to implement the sorts of sweeping reforms that many of my interviewees sought, such as slashing the paperwork burdens on federal grants.

Those benefits may be realized, but the new rule would also constitute the most extreme centralization of American research probably since World War II. The executive branch would have the authority to dramatically steer, curtail, or limit research as it sees fit, and when it comes to the balance between scientific freedom and top-down direction, the rule appears to be the worst-case scenario that many of my interviewees worried about. In the words of Tabarrok:

“A decentralized funding system is inefficient the way markets and federalism are inefficient—we give up some economies of scale and get experimentation, error correction, and robustness in return. A system in which every award advances ‘the President’s policy priorities’ is efficient the way ministries of science are efficient. We know how that experiment ends.

“America is moving in the wrong direction. We should double down on what made America great. Instead, we are adopting all of the loser policies of authoritarian nations.”

Gold Standard Science

In May 2025, the Trump administration issued an executive order on “Restoring Gold Standard Science.” On its face, the order is a noncontroversial attempt to make federally funded scientific output more rigorous to combat the Replication Crisis and restore public faith in science. It mandates that federal research abide by well-established but often neglected scientific standards, such as disclosing conflicts of interest, maintaining skepticism, reporting negative results, etc.62

In August 2025, the NSF announced its Gold Standard Science implementation plan, which mostly states how the NSF already complies with the executive order (for instance, the NSF is transparent in its publication of internal data and grant database), but also presents a vague list of relevant “Plans for Future Activities.” These include scientific integrity courses, putting Gold Standard tenets in grant contracts, and, more interestingly, “[developing] the culture of acceptance of negative or null results as positive outcomes in the research community.”63

So far, the impact of the Gold Standard Science order on the NSF appears minor, and only one interviewee brought it up.

Merit Review Reform

In December 2025, the NSF announced changes to the NSF Merit Review system. Summarizing these changes is difficult because they mostly tweak aspects of the mechanics of the review process, but their broad goal is to streamline the NSF’s grant review to make it faster, require fewer personnel, and give more discretion to Program Officers. For instance, Program Officers were previously required to put grant proposals through three or more outside reviews, but now there is a two-review minimum, and one of the reviews can be internal (conducted at the NSF).64, 65

Unlike most Trump-era reforms, these changes are within the scope of reform that many of my NSF interviewees support. Some critical commenters have cast the reforms as “[lowering] grant requirements”66 at the NSF, but they seem to be well-intentioned efforts to let Program Officers put good proposals through fewer review processes to expedite their approvals.

Restructuring67, 68, 69, 70

In May 2025, NSF employees were reportedly informed that the agency intended to restructure the 37 Divisions within its eight Directorates. Initially, the Divisions were to be replaced by “Clusters” based on a smaller staff and an emphasis on the Trump administration’s research priorities: artificial intelligence, quantum information science, biotechnology, nuclear energy, and translational science. In December, the NSF formally announced an agencywide “organizational realignment.” By early 2026, the restructuring was underway, but in a somewhat different form, with several Directorates replacing their Divisions with units called “Sections,” while the Directorate for STEM Education consolidated four divisions into two “Clusters.”

The administration’s FY2027 budget request proposed more drastic restructuring based on reducing the NSF’s budget from $8.75 billion to approximately $3.96 billion and closing the Directorate for Social, Behavioral and Economic Sciences (SBE). Nature reported on April 3, 2026, that NSF leaders had told employees they intended to dissolve the SBE, with some of its functions being transferred elsewhere. However, as of September 2026, the SBE is still operational, and the full extent of future NSF restructuring remains unclear.

Indirect Costs

In May 2025, in response to Trump administration directives, the NSF announced a policy to cap all future grant indirect costs, or funds paid to institutions by science funding agencies to house and support researchers, at 15% of modified direct costs. Research institutions individually negotiate their indirect cost rates, but the national average is estimated at 53%, and elite institutions have gotten much higher figures.71 In June, a court struck the policy down, and the NSF has reverted to a negotiated indirect cost rate.

I did my best to cover the famously fraught topic of indirect costs in my NIH report. Some people think they are a vital aspect of federal science funding that has nurtured the American research ecosystem into the greatest science-producing force on earth; other people think it is a de facto scam that lets universities buy research with tax dollars; and plenty of people have moderate opinions in between.

I chose not to focus on indirect costs in this paper because I think it would be retreading the same ground, and ultimately, I believe it is impossible to evaluate the full impact and implications of the current indirect cost policy without extensive auditing of universities and experimentation with different indirect cost structures.


Part 2 – Why the National Science Foundation Needs Metascience and Experimentation

I spoke with 31 researchers and scientific observers about their experiences with the National Science Foundation and their suggestions to reform it. Their ideas ran across a huge range of topics, including grant types, peer review methodology, counters to the Replication Crisis, Directorate and Division shuffling, and even complete restructurings of the foundations of American science funding.

While nearly all these suggestions are worth consideration, I kept coming back to the same conclusion that it simply is not possible to comprehensively evaluate them in the status quo system. There is little or no good empirical evidence to determine whether an NSF-style peer review system based on stronger Program Officer discretion funds better science than an old NIH-style peer review system based on paylines. There is also not enough good empirical evidence to suggest that the status quo project-based grant system works better than block grants, that targeting incremental research underperforms high-risk/high-reward research, that federal science funding outperforms philanthropic science funding, etc.

There is not enough experimentation or evidence to confidently determine whether one reform suggestion is better than any other in most cases. Virtually all suggestions from my interviewees were speculative and conjectural. This is why, after months of studying the NSF, I have come to one main conclusion: there are many promising reforms worth considering, but we do not yet know which actually work. Therefore, above all, the NSF should prioritize metascience research and experimentation to identify which reforms are effective.

The Central Question in Metascience

To demonstrate this point, consider the following question: Does the National Science Foundation do its job well? Or to put it another way, does the NSF produce sufficiently valuable research to justify its cost to taxpayers?

When I asked my interviewees whether they thought the NSF was producing good work, nearly all said “yes,” but only on what I consider speculative grounds.

The most common type of evidence that my interviewees or online publications72 pointed to as an assertion of the value of the NSF was anecdotal. Typically, they would say that NSF-funded research helped lead to a scientific discovery, like gravitational waves or the internet. Numerous interviewees even suggested that the NSF should put more effort into advertising these success stories to demonstrate its value to taxpayers and the public.

While these anecdotes may have some marketing value, they have little bearing on making a cost-benefit analysis of NSF funding as a whole. It is plausible that NSF funding has at times led to great scientific breakthroughs while mostly being useless. It is also plausible that these scientific breakthroughs would have been discovered eventually without NSF funding, or possibly even sooner if NSF funding has a distortionary impact on scientific output (as I argued that NIH funding does to some degree in my previous report).

As a brief aside, I find the anecdotal reports of NSF funding success to be sometimes, or even often, misleading. In my experience, proponents of the NSF sometimes try to give the NSF too much credit for particular discoveries by conflating funding research that made a small contribution to a discovery with making the discovery itself.

For instance, the NSF website lists CRISPR as an innovation or discovery “brought to you by the NSF” (though another page says “NSF-supported research has advanced” the technology). In 1987, Japanese researchers first discovered the DNA sequence that would later be identified as CRISPR, primarily French researchers later revealed CRISPR’s functionality, and CRISPR-Cas9, the gene-editing tool, was developed in 2012 by a team led by American researcher Jennifer Doudna and French researcher Emmanuelle Charpentier. The NSF’s claim to have assisted in this development process consists of funding some of Doudna’s earlier work on RNA and then funding some of her CRISPR work after her key discovery. In her crucial 2012 CRISPR-Cas9 paper, Doudna lists her funders and does not mention the NSF.73 The NSF’s claim that “NSF-supported research has advanced one of the sharpest tools in gene technology” is probably technically true, but the agency does not deserve too much credit for CRISPR, and a similar argument could be applied to various degrees to many of the famous discoveries often credited to the NSF.74, 75

Beyond anecdotes, my interviewees sometimes referred to metrics to demonstrate the value of the NSF’s work, like the amount of money the NSF spends on research compared to other science funding organizations, the number of projects it funds, the number of students and postdocs cultivated by NSF money, or the number of citations produced by NSF-funded work. Other interviewees emphasized their personal experiences with the NSF, either as a grant recipient or employee, and spoke about seeing the high quality of the agency’s work first-hand.

These standards of evaluation may have merit, but they are all ultimately conjectural. Most of these metrics are based on inputs without enough consideration of output, and many of these claims are based on, in Bastiat’s terms, the seen without consideration for the unseen.

My conclusion from my interviews and research is that there have been few serious efforts to quantify the value of the NSF’s research to the American economy, society, or government, nor have there been serious attempts to compare this value to alternative science funding institutions or methods, and therefore assessments of its current and past value are highly speculative.

I also recognize that this is not an easy issue. If we want to know whether an ordinary for-profit corporation is functioning, we can look at its profits, and if we want to know whether an ordinary for-profit research institution (like a pharmaceutical company) is functioning, we can look at how often its research results in profitable products relative to the research of its competitors. But the NSF is not a for-profit entity; it is a government agency, and one that is tasked with funding scientific matters that are explicitly distant from profitable uses in commercial markets (“basic science”). There is no readily available method for evaluating the value of its output compared to its costs.

The importance of this issue goes deeper than evaluations of the NSF as a whole. Many interviewees claimed that the NSF would produce better research overall if it funded more high-risk, high-reward proposals. That may be true, but to verify this hypothesis, we would need to run an experiment comparing the outcomes of high-risk, high-reward proposals against the status quo of more incrementalist proposals, and judge which group produced better results over a long period of time. But if we have no way to measure results, then we can’t compare them.

Nearly all reform proposals I have heard for the NSF and NIH depend, at least to some degree, on figuring out how to measure research outcomes.

If we don’t have a good way to measure research outcomes, then discussing NSF reforms puts us in the position of (as one interviewee put it) officials of the Soviet Gosplan, or State Planning Committee. NSF administrators or Congress can implement reforms, but they would be no more than educated guesses, and we would lack feedback loops to determine which ones work and which ones don’t.

When I asked my interviewees how we could or should measure research outcomes at the NSF, I heard an interesting mix of answers that I’ll split into three general responses.

The most common answer I heard, particularly from researchers, was some version of “it can’t be done.” The NSF is devoted to basic science, which, by its nature, is open-ended, unpredictable, and prone to either complete failure or difficult-to-measure success, especially since the value (commercial or otherwise) of basic research projects may not be realized until years or decades after their completion. To quote two of my interviewees: “How do we measure the value of gravitational waves?” and “A probe to Pluto has no market value.”

Two other interviewees evoked a need for “faith” in the value of scientific research, or trust that researchers are competent, well-meaning, and intelligent people who generally do their best to advance scientific progress.

A common addendum to this answer was Goodhart’s Law, or “when a measure becomes a target, it ceases to be a good measure.” In other words, if concrete metric goals are attached to research outcomes, then researchers will target these metrics, which will lead to distortions in the research, and the metrics will no longer serve as good proxies for research quality. I heard a strong consensus that standard researcher quality measurements like citations or the h-index are not well liked by scientists. Multiple interviewees again reminded me that scientists tend to be very smart people, and very smart people will tend to game systems that determine their salaries and prestige.

I don’t discount the “It can’t be done” answer. Measuring research output is extremely complicated, and if there were an easy and effective solution, it probably would have been found already. I trust individual researchers with relevant domain knowledge to be able to read a single research paper and give it a qualitative assessment, but I understand how difficult and complex it is to evaluate dozens or thousands of papers produced by a single NSF division or the NSF as a whole.

But at the same time, we have to consider how self-serving this response might be. If scientific research outcomes can’t be measured, then there is no way to determine which scientists, Program Officers, or science funding agencies are underperforming, and no way to filter them out. I have a general faith and trust that most scientists are smart people trying to do their best, but I also trust in economic incentives, and if the science funding incentive structure doesn’t punish failure, then we will probably have too much failure.

In my opinion, it is plausible that the “it can’t be done” answer is closest to the truth and that any attempts to impose retrospective measurements on NSF-type science will cause worse problems than they solve, but I don’t think the problem has been studied for long or hard enough to make that determination, and the costs of settling for this answer could be very high.

A softer form of the “it can’t be done” answer from interviewees was “it can be done, but only over very long periods of time, and only with qualitative analysis.”

This sort of assessment process could take many forms and structures, but would end up as some type of retroactive peer review. For instance, the NSF could split its ordinary grantmaking budget between the status quo incrementalist grant style and high-risk, high-reward grantmaking, run both systems in parallel for ten years, and then host peer review panels to go over the research outputs of both systems to determine which one performed better by the subjective committee preferences. There is a lot that would need to be filled in to make such an experiment work, but it seems feasible. At the very least, retroactive peer review could be applied to existing Divisions, Directorates, and special funding programs (like EAGER or RAPID) to evaluate their historical performances at relatively little cost.

There is also the critical question of who would do the evaluating. The most obvious choice is to pick expert researchers to evaluate their own fields, but this could potentially breed biases. If a group of physicists (to pick a random domain of science) is selected to do a retrospective peer review evaluation of the Division of Physics in the Directorate for Mathematical and Physical Sciences, they will have an incentive to confirm that the Division is indeed doing amazing work and therefore should probably get a funding allocation increase for the next ten years.

To a few interviewees, I floated the idea of bringing in McKinsey/Boston Consulting Group-type consultants to do these retrospective evaluations, and this was generally not favored (though one interviewee told me that DARPA sometimes brings in these groups). Alternatively, the NSF or another government agency entirely could develop its own internal science review evaluation teams with long-term staff positions designed to provide insulation from the wider scientific funding ecosystem.

Most researcher interviewees thought that there was no satisfying way to measure basic science outcomes, while some thought it could only be done with long time horizons and qualitative analysis. But other interviewees, particularly those interested in metascience and science policy, believed that developing retrospective evaluation metrics was possible and highly advisable. Metascience is still in its infancy as a field, and with the right mixture of science, economics, and statistics, retrospective scientific evaluations could be quantified effectively.

For instance, citations are a brute-force method for evaluating the impact of a paper and are considered highly gameable, but they usually have some value in demonstrating that a paper is influential. The h-index, developed in 2005, was an attempt to augment the evaluative powers of citations by combining a researcher’s publication output with citation counts, and while it is still gameable and blunt, it is not a bad way to quickly gauge a researcher’s prominence. There will always be biases and shortcomings, but more work can be done in this field to develop analytical tools to evaluate researchers and the influence of their papers.

For example, Evidence, which grew out of the NeuroLibre platform, is developing a new model of scientific publishing based on “dynamic papers.” Rather than separating a static article from the data, code, and software that produced its findings, a dynamic paper integrates these materials into an interactive, reproducible web publication. Evidence envisions a “woven literature” in which publications can reuse one another’s data, methods, figures, and computational workflows. If such a platform were adopted at scale and linked to NSF grant records, this infrastructure could give evaluators better evidence about whether particular papers are reproducible, reused, adapted, or extended by later work. These measures would not necessarily directly reveal scientific quality, but they could supplement citation counts and expert review in evaluating NSF programs.

I tried searching beyond my interviewees for research into evaluations of the NSF’s output and didn’t find much.

There is a decent amount of research on the economic impacts of federal research spending as a whole. Matt Clancy of the Institute for Progress has a good overview of the literature on the subject and makes a “reasonable guess” at an average return on investment of $5.50 in GDP-based benefits for every $1 of American R&D spending (public and private), with a marginal value of $2-$5 at current spending levels for government R&D.76 From the sources Clancy cites and beyond, there is a strong consensus in the academic literature that federal research spending is highly productive.

But that’s total federal research spending, which includes not just the NSF, but the NIH, DARPA, the Department of Energy, and many other agencies, and it includes not just basic research funding, but also the other roughly 75% of non-basic federal research spending.77

To try to disaggregate the NSF’s economic impact or research quality assessment, I tried looking at the NSF’s internal documents. Within the NSF, one of the two main sources of internal evaluations is the Committees of Visitors (COV), which consist of groups of experts from a given field who visit the NSF, monitor its operations (mostly its grantmaking in the relevant field), and write a report to assess its functioning. One of my interviewees called these assessments “useless,” and maybe that’s a little strong, but I’m inclined to agree. I read through a few, and they all struck suspiciously similar tones to suggest that the NSF runs very well but could always use a few minor procedural tweaks to improve. Most importantly, the COVs rarely do any retrospective evaluations; they usually only monitor the inputs in the NSF’s science funding process and use mostly conjectural subjective judgments to assess them (nearly always positively).

The NSF’s other main evaluative body is Evaluation and Assessment Capability (EAC), a unit that puts out a few papers each year on aspects of the NSF’s operations. For instance, EAC used to do an annual review of the Merit Review system, though a new report has not come out since 2024.78

While some of these reports have value, they also rarely do any sort of retrospective analysis of the NSF. For instance, the June 2022 “Rotator Study” describes what NSF rotators are, their institutional diversity (research intensive vs. non-research intensive), their demographic diversity (race and ethnicity), and the impact of a rotator rule change on costs to the NSF.79

There are many interesting studies that could be done on NSF rotators: does working as a rotator improve researcher careers and salaries?; do Divisions with more rotators have different research funding priorities or outcomes?; are rotator Program Officers more likely to cultivate cliques than long-term employees by inviting friends as reviewers?; do rotators outperform long-term NSF employees as Program Officers or in other positions?; etc. But instead of attempting to tackle any of these admittedly difficult questions, the study mostly just reports on demographics.

Other EAC reports look at retrospective outcomes, but not in an experimental manner. For instance, the October 2023 “Long-term Educational Outcomes of NSF Research Experiences for Undergraduates (REU) Site participants (2013 and 2014 Cohorts) Fact Sheet” describes the long-term education outcomes of students who went through the REU program, including the percentages of students who received bachelor’s or graduate degrees in STEM fields. However, there is no attempt to look at a control group of comparable students who didn’t attend the program.80

Similarly, the May 2024 “Evaluation of NSF Grants to Broaden Participation in STEM: American Innovation and Competitiveness Act Section 305” looked at the results of numerous STEM Education programs and recorded descriptive outcomes, but used no control group to demonstrate the efficacy of the interventions.81 Both studies demonstrate that giving money to people to study STEM will make them study STEM, but there is no indication that they wouldn’t have studied STEM or would have been worse at STEM without NSF money.

Even if the NSF or some other organization develops a good method for measuring the comparative quality of scientific output across science funding organizations and individual scientific projects, that’s only the beginning of the metascience evaluation process. Evaluations of scientific output need to be measured against specific and broad goals; for instance, how does the NSF’s research stimulate the economy, improve U.S. military capabilities, help the U.S. compete with China, or achieve any other federal government policy goal?

I don’t have models for these evaluation processes, but I’m not sure anyone else does either. That is why my primary aim in this paper is to encourage the NSF and other science funding agencies to invest in metascience, principally the metascience question of how to evaluate the quality of research output. Until we have a good model for this, we won’t have a good way to evaluate reform proposals at the NSF or the value of the NSF as a whole.


Part 3 – Potential NSF Reforms and Experiments

All 31 of my interviewees had suggestions for how to improve the National Science Foundation. I will go through these suggestions and describe their structures, rationales, and relative popularity with my interviewees. Ideally, this should be thought of as a menu for selecting possible experiments that can be set up in a future NSF that is committed to metascience and optimizing the agency’s research output.

Many of these reform proposals are overlapping or build off each other, but I’ll do my best to put them into neat delineations. Also, while all of these proposals have tradeoffs and contrarian views, whether I will discuss the opposition viewpoint depends on the extent of feedback I got from my interviewees. I will break the reform suggestions into eight categories: Grant Funders, Grant Funding Mechanisms, the Bell Labs Model, Grant Funding Overhauls, Bureaucracy, the Replication Crisis, Intellectual Property, and Dismantling the Status Quo Entirely.

Grant Funders

The NSF relies heavily on the work of its Program Officers to review and select grants to fund. The following are reform proposals concerning the power, structure, and responsibility of these Program Officers:

The Program Officer/Peer Review Balance

Both the NSF and NIH use a combination of Program Officers and peer review to make grant funding decisions. The NSF is known to put more power in the hands of Program Officers; the NIH prioritizes peer review scores. While most of my interviewees at both the NSF and NIH favor the NSF system, nearly all interviewees acknowledge that both sides have tradeoffs. For instance, the NSF system generally encourages more ambitious project funding, but bad individual Program Officers have a worse impact on grant funding outcomes than the more consensus-based and committee-based NIH system.

The NSF could set up experiments that test different points along the Program Officer-peer review spectrum. For instance, the NSF could set aside money for a Division that puts funding decisions entirely in the hands of Program Officers who are given complete discretion to assemble as much or as little peer review as possible, or even to not conduct peer review at all. On the other end of the spectrum, the NSF could try an extremely broad form of peer review where they survey as many researchers in a field as possible on a group of proposals and make funding decisions entirely based on averages. The outcomes of these experimental structures and any other along the spectrum could be compared to find best practices.

Portfolios or Venture Capital Models

One of the most popular reforms suggested by my interviewees was to reorient the NSF’s grantmaking system to portfolios, or at least a portfolio-type mindset. Instead of thinking of research projects individually, Program Officers should think of themselves and their Divisions as building a portfolio of investments with various potential synergies.

For instance, part of the reason that NSF grants tend to be too conservative is that there is an expectation that project proposals will achieve their stated aims. Multiple interviewees suggested that Program Officers could be incentivized to fund higher-risk, higher-reward projects if they were given a mandate to have a certain percentage of funded projects fail to meet their stated aims. Portfolios with excessively high project success rates could be rebalanced to prioritize riskier proposals.

On the more radical end, some interviewees suggested embracing an explicit venture capital model of science funding. If sufficient metrics or evaluation processes were developed, Program Officers could compete against one another to hit targets based on the outcomes of the projects they funded, and successful Program Officers and Divisions could be rewarded with higher budgets or even personnel bonuses.

Grant Funder Skill Cultivation

Building off the venture capital model, numerous interviewees posited that the value of a grant funder qua grant funder is underrated. NSF Program Officers consist primarily of scientists (both rotators and long-term personnel), and while scientists tend to have the expertise to evaluate the scientific merits of grants, good scientists don’t necessarily make good selectors of grant proposals, just as good businessmen don’t necessarily make good venture capitalists.

In other words, maybe skill in grant selection should be conceptually decoupled from researcher skill. The NSF could consider creating separate career tracks for scientists who are particularly skilled at picking the right grants to cultivate and reward their abilities, and the NSF could more aggressively filter out great scientists who happen to be bad at picking grants from Program Officer roles. Alternatively, the NSF could look outside of science for potential grant selectors, like McKinsey/Boston Consulting Group-style consultants or even prediction markets.

There could also be useful cultural reforms to these ends. The scientific community and wider culture rightfully celebrate scientists who make brilliant discoveries; maybe there should be more celebration of great science funders just as there has been more recognition in recent decades of great venture capitalists. Everyone in the finance and tech world knows Andreessen Horowitz and Sequoia Capital; if the best grant proposal funders at the NSF and elsewhere achieved higher levels of prestige, it would encourage greater performance. At a smaller level, the NSF could issue certificates, medals, or other forms of recognition to its best Program Officers.

Grant Funding Mechanisms

Grant funders are just one component in the grant funding mechanisms at the NSF. The current system, which has remained largely untouched by Trump administration reforms, primarily funds extramural research through project-based grant proposals from researchers at external institutions (usually universities).

The following proposals are suggestions to reform the mechanisms of how these proposals are chosen:

People Over Projects

The NSF and NIH both have a project-based funding model, wherein researchers send in proposals for specific projects that are evaluated for funding. The identity of the researcher and their background and track record are factors in the evaluation, but the primary factor is the project itself.

One of the most popular reform proposals I heard from both NSF and NIH interviewees is to shift from a project-based funding model to a people-based funding model, wherein researchers apply for funding primarily based on their backgrounds, track records, and general research ideas, rather than specific project proposals.

This is another reform idea designed to encourage more high-risk, high-reward research. One of the main incentives for risk-averse research in the current system is that grant reviewers tend to reward projects with lots of preliminary data and defined goals, neither of which would be included in a person-oriented review process.

Project-based grants also discourage ambitious research and basic research by penalizing research flexibility; researchers are incentivized to pick a project design and see it through even if they realize weaknesses in the design or find uninteresting results. A person-based system would permit more flexibility in project designs.

Furthermore, some researchers have confessed to me that it is an open secret that most of the time when researchers make project-based grant proposals, they will already have done “about 80%” of the project and are actually just getting funding to complete the project and start their next project. Presumably this is more the case in some scientific domains than others (and at the NIH more than the NSF), but a person-oriented proposal system would do away with this charade.

Golden Tickets

Golden Ticket funding mechanisms were very popular among my metascience interviewees. The idea is most associated with a trial run by the Villum Foundation, a Danish nonprofit science funder, in which each grant proposal reviewer was given one “decisive vote” per review round; proposals which received a decisive vote were automatically funded regardless of the scores given by other reviewers. In theory, this incentivized the funding of more novel, interesting, even unpopular, high-risk, high-reward projects, and also streamlined the grant review process. In 2023, the NSF considered a Golden Ticket pilot with a similar structure,82 though other Golden Ticket models could also be considered, such as one that gives Golden Tickets to Program Officers rather than reviewers.

The same metascientists who championed this proposal told me that it had strong opponents in the scientific community, and understandably so. Golden Tickets could empower reviewers or Program Officers to fund more bad projects or more easily support “cliques” by funding friends and allies in their scientific specialties. Over the long run, Golden Tickets could also box new researchers out of NSF money as funding opportunities were more consistently given to established scientists with good track records.

Lotteries

Nearly every former NSF employee (rotator or long-term) I talked to told me that the NSF received so many good grant proposals that after filtering out roughly the bottom 80% of grant applications, it became a “crapshoot” (as two interviewees put it) to choose which grants to fund.

The NSF could try to institute a lottery system to randomly distribute funding to grants that meet a certain threshold in the peer review evaluations or at the Program Officer’s discretion. The threshold could be experimentally raised or lowered to test variable degrees of randomness. If the higher-randomness portfolios outperform the lower-randomness ones, then this indicates that Program Officers are less valuable and important than most people believe (at least at their current capability levels). These experimental results could also inform the amount of structuring needed in the grant review process – if higher-randomness portfolios do well, then the NSF can cut down on the amount of peer review and Program Officer oversight needed, which could cut costs and speed up the grant review process.

ARPAs

ARPAs, or Advanced Research Projects Agencies, are a model of science funding most associated with DARPA, or the Defense Advanced Research Projects Agency, a science funding arm of the Department of Defense. In 2022, the Department of Health and Human Services launched its ARPA-H (ARPA for Health), which coordinates with the NIH but operates independently of it.

ARPAs are federal science funding agencies that outline specific research goals, usually technology development, and then aggressively fund research toward these specific ends. They tend to strongly empower Program Officers and give relatively little deference to peer review, and they tend to have fast turnaround times for grant applications and responses. The idea is to erect a smaller, nimbler science funding agency that operates in parallel to the primary agency to bypass the larger agency’s more burdensome structures to achieve narrower but more time-sensitive goals.

The prime example, DARPA, was central to the development of ARPANET, the precursor to the internet, and DARPA-funded research also contributed to GPS, anti-radar stealth, onion routing, and many forms of cybersecurity. Almost all of my NIH and NSF interviewees who interacted with DARPA had nothing but praise for the agency, particularly its speed and efficient administration.

While an ARPA is probably better suited for the more applied research-oriented NIH, an NSF ARPA could fill a useful role by organizing researchers to pursue specific outcomes or technologies for economic or national interests. If there were an NSF ARPA at the Division or even Directorate level, it could enhance the NSF’s overall responsiveness to research developments and help the NSF translate its research more efficiently to practical ends.

An interesting take from one interviewee is that the popularity of ARPAs is a condemnation of federal science funding models. Science funding agencies like the NIH should have sufficient adaptability and speed built into their research funding structures; the need to build ARPAs to supplement such agencies is evidence that they are too cumbersome.

I think there is some truth to that, but it is also plausible that research funding works best when you have multiple tracks of reactivity. For instance, maybe the NIH’s status quo funding mechanisms are too slow and not adaptable enough, but if all of the NIH were under an ARPA model, that might push it too far away from exploratory, basic science funding, so keeping an ARPA division alongside the status quo could be optimal. Only metascience experimentation can find the right combination.

The Bell Labs Model

I am not quite sure how to categorize this particular cluster of reform suggestions, so I will make a separate section for it.

Across 31 NSF interviews, Bell Labs was brought up by interviewees without my prompting at least a dozen times. The modern Bell Labs descends from a research institution set up by Alexander Graham Bell in the 19th century. Over the course of numerous acquisitions and mergers, Bell Labs emerged in the 1940s as the primary scientific research center of AT&T, and over the course of the next forty years, until the breakup of the state-backed AT&T telephone monopoly in 1984, Bell Labs contributed to the development of some of the most important technologies of the modern era, particularly the transistor and the C programming language. Bell Labs continues its operations to this day (now as a part of Nokia) but has long since passed its peak output and prestige.

From this historical record, a Myth of Bell Labs has emerged. Just as every armchair general wishes they could have led an army during World War II, and every classical artist wishes they had been patronized by the Medicis in Renaissance Florence, every researcher wishes they had worked in Bell Labs at its peak. And three of my interviewees actually did work at Bell Labs.

While Bell Labs is often portrayed as the best place ever to do science, as with all good myths, the exact telling of it is variable. Some of my interviewees described Bell Labs as a space where brilliant researchers were invited, given near-unlimited funding, and set free to explore any questions they wanted in any way they wanted, making Bell Labs the ultimate paradise of laissez-faire scientific exploration and passion. Other interviewees agreed that Bell Labs employed the best researchers, had the best facilities, and permitted plenty of freedom in scientific pursuits, but they emphasized that Bell Labs laid out goals for researchers to pursue, so their exploratory freedom was channeled toward concrete purposes (in telecommunications). These interviewees also described Bell Labs as having an elite scientific administrative staff that provided the perfect balance of support and pressure on its researchers to get the best work out of them.

I also spoke with two other interviewees who had a more mixed perspective on Bell Labs; they attributed much of Bell Labs’ best work to the fact that AT&T had government-backed monopoly power. I also spoke with two interviewees who said that Bell Labs was massively overrated and actually slowed telecommunications development by failing to pursue research that could have resulted in developing fiber optics and the internet decades ahead of time.

The point being that both the myth and reality of Bell Labs are complicated. Books have been written about its history, reality, and legacy, and though that is beyond the scope of this essay, it is worth considering how the Myth of Bell Labs could shape reforms at the NSF.

Put simply: how should a science funding agency like the NSF structure researcher freedom? Should it establish a funding model that gives maximum freedom to researchers to pursue their curiosities? Or should it establish a funding model with clear top-down directives aligned with the national interest, military defense, economic growth, or the most pertinent expected scientific breakthroughs according to leading experts? Or should the research freedom be set somewhere in between? Or should an agency like the NSF try to cultivate a variety of funding structures and mechanisms that complement both higher and lower levels of research freedom?

Just as we need more metascience experiments to determine the optimal funding mechanisms or the balance between Program Officers and study sections, we need more metascience research and discussions on the principles underlying these reform proposals. Many of my interviewees advocated for aligning NSF reforms with a Bell Labs model that promoted researcher freedom, while others advocated for reforms to give researchers more direction and goal-orientation. Only experimentation and empirical evidence will determine which research philosophy is optimal.

Grant Funding Overhauls

The previous section looked at tweaks to the existing NSF grant review structure; this section looks at complete overhauls to the structure. However, it is important to remember that most of these reforms are not mutually exclusive. With the right metascience funding framework, the NSF, NIH, and other federal grantmaking agencies can run concurrent experiments on how grants are funded that have wildly different approaches and likely outcomes.

Bounties

The standard NSF grant solicitation either asks for researchers to submit proposals based on any topic they want within a given sub-domain or asks researchers to submit proposals based on responses to a specific prompt.

A bounty system would invert this structure. Instead of paying researchers to try to find results and answers, the NSF could pose a question and then pay researchers who have completed research that adequately solves a problem, improves science, or finds an answer to a lingering question. For instance, the NSF could issue bounties like:

“The National Science Foundation will award a $500,000 to $3 million prize to the creators of an openly available scientific tool that is successfully used and independently validated by at least 20 research groups and substantially reduces the cost, time, or error rate of a scientific procedure.”

“The National Science Foundation will award a $50 million prize to the first eligible research team to develop and publicly disclose a material that exhibits reproducible superconductivity at room temperature and ordinary atmospheric pressure.”

“The National Science Foundation will award a $10 million prize for a rigorous resolution of the existence-and-smoothness problem for the three-dimensional incompressible Navier–Stokes equations, which describe the motion of fluids such as air and water.”

A bounty system would introduce an interesting twist in the NSF’s research priorities. Basic science is typically open-ended and has unpredictable results, but there are many questions that basic science is trying to answer. A bounty system could incentivize researchers to better pursue useful results, and since the actual research would likely be funded by non-NSF sources, like private philanthropy or even for-profit companies, the NSF could steer the broader scientific community toward its research priorities (whether or not that is a useful goal is a matter for debate).

NSF bounties could also introduce a new element of prestige into the scientific community. Successfully claiming bounties could not only be extremely financially beneficial, but a great status marker for researchers.

Federal science agencies, including the NSF, have put out a few bounties over the years. For instance, the Department of Energy’s 2022 American-Made Solar Forecasting Prize evaluated competitors’ forecasts against a benchmark, and the NSF co-ran a bounty with the National Institute of Standards and Technology in 2022–2023 to develop privacy-enhancing technologies in which independent teams tested working prototypes for privacy vulnerabilities.

Self-Organized Funding Allocation

Two interviewees suggested a concept called “Self-Organized Funding Allocation” (SOFA) as an alternative to the status quo system, though I could not find any examples of significant science funders using the model.

A SOFA system can operate in many different ways, but the basic concept is to assemble a relatively large group of researchers in a given field, assign them an equal and automatic sum of money each year, and then mandate that every researcher give away a portion of that sum to other researchers in their pool. So every researcher would keep some money for themselves for their own research and then decide who else to fund in their field based on their own criteria.

In theory, this model would create a more organic, adaptable, and autonomous funding process that is highly responsive to scientific advances. The NSF or whatever science funder administered the grants could offload most of its bureaucracy since it would just have to assemble the researcher pool and then distribute the funds, and then the researchers could rely upon their decentralized knowledge and professional networks to distribute the money, and hopefully scientists would shift funding annually based on which researchers, proposals, and research strands produced the best output. This could create a more collegial system that circumvents the Grant Treadmill and shifts the balance of power away from agency administrators, university administrations, and political forces, and back toward researchers.

As always, there are tradeoffs to such proposals. A SOFA system would risk inverting the risks of empowering Program Officers by empowering the equivalent of peer reviewers, or rather, by creating the equivalent of dozens of Program Officers. In the worst-case scenario, this could lead to the sort of groupthink and cliquish dynamics that detractors claim plague the NIH grant review process.

Block Grants

Block grants are simultaneously one of the most plausible overhaul reforms for the NSF and one of the most controversial. Many interviewees were enthusiastic supporters of block grants and thought they could place the entire federal science funding system on the right track, while other interviewees thought that implementing system-wide block grants would be a disaster.

Block grants are grants that are given to institutions with broad discretion for how they use the funds, as opposed to the typical NSF or NIH grants today, which are given to institutions but earmarked for use by specific researchers. The NSF and NIH both have block grant programs, mostly in the form of cultivating large-scale research labs at institutions, such as the NSF’s AI Materials Institute at Cornell University. But proponents support expanding the block grant funding mechanism to replace much or all of the standard individual grants so that it becomes the basis of federal science funding.

As with many of these other proposals, there are many ways a broad block grant system could operate. A good, large-scale example would be for the NSF to cease its current grant application model and instead solicit research funding proposals from research institutions, like universities and private labs. These institutions could put together proposals to demonstrate their research capabilities in terms of facilities, personnel, administrative support, track record, and any other relevant criteria. The NSF could then distribute the funds to these institutions on a long-term basis, maybe five or even ten years, with guaranteed annual outlays, and it would be up to the institutions to decide how to spend or distribute the funds to their researchers. At the end of the funding periods, the NSF could do a large-scale review of scientific output and then adjust funding accordingly for the next period, thereby rewarding successful institutions and punishing unsuccessful ones to create competition in the research ecosystem.

I can see a lot of advantages in this model. In essence, it would offload most of the bureaucracy at the NSF (or NIH) to the research institutions, and at least theoretically, the institutions would have better localized knowledge for making grant-funding decisions. And with the long time frames and set-payment structure, the research institutions would be incentivized to optimize their research output in terms of quantity, quality, and costs. In a sense, this would turn the entire federal funding ecosystem into a market-like structure, and this would incentivize the players in the system to do metascience experimentation to improve the quality of their research output to increase their federal funding. Maybe Harvard would use a lottery-based grantmaking system, MIT would shift all grantmaking authority to specially chosen Program Officers, Stanford would use a venture capital grantmaking model, etc., and over the course of decades, winning models would emerge. There’s a lot of theory and ifs in this setup, but also lots of potential upside.

Detractors argue that a larger block grant system would create a giant host of new bad incentives and problems in federal science funding. The internal politics of the NSF (which are minor) or the NIH (which are considerable) would mostly be replaced by the often cutthroat politics of universities, which also have their own notoriously bloated bureaucracies. Therefore, it’s plausible that whatever research funding distribution systems they set up would be less effective than the comparatively aloof and professional systems in the federal government. Additionally, with federal money more directly given to institutions and in larger sums than under the current indirect cost system, universities would have a strong incentive to lobby for unfair payment distributions. This could extend all the way to Congress where Representatives from districts with large research institution concentrations would have a strong incentive to meddle in funding affairs.

Even with these valid counterarguments, most block grant proponents and opponents agree that there is some role for block grants in the federal science funding system, with the NSF-funded large research projects being particularly popular.

X-Labs and Bell Labs X

X-Labs and Bell Labs X are two intriguing forms of block grant overhauls of the NSF.

X-Labs (which was previously known as “Tech Labs”) is an initiative of the NSF Directorate for Technology, Innovation and Partnerships (TIP), first announced in December 2025. The idea is for the NSF to fund the creation of numerous new research institutions across the United States outside the bounds of traditional research organizations like universities and for-profit companies. In May 2026, the NSF announced that it would spend $1.5 billion “over the next decade” on X-Labs, and the first round of funding would focus on “Scientific Instrumentation for Sensing and Imaging” and “Quantum Systems: Interconnects and Integrated Photonics.”83

Bell Labs X is a proposal by Jeff Tsao, a former senior scientist at Sandia Labs (a research institute of the Department of Energy), for the federal government to co-finance the launch of new research institutions at for-profit corporations, with the goal of creating new “Bell Labs” across the country. Tsao’s proposal floats an example of setting up a corporate lab with a $120 million budget, with the federal government paying $100 million and the for-profit company paying $20 million.84

Both X-Labs and Bell Labs X could cultivate a market-oriented competitive science research field that could not only produce good scientific output but also serve as laboratories for metascience designs.

Furthermore, these proposals consider the complicated variables at play in designing research institutions. Universities, independent labs, and corporate labs all have advantages and disadvantages in their models; for instance, corporate labs have good incentives for developing IP and ruthlessly filtering out low-quality researchers, but they have weak incentives for cultivating longer-term, open-ended basic research, while universities generally have the opposite incentive structure. Proposals like X-Labs and Bell Labs X shift the variables of profit orientation, short-term vs. long-term thinking, personnel types, and other factors to create new institutional alchemies. This is the sort of experimentation we need in federal science funding.

Intramural Programs

Over 80% of NIH funding is directed toward extramural grants, while about 10% goes to the intramural program consisting of the NIH’s own labs and researchers. The NSF does not have an intramural program, but it could develop its own.

During my NIH project, the intramural program got rave reviews, with many researchers calling it the ideal open-ended research environment. During my NSF interviews, whenever I floated the idea of an NSF intramural system, I got cooler responses. In part, the NSF’s varied domains make it more difficult to build a large-scale intramural program, whereas the NIH is solely focused on bioscience. Some interviewees argued that the NSF’s large-scale institutional projects like LIGO already fulfill the role of an intramural program.

Bureaucracy

“Bureaucracy” is a vague term that I will use to group NSF reform proposals concerning the administrative requirements put on researchers applying for NSF funding and the structure of staffing at the NSF.

Cut Grant Proposal Size and Maintenance

I am pretty sure that every interviewee across my NIH and NSF projects agreed that their federal science agencies should reduce the amount of paperwork involved in grant applications and maintenance.

The average NSF grant takes about 120 hours to write,85 and some interviewees claimed that NIH grants take even longer. Once grants are awarded, recipients are usually required to update the NSF annually on their progress. My interviewees generally found both of these processes to be overly convoluted, annoying, and time-consuming distractions from their actual scientific work. Additionally, the bureaucratic burden of these processes has contributed to the tremendous administrative bloat at modern universities and the associated indirect cost conundrum.

While nearly all of my interviewees wanted this paperwork to be cut down, most were aware of the complexity of such cuts. It is the nature of bureaucracy that most individual compliance requirements are reasonable, but the accumulated mass of requirements becomes unbearable. The NSF wants grant applicants to submit conflict of interest disclosures, collaborator spreadsheets, data management and sharing plans, mentoring plans for how to guide their graduate student and postdoc workers, fieldwork safety plans, and human and animal testing subject protocols, and the NSF wants its existing grantees to submit annual reports, final reports, outcome reports, updates to researcher disclosures, notifications on research pathway changes, ethics training records, and many other forms, papers, and spreadsheets that have completely reasonable justifications for a federal agency financed by tax dollars, but nonetheless can drive an honest scientist to madness.

Whether by Congressional mandate or administrative fiat, ideally the NSF could put together a committee to go through the status quo grant application and administrative forms, and slash them down to a more manageable level.

Reducing Funding Response Times

In 2024, according to the NSF’s own figures, the average wait time from NSF proposal submission to award decision was 5.87 months, which had been steady since at least 2021, though the average time jumped to 6.49 months in 2025, likely due to the Trump administration cuts and reforms. The NSF says that it strives to keep the figure below six months.86 After award decisions are made, it takes another few weeks to a month to confirm the award after it goes through further NSF review.

For comparison, the NIH estimates the time gap between applying for grants and receiving them at 9-10 months,87 with high variance outside that range presumably being a product of the NIH’s many institutes and funding programs.

Comparing federal science funding to nonprofit science funding is difficult because the former has more oversight requirements since it uses taxpayer dollars, and the nonprofits tend to use simpler funding mechanisms than the open solicitations for project proposals favored by the NSF and NIH. But with these caveats in mind:

Based on these examples, the NSF is not at the longer end of the grant application turnaround timelines. However, it is almost certainly useful for the NSF to experiment with different grant evaluation timings to discover what is optimal for both the researchers trapped in the Grant Treadmill and scientific progress as a whole. It might be best for the NSF (and NIH) to establish different grant programs with variable levels of grant application timings to balance agency adaptability and grant review rigor.

Hire More NSF Staff

The NSF takes pride in running a lean operation; in 2024, only 5% of the agency’s costs went to administrative overhead, while nearly all the rest of the funding went to research or STEM education.

A few interviewees suggested that the NSF’s staff was actually too lean, and efficiency could be gained by hiring more administrators; for instance, more staff could cut grant application response times. If true, this issue has been exacerbated by the Trump-era cuts to the NSF staff, particularly to the rotator cohort and higher-level administrators.

The Replication Crisis

The basis of the scientific method is that we can discover causal trends by designing experiments that produce the same results under the same conditions. Generally, if a scientific experiment’s result does not replicate, then its usefulness is limited.

Starting in the early 2010s, many scientists and observers have claimed that the American and perhaps global scientific community is in a Replication Crisis, with a substantial number of research papers failing to replicate. For instance, a 2015 paper found that only 36% of papers from top psychology journals replicated,94 and over ten years later, a 2026 SCORE paper estimated that about half of all social science papers do not sufficiently replicate.95

This phenomenon exists across scientific domains. The causes of these replication failures include statistical manipulation (like p-hacking), publication bias (particularly an aversion to publishing null results), cultural issues, and fraud.

The current and past presidential administrations have taken some minor steps to address the Replication Crisis. In 2022, the Biden administration issued the Nelson Memo,96 which ordered research agencies to end embargoes on federally funded research papers, a move that was enthusiastically welcomed by open-science advocates. The second Trump administration’s Gold Standard Science and its subsequent enforcement directives are also seen as moves to tighten up scientific standards. And in February 2026, the NIH launched the Strengthening Replication and Reproducibility of NIH-funded Research program.

At the NSF, there have been plenty of discussions of the Replication Crisis but relatively little effort to implement policies to address it. In 2019, the NSF helped fund a consensus report on the crisis at the National Academy of Sciences;97 in 2020 and 2022, the NSF issued Dear Colleague letters aimed at encouraging science quality and replicability;98, 99 in 2024, the NSF launched the Cyberinfrastructure for Public Access and Open Science program to encourage the sharing of research results to promote higher quality; etc.100

I asked most of my interviewees what they thought of the Replication Crisis and whether there was anything a federal science funding agency like the NSF could or should do about it, and I received many complex answers across a wide range of perspectives.

Broadly speaking, most of my NSF researcher interviewees downplayed the Replication Crisis. They did not deny that it was happening or that it was a problem, but they emphasized the complexity of the scientific process and suggested that the more extreme critics of the Crisis misunderstood the nature of replication. Even in harder sciences where replication is easier, research projects often concern subjects with so many variables that replication is not a reasonable expectation. I think one interviewee put it best by explaining that in many scientific domains, researchers aim for a paper to be directionally correct rather than strictly replicable. These interviewees argued that while the Replication Crisis is concerning, it should not be considered damning for most modern scientific output.

Again broadly speaking, another substantial portion of my interviewees, mostly non-researchers, were more cynical about the Replication Crisis. They claimed that it indicated widespread low-quality output from the scientific community, which called into question the entire science funding ecosystem. One of my interviewees previously worked in the NIH’s extramural department for many years, and I asked this individual: in a given year while you worked at the NIH, what percentage of extramural grants qualified as what you would consider “good science”? My interviewee’s answer was 10% to 15%. A minority of interviewees likewise expressed dismal views of the value of most scientific papers produced in a given recent year.

Determining the extent of the Replication Crisis and what it says about modern scientific output is beyond the purview of this essay, but I will relate a few suggestions from interviewees for how to ameliorate replication failure and low-quality research. However, most interviewees emphasized that the Replication Crisis was a product of both policy and culture, and that a science funding body like the NSF could only marginally improve matters.

A popular suggestion was for the NSF or NIH to set aside a portion of its budget to finance replication efforts, with priority given to trying to replicate high-impact papers. This proposal has gotten a lot of attention over the past few years, and there is a lively debate on how much money should be devoted to such efforts. Secretary of Health and Human Services Robert F. Kennedy Jr. has floated the idea of directing 20% of the NIH’s budget to replications.101 In a 2025 post for the Institute for Progress, Jordan Dworkin applied some metascience and econometric analysis to the value of replications and concluded that the NIH should set its replication budget to 1.4%.102 At the Good Science Project, Stuart Buck suggests that Dworkin is undervaluing the impact of large-scale replication efforts because of the dynamic effect they can have by encouraging researchers to avoid publishing papers that fail replication.103

Many interviewees suggested that science funding organizations like the NSF should establish explicit protocols for punishing researchers, at least for cases of known fraud, but also possibly for engaging in p-hacking or other methods to publish low-value results. Punishments could come in the form of blocking access to future funding activities or even making public pronouncements to shame low-quality researchers.

Another suggestion was to create more stringent expectations for how research projects are conducted and reported. All NSF projects could be required to have predefined experimental conditions, with no or only limited allowances for changing designs, and researchers could be required to affirm this continuity in their update reports to the NSF. This would ensure that more projects properly end with and report null results. From the other end of the problem, one researcher suggested that the NSF should hire an “army of statisticians” to crawl through the results of every paper and root out fraud and statistical malfeasance.

One of the issues that all of these proposals wrestle with is the interaction between science policy and culture. Some interviewees went as far as to say that there are no policies that will sufficiently subdue the Replication Crisis; only a shift in the culture of science will suppress the incentives underlying the crisis. One interviewee asserted that the fundamental cause of the Replication Crisis was a lack of moral virtue and courage among researchers.

Some interesting suggestions from interviewees were to use science policy to adjust science culture. For instance, one of the drivers of the Replication Crisis is that scientists gain prestige (and subsequently, higher salaries and better jobs) by publishing eye-catching affirmative experimental results; this has the inadvertent effect of discouraging the publishing of null results, which incentivizes researchers to engage in statistical trickery or redesign experimental parameters to find positive results.

Perhaps the NSF, NIH, or other science institutions could conceive of policies that boost the prestige of papers that find null results. For instance, the NSF could offer bounties for well-constructed, interesting research projects that ended with null results. Or prestigious journals like Nature could set aside a portion of their publication pages for high-profile null results.

Science institutions could also boost the prestige associated with replications. An interesting idea floated by two interviewees was to encourage the creation of superstar teams of replication challengers, either at the research institution level or through science funding mechanisms. In the world of finance, there are famous short-selling firms like Hindenburg Research (which is now defunct) and Kynikos Associates that direct public attention to major companies suspected of fraud or excessive detachment from core economic metrics; the scientific community could produce similar teams of replication challengers that publicly attempt to verify the scientific soundness of high-profile papers.

As a final note, it is an open question whether the NSF or other science funding agencies should try to tackle the Replication Crisis. One of my interviewees reported that he had discussed pro-replication policies with multiple high-ranking officials in federal bodies, and their consensus view was that it is not the responsibility of the government to “police” scientists. It is plausible that efforts to promote replication and reduce fraud could actually hinder scientific output by creating overly restrictive experimental parameters, for instance, by locking researchers into experimental designs that they realize will not work partway through their grant projects.

Intellectual Property

The NSF and NIH spend billions of dollars annually funding science conducted at non-governmental institutions, primarily universities. This research sometimes leads to the development of intellectual property (IP) with commercial value; the 1980 Bayh-Dole Act generally permits institutions to retain IP developed by government-funded research. In the context of the NSF, this system incentivizes universities, research nonprofits, and corporate partners to use federal money to develop commercially viable products for their benefit in a quasi-market structure.

A few interviewees suggested tweaking or overhauling the Bayh-Dole Act. One policy analyst interviewee argued that universities in particular have historically underutilized the IP potential of their federally funded research, and therefore are not producing optimal benefits to the American economy. The interviewee suggested that the federal government could alter the Bayh-Dole Act to give a (new or existing) federal agency the discretionary authority to develop or acquire IP derived from federally funded research, and then such IP could be released to the public or licensed out.

A more radical idea floated by an interviewee was to repeal the Bayh-Dole Act entirely and replace it with new legislation that gives the federal government ownership of IP produced by research it funds at private institutions. The government could then license out the IP, perhaps with a first-buyer option to the research institute that developed it, and use the proceeds to fund further federal research.

An even more radical idea floated by one interviewee and considered by another was to repeal Bayh-Dole and incorporate the federal IP gains into a venture capital-style science funding system. If the individual NSF Directorates and NIH Institutes could keep the IP their funding generated and license it out for revenue generation, then they would have a strong incentive to fund good research.

Dismantling the Status Quo Entirely

Dr. J. Scott Turner is a physiologist and evolutionary biologist who retired in 2019 after decades of research (including grants from the NSF, NIH, U.S. Army Research Office, etc.) and teaching work, primarily at the SUNY College of Environmental Science and Forestry. I asked for and was granted an interview with Dr. Turner after reading his recent publication, Rescuing Science: Recovering Science as a Civic Virtue,104 at the National Association of Scholars, and he offered me permission to use his name in this report.

Rescuing Science offers a comprehensive and fascinating criticism of the NSF, NIH, and federal science funding that argues for a dismantling of the status quo system and a return to a “Small Science Ecosystem” based on private funding largely separated from the government and academia. By Dr. Turner’s account, before World War II, the American science ecosystem was highly decentralized, diversified, and successful in producing enormous scientific breakthroughs. But in the middle of the 20th century, basic science research largely fell under the purview of federal science funders, particularly the NSF and NIH, which established the current system wherein federal tax dollars flow from government agencies primarily to universities that employ researchers.

According to Dr. Turner, inadvertent flaws in this system created a web of bad incentives that underlie the major faults in modern science funding. Namely, research institutions (primarily universities) became financial beneficiaries of the government, while researchers became mere assets to be utilized by the institutions for financial gain. This perverse relationship led to the development of the Grant Treadmill, hyper-competitiveness between researchers, the utilization of large pools of graduate students as cheap labor, indirect costs and administrative bloat, the Replication Crisis, and political and ideological manipulation of science policy, among other commonly maligned science funding trends that have made research a challenging career path often far removed from the curiosity-driven appeal of science.

On Dr. Turner’s account, which was heavily influenced by a 2020 paper from Stephen Turner and Daryl Chubin on the development of scientific culture,105 the current federal science structure has fundamentally eroded the ambition and spirit of scientists, to be supplanted by careerism and bureaucracy navigation, or a transition from an “ethic of discovery” to an “ethic of production.” In other words, most researchers are more focused on their careers than scientific exploration, and most science institutions are focused on their “endless quest for money,” and the quality of science has suffered as a result. Dr. Turner argues that to reverse the ethic drift, major policy changes are needed to fix science funding incentives, and only then will the culture of science shift back to one of civic virtue.

The vast majority of my interviewees would disagree with Dr. Turner’s conclusion that basic federal science funding would ideally be wound down, or at least massively overhauled. But Rescuing Science might be the best-written explanation I have seen for the systemic flaws in the current science funding system, and I think that most researchers and science funding administrators would agree with many of his criticisms, if not necessarily to the same extent.

It is notable that Dr. Turner is not alone in his take. A handful of other interviewees expressed similarly dire views of the scientific status quo, whether concerning the average researcher’s incentive structure or the low quality of most scientific output. I am still shocked that a former high-level, long-time NIH employee estimated that 85%-90% of all NIH extramural grants in a given year are bad science.

As much as I respected all of my interviewees, I think there was plenty of status quo bias in their opinions; basic science funding has been dominated by a federally backed model for decades, and most researchers have not given serious consideration to fundamental alternatives. Whether it be Dr. Turner’s vision of a return to a “Small Science Ecosystem,” or more aggressively market-based private science funding alternatives, or greater pushes for philanthropic science funding, there are interesting alternatives to the federal basic science funding system that are worth consideration.

AI Addendum

In February 2026, Matt Yglesias wrote a blog post about how it is difficult to do public policy analysis these days because AI might render all current analysis entirely outdated within the next few years: “Questions about basically every medium-run policy debate collapse into arguments about the future trajectory of A.I.”106

This same conundrum applies to this project. Any current suggestions for reforming the NSF or implementing metascience policies may be fundamentally altered or made irrelevant by the impacts of AI. For instance, the shortcomings of citations or the h-index as a measure of researcher impact may be revolutionized by AI tools that analyze hundreds of thousands of research reports and feed sophisticated models that can accurately gauge the relative impact of science papers better than any mere human model. Or maybe most discussions of grant models and Program Officers can eventually be thrown out in favor of feeding all grant proposals to an AI that selects the best applications based on sophisticated machine-learning-based algorithms.

AI occasionally came up in my interviews, but mostly in relation to the Trump administration pushing for more AI research. I don’t have enough AI expertise, nor did I pursue the issue enough in my project to predict the many ways that AI will impact science funding, but it could make the whole sphere unrecognizable within a decade or two. At the very least, researchers and science funders should be open to the many opportunities for efficiency gains offered by using AI.


Part 4 – How the NSF Can Implement Metascience

The National Science Foundation should do more metascience experimentation on its science funding mechanisms and evaluative processes. While Part 3 listed policies with which the NSF could consider experimenting, Part 4 will discuss some possibilities for how the NSF could develop its new Office of Metascience and use it to test funding reforms.

Existing Experimentation at the NSF

I have criticized the NSF for failing to sufficiently engage in metascience experimentation, resulting in an agency that has changed relatively little for decades. However, I want to acknowledge that the NSF, or at least parts of the organization, does periodically engage in experimentation.

In most instances, these experiments take the form of pilot programs. For example, in the early to mid-2010s, the NSF launched eight pilot programs testing alterations to peer review, including virtual panels, asynchronous reviewer discussions, and anonymous electronic polling for panel members.107 Parts of the NSF have also tried eliminating deadlines for grant applications, including three programs in the Division of Chemistry since 2022,108 many of the funding opportunities in the Directorate for Biological Sciences since 2018,109, 110 and multiple divisions in the Directorate for Engineering also since 2018.111 Additionally, nearly all of my interviewees who worked at the NSF reported that experimental suggestions or pilots were commonplace on small scales.

The historical picture that emerged from my research and interviews was of an NSF that gave Program Officers and other employees considerable latitude to try experiments, but offered limited support for systematically designing, evaluating, and publishing them. Many pilots were small and decentralized: of the eight peer review pilots in the early to mid-2010s, only one was NSF-wide, while the rest were confined to individual Divisions or Directorates. Interviewees also described substantial siloing across the agency. In my assessment, published evidence connecting these experiments to subsequent scientific outcomes remained limited. These are problems that a dedicated metascience office could help address.

Fortunately, some developments over the last few years indicate that the NSF has begun to put a greater emphasis on experimentation. In 2023, the NSF announced a partnership with the Institute for Progress, a progress studies think tank, explicitly “to test new mechanisms for funding research and innovation,”112 though, as of September 2026, little information about the outputs of this partnership has been released publicly. Additionally, in December 2025, the Directorate for Technology, Innovation and Partnerships (TIP) announced that, at the behest of Congress, it would “rigorously experiment in the proposal and award process as part of its new initiatives.” The first public sign of this experimentation in action was revealed in the Trump administration’s Science: A New Golden Age report, which states that TIP has begun Golden Ticket experimentation (though I heard an anecdote from an interviewee that the experiment was abandoned).

On September 10, 2026, the NSF announced its new Office of Metascience, which is intended to evaluate the effectiveness of new initiatives and help the agency expand successful approaches and discontinue unsuccessful ones. Later that month, Jason Gallo appeared publicly as the office’s newly appointed director, with the office housed within the NSF’s Office of the Director. This is a welcome development that addresses one of this essay’s central recommendations. The remaining questions concern its resources, authority, methods for evaluating scientific outcomes, and commitment to publishing and acting on its findings. The proposals below should therefore be read as suggestions for developing the new office and extending metascience efforts across federal science funding.113, 114

Do Scientists Want Metascience Experimentation?

I asked many of my interviewees how they or their colleagues felt about experimenting with federal science funding mechanisms or other potential metascience programs, and I heard an interesting mix of answers.

Most interviewees responded that young scientists tend to be eager for metascience. As scientists, they recognize the value of experimentation as a means of optimizing a process, so they understand that experimenting with science funding is the best way to improve science as a whole. Additionally, younger scientists tend to bear the brunt of the worst parts of the status quo science funding system, and favor reforms to get more and new opportunities.

However, both from interviewee testimonies and my own observations of interviewees, I noticed that older, more established researchers tended to be less sanguine about metascience experiments. There are many complaints about the Grant Treadmill, excessive research competition, funding projects over people, etc., but there are still winners who come out on top of this system and have strong careers with decades of ample funding. To paraphrase one interviewee, “some of them think that if they’ve been rewarded by the system, then it can’t be that bad.”

But on the whole, my interviewees, even the older and more established ones, tended to support metascience experimentation. I wish they were even more enthusiastic given the potential benefits of metascience, but I believe that if significant metascience reform were to be instituted at the NSF and NIH, and the status quo funding mechanisms were greatly disrupted or overhauled, most researchers today would support the process as long as it was built on a solid foundation and overall funding levels did not decline.

Potential Metascience Reforms at the NSF

Just as the process of figuring out how to reform the NSF to increase the value of its research output will require much experimentation, the process of reforming the NSF to figure out how to best reform the NSF will also require much experimentation. I asked most of my interviewees how we can approach implementing experimentation and metascience at the NSF and in federal science funding more generally, and the following are some of their considerations:

Copy Successful Metascience Structures at Other Science Funding Organizations

The NSF and NIH are lagging behind other science funding organizations in metascience reform. These federal bodies could reach out to these groups to learn from their structures and trials to refine their own approaches. For instance, major philanthropic funders like Coefficient Giving and the Gates Foundation have some publicly available but mostly private data on their grantmaking processes, results, and experiments.

Or more comparably, the American government could look to efforts in other governments for guidance, though, with one exception, the efforts so far have been paltry. In 2024, the United Kingdom created a Metascience Unit with a £10 million, three-year budget (later increased to £49 million over three years) to oversee metascience development in UK Research and Innovation. In June 2025, the Metascience Unit released an annual report on the first year of its efforts, and it looks like it is doing exactly the type of things that my metascience interviewees and I would want a government metascience body to do, including:115

Aside from the UK, there have been some minor metascience efforts in other governments. For example:

The Question of Scale

Nearly all of my interviewees agreed that the NSF would benefit from a group dedicated to running metascience experiments. The new Office of Metascience provides an organizational home for this work, but its creation leaves important questions about the scale of the effort, the resources devoted to it, and its relationship with the rest of the agency.

Some interviewees suggested that each NSF Directorate should have its own metascience group. This could allow experimentation suited to the structures and needs of different sciences, like engineering or biology. Such groups could complement a central office by helping develop and conduct experiments within their fields.

Most interviewees, however, emphasized the limitations of small, decentralized experiments and favored stronger central coordination. The NSF’s decision to place the new office within the Office of the Director is consistent with this preference. The question now is whether it will have the authority and resources to coordinate experiments across Directorates, support external metascience research, and help implement findings across the agency.

Finally, some interviewees favored a metascience group or agency outside the NSF entirely. A federal body could design, coordinate, and oversee experiments across science funding agencies, including the NSF, NIH, the Department of Energy, and DARPA. The creation of an NSF office does not settle whether this broader federal effort would also be valuable.

Carve Out Appropriations for Metascience

The NSF and NIH are funded by Congressional appropriations; a portion of their funding should be earmarked for metascience. Ideally, this sum would be added to existing budgets, but considering the long-term value of improving scientific funding mechanisms, even taking funding out of the existing funding pool will likely benefit federal science funding agencies.

It is also worth considering that metascience is fairly cheap as far as science goes. Metascience projects usually don’t require hadron colliders or Antarctic research stations; they just require smart people, statistical modeling, and usually a lot of time. Metascience could provide an incredible bang for its buck for federal science funders.

Encourage a Culture of Experimentation at the NSF

While many interviewees described a good experimental spirit at the NSF, the agency could do more to cultivate and incentivize interesting experimental projects. The new Office of Metascience could solicit suggestions from NSF personnel and grant recipients for experimental designs. Ideally, proponents of particular reform measures, like Golden Tickets, could gain funding and organizational support to design and run science funding experiments.

External Metascience Partnerships

While my policy-oriented interviewees and I agreed that the federal government should be developing its own metascience groups, federal science funding agencies should also explore partnerships with external metascientists. This can come in the forms of initiatives like the one recently launched between the NSF and the Institute for Progress, extramural funding of metascience work, or inviting external groups into the NSF to conduct metascience experiments like pilot programs.120

On the crucial question of how to evaluate the quality of scientific research, I think the NSF should explore potential partnerships with groups outside of science or metascience, like consulting or financial firms. Venture capital firms could also be a source of insight into how to pursue more efficient, portfolio-based science funding systems.

Data Sharing

One of the easiest metascience reforms the NSF or NIH could implement would be to collect and publish its vast troves of metascience data.

Over decades, these federal research agencies have received millions of research proposals, funded hundreds of thousands of research projects, and collected millions of reports on research progress and results. All of this is metascience data. All of this could be studied and combed for trends in science funding. While the NSF and NIH currently publish broad statistical reports on their science funding activity, like how many proposals are funded by each Directorate or Institute, funding rates, demographic data, etc., this enormous stockpile of data has not been adequately distributed and investigated for metascience purposes.

Besides a lack of current political will, there are obstacles to sharing this data, including confidentiality concerns and proprietary and trade-secret information. Given that the NSF and NIH are federal agencies, there are a host of privacy and clearance regulations governing the collecting and publishing of their federally funded research. Hopefully, Congress or scientific administrators could alter these regulations, construct frameworks with adequate protections and carve-outs, or at least provide temporary clearances to dedicated metascience researchers who jump through the right hoops to get permission to access this crucial data.

Who Do We Need to Convince to Do More Metascience at the NSF?

I think the National Science Foundation should invest far more of its resources into metascience, and I have laid out numerous reforms worth testing and some structural ways to do so. However, the federal government is a notoriously bureaucratic entity by design, and reform is no simple matter. So I asked numerous policy-oriented interviewees, “who do we need to convince in the federal government to get federal science funding agencies to do more metascience?”

The answers were all variations of, “it’s very complicated.” As with any other federal agency, different aspects of the NSF are controlled by different entities, both directly and indirectly, and we would ideally want an alignment of numerous entities within the federal government to implement a large-scale metascience process at the NSF or elsewhere.

Sustained support from the White House will be important for developing these metascience efforts. The President’s Office of Science and Technology Policy advises the President on federal science policy, and its director (currently Michael Kratsios) is considered to be the chief scientific advisor to the president. Additionally, the President himself has considerable power to impact federal science policy by setting policy agendas and issuing executive orders, as we have seen with President Donald Trump’s initiation of Gold Standard Science, elimination of DEI, and proposed cuts to the NSF.

The next most important decisionmaker is probably Congress. Legislative champions of science could propose higher budgets for the NSF and other federal science funding agencies, and propose earmarking funds for specific metascience endeavors.

Next, we need the leaders of the federal science funding agencies to be on board with metascience reforms. Energetic leadership could go a long way to quickly and effectively implementing metascience reforms proposed and funded elsewhere.

Finally, the non-leadership personnel of the NSF, NIH, and other science funding agencies need to be enthusiastic about enacting metascience reforms for them to be truly effective. If experimentation is pursued to a great extent, it could and probably should disrupt many status quo science funding structures and their beneficiaries, which include both prestigious researchers and science institutions like major universities. There will be pushback against experimentation that proposes alternatives to systems with major beneficiaries, but mid- and lower-level NSF personnel, down to the Program Officers, can be a positive force in encouraging metascience and reaping its ultimate benefits.

Conclusion

In July 2026, while I was wrapping up my final edits on this essay, Michael Kratsios, the director of the White House Office of Science and Technology Policy, released “Science: A New Golden Age” as a recommendation for federal science policy to the Trump administration. His report effectively preempted most of my policy recommendations, which should not be too surprising since we drew upon some of the same literature and scholarly work, particularly from the Institute for Progress and the wider progress studies community.

I recommend reading the entirety of “Science: A New Golden Age” (SANGA), but in essence, it is an attempt to reform and advance the core doctrine of federal science funding laid out by Vannevar Bush in 1945 in Science—The Endless Frontier. Bush argued that basic science research develops a corpus of knowledge that is then turned into practical development, such as new technologies, by applied science. Bush believed that the private market lacked the incentive to conduct sufficient basic research, and so the government should fund scientific research (primarily in academia) to optimize scientific progress and technological growth.

SANGA supports federal financing of private research but argues for spreading more funding outside academia, giving the federal government a more active role in directing research priorities, and actively connecting basic research output to specific technological domains (engineering, medicine, etc.) to facilitate technological growth. SANGA argues that American research progress has slowed in part due to deficiencies in the current federal science funding models, and that major reforms are needed to science funding processes based on cutting-edge and future metascience theories and experimentation.

I was initially disheartened that SANGA might completely steal my work’s thunder, but after reading it, I think my essay is a perfect companion piece to it. With a few key caveats, my report is in very strong agreement with SANGA, and I believe it can serve as a useful complement to its goals and attempts at gaining popular traction. While SANGA’s reach is broader, as it makes recommendations for all federal science funding, my essay is mostly directed toward policy recommendations for the National Science Foundation and maybe could serve as guidance for implementing many of the policies proposed by SANGA.

The scope of the paper is too broad for me to comment on all of its arguments and claims, but I will address a few key issues on which I agree or disagree, or toward which I am ambivalent based on my research and interviews.

I strongly agree with SANGA regarding many of the systemic problems afflicting federal science funding and the research system built around it, including the Grant Treadmill, excessive grant application and maintenance burdens, excessive bureaucracy, hyper-competition among researchers, excessive conservatism in research goals, excessive risk aversion and careerism among researchers, etc. However, I am agnostic on SANGA’s arguments that the systemic problems within federal science funding have led scientific research as a whole to be less productive or to slow down, as I don’t believe research quality and progress can be effectively measured at this time.

I most strongly agree with SANGA regarding the need to implement metascience reforms and experimentation in federal science funding agencies. SANGA explicitly recommends creating metascience units within federal science funding agencies like the NSF and NIH that would be devoted to developing experiments on federal science funding to discover best practices. SANGA recommends having these metascience units report to the agency heads and have the power to require Program Officers to participate in their experiments. The metascience units would also be tasked with building infrastructure to collect and analyze science funding data produced by the agencies for metascience study, and the results would be published. Like my paper, SANGA points to the UK Metascience Unit as a small-scale model for metascience reform. The NSF’s announcement of its new Office of Metascience is a welcome step toward this recommendation; the priority now is to give it the resources, data access, and institutional support to produce useful evidence and act on its findings.

I strongly agree with SANGA regarding the potential value of many of the science funding policies that could be implemented in agencies like the NSF and NIH, including Golden Tickets, funding people over projects, portfolio-based grantmaking, longer time horizons for investigators, elevating the status of Program Officers, etc.

I agree with SANGA that federal science policy should try to spread more of its funding outside of academia. Institutional diversity is important for maintaining research ecosystem vibrancy, and it is likely that some of the systemic issues in federal science funding are downstream of the incentive structures of academia.

I slightly disagree with SANGA’s recommendations to implement or expand many specific science funding reforms, such as favoring Program Officer-based funding decisions over models that favor peer review, using longer time horizons for research grants, targeting higher-risk research projects, implementing a portfolio-based research funding model, etc. All of these reforms, and other recommendations, are popular among many scientists and have good theoretical backing, but I believe that they currently lack sufficient empirical evidence to declare them superior to the status quo models of federal science funding. They are all worth implementing in experimental forms, but more metascience research is needed before implementing these reforms on a larger or more permanent basis.

In my opinion, SANGA does not fully resolve what I consider the fundamental problem in metascience: how should the quality and value of scientific output be measured? The report recognizes the limitations and gameability of conventional metrics, including Goodhart’s Law. It also proposes important steps toward better evaluation, including controlled grantmaking experiments and agency metascience units. However, it does not offer a sufficiently developed account of what should count as a strong scientific outcome or how agencies should compare radically different forms of value, especially in open-ended basic research whose benefits may emerge decades later. I do not fault Kratsios for failing to solve an extremely difficult problem, but federal metascience units should treat the development and validation of better outcome measures as one of their central research programs.

I am agnostic about SANGA’s arguments regarding how to conceptualize or balance the relationships between basic, translational, and applied scientific research. My interviewees often discussed this issue and presented a wide variety of views on it, from which I could derive no consensus. I am open to SANGA’s claims that the relationships can be restructured to make scientific progress and technological development faster and more efficient.

I am ambivalent about SANGA’s proposed policies for confronting the Replication Crisis. SANGA gives good recommendations for attempting to set and enforce tighter standards for federally funded research, but I believe my report gives more consideration to how the Replication Crisis impacts different scientific fields and the complexity of evaluating the value of replication in specific domains. I read SANGA’s Replication Crisis policies as too uniform and potentially at cross-purposes with its goal of reducing federal scientific bureaucracy.

My greatest disagreement with SANGA concerns its stronger commitment to politically directed priority-setting within federally funded research. I agree that elected officials have a legitimate role in allocating taxpayer resources and identifying broad national priorities, particularly in areas such as defense, energy, biotechnology, and artificial intelligence. SANGA also appropriately distinguishes broad political direction from scientific execution, which it would generally leave to researchers. However, many of my interviewees expressed concern that excessive political control could suppress scientific curiosity, expose research priorities to short-term political swings, and weaken the decentralized character of the American research ecosystem. I believe government-defined missions should exist for specific goals, but not as the organizing principle for the entire system.

Footnotes

  1. https://www.nsf.gov/about/history ↩︎

  2. https://www.pi.infn.it/~giorgio/INFN/3M/SciencetheEndlessFrontier.pdf ↩︎

  3. https://www.nationalacademies.org/read/25990/chapter/3 ↩︎

  4. https://www.nas.org/report/rescuing-science/full-report/ ↩︎

  5. https://www.nsf.gov/impacts/mri ↩︎

  6. https://www.nsf.gov/impacts/doppler ↩︎

  7. https://www.nsf.gov/impacts/3d-printing ↩︎

  8. https://www.nsf.gov/impacts/internet ↩︎

  9. https://www.nsf.gov/about/history ↩︎

  10. https://www.nsf.gov/about/history ↩︎

  11. https://www.nsf.gov/impacts/space-time ↩︎

  12. https://www.nsf.gov/impacts/space-time ↩︎

  13. https://www.nsf.gov/impacts/crispr ↩︎

  14. https://www.nsf.gov/science-matters/invention-impact-story-lasik-eye-surgery ↩︎

  15. https://www.nsf.gov/about/history ↩︎

  16. https://www.nsf.gov/about/history ↩︎

  17. https://www.nsf.gov/about/history ↩︎

  18. https://www.nsf.gov/about/history ↩︎

  19. https://nsf-gov-resources.nsf.gov/pubs/2025/nsf25002/pdf/nsf25002.pdf ↩︎

  20. https://ncses.nsf.gov/pubs/nsf26314 ↩︎

  21. https://www.science.org/content/article/despite-trump-chaos-nsf-avoided-feared-dip-research-financing ↩︎

  22. https://www.nsf.gov/funding/overview ↩︎

  23. https://grants.nih.gov/news-events/nih-extramural-nexus-news/2025/11/implementing-a-unified-nih-funding-strategy-to-guide-consistent-and-clearer-award-decisions ↩︎

  24. https://www.everycrsreport.com/reports/R46753.html ↩︎

  25. https://nsf-gov-resources.nsf.gov/pubs/2025/nsf25002/pdf/nsf25002.pdf ↩︎

  26. https://report.nih.gov/nihdatabook/report/5 ↩︎

  27. https://comptroller.war.gov/Portals/45/Documents/defbudget/FY2027/budget_justification/pdfs/03_RDT_and_E/RDTE_Vol1_DARPA_MasterJustificationBook_PB_2027.pdf ↩︎

  28. https://newsletter.goodscienceproject.org/p/minimum-viable-bureaucracy ↩︎

  29. https://globalresearchcouncil.org/fileadmin//documents/Library/COVID-19/Letter_on_the_Coronavirus_Disease_2019__COVID-19___nsf20052____NSF_-_National_Science_Foundation.pdf ↩︎

  30. https://www.statnews.com/2022/03/29/nih-long-covid-sluggish-study/ ↩︎

  31. https://www.statnews.com/2023/08/09/long-covid-nih-trials/ ↩︎

  32. https://www.statnews.com/2024/05/31/long-covid-nih-recover-initiative-falls-short-on-causes-treatments/ ↩︎

  33. https://www.nsf.gov/policies/pappg/24-1/privacy-act-public-burden ↩︎

  34. https://www.nsf.gov/funding/overview ↩︎

  35. https://thefdp.org/demonstrations-resources/faculty-workload-survey/ ↩︎

  36. https://thefdp.org/demonstrations-resources/faculty-workload-survey/ ↩︎

  37. https://ncses.nsf.gov/pubs/nsf26316/assets/data-tables/tables/nsf26316-tab005.pdf ↩︎

  38. https://www.nature.com/articles/d41586-026-01105-7 ↩︎

  39. https://nsf-gov-resources.nsf.gov/pubs/2025/nsf25002/pdf/nsf25002.pdf ↩︎

  40. https://www.npr.org/2025/01/31/nx-s1-5282162/scientists-grants-frozen-trump-executive-actions-dei-deia ↩︎

  41. https://www.statnews.com/2025/02/02/nsf-national-science-foundation-payments-resume-after-trump-freeze/ ↩︎

  42. https://www.aip.org/newsletter/00000196-6e5d-d851-a9d7-ffffc1e8000a ↩︎

  43. https://www.nature.com/immersive/d41586-026-00088-9/index.html ↩︎

  44. https://eos.org/research-and-developments/after-restructuring-nsf-wants-to-hire-more-staff-but-reduce-solicitations ↩︎

  45. https://www.nextgov.com/people/2025/03/nsf-begins-reinstating-probationary-employees/403414/ ↩︎

  46. https://www.npr.org/2025/02/18/nx-s1-5301049/national-science-foundation-fires-roughly-10-of-its-workforce ↩︎

  47. https://www.nextgov.com/people/2025/03/nsf-begins-reinstating-probationary-employees/403414/ ↩︎

  48. https://www.govexec.com/management/2025/07/nsf-slashes-most-career-executive-roles-after-shedding-one-third-staff/407066/ ↩︎

  49. https://nsf-gov-resources.nsf.gov/files/FY-2027-NSF-Budget-Request-to-Congress.pdf ↩︎

  50. https://www.science.org/content/article/nsf-slashes-number-rotators-and-well-paid-managers-part-restructuring ↩︎

  51. https://www.aip.org/fyi/the-week-of-may-12-2025 ↩︎

  52. https://www.oversight.gov/sites/default/files/documents/reports/2024-11/Management%20Challenges%20for%20NSF%20in%20FY%202025.pdf ↩︎

  53. https://nsf-gov-resources.nsf.gov/files/FY-2027-NSF-Budget-Request-to-Congress.pdf ↩︎

  54. https://www.nsf.gov/updates-on-priorities ↩︎

  55. https://www.nsf.gov/news/nsf-announces-100-million-investment-national-artificial ↩︎

  56. https://www.nsf.gov/focus-areas/ai ↩︎

  57. https://www.nsf.gov/updates-on-priorities ↩︎

  58. https://www.urban.org/urban-wire/nsf-has-canceled-more-1500-grants-nearly-90-percent-were-related-dei ↩︎

  59. https://x.com/4wavePepe/status/2033628407108432125 ↩︎

  60. https://marginalrevolution.com/marginalrevolution/2026/06/the-nationalization-of-american-science.html ↩︎

  61. https://www.federalregister.gov/documents/2026/05/29/2026-10817/regulation-for-federal-financial-assistance ↩︎

  62. https://www.whitehouse.gov/presidential-actions/2025/05/restoring-gold-standard-science/ ↩︎

  63. https://nsf-gov-resources.nsf.gov/files/NSF-GSS-Implementation-Plan-2025.pdf ↩︎

  64. https://www.research.utah.edu/resources-opportunities/changes-to-nsf-peer-review-process/ ↩︎

  65. https://cossa.org/national-science-board-releases-anticipated-merit-review-report/ ↩︎

  66. https://www.insidehighered.com/news/quick-takes/2025/12/17/nsf-lowers-grant-review-requirements-nih-hunts-phrases ↩︎

  67. https://www.washington.edu/federalrelations/2025/05/13/nsf-restructuring/ ↩︎

  68. https://www.nsf.gov/od/updates/nsf-announces-organizational-realignment ↩︎

  69. https://www.nsf.gov/about/budget/fy2027 ↩︎

  70. https://www.nature.com/articles/d41586-026-01105-7 ↩︎

  71. https://mindingthecampus.org/2025/04/09/15-billion-saved-from-indirect-costs-boosts-research/ ↩︎

  72. https://www.nytimes.com/2025/05/16/science/federally-funded-science-breakthroughs.html ↩︎

  73. https://innovativegenomics.org/wp-content/uploads/2024/08/Jinek-2012.pdf ↩︎

  74. https://www.nsf.gov/impacts ↩︎

  75. https://www.nsf.gov/impacts/crispr ↩︎

  76. https://www.newthingsunderthesun.com/pub/d4ggviu4/release/2#whats-a-reasonable-guess-at-the-average-return-on-investment-roi-of-rd ↩︎

  77. https://ncses.nsf.gov/pubs/nsb20257/federal-support-for-u-s-r-d ↩︎

  78. https://www.nsf.gov/od/oia/eac/publications-date ↩︎

  79. https://nsf-gov-resources.nsf.gov/2023-04/BETA%20SITE%20IPA%20Rotator%20Study_508c_0.pdf ↩︎

  80. https://nsf-gov-resources.nsf.gov/2023-10/ETAP-REU-Infographic-508.pdf ↩︎

  81. https://nsf-gov-resources.nsf.gov/files/NSF-AICA-Report.pdf ↩︎

  82. https://www.nature.com/articles/d41586-023-00579-z ↩︎

  83. https://www.nsf.gov/news/nsf-announces-15b-nsf-x-labs-initiative-pursue-generational ↩︎

  84. https://fas.org/publication/rebuild-corporate-research/ ↩︎

  85. https://www.nsf.gov/policies/pappg/24-1/privacy-act-public-burden ↩︎

  86. https://nsf-gov-resources.nsf.gov/files/FY-2027-APP-and-FY-2025-APR.pdf ↩︎

  87. https://grants.nih.gov/grants/guide/pa-files/PA-25-301.html ↩︎

  88. https://coefficientgiving.org/our-approach/grantmaking-process/ ↩︎

  89. https://gcgh.grandchallenges.org/grant-opportunities/faq ↩︎

  90. https://wellcome.org/research-funding/guidance/prepare-to-apply/scheme-application-deadlines ↩︎

  91. https://chanzuckerberg.com/rfa/synthetic-biology-principles-immunology/ ↩︎

  92. https://www.simonsfoundation.org/grant/targeted-grants-in-mps/?tab=rfa ↩︎

  93. https://marginalrevolution.com/marginalrevolution/2020/04/fast-grants-a-project-of-emergent-ventures-against-covid-19-status-update.html ↩︎

  94. https://www.science.org/doi/10.1126/science.aac4716 ↩︎

  95. https://news.ki.se/half-of-social-science-results-cannot-be-replicated ↩︎

  96. https://bidenwhitehouse.archives.gov/ostp/news-updates/2022/08/25/ostp-issues-guidance-to-make-federally-funded-research-freely-available-without-delay/ ↩︎

  97. https://www.nationalacademies.org/projects/DBASSE-BBCSS-17-03 ↩︎

  98. https://www.nsf.gov/funding/opportunities/dcl-open-science-research-data ↩︎

  99. https://www.nsf.gov/funding/opportunities/dcl-reproducibility-replicability-science ↩︎

  100. https://www.nsf.gov/funding/opportunities/ci-paos-cyberinfrastructure-public-access-open-science/506307/pd24-7414 ↩︎

  101. https://www.axios.com/2025/03/24/medical-studies-rfk-nih-replication ↩︎

  102. https://ifp.org/how-much-should-we-spend-on-replication/ ↩︎

  103. https://goodscience.substack.com/p/back-and-forth-on-the-value-of-replication ↩︎

  104. https://www.nas.org/report/rescuing-science/full-report/ ↩︎

  105. https://www.nas.org/report/rescuing-science/full-report/ ↩︎

  106. https://www.slowboring.com/p/ai-progress-is-giving-me-writers ↩︎

  107. https://nsf-gov-resources.nsf.gov/pubs/2015/nsb201514/nsb201514.pdf#page=43 ↩︎

  108. https://www.nsf.gov/funding/opportunities/che-drpndp-division-chemistry-disciplinary-research-programs-no/nsf22-606/solicitation ↩︎

  109. https://www.nsf.gov/funding/opportunities/dcl-implementation-no-deadline-full-proposal-submission-process/nsf18-011 ↩︎

  110. https://www.nsf.gov/funding/opportunities/ios-division-integrative-organismal-systems-core-programs/nsf24-546/solicitation ↩︎

  111. https://www.nsf.gov/funding/information/dcl-removal-deadlines-core-programs-directorate-engineering ↩︎

  112. https://www.nsf.gov/tip/updates/nsf-partners-institute-progress ↩︎

  113. https://www.nsf.gov/news/nsf-leadership-new-golden-age-science ↩︎

  114. https://researchonresearch.org/rori-convenes-global-metascience-panel-at-oecd-blue-sky-forum/ ↩︎

  115. https://www.gov.uk/government/publications/a-year-in-metascience-2025 ↩︎

  116. https://sshrc-crsh.canada.ca/en/news/2025/funding-to-strengthen-national-capacity-in-metaresearch.aspx ↩︎

  117. https://www.nature.com/articles/nature.2016.20287 ↩︎

  118. https://www.openscience.nl/en/researchprogrammes/replication-studies ↩︎

  119. https://www.snf.ch/en/JyifP2I9SUo8CPxI/news/news-210331-drawing-lots-as-a-tie-breaker ↩︎

  120. https://www.nsf.gov/tip/updates/nsf-partners-institute-progress ↩︎