Investigating the Investigator Grants 2019

Introduction

The hottest topic on Australian researchers lips this month has been the recent NHMRC Investigator Grant announcement. After all the commotion around the new scheme and the distribution of research dollars, I set about flexing my data-science muscles to see what the outcomes of the scheme were overall, and what a successful application might look like for the next round.

What are Investigator Grants?

If you are a scientific researcher in Australia, chances are you were aware of the NHMRC announcement a few weeks ago of their latest round of funding. For those who are outside health-focused research, this was a highly-anticipated announcement after a complete re-structure of the NHMRC Funding Schemes over the previous year. The reform included replacing the Fellowship system (funds to support individual researchers to establish their own research programme) under the unified banner of Investigator Grants.

Historically, previous incarnations of these Fellowships (Early Career, Career Development and Research Fellowships) have supported the very best and brightest researchers in Australia with varying levels of lottery-luck in their award. Importantly, having been successful at the first rungs on the Fellowship ladder, researchers were then much more likely to gain the next level of support. Therefore, the award of these Fellowships to emerging researchers is a crucial support system that could make or break a career.

In particular, the NHMRC website lists better support for Early and Mid-Career Researchers (EMCRs) as a key goal of their reform:

Early and mid-career researchers are being discouraged from pursuing a research career. […] In response to these issues, NHMRC reviewed the structure of its research funding and has reformed its grant program.

NHMRC, 2019

Why the fuss?

For researchers who spend many (many) weeks (months) working on the application for these schemes, the outcomes are of intense interest. The research funding environment in Australia is extremely competitive, and often the difference between funded and not funded can be little more than pot-luck. Early reports during the peer review process this year suggested that the new scheme had an abundance of issues and was likely to be even more of a lottery than previous years.

After the announcement, scientists flooded Twitter with their thoughts, interpretations and statistical analysis of the overall outcomes. There were many deserving people who were lucky enough to make the grade this time around. However, there were also an abundance of leading researchers who missed out and cursory analyses highlighted an abundance of inequities and unforeseen results of the reform.

Many of the numbers you will see online come from the key performance data released by the NHMRC (data on gender, age and state awardee rates has been made available since at least 2013). It is my firm belief that, as scientists, we should use science (including data science) to drive our decision making. The trends reveal biases/underlying issues in the grant scheme, and provide rationale/direction for additional reform. In addition, the profile of previously successful applicants can help researchers like myself understand how we fit within the scheme and, according to whether (or not) we fit the profile, inform our decision to apply in the next round (which is not that far away!).

Crunching the numbers

Most of the raw data used in this analysis came from the NHMRC grant outcomes website. I collected some extra data on the Field of Research codes and individual researchers from additional sources including the Australian Bureau of Statistics and Scival.

After initial cleaning of the raw data, I decided to equate the new and old schemes by ‘matching’ the tiers as best as possible. According to a fact sheet provided by the NHMRC, equivalent schemes are as follows: Early Career Fellowships map to Emerging Leadership level 1, Career Development Fellowships map to Emerging Leadership level 2, and Research Fellowships map to Leadership levels.

Due to differences in eligibility (according to years post-PhD) as shown above, the correlation is not perfect. But it serves the purpose of being able to compare as the NHMRC intended at least.

For the nitty-gritty details of the number crunching and visualisation techniques that went into the makings of this post, be sure to check out the Behind the Scenes post soon. If you’re simply here for the pretty pictures and insights, read on!

Key insights

Overall trends

At first glance, it appears as though the new investigator scheme has seen an injection of more money into the Fellowships portion of NHMRC funding. However, this funding includes more money per person at the upper tiers without a substantial increase in the number of applications funded. Moreover, the number of successfully funded applications and dollars was skewed toward the Leadership levels with an overall decline in the number of level 1 and 2 (Emerging Leader) applications funded.

The decline in the number of applications funded was matched by a substantial increase in the number of applications submitted, leading to a nosedive in the success rates particularly in the EL2 and L1 brackets. This increase in applications was thought to reflect the changes in eligibility structure meaning everyone had a go and often at levels far below what was appropriate for their research experience. The NHMRC is reported to be hopeful the number of applicants will decrease in the next round (thus artificially inflating the success rate).

Location, location, location!

With the conglomerate of research institutes, personnel and equipment in Melbourne, it has always been tough to beat in funding success. The same was true of this year, with Victoria the overwhelming leader in the number of awarded applications and second-highest success rate. In fact, Victoria received more than 45% of the awarded Fellowships in 2019.

The large number of applications originating in Victoria, coupled with their high success rate, suggests that not only is there a critical mass of outstanding researchers concentrated around the Melbourne biomedical hubs, but that they benefited immensely from the support processes in place in the lead up to submission. Hopefully, other institutes around the country can find value in their strategies for the following rounds.

Gauging the gender gap

Gender has always been a touchy topic when it comes to Fellowship funding in Australia, especially at the later stages of academia. There were concerted efforts made to target gender equality in the new scheme. However, at first glance, there was an enormous bias in the total number and dollars awarded. In fact, this disparity is the worst that the Fellowships scheme has seen in the last five years.

If we dig a little deeper, it becomes clear that this skew is due mainly to biases in the upper tier of the scheme. In fact, at the first and second tiers (level 1 and 2), there has been progress toward equality and, in some cases, even over-representation of women for the last five years. However, the successful males outnumber females two to one at the highest tier (level 3).

Two to one. The new scheme, if anything, has made this worse with the proportion of awards for women the lowest it has been since 2015. This points toward a systemic issue with how relative to opportunity is assessed and the lack of support for women entering the upper echelons of academia that is still ingrained in the research culture in Australia. Moreover, the extended 5-year term of these awards means that these differences will permeate University faculty for many years to come.

Titles and track records

As a general benchmark, academic titles say something about a researcher’s seniority. Considering the distribution of titles among successful awardees, there is a clear trend toward more and more senior researchers finding success at lower levels of the Fellowship scheme. For example, Associate Professors have taken a share of the level 1 funding for the last two years and for the first time, there was an Emeritus Professor awarded level 3 funding in 2019.

Similar trends can be seen in the track record of successful awardees, especially their publication history. While this is imperfect due to author name mismatches, overall the median number of publications for level 2 awardees has held steady around 50. In contrast, level 3 saw a sharp increase this year for the first time. The median number of publications for level 1 awardees has also steadily increased, meaning that to be competitive ECRs now need on average twice as many publications as they did 5 years ago.

With the revamp of funding levels and removal of specific ‘years-post-PhD’ ranges for level 1 and 2, this was always a concern. While I have (many, many) issues with using the years-post-PhD award as a ruler to measure relative success, at the bare minimum I do believe this distinction helped stratify junior researchers in the eyes of reviewers and assisted their assessment relative to opportunity. Unfortunately, the lack of boundaries this year left many researchers unsure of the appropriate level of funding they should apply for and moreover allowed many to take advantage of the lower levels in the scheme. This placed a large responsibility on reviewers to fairly evaluate an individuals trajectory against others with up to 10 years longer in research. A big ask!

Trendy topics

With every application, researchers include up to five keyword phrases describing the focus of their proposed research. By looking at the most popular keywords, I wanted to understand the research themes attracting the most funding and potentially consider how this has evolved over the last few years.

• 

• 

Interestingly, the top five keywords for the previous five years have typically been some iteration of health, disease, biology and cancer. While these seem very general, reading between the lines reveals a transition from ‘disease’-driven research to ‘health’-centric. Interestingly, epidemiology has emerged as a prominent focus this year. Cancer research has also held a steady proportion of funded applications, suggesting an area of high priority either among reviewers or impressive researchers embedded in this field.

What is unclear from these trends is the type of research being funded in these proposals. This is captured in the NHMRC reporting process as ‘broad research themes’. Traditionally, fundamental (Basic) research has held a large share of the total funding. However, the last five years have seen a steady decline in this proportion and a corresponding increase in the more translational themes.

In particular, clinical medicine and science enjoyed a sizeable bump in the proportion of funding awarded such that it has almost reached parody with basic science. This likely stems from the strong emphasis on ‘research impact’ that permeates all aspects of the new scheme, and at face value is not terribly alarming. However, our translational research must be underpinned by quality fundamental understanding and our ability to fund this type of research from the NHMRC scheme appears to be questionable in the future.

What does this mean for science, and ECRs, in Australia?

Overall, there has been a shift toward more senior and established researchers in all three levels of award. This is not merely a consequence of the new scheme, but has definitely been accentuated by it. This is likely reflective of the desire to invest in researchers who can demonstrate their previous impact has a direct and immediate public benefit.

This is also reflected in the share of the pie gobbled up by each of the four broad research areas, where Clinical and Medical Science has seen a steady increase at the expense of Basic Science.

While I made every effort to take a comprehensive snapshot of the available awardee data, these insights were limited in part by fragmentation of the data. The raw data provided by the NHMRC includes summary totals e.g. by gender or by state, but often the per-applicant information, due to obvious privacy reasons, is not available.

Some of the missing data, such as the years-post-PhD for successful applicants, could, in theory, be provided anonymously and would provide tremendous insight for potential applicants. At this stage, the closest proxy for years post-PhD is the mean age of awardees (although this as a measure is complicated by relative to opportunity). This year the mean age for level 1 awardees increased from 35 in 2018 to 37, agreeing with the overall sentiment of increasing seniority among successful applicants. With the restructuring in 2019, the NHMRC has all but done away with the ECR funding dedicated for those entering the postdoctoral workforce and require us instead to relying on the provision of project funding by senior researchers for many years before being deemed worthy and ‘impactful’.

There have been a number of statements released by associations commenting on this and other perceived failures of the new scheme, and suggesting changes to overcome these issues. For example, the National Association of Research Fellows released a host of recommendations aimed at alleviating many of the pain points ranging from the application structure to review processes. However, the short turnaround time (given applications for the next round open tomorrow and close in a little under two months) means that these recommendations are extremely unlikely to be implemented until next year. Moreover, this is a general and increasingly-acknowledged failure of research funding schemes world-wide. At this point, like so many other ECRs in my position, it is now time for me to think about whether my time, energy and effort are best placed in the hands of the NHMRC, other Australian funding schemes or – like so many in my position – whether I should instead focus on opportunities abroad. In the spirit of data-driven decision making, hopefully these insights have helped guide your thoughts as they have mine.

Resources

• The original data was sourced from the NHMRC website
• For more on the initial guidelines provided during the scheme restructure, check out the NHMRC Factsheet
• Author track record information, including publication number and field-weighted citation impact, were collected from SciVal. If you are considering an application in the upcoming round, it’s a great idea to benchmark yourself against previous successful applicants.
• For more info on the specific number crunching and data visualisation techniques used here, don’t forget to keep an eye out for my Behind the scenes post.

---

Are you thinking of applying in the next round of Investigator Grants? Did any of these stats surprise you, or were they helpful in your decision of whether or not to apply in the next round? Head over to the contact page, or let me know on Twitter.

Image credits: inspecting gears with magnifying glass | @ pluyer via unsplash