Why peer review should be fun but can make you cry

The life of an academic is very much focused on and surrounded by the process of peer review.

With peer review, I mean making and receiving comments on manuscripts that we have written or have been asked to read and assess for scientific journals.

Most of us review papers for scientific journals in the hope that we can support a rigorous scientific process in producing knowledge that has been considered from all angles.

I am yet to go through a peer review process that did not add value and improve my own writing and communication of my research.

But it is a tricky process in many ways and just having gone through it again, I wanted to reflect here on why we do peer review and how each of us could perhaps structure feedback in ways that is constructive and has the potential to increase the quality of work that is being assessed.

The challenges of conducting peer review

Peer review has a tremendously important role in keeping science honest and to make sure that what is being published is an adequate account of an issue under scrutiny.

But sometimes peer review and in particular getting review comments back is far from fun. Most academics winch when they open the dreaded email and know that time has come to read what someone thinks about our research.

The opinions can indeed be widely divergent. For example, in a recent peer review of a journal article I am writing with colleagues, we received 11 pages of comments from 3 different reviewers.

Many of these comments were helpful as these sought to clarify our terminology, analysis and presentation of the results.

However, as we have to respond to each comment, the 11-page document turned into 25 pages, and this is not including the changes we made in the revised manuscript.

This time around I felt that I was writing two new papers: revising the actual manuscript and then writing responses to the reviewer comments. 

One author noted that our conclusions were the best piece of the manuscript whereas the other argued that the conclusions were merely sweeping generic statements that lacked the insights presented in the paper.

One reviewer was very happy with the terminology whereas two others wanted us to use similar terms they would use.

In the peer review process, we have a double-blind peer review: neither the author/s or the reviewer know each other’s identity, or blind peer review where the author/s name is shown but reviewer names stay hidden.

Both of these have been put in place in order to maintain some level of integrity and reduce bias.

Although at times one can easily guess who the author and the reviewer are: just count the most number of publications starting with the same last name.

But anonymity can also lead to overly negative comments and sometimes to even outright manoeuvring especially when people work in the same area of research and might be competing for the same grants.

My first cry and peer review

When I was submitting my very first journal paper, I had worked hard in condensing my master thesis research into a journal article that was submitted as part of a special issue.

By the time I got my first review comments back, I was holding back tears when a professor at the department read them out to me.

Apparently my work added no new value to the field and was not publishable.

All the other submissions in the special issue moved into revision (opportunity to revise before final submission and publication).

Except mine.

I was devastated. At the time I did not even know how to submit a paper to another journal (I was fresh out of master’s degree and didn’t even know what the process looked like).

So I just let it go.

My next attempt was a journal paper based on a project I had been leading in Africa.

The peer review comments were conflicting: one reviewer thought the paper was very good, another thought that literature review was excellent but the results needed more structure, but the third…

The third reviewer said my paper was basically rubbish and there was nothing new or valuable in it.

The editor often intervenes in such cases and points out which comments the author needs to address. Here this was not the case.

But with the support of my professor, I persisted with the comments and re-wrote what needed rewriting and learned the submission process by heart.

And alas, my first paper was shortly published after I started my PhD.

I was so full of joy and pride that I had managed to publish my first paper.

But part of the experience also left me upset again. I could not understand how my paper did manage to see the light of day given the very conflicting comments.

With more experience, I do now understand the process better however and I actually find reviewing papers exciting and interesting, and quite fun.

Why reviewing is rewarding

There are several reasons why reviewing is actually a good thing to do (and we do it for free btw).

Good thing about reviewing other scientists’ work is that you get to see what is novel and new in the field and the kinds of submissions that are in the field at that point in time.

This helps in distinguishing your own thoughts often about which directions are emerging in a field of research, and keeps you on your toes as well as to what a robust scientific paper should look like.

I find that reviewing papers is actually a good way to keep up to date with the literature and also some of the more innovative methods that authors have been testing in the field.

I encourage my students to review when they can as this gives also access to how to situate their own research in the field and what they could be thinking about.

Also engaging in a robust and helpful review of others work helps you to become established as an expert and people might seek your advice also in the future when it comes to seeking particular experts for committees or initiatives.

At the same time, when I review someone’s work, I also have the responsibility of making sure that whatever is published contributes to what has already been discovered before.

This of course is a grey area in many ways because novelty can be claimed in many ways: for example a new method, a new angle, a new geographic region or sector.

But the main aim is to increase the quality of science.

How to be a helpful reviewer

Although this post has focused on the scientific peer review process, many of these relate to the basic principles of giving feedback on reports, which are essential skills in leadership and management as well.

In reviewing, I always look at several things that can improve the manuscript:

  1. Clarity of writing/ideas: is the writer explaining his or her logic clearly, are there concepts and sentence structures that make it easy to understand what the main aim of the manuscript is? Do the methods, results, and discussion contribute to the main aim of the article?
  2. Novelty of research/ideas: how is the manuscript contributing to knowledge in this area? Are they citing the main sources and building upon what is already known? Are the methods novel and insights different from what has been published before?
  3. Structure of the manuscript/report: are there specific elements missing from the structure? Is the flow between sections good?
  4. Recommendations: what kinds of recommendations emerge based on the research? How could these be implemented innovatively? What are the next steps/knowledge gaps identified?

On top of these, the tips I would give for someone starting out in academia is to understand the field (who has published what, which are the major debates you wish to contribute to) as this helps you to situate your own research in a niche or an area that has been overlooked.

This is particularly why PhDs are often required to spend most of their first year just going through literature and finding the gaps. In some areas this is easier and in others it is not.

Mind you, although I know all of this, it does not mean that all of this comes naturally to me. I find it even more difficult to judge my own writing but I do try to keep these guidelines when I start drafting articles and reports.

The point is that to the contrary of science being portrayed as a mere rational non-emotional process, peer review in particular raises often lots of emotions (not necessarily always tears, mind you).

And that’s ok because it also signals how much our work and that of others means for us and that we are putting our hearts and souls in the papers we write and science that we produce.

 

Is 1.5 degree goal an illusion?

This past week has been quite eventful given that major issues have been either discovered or experienced when it comes to climatic changes.

Japan has been experiencing significant floods with never before seen rain events while cities around the word have reported having broken all time heat records.

A new study published in Nature used paleoclimatic data (past climate records) to show that it is highly likely that we have underestimated the degree or level of climate change that we are already committed to.

This new study sets the Paris Agreement’s goal to stay below 1.5 degrees into question, a temperature goal that is also the focus of Intergovernmental Panel on Climate Change’s (IPCC) 1.5 degree report to be released later this year in October.

Yet, for those who do not necessarily work on climate change or are not following this area of research and policy that closely, what does it really matter if it’s 1.5. or 2 degrees or 4?

Sadly, this is not some kind of a game where we just pick goals and then see who comes closest to what.

The difference between degrees of warming is drastic as some ecosystems and their services that we rely on can stop functioning even with 1.5. degrees of warming, let alone 2 degrees or 4.

This is a world most of us do not want or cannot fathom.

Adaptation Limits and Planetary Boundaries

Some critics have noted that perhaps we need to accept that focusing on 1.5 degrees is futile as a goal given the latest science and changes we are already seeing; that the science community would do better to tackle the real big questions as to what actually happens with more drastic warming of the planet.

Others have voiced concern over talking about 4 degrees as this could take away focus and commitment from the 1.5 goal that has now been globally agreed on, and what changing extremes mean for adaptation limits.

What might be useful in this discussion is the link to another body of research, that of earth and planetary boundaries.

These concepts have been researched by Professor Will Steffen and others and are often used to provide a global set of trends in areas such as population growth, and ecosystem and biodiversity loss etc.

While we might look at our own city, our own neighbourhoods, state or country and try to determine what kinds of limits we face in adapting to climate change, all of these are part of a much bigger complex system that has its own diverse limits and thresholds that we seem to be stepping closer and closer to.

The intersection of these is that we need to keep in mind both a broader planetary boundary perspective but also the context specific details of where adaptation takes place.

Yet, often there is strong focus on one kind of risk for example on rising sea levels but not compounding risks necessarily where for example seas begin to rise, erosion increases, cyclones become more intense, droughts become longer and common plants and ecosystem services change.

We cannot cover all risks at once and we do not have the data or capacity to provide exact specific details how all of these risks and impacts accumulate and interact over space and time.

But what we can say is that countries, communities, households and individuals need to at least begin to consider particular risks while also understanding the interconnections that different risks and systems have.

 Geostorm Leadership?

I recently saw the movie Geostorm that describes the world in 2019/2022 and how particular kinds of geoengineering efforts including using a global satellite system to control weather are failing.

The world the movie describes shows a high level of political cooperation on the “fixing strategies” to create this shared system of planetary control.

In Fred Kohman’s book on Transcendent leadership, he notes how in a marathon people are not running to follow a leader. People run because they have a goal and leaders are often simply just closer to the goal.

What this means is that transcendental leaders actually get people’s commitment to the mission, and not just to the leader.

It is the mission that people believe in and the leader’s job is mainly to excel in helping people to reach and spread that mission.

This has led me to reflect what exactly is our mission when it comes to climate change and temperature goals, and what leadership style can deliver the most support for such mission.

And if we have already moved past 1.5 or 2 degrees of warming, what then?

This calls for a discussion on the role that science can contribute to urgent policy decisions but also deeper reflections on how we all see our own roles in producing or using knowledge in a warming world.

At this point, I have no real answers except that as a scientist I am committed to exploring these questions further and hopefully some of the science that we produce can answer these questions.

 

 

 

 

 

6 essentials in developing an academic career

 I’ve recently been approached by several early career researchers (ECRs) about how to network, how to focus their careers and how to start building their profile while they are still in an early phase in their career. In the university system, there is often not much advice on personal branding for academics or how to effectively network or on the use of social media platforms as often the expectation is that your published research speaks for itself.

But in today’s world where much of our visibility is increasingly linked to an online presence, ECRs need to think about how they get noticed while they are still getting to know their area of expertise. This means being more strategic where to invest your energy in career development.

I’ll share my lessons learned here both based on my own personal experience as an ECR but also from the management and leadership literature and podcasts from which I have picked up many of these ideas.

Know yourself

This is such a cliché yes but so true at the same time. Everyone should spend some time mapping out their interests, values, and what inspires them. I came across recently a Passion Assessment, which basically asks 6 questions e.g. what are you passionate about, what cannot you stop doing, etc. There are a number of different assessments out there (some paid, some free) where you can reflect over your values and strengths.

I have for example done the the Strength Finder Assessment  to have a better idea where my key strengths lie and what that means as to what I am good at and like to do. It all sounds very simple and often it is, you don’t need an assessment necessarily but rather inner reflection as to where you want to go and what you are interested in. This relates also to your personal brand: what do you want people to know you by?

Linked In

Regardless what people say about LinkedIn, over 90% of managers and employers go online to find you when your CV comes in. I think of LinkedIn profiles as living CVs where you state your experience but also tell about what you have learned in which job or degree, and showcase your skills. There is a great article by Brenda Bernstein on what this exactly means in regards to structuring your profile and what things should stand out.

One essential is the obvious: a picture. People want to know who you are and uploading a photo will make you more human and not just a name. It’s worth investing your time in creating an updated profile that also tells a story of who you are, what your interests are, and where you see yourself contributing the most. One specific profile I love is Mara Bun, she manages to write in a very funny but insightful way why she is now where she is and how she got there.

What I find most useful in LinkedIn is the ability to keep up to date with my field and networks but also to connect with new people who I might not be able to meet somewhere else. Recently a maple farmer who is also a decision researcher got in touch after seeing my comments on decision-making in LinkedIn. Now we are discussing different aspects of decision-making and it’s a great way to learn more through such personal connections.

Note to self: If you want to connect with someone who you do not personally know, send a connection request with a personal message. This will indicate that you are actually interested in the person and it is not just about growing your network. I rarely accept LinkedIn invitations without a message because I want to know the people I connect with. Brenda has more to say on this so listen here for her interview in Coaching for Leaders podcast, it’s brilliant!.

 

Twitter

Many academics might not see the necessity of Twitter, especially since it requires active use if you want to stay ‘visible’ to broader number of people. I began using Twitter only last year but I have found that it’s a great way to follow other people and topics in my field.

Twitter also makes it easy to stay up to date what is happening in conferences and workshops that I cannot attend in person. For example, during United Nations Framework Convention on Climate Change CoP 23 last year, it was easier to get an understanding where interesting side events were by following the Twitter feed.

Posting about your own research on Twitter is also good for a number of reasons: it has a greater likelihood to be picked up by more people, it can increase your citations, but also get you noticed by more prominent people in your field. Re-tweeting important news and articles also stores these in your profile that you can check later for references.

Make a plan how you want to use Twitter (daily? weekly? during conferences?) and be consistent in what you tweet about and how often. Note also that bad tweets can ruin reputations so have a think what you write and how that could be perceived.

 

Conferences

Conferences have so many opportunities for career development and I always encourage PhDs to attend as many conferences as they can. These are great ways to stay up to date with the recent science in your field but also meet the people whose papers you read.

Volunteering at conferences is good because you often get to do many different roles, learn about event management, and assist experts who you might not otherwise have a chance to talk to.

Many early career researchers get discouraged when your abstract is accepted to a conference “only” as a poster. But poster sessions have many benefits that you don’t get in regular oral presentation sessions: people have more time to discuss, are more relaxed, and you learn as much about them as they learn about you.

Something that is not possible when giving a somewhat rushed 12-minute oral presentation to a large group of people where the majority is already busy checking the program which session to attend next.

 

Networking

Online platforms such as Twitter and LinkedIn are useful in networking. But pretty much the best way to reach a person is still email (Dave Stachowiak: indeed!). Why? Because everyone reads their email.

If you have met someone at a conference or have read their work and have questions or just want to connect, find out the person’s email and write to them.

Sending an email to an author of a paper you found particularly compelling is a great way to connect but also to establish a conversation about the topic. Some people might not respond but the right people will. And those are the ones you want to have conversations with. There are also limits to this: bugging an author vs. being respectful of their time.

During my PhD, I once emailed a professor ahead of a conference we were both attending and asked whether he could answer some questions I had regarding his theoretical framework as I was developing mine. He never replied. At the conference, I bumped into him and made a comment about the email. It was awkward but it turned into a funny conversation since he was so embarrassed. Now we’ve been working together on papers for the past six years.

 

Make your research relevant

One of the best ways to contribute to your field is to make sure your research is addressing key knowledge gaps in the research field. I know most PhDs worry about whether their research is going to be novel enough, and spend often the first year busily reading literature to make sure they can find a gap to contribute to.

There are several quick ways to check where the field is at. Intergovernmental Panel on Climate Change assessment reports are a great source as often these have identified which issues are still unclear, which concepts we need a better understanding of, and what is still not yet known. The assessments reports are also good as they summarise the literature and main arguments, and give you a good indication which authors to follow.

Likewise, documentation from the United Nations Framework Convention on Climate Change (UNFCCC) on climate adaptation and different agenda items, and reports such as the UNEP Adaptation Gap Report 2017 , all note areas where new research is necessary.

As an avid podcast listener, I would also encourage listening to different podcasts on your field and even others that deal with interesting topics. True innovation often comes from blending and merging ideas from different disciplines together in a new novel way.

To sum up:

  • Know thyself: Think about how you want people to perceive you and know you for; what are your core strengths and values? How can you leverage those in developing your career?
  • LinkedIn: This is your living CV. Make sure it’s up-to-date, with a profile picture, and use the sections to tell a story who you are, and what expertise and experience you have.
  • Twitter: Make Twitter part of your career strategy. What keywords can people find you with? What are you tweeting about? Be consistent.
  • Conferences: Use conferences and poster sessions as opportunities to connect with your peers. Volunteer if you can as this will be great for your CV but you can get to meet experts you would not otherwise have a chance to talk to.
  • Networking: Go “old school” and email people who you admire in your field.
  • Make your research relevant: check what the latest major science reports have identified as key knowledge gaps.

It’s never too early or too late to start developing your career and brand, even if it is just to get your research noticed by a broader audience. There is much more to this of course than what I’m detailing here, but these are at least some of the first basic steps that can be helpful in starting that journey.

Entrepreneurial Editor: How to edit a book more or less gracefully

I have been reading Dorie Clark’s Entrepreneurial You , which is an amazing collection of hands on advice from podcasting to writing blogs to authoring books. Dorie’s main message is that there are multiple ways that we can contribute and develop our careers. What makes the book fascinating is that much of the advice is grounded in her own experiences but she has also interviewed a great number of others who have succeeded in developing a business and built careers.

In this spirit of openness about her lessons learned, I thought I’d share my recent experience in co-editing a book and what lessons I can take from that might be helpful to others. I’d like to make a disclaimer here to say that this blog post is very much about my lived experience and not that of the publisher or my co-editor. For me, capturing experience-based knowledge is very crucial and that is what I’ll focus on here.

Three main ways dominate in how a book gets authored: you either reach out to your contacts whom you know and ask them to contribute, you issue an open call and see what you get (cast a wider net), or you go at it alone. Each approach has their merits and constraints: sole authorship has full control, asking your friends and colleagues means you might get more accountability in quality and timeliness (they don’t want to disappoint you) and casting a wider net means you are able to capture more diversity in authorships, regions and countries. But sole authorship means you might miss some linkages and issues, asking your friends could mean you all share the same way of thinking, and casting a wider net might result in significant delays and controversies what everyone is supposed to be writing about.

The idea for the book Limits to Climate Change Adaptation was based on the Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report (2014) that had identified a knowledge gap in regards to the concept of “adaptation limits” and what that means in both practice and theory.. Given that there was not much literature on this, we decided to issue an open call to see who identified firstly their research with the topic and which countries and regions we could cover. The number of responses was great with many people wanting to contribute. But the end result was that once the excitement died down and the reality hit about writing an actual chapter, many early enthusiasts did not return a manuscript draft and/or stopped responding to emails.

As an editor, this was frustrating as I could see the potential in many of the abstracts that had been submitted but I was not able to muster up their courage to get them over the line to start writing. Still, we got more chapters than we had expected and I am very thankful for all the authors for sticking with the process and providing such good insights from their countries and disciplinary backgrounds. I am also pleased that we did an open call because it helped to connect to new people whose research I was not previously aware of.

Because there is not much empirical evidence on what actual adaptation limits are, how they differ from thresholds, and what the actual line is between adaptation constraints and limits, it was challenging to get all the chapters to consider these concepts in the same way. We asked all the authors to build on the IPCC Fifth Assessment Report (Working Group II, Chapter 16) so that we could provide a more coherent approach that could also be helpful in informing the IPCC Sixth Assessment Report. I strongly believe that the book has contributed to this gap, with many specific chapters on relocation, migration, agriculture, coastal infrastructure, and even psychology. The chapters truly showcase how different disciplines approach climate adaptation differently including methods, analysis and the way results are derived.

One thing that I cannot emphasise enough is that a good book cannot happen without great reviewers. I am truly thankful for the people who reviewed and provided comments and their time to improve the chapters. I am lucky that many of the reviewers we contacted were interested in the topic and really took the time to help us to assess the chapters.

One thing that I had not prepared for was the long process that editing a book truly is. This may sound naïve but most editors and authors make a plan in the beginning of the milestones that should be achieved but these often are constantly shifted when people’s lives get busy, reviews take time, and manuscripts have to go through final edits. I was lucky enough to have a co-editor who already has an established contact with the publisher. This made things much easier as he knew the procedures and protocols on agreements, something a newbie to publishing must learn as part of the process. Co-editing with someone more experienced is a great way to learn these things. If you have an idea for a book, it’s worth asking around those in your field whether they would be interested in collaboration.

Getting the book out however is half of the job. If you want your book to contribute to the current discussions on the topic, you need to make sure it ends up in the hands and minds of a broad audience and eventually the right people. I haven’t figured out yet a perfect way to do that but given my work is usually with professional scientific community and policymakers, I’m using LinkedIn and Twitter to spread the news and I am also writing to my contacts who might be interested in the book. I am not getting paid for the book so this is different from those authors/editors who derive income from how many copies are sold. My contribution and passion to have the book read has more to do with wanting to contribute to the scientific and policy discussions on climate adaptation.

Given this is the first published book on the concept of adaptation limits to my knowledge, I do have a need also from the scientific perspective to make sure the knowledge we have collated in the book can feed into science and policy discussions whether it is Intergovernmental Panel on Climate Change (IPCC), the United Nations Framework Convention on Climate Change (UNFCCC), and the work that research organisations and NGOs do on the ground.

It would be great to hear from you who have editing and publishing experience how you’ve managed the process of showcasing your work and getting it out there. Any tips on what you’ve found effective? This can be in terms of letting people know about your work or how your research has fed into policy and science processes.

One final thought: people always say look at the competition and see where your ideas fit and can add value. In science, picking up a less examined concept or a knowledge gap can be challenging but it is a good opportunity to contribute to your field and broader scientific community.