Tag: Research management

Is AI ready to mass-produce lay summaries of research articles?

[ad_1]

Generative AI might be a powerful tool in making research more accessible for scientists and the broader public alike.Credit: Getty

Thinking back to the early days of her PhD programme, Esther Osarfo-Mensah recalls struggling to keep up with the literature. “Sometimes, the wording or the way the information is presented actually makes it quite a task to get through a paper,” says the biophysicist at University College London. Lay summaries could be a time-saving solution. Short synopses of research articles written in plain language could help readers to decide which papers to focus on -— but they aren’t common in scientific publishing. Now, the buzz around artificial intelligence (AI) has pushed software engineers to develop platforms that can mass produce these synopses.

Scientists are drawn to AI tools because they excel at crafting text in accessible language, and they might even produce clearer lay summaries than those written by people. A study¹released last year looked at lay summaries published in one journal and found that those created by people were less readable than were the original abstracts -— potentially because some researchers struggle to replace jargon with plain language or to decide which facts to include when condensing the information into a few lines.

AI lay-summary platforms come in a variety of forms (see ‘AI lay-summary tools’). Some allow researchers to import a paper and generate a summary; others are built into web servers, such as the bioRxiv preprint database.

AI lay-summary tools

Several AI resources have been developed to help readers glean information about research articles quickly. They offer different perks. Here are a few examples and how they work:

– SciSummary: This tool parses the sections of a paper to extract the key points and then runs those through the general-purpose large language model GPT-3.5 to transform them into a short summary written in plain language. Max Heckel, the tool’s founder, says it incorporates multimedia into the summary, too: “If it determines that a particular section of the summary is relevant to a figure or table, it will actually show that table or figure in line.”

– Scholarcy: This technology takes a different approach. Its founder, Phil Gooch, based in London, says the tool was trained on 25,000 papers to identify sentences containing verb phrases such as “has been shown to” that often carry key information about the study. It then uses a mixture of custom and open-source large language models to paraphrase those sentences in plain text. “You can actually create ten different types of summaries,” he adds, including one that lays out how the paper is related to previous publications.

– SciSpace: This tool was trained on a repository of more than 280 million data sets, including papers that people had manually annotated, to extract key information from articles. It uses a mixture of proprietary fine-tuned models and GPT-3.5 to craft the summary, says the company’s chief executive, Saikiran Chandha, based in San Francisco, California. “A user can ask questions on top of these summaries to further dig into the paper,” he notes, adding that the company plans to develop audio summaries that people can tune into on the go.

Benefits and drawbacks

Mass-produced lay summaries could yield a trove of benefits. Beyond helping scientists to speed-read the literature, the synopses can be disseminated to people with different levels of expertise, including members of the public. Osarfo-Mensah adds that AI summaries might also aid people who struggle with English. “Some people hide behind jargon because they don’t necessarily feel comfortable trying to explain it,” she says, but AI could help them to rework technical phrases. Max Heckel is the founder of SciSummary, a company in Columbus, Ohio, that offers a tool that allows users to import a paper to be summarized. The tool can also translate summaries into other languages, and is gaining popularity in Indonesia and Turkey, he says, arguing that it could topple language barriers and make science more accessible.

Despite these strides, some scientists feel that improvements are needed before we can rely on AI to describe studies accurately.

Will Ratcliff, an evolutionary biologist at the Georgia Institute of Technology in Atlanta, argues that no tool can produce better text than can professional writers. Although researchers have different writing abilities, he invariably prefers reading scientific material produced by study authors over those generated by AI. “I like to see what the authors wrote. They put craft into it, and I find their abstract to be more informative,” he says.

Is ChatGPT making scientists hyper-productive? The highs and lows of using AI

Nana Mensah, a PhD student in computational biology at the Francis Crick Institute in London, adds that, unlike AI, people tend to craft a narrative when writing lay summaries, helping readers to understand the motivations behind each step of the study. He says, however, that one advantage of AI platforms is that they can write summaries at different reading levels, potentially broadening the audience. In his experience, however, these synopses might still include jargon that can confuse readers without specialist knowledge.

AI tools might even struggle to turn technical language into lay versions at all. Osarfo-Mensah works in biophysics, a field with many intricate parameters and equations. She found that an AI summary of one of her research articles excluded information from a whole section. If researchers were looking for a paper with those details and consulted the AI summary, they might abandon her paper and look for other work.

Andy Shepherd, scientific director at global technology company Envision Pharma Group in Horsham, UK, has in his spare time compared the performances of several AI tools to see how often they introduce blunders. He used eight text generators, including general ones and some that had been optimized to produce lay summaries. He then asked people with different backgrounds, such as health-care professionals and the public, to assess how clear, readable and useful lay summaries were for two papers.

“All of the platforms produced something that was coherent and read like a reasonable study, but a few of them introduced errors, and two of them actively reversed the conclusion of the paper,” he says. It’s easy for AI tools to make this mistake by, for instance, omitting the word ‘not’ in a sentence, he explains. Ratcliff cautions that AI summaries should be viewed as a tool’s “best guess” of what a paper is about, stressing that it can’t check facts.

Broader readership

The risk of AI summaries introducing errors is one concern among many. Another is that one benefit of such summaries — that they can help to share research more widely among the public — could also have drawbacks. The AI summaries posted alongside bioRxiv preprints, research articles that have yet to undergo peer review, are tailored to different levels of reader expertise, including that of the public. Osarfo-Mensah supports the effort to widen the reach of these works. “The public should feel more involved in science and feel like they have a stake in it, because at the end of the day, science isn’t done in a vacuum,” she says.

But others point out that this comes with the risk of making unreviewed and inaccurate research more accessible. Mensah says that academics “will be able to treat the article with the sort of caution that’s required”, but he isn’t sure that members of the public will always understand when a summary refers to unreviewed work. Lay summaries of preprints should come with a “hazard warning” informing the reader upfront that the material has yet to be reviewed, says Shepherd.

Why scientists trust AI too much — and what to do about it

“We agree entirely that preprints must be understood as not peer-reviewed when posted,” says John Inglis, co-founder of bioRxiv, who is based at Cold Spring Harbor Laboratory in New York. He notes that such a disclaimer can be found on the homepage of each preprint, and if a member of the public navigates to a preprint through a web search, they are first directed to the homepage displaying this disclaimer before they can access the summary. But the warning labels are not integrated into the summaries, so there is a risk that these could be shared on social media without the disclaimer. Inglis says bioRxiv is working with its partner ScienceCast, whose technology produces the synopses, on adding a note to each summary to negate this risk.

As is the case for many other nascent generative-AI technologies, humans are still working out the messaging that might be needed to ensure users are given adequate context. But if AI lay-summary tools can successfully mitigate these and other challenges, they might become a staple of scientific publishing.

[ad_2]

Source link

March 20, 2024
Is the Mars rover’s rock collection worth $11 billion?

[ad_1]

The Perseverance rover drills a rock core from the edge of the ancient river delta in Jezero Crater on Mars.Credit: NASA/JPL-Caltech

The Woodlands, Texas

Scientists are on edge as they wait for NASA to answer two of the most consequential questions in Mars exploration. Where on the red planet will the Perseverance rover collect its final rock samples? And can NASA and the European Space Agency (ESA) even afford to fly the mission’s hard-won samples — the prize at the end of a decades-long quest — back to Earth?

Flying Mars rocks to Earth could cost an astronomical $11 billion

Over the past few years, Perseverance has been exploring an ancient river delta in Mars’s Jezero Crater, with the aim of finding signs of past life. The rover’s belly is now stuffed with 17 tubes of Martian rock, dirt and air that scientists say represent an astounding geological collection. “The science is only getting better as we see what Perseverance keeps collecting,” says Laurie Leshin, director of NASA’s Jet Propulsion Laboratory (JPL) in Pasadena, California. But the rover’s instruments aren’t sophisticated enough to determine whether molecules in the samples point to signs of life, or to determine the samples’ age, and so reveal something about the history of Mars. For that, laboratories on Earth are needed.

However, bringing Perseverance’s samples back could cost as much as US$11 billion, an independent panel concluded in a scathing engineering analysis last year. That’s more than NASA can afford. By the end of this month, it and ESA are supposed to find a cheaper way to achieve Mars sample return — or risk leaving the carefully collected rocks where they are.

Adding to the drama, Perseverance’s planners are debating what other science the rover should do before it has to stop exploring. The original mission plan was to explore the ancient river delta and then drive up out of the crater — where there are even older rocks that could tell scientists more about the history of Mars. But as Perseverance approaches Jezero’s rim (see ‘Epic journey’), some engineers are advocating for it to turn around and wait at a lower altitude, where it might be safer and cheaper to pick up the samples.

Source: Nature adaptation from NASA/JPL-Caltech/MSSS/JHU-APL/Brown University

John Mustard, a planetary scientist at Brown University in Providence, Rhode Island, wants the rover to stick to the original plan. The rocks currently on board are “great, but they’re not sufficient to be the transformative samples that we want them to be”, he says. “They’re not Apollo-scale,” he adds, referring to the Moon rocks collected by Apollo astronauts in the 1960s and 1970s that revolutionized scientific understanding of the Moon and Earth.

He and other scientists pressed the case for exiting the crater last week at the Lunar and Planetary Science Conference in The Woodlands, Texas. All eyes are now on NASA to see what it decides.

“Right now what we can say is, we’re committed to [Mars sample return] being the best value,” says Lindsay Hays, acting lead scientist for Mars sample return at NASA headquarters in Washington DC. “My focus is really on making sure that we get as much science out of what we can get.”

A long quest

NASA has been working on various concepts for bringing rocks back from Mars since the 1980s. Perseverance, the fifth in a string of increasingly sophisticated Mars rovers from the agency, landed in Jezero in 2021 to maximize scientists’ chances of finding signs of past life, if it ever existed. Jezero was once filled with water: a river flowed into it that created an ancient delta similar to those on Earth, which can preserve organic material — usually the remnants of plants and other organisms that came from upstream.

NASA Mars rover to cache first rock samples for delivery to Earth

So far, Perseverance hasn’t spotted any obvious signs of ancient life, such as fossils, with its cameras. The best chance of finding past Martian life would be to analyse the rocks the rover has collected for materials rich in carbon, including organic compounds, that might have been created by the decay of long-dead organisms, says Tanja Bosak, a geobiologist at the Massachusetts Institute of Technology in Cambridge. This analysis would need to happen on Earth.

Two of the rock cores are particularly promising for this; they are fine-grained mudstones from the delta that could have trapped organic material. Other cores collected by Perseverance include once-molten rocks from the crater floor that could be analysed to determine the age of that region; sedimentary rocks from the river delta that hold a history of how Mars’s climate and habitability changed through time; and rocks from the delta’s edges that appear to have interacted with deep groundwater, another potentially habitable environment, for long periods.

Stay or go?

The rover is currently exploring a narrow band of rock near the crater’s rim that is rich in carbonate minerals. On Earth, carbonates commonly form along lake shorelines and can preserve evidence of life. But scientists are still debating whether Jezero’s band represents an ancient shoreline.

In the coming months, the rover will roll onto the rim; after that, the question is whether it will leave the crater. If so, it would explore ‘basement’ rocks from around 4 billion years ago — older than the 3.5-billion-year-old delta — and fossilized hydrothermal vents that could have been a haven for Martian life.

When Perseverance drills a rock core such as this one, collected in October 2023, with its robotic arm, it then seals the specimen in a sample tube for safekeeping.Credit: NASA/JPL-Caltech/ASU

But going to this region, known as Nili Planum, might involve more risk than NASA is now willing to take. One concern is that Nili Planum is several hundred metres higher than the crater floor, so the atmosphere above it is thinner, making it more difficult — and expensive — for a sample-retrieval mission to land there.

Scientists are also concerned about how much farther the rover can physically roll before it gives out. Perseverance has travelled nearly 25 kilometres since landing, but mission scientists think it might be able to cover another 70–90 kilometres. If this is confirmed by testing at JPL, it might be able to reach some of Nili Planum’s most intriguing rocks, which are around 16 kilometres from the rover’s current location, and then make it back into the crater for pick up. If Perseverance does die unexpectedly, it has already left a backup collection of ten cores on the floor of Jezero Crater.

Budget constraints

Now the focus turns to money and how much NASA can invest in bringing the samples back. The mission is part of NASA’s planetary sciences portfolio, which currently spends $2.7 billion annually.

Mars rocks await a ride to Earth — can NASA deliver?

NASA has said it doesn’t want to spend more than 35% of its budget on the mission to retrieve the samples in any given year. “Whatever we implement for Mars sample return is going to be done in the context of a balanced planetary science portfolio,” Lori Glaze, director of NASA’s planetary sciences division, told the conference. But the uncertainty about how much funding might be available to work on Mars sample return forced JPL to lay off 8% of its employees last month.

Much of the cost for Mars sample return comes from its complexity. According to current plans, NASA would build a lander to retrieve the samples and a rocket to carry them off the surface to orbit Mars. ESA would contribute a spacecraft that would capture the samples in Mars orbit and transfer them to Earth. ESA has not discussed its budget for Mars sample return as publicly as NASA has, but European planetary scientists have expressed “consistent and strong science support” for the programme, says Gerhard Kminek, ESA’s lead scientist for Mars sample return in Noordwijk, the Netherlands.

If NASA and ESA can figure out a path forwards, the rock collection would touch down on Earth no earlier than 2033. Meanwhile, the agencies have competition: China has announced plans to return Mars rocks to Earth at around the same time.

[ad_2]

Source link

March 19, 2024
an early-career researcher’s checklist for prioritizing projects

[ad_1]

Deciding which projects to pursue can be daunting, but a simple framework can help you to decide which ones are the best fit.Credit: Biwa Studio/Getty

All three of us have, at some point in our academic careers, taken on one too many projects. For example, M. P. is finishing his PhD programme in management at the Wharton School of the University of Pennsylvania in Philadelphia. In his first three years there, he got involved in 12 projects. Eventually, he realized that he needed to narrow his focus to just a few, to get them over the finish line. In his fourth year, he dropped all but three — and the results of those projects have since been published in leading academic journals.

When E. T., who earned a PhD at the University of Virginia in Charlottesville, and J. M. C., whose PhD is from the University of California, Santa Barbara, first entered graduate school, they followed their broad interests. They were pulled in too many directions before carefully choosing where to expend their efforts.

Over our careers, we have learnt that time, energy and resources are precious. The path to success involves strategically focusing on select projects — not scattering attention across multiple fronts. Graduate students must be ruthless when working out which projects to prioritize.

People, passion, publishable

We’ve created a framework that we call the ‘3 Ps’ — people, passion and publishable — to determine which research lines to pursue and which to cast aside. Prioritize projects that involve people you like, as well as those that you are passionate about and that have strong potential to provide good, publishable results.

PhD student McKenzie Preston.Credit: McKenzie Preston

People. Work is more enjoyable, and feels easier, when it is done with friends or trusted colleagues whom you feel comfortable around. And having co-workers you like can make work more efficient, because you can communicate without overthinking about how you come across. The quality of your group’s work also benefits: in established relationships, people are open and receptive to feedback and ideas and can speak freely about potential issues before they become bigger problems.

Power dynamics can make it difficult for early-career scientists to freely choose whom they work with, and finding ways to manage conflict and avoid bad managers can be very difficult. But when you are able to choose your projects, good interpersonal relationships should be very high on your list of things to look for.

Underpaid and overworked: researchers abroad fall prey to bullying

Passion. Work on projects you care about. To publish a paper, you will have to stay interested in the concept for a long time. And at some point, you will have to work really hard. If you do not like or care about the topic, it can be hard to pull that off. E. T. started a project with two colleagues whom she respected, despite the research topic being outside her primary area of expertise. When both collaborators relocated, she was left in charge — and the project felt draining to work on. It fizzled out over time.

Passion comes from a few sources. You might have heard that other people’s passion can rub off on you, and you won’t realize how much you care until you get involved in a project. That can be true. But it is worth assessing your interest in a project periodically. You can also think about the project in terms of your professional goals and whether it will help you to meet them.

Publishable. Focus on the projects that are likely to produce the best results. What form those will take is, of course, dependent on your field. But for most scientists, getting the best results is required to publish papers in good journals.

Jacqueline M. Chen is an associate professor in psychology.Credit: University of Connecticut

For quantitative research, this could mean prioritizing projects that will yield robust, reproducible effects. If you are considering writing a registered report, which reviewers will decide whether to accept on the basis of the idea and methods alone, assess whether you can obtain results that pass quality checks. For qualitative research, this could mean prioritizing projects that uncover fresh perspectives, themes or phenomena. Remember, to publish, your goal is to obtain results that could inform theory, practice or policy. Data that are mixed, convoluted or not robust could be a major barrier to publication.

Should I work on this project? A checklist

Where should you expend your energy? This checklist can help scientists choosing between projects in their graduate studies. The first two items relate to people, the second two to passion and the last two to whether the data are publishable. List each project you are working on, and for each of the following statements, rate the project on a scale of 1 (strongly disagree) to 5 (strongly agree).

1. This project is with people I trust to be good scientists or scholars.

2. I look forward to meetings with collaborators about this project.

3. The topic of this project is interesting to me.

4. This project fits with my desired professional identity.

5. Data collection for this project is going well or is likely to go well.

6. The results seem to be robust or are likely to be robust.

Immediately disregard any projects that score a 1 in any category. Then charge ahead with the project with the highest score.

Competing Interests

This work was supported by US National Science Foundation grant number DGE-1943041 to J. M. C..

[ad_2]

Source link

March 15, 2024
Being a parent is a hidden scientific superpower — here’s why

[ad_1]

Being a parent is often seen as a career obstacle, but it can actually make you a better scientist, says nutrition epidemiologist Lindsey Smith Taillie.Credit: Paul Taillie

More than once in the past few years, in a variety of informal settings, I’ve overheard senior scientists recommend hiring people without children over those who are parents. Their reasoning, I gather, is that a parent might be smart and well-trained, but wouldn’t have the time or dedication to cut it in research. As a mid-career scientist with two young children, these comments floored me.

In my experience, these assumptions, typically aimed at faculty members or postdocs, are all too frequent. And although people tend to phrase their concerns in a gender-neutral way, about ‘parents’, they’re almost always talking about women. Women, who comprise only 33% of full professors despite accounting for more than 50% of the PhDs awarded each year, and who consistently have lower salaries than men across all ranks. Women, who still disproportionately do the bulk of domestic work, including childcare, around the globe. Although I’ve heard these comments more often from men, I’ve also heard female scientists essentially dismiss someone if they become pregnant, as if their career is over before really getting started.

The superhero skills needed to juggle science and motherhood

It’s true that being an academic woman with children is hard. In my field of global nutrition, it’s very common to have meetings at odd hours or need to travel at short notice. Dealing with school closures and frequent illnesses feels similar to playing whack-a-mole, needing to keep research moving while juggling childcare.

I have benefited from being white, heterosexual, married, neurotypical and working at a prestigious university. Crucially, I also benefit from having a husband, also a scientist, who does at least half of the childcare, cooking and cleaning, something that I think is still rare in heterosexual co-parenting relationships. Still, even with all of this privilege, it’s hard: there are many days when my brain feels shattered.

But, becoming a parent has also undoubtedly helped my career; both my rate of publishing and my number of grants won have increased substantially since my first daughter was born in 2017. I’ve become a more productive scientist. Here’s why.

Time scarcity

Those senior scientists who say that parents have less time are probably right: before I had children, I worked longer hours. I would go down rabbit holes into the early evening and often on weekends. I felt like I was always working and filling up all of my available time with research. But now, I write e-mails, papers and grant drafts like I am taking an exam: with intense focus and high speed. Having time constraints has forced me into a mindset of relentless prioritization, which has increased my scientific acumen and decision-making.

For example, last December, I was asked to present my research at a US Senate committee hearing on type 2 diabetes. I had only four days to put together a written testimony summarizing decades of data and build a case for why nutrition matters in diabetes prevention. My husband was out of town and, in a cruel twist of fate, one of my children got a throat infection. It was stressful, but I was able to draft the entire testimony in a single workday — something that, before having children, would have easily taken the entire four days. Also, because I knew that I’d need to rush off any second to tend to my sick child, I was able to push through my anxiety about writing such an important document and focus on getting pen to paper.

Why two scientist-mums made a database of parental-leave policies

Arguably, you could achieve this effect without children by having stronger work–life boundaries. That’s great, but it never worked for me. Having a non-negotiable deadline of school or day-care pick-up forced me to let go of my perfectionist tendencies, supercharging my productivity.

A fresh perspective

Becoming a parent also gave me a first-hand perspective on my field of nutrition. For example, similar to most young children, my three- and six-year-olds are picky eaters, and it’s been a challenge for me to get them to try new foods and eat veggies while also keeping food waste to a minimum. From social media, I discovered that giving my daughters tiny portions presented in a cute way — for example, a single broccoli floret with a toothpick and dip or a few spoons of soup in a colourful cupcake tin — helped with this. These experiences with my own children have helped me to incorporate families’ perspectives into my research design and to test interventions to prevent household food waste, increasing the chances that our interventions will be more effective for more people.

Parent networks

Even more importantly, becoming a parent has allowed me to create networks. I collaborate with colleagues who are also parents, and sharing our experiences has helped us to become friends, able to empathize and help each other out in a pinch, with work or with parenting.

Lindsey Smith Taillie’s experiences as a mother have helped to improve her food-waste intervention designs.Credit: Hacienda Mucuyche

This network has extended far beyond my immediate colleagues, too. Through the social-media platform Facebook, I have found an online community of academic mothers, which has become a treasure trove of help and advice. More than just tips on sippy cups or football clubs, people in the group share the hidden rules of playing the academic game, from handling job searches as a couple of two academics to going up for tenure or accepting tough grant reviews.

The networking benefits of parenthood translate to the team science, too. Sharing experiences about children helps to build rapport with collaborators — we’re able to bond over our common scientific challenges and laugh about our children’s silly stunts.

Emotional intelligence

Parenting has also made me a more effective teacher. For example, because my older daughter is obsessed with mythical creatures, I’m the proud owner of a giant inflatable pink unicorn costume — something that I have worn in class to demonstrate the power of food marketing, when discussing the Starbucks pink unicorn frappuccinos. It was silly, but that silliness has been helpful for connecting with students. Beyond pink unicorns, telling stories about my children in the classroom has made me more relatable and helped me to show key points about nutrition by invoking real-world examples. Parenting has helped me to expand my horizons and relate more to my students.

Could roving researchers help address the challenge of taking parental leave?

Being a parent has made me a better mentor, more able to support students who have children and helping me to treat all students as whole people, with a life outside science — whether or not that includes children. Because of my own experience, I feel better equipped to help my students to integrate the facets of their lives and find what balance looks like for them. It’s been difficult to speak out publicly about both the challenges and merits of parenting as a scientist. When I push back against things such as out-of-hours meetings, I worry about increasing biases against parents and especially mothers, perpetuating challenges to hiring and retaining them in the scientific workforce. But as time goes on, and I see these biases persist, I think that now is the time to speak up and be clear. Parenting isn’t my scientific kryptonite; it’s my superpower.

[ad_2]

Source link

March 13, 2024
Peer-replication model aims to address science’s ‘reproducibility crisis’

[ad_1]

An independent team could replicate select experiments in a paper before publication, to help catch errors and poor methodology.Credit: SolStock/Getty

Could the replication crisis in scientific literature be addressed by having scientists independently attempt to reproduce their peers’ key experiments during the publication process? And would teams be incentivized to do so by having the opportunity to report their findings in a citable paper, to be published alongside the original study?

These are questions being asked by two researchers who say that a formal peer-replication model could greatly benefit the scientific community.

Anders Rehfeld, a researcher in human sperm physiology at Copenhagen University Hospital, began considering alternatives to standard peer review after encountering a published study that could not be replicated in his laboratory. Rehfeld’s experiments¹ revealed that the original paper was flawed, but he found it very difficult to publish the findings and correct the scientific record.

“I sent my data to the original journal, and they didn’t care at all,” Rehfeld says. “It was very hard to get it published somewhere where you thought the reader of the original paper would find it.”

The issues that Rehfeld encountered could have been avoided if the original work had been replicated by others before publication, he argues. “If a reviewer had tried one simple experiment in their own lab, they could have seen that the core hypothesis of the paper was wrong.”

Rehfeld collaborated with Samuel Lord, a fluorescence-microscopy specialist at the University of California, San Francisco, to devise a new peer-replication model.

The replication crisis won’t be solved with broad brushstrokes

In a white paper detailing the process², Rehfeld, Lord and their colleagues describe how journal editors could invite peers to attempt to replicate select experiments of submitted or accepted papers by authors who have opted in. In the field of cell biology, for example, that might involve replicating a western blot, a technique used to detect proteins, or an RNA-interference experiment that tests the function of a certain gene. “Things that would take days or weeks, but not months, to do” would be replicated, Lord says.

The model is designed to incentivize all parties to participate. Peer replicators — unlike peer reviewers — would gain a citable publication, and the authors of the original paper would benefit from having their findings confirmed. Early-career faculty members at mainly undergraduate universities could be a good source of replicators: in addition to gaining citable replication reports to list on their CVs, they would get experience in performing new techniques in consultation with the original research team.

Rehfeld and Lord are discussing their idea with potential funders and journal editors, with the goal of running a pilot programme this year.

“I think most scientists would agree that some sort of certification process to indicate that a paper’s results are reproducible would benefit the scientific literature,” says Eric Sawey, executive editor of the journal Life Science Alliance, who plans to bring the idea to the publisher of his journal. “I think it would be a good look for any journal that would participate.”

Who pays?

Sawey says there are two key questions about the peer-replication model: who will pay for it, and who will find the labs to do the reproducibility tests? “It’s hard enough to find referees for peer review, so I can’t imagine cold e-mailing people, asking them to repeat the paper,” he says. Independent peer-review organizations, such as ASAPbio and Review Commons, might curate a list of interested labs, and could even decide which experiments will be replicated.

Lord says that having a third party organize the replication efforts would be great, and adds that funding “is a huge challenge”. According to the model, funding agencies and research foundations would ideally establish a new category of small grants devoted to peer replication. “It could also be covered by scientific societies, or publication fees,” Rehfeld says.

A controlled trial for reproducibility

It’s also important for journals to consider what happens when findings can’t be replicated. “If authors opt in, you’d like to think they’re quite confident that the work is reproducible,” says Sawey. “Ideally, what would come out of the process is an improved methods or protocols section, which ultimately allows the replicating lab to reproduce the work.”

Most important, says Rehfeld, is ensuring that the peer-replication reports are published, irrespective of the outcome. If replication fails, then the journal and original authors would choose what to do with the paper. If an editor were to decide that the original manuscript was seriously undermined, for example, they could stop it from being published, or retract it. Alternatively, they could publish the two reports together, and leave the readers to judge. “I could imagine peer replication not necessarily as an additional ‘gatekeeper’ used to reject manuscripts, but as additional context for readers alongside the original paper,” says Lord.

A difficult but worthwhile pursuit

Attempting to replicate others’ work can be a challenging, contentious undertaking, says Rick Danheiser, editor-in-chief of Organic Syntheses, an open-access chemistry journal in which all papers are checked for replicability by a member of the editorial board before publication. Even for research from a well-resourced, highly esteemed lab, serious problems can be uncovered during reproducibility checks, Danheiser says.

Replicability in a field such as synthetic organic chemistry — in which the identity and purity of every component in a reaction flask should already be known — is already challenging enough, so the variables at play in some areas of biology and other fields could pose a whole new level of difficulty, says Richard Sever, assistant director of Cold Spring Harbor Laboratory Press in New York, and co-founder of the bioRxiv and medRxiv preprint servers. “But just because it’s hard, doesn’t mean there might not be cases where peer replication would be helpful.”

How to make your research reproducible

The growing use of preprints, which decouple research dissemination from evaluation, allows some freedom to rethink peer evaluation, Sever adds. “I don’t think it could be universal, but the idea of replication being a formal part of evaluating at least some work seems like a good idea to me.”

An experiment to test a different peer-replication model in the social sciences is currently under way, says Anna Dreber Almenberg, who studies behavioural and experimental economics at the Stockholm School of Economics. Dreber is a board member of the Institute for Replication (I4R), an organization led by Abel Brodeur at University of Ottawa, which works to systematically reproduce and replicate research findings published in leading journals. In January, I4R entered an ongoing partnership with Nature Human Behaviour to attempt computational reproduction of data and findings of as many studies published from 2023 onwards as possible. Replication attempts from the first 18 months of the project will be gathered into a ‘meta-paper’ that will go through peer review and be considered for publication in the journal.

“It’s exciting to see how people from completely different research fields are working on related things, testing different policies to find out what works,” says Dreber. “That’s how I think we will solve this problem.”

[ad_2]

Source link

March 13, 2024
Artificial intelligence and illusions of understanding in scientific research

[ad_1]
Crabtree, G. Self-driving laboratories coming of age. Joule 4, 2538–2541 (2020).

Article
CAS

Google Scholar
Wang, H. et al. Scientific discovery in the age of artificial intelligence. Nature 620, 47–60 (2023). This review explores how AI can be incorporated across the research pipeline, drawing from a wide range of scientific disciplines.

Article
CAS
PubMed

Google Scholar
Dillion, D., Tandon, N., Gu, Y. & Gray, K. Can AI language models replace human participants? Trends Cogn. Sci. 27, 597–600 (2023).

Article
PubMed

Google Scholar
Grossmann, I. et al. AI and the transformation of social science research. Science 380, 1108–1109 (2023). This forward-looking article proposes a variety of ways to incorporate generative AI into social-sciences research.

Article
CAS
PubMed

Google Scholar
Gil, Y. Will AI write scientific papers in the future? AI Mag. 42, 3–15 (2022).

Google Scholar
Kitano, H. Nobel Turing Challenge: creating the engine for scientific discovery. npj Syst. Biol. Appl. 7, 29 (2021).

Article
PubMed
PubMed Central

Google Scholar
Benjamin, R. Race After Technology: Abolitionist Tools for the New Jim Code (Oxford Univ. Press, 2020). This book examines how social norms about race become embedded in technologies, even those that are focused on providing good societal outcomes.
Broussard, M. More Than a Glitch: Confronting Race, Gender, and Ability Bias in Tech (MIT Press, 2023).
Noble, S. U. Algorithms of Oppression: How Search Engines Reinforce Racism (New York Univ. Press, 2018).
Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S. On the dangers of stochastic parrots: can language models be too big? in Proc. 2021 ACM Conference on Fairness, Accountability, and Transparency 610–623 (Association for Computing Machinery, 2021). One of the first comprehensive critiques of large language models, this article draws attention to a host of issues that ought to be considered before taking up such tools.
Crawford, K. Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence (Yale Univ. Press, 2021).
Johnson, D. G. & Verdicchio, M. Reframing AI discourse. Minds Mach. 27, 575–590 (2017).

Article

Google Scholar
Atanasoski, N. & Vora, K. Surrogate Humanity: Race, Robots, and the Politics of Technological Futures (Duke Univ. Press, 2019).
Mitchell, M. & Krakauer, D. C. The debate over understanding in AI’s large language models. Proc. Natl Acad. Sci. USA 120, e2215907120 (2023).

Article
PubMed
PubMed Central

Google Scholar
Kidd, C. & Birhane, A. How AI can distort human beliefs. Science 380, 1222–1223 (2023).

Article
CAS
PubMed

Google Scholar
Birhane, A., Kasirzadeh, A., Leslie, D. & Wachter, S. Science in the age of large language models. Nat. Rev. Phys. 5, 277–280 (2023).

Article

Google Scholar
Kapoor, S. & Narayanan, A. Leakage and the reproducibility crisis in machine-learning-based science. Patterns 4, 100804 (2023).

Article
PubMed
PubMed Central

Google Scholar
Hullman, J., Kapoor, S., Nanayakkara, P., Gelman, A. & Narayanan, A. The worst of both worlds: a comparative analysis of errors in learning from data in psychology and machine learning. In Proc. 2022 AAAI/ACM Conference on AI, Ethics, and Society (eds Conitzer, V. et al.) 335–348 (Association for Computing Machinery, 2022).
Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206–215 (2019). This paper articulates the problems with attempting to explain AI systems that lack interpretability, and advocates for building interpretable models instead.

Article
PubMed
PubMed Central

Google Scholar
Crockett, M. J., Bai, X., Kapoor, S., Messeri, L. & Narayanan, A. The limitations of machine learning models for predicting scientific replicability. Proc. Natl Acad. Sci. USA 120, e2307596120 (2023).

Article
CAS
PubMed
PubMed Central

Google Scholar
Lazar, S. & Nelson, A. AI safety on whose terms? Science 381, 138 (2023).

Article
PubMed

Google Scholar
Collingridge, D. The Social Control of Technology (St Martin’s Press, 1980).
Wagner, G., Lukyanenko, R. & Paré, G. Artificial intelligence and the conduct of literature reviews. J. Inf. Technol. 37, 209–226 (2022).

Article

Google Scholar
Hutson, M. Artificial-intelligence tools aim to tame the coronavirus literature. Nature https://doi.org/10.1038/d41586-020-01733-7 (2020).

Article
PubMed

Google Scholar
Haas, Q. et al. Utilizing artificial intelligence to manage COVID-19 scientific evidence torrent with Risklick AI: a critical tool for pharmacology and therapy development. Pharmacology 106, 244–253 (2021).

Article
CAS
PubMed

Google Scholar
Müller, H., Pachnanda, S., Pahl, F. & Rosenqvist, C. The application of artificial intelligence on different types of literature reviews – a comparative study. In 2022 International Conference on Applied Artificial Intelligence (ICAPAI) https://doi.org/10.1109/ICAPAI55158.2022.9801564 (Institute of Electrical and Electronics Engineers, 2022).
van Dinter, R., Tekinerdogan, B. & Catal, C. Automation of systematic literature reviews: a systematic literature review. Inf. Softw. Technol. 136, 106589 (2021).

Article

Google Scholar
Aydın, Ö. & Karaarslan, E. OpenAI ChatGPT generated literature review: digital twin in healthcare. In Emerging Computer Technologies 2 (ed. Aydın, Ö.) 22–31 (İzmir Akademi Dernegi, 2022).
AlQuraishi, M. AlphaFold at CASP13. Bioinformatics 35, 4862–4865 (2019).

Article
CAS
PubMed
PubMed Central

Google Scholar
Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).

Article
CAS
PubMed
PubMed Central

Google Scholar
Lee, J. S., Kim, J. & Kim, P. M. Score-based generative modeling for de novo protein design. Nat. Computat. Sci. 3, 382–392 (2023).

Article
CAS

Google Scholar
Gómez-Bombarelli, R. et al. Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach. Nat. Mater. 15, 1120–1127 (2016).

Article
PubMed

Google Scholar
Krenn, M. et al. On scientific understanding with artificial intelligence. Nat. Rev. Phys. 4, 761–769 (2022).

Article
PubMed
PubMed Central

Google Scholar
Extance, A. How AI technology can tame the scientific literature. Nature 561, 273–274 (2018).

Article
CAS
PubMed

Google Scholar
Hastings, J. AI for Scientific Discovery (CRC Press, 2023). This book reviews current and future incorporation of AI into the scientific research pipeline.
Ahmed, A. et al. The future of academic publishing. Nat. Hum. Behav. 7, 1021–1026 (2023).

Article
PubMed

Google Scholar
Gray, K., Yam, K. C., Zhen’An, A. E., Wilbanks, D. & Waytz, A. The psychology of robots and artificial intelligence. In The Handbook of Social Psychology (eds Gilbert, D. et al.) (in the press).
Argyle, L. P. et al. Out of one, many: using language models to simulate human samples. Polit. Anal. 31, 337–351 (2023).

Article

Google Scholar
Aher, G., Arriaga, R. I. & Kalai, A. T. Using large language models to simulate multiple humans and replicate human subject studies. In Proc. 40th International Conference on Machine Learning (eds Krause, A. et al.) 337–371 (JMLR.org, 2023).
Binz, M. & Schulz, E. Using cognitive psychology to understand GPT-3. Proc. Natl Acad. Sci. USA 120, e2218523120 (2023).

Article
CAS
PubMed
PubMed Central

Google Scholar
Ornstein, J. T., Blasingame, E. N. & Truscott, J. S. How to train your stochastic parrot: large language models for political texts. Github, https://joeornstein.github.io/publications/ornstein-blasingame-truscott.pdf (2023).
He, S. et al. Learning to predict the cosmological structure formation. Proc. Natl Acad. Sci. USA 116, 13825–13832 (2019).

Article
MathSciNet
CAS
PubMed
PubMed Central

Google Scholar
Mahmood, F. et al. Deep adversarial training for multi-organ nuclei segmentation in histopathology images. IEEE Trans. Med. Imaging 39, 3257–3267 (2020).

Article
PubMed
PubMed Central

Google Scholar
Teixeira, B. et al. Generating synthetic X-ray images of a person from the surface geometry. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 9059–9067 (Institute of Electrical and Electronics Engineers, 2018).
Marouf, M. et al. Realistic in silico generation and augmentation of single-cell RNA-seq data using generative adversarial networks. Nat. Commun. 11, 166 (2020).

Article
CAS
PubMed
PubMed Central

Google Scholar
Watts, D. J. A twenty-first century science. Nature 445, 489 (2007).

Article
CAS
PubMed

Google Scholar
boyd, d. & Crawford, K. Critical questions for big data. Inf. Commun. Soc. 15, 662–679 (2012). This article assesses the ethical and epistemic implications of scientific and societal moves towards big data and provides a parallel case study for thinking about the risks of artificial intelligence.

Article

Google Scholar
Jolly, E. & Chang, L. J. The Flatland fallacy: moving beyond low–dimensional thinking. Top. Cogn. Sci. 11, 433–454 (2019).

Article
PubMed

Google Scholar
Yarkoni, T. & Westfall, J. Choosing prediction over explanation in psychology: lessons from machine learning. Perspect. Psychol. Sci. 12, 1100–1122 (2017).

Article
PubMed
PubMed Central

Google Scholar
Radivojac, P. et al. A large-scale evaluation of computational protein function prediction. Nat. Methods 10, 221–227 (2013).

Article
CAS
PubMed
PubMed Central

Google Scholar
Bileschi, M. L. et al. Using deep learning to annotate the protein universe. Nat. Biotechnol. 40, 932–937 (2022).

Article
CAS
PubMed

Google Scholar
Barkas, N. et al. Joint analysis of heterogeneous single-cell RNA-seq dataset collections. Nat. Methods 16, 695–698 (2019).

Article
CAS
PubMed
PubMed Central

Google Scholar
Demszky, D. et al. Using large language models in psychology. Nat. Rev. Psychol. 2, 688–701 (2023).

Article

Google Scholar
Karjus, A. Machine-assisted mixed methods: augmenting humanities and social sciences with artificial intelligence. Preprint at https://arxiv.org/abs/2309.14379 (2023).
Davies, A. et al. Advancing mathematics by guiding human intuition with AI. Nature 600, 70–74 (2021).

Article
CAS
PubMed
PubMed Central

Google Scholar
Peterson, J. C., Bourgin, D. D., Agrawal, M., Reichman, D. & Griffiths, T. L. Using large-scale experiments and machine learning to discover theories of human decision-making. Science 372, 1209–1214 (2021).

Article
CAS
PubMed

Google Scholar
Ilyas, A. et al. Adversarial examples are not bugs, they are features. Preprint at https://doi.org/10.48550/arXiv.1905.02175 (2019)
Semel, B. M. Listening like a computer: attentional tensions and mechanized care in psychiatric digital phenotyping. Sci. Technol. Hum. Values 47, 266–290 (2022).

Article

Google Scholar
Gil, Y. Thoughtful artificial intelligence: forging a new partnership for data science and scientific discovery. Data Sci. 1, 119–129 (2017).

Article

Google Scholar
Checco, A., Bracciale, L., Loreti, P., Pinfield, S. & Bianchi, G. AI-assisted peer review. Humanit. Soc. Sci. Commun. 8, 25 (2021).

Article

Google Scholar
Thelwall, M. Can the quality of published academic journal articles be assessed with machine learning? Quant. Sci. Stud. 3, 208–226 (2022).

Article

Google Scholar
Dhar, P. Peer review of scholarly research gets an AI boost. IEEE Spectrum spectrum.ieee.org/peer-review-of-scholarly-research-gets-an-ai-boost (2020).
Heaven, D. AI peer reviewers unleashed to ease publishing grind. Nature 563, 609–610 (2018).

Article
CAS
PubMed

Google Scholar
Conroy, G. How ChatGPT and other AI tools could disrupt scientific publishing. Nature 622, 234–236 (2023).

Article
CAS
PubMed

Google Scholar
Nosek, B. A. et al. Replicability, robustness, and reproducibility in psychological science. Annu. Rev. Psychol. 73, 719–748 (2022).

Article
PubMed

Google Scholar
Altmejd, A. et al. Predicting the replicability of social science lab experiments. PLoS ONE 14, e0225826 (2019).

Article
CAS
PubMed
PubMed Central

Google Scholar
Yang, Y., Youyou, W. & Uzzi, B. Estimating the deep replicability of scientific findings using human and artificial intelligence. Proc. Natl Acad. Sci. USA 117, 10762–10768 (2020).

Article
CAS
PubMed
PubMed Central

Google Scholar
Youyou, W., Yang, Y. & Uzzi, B. A discipline-wide investigation of the replicability of psychology papers over the past two decades. Proc. Natl Acad. Sci. USA 120, e2208863120 (2023).

Article
CAS
PubMed
PubMed Central

Google Scholar
Rabb, N., Fernbach, P. M. & Sloman, S. A. Individual representation in a community of knowledge. Trends Cogn. Sci. 23, 891–902 (2019). This comprehensive review paper documents the empirical evidence for distributed cognition in communities of knowledge and the resultant vulnerabilities to illusions of understanding.

Article
PubMed

Google Scholar
Rozenblit, L. & Keil, F. The misunderstood limits of folk science: an illusion of explanatory depth. Cogn. Sci. 26, 521–562 (2002). This paper provided an empirical demonstration of the illusion of explanatory depth, and inspired a programme of research in cognitive science on communities of knowledge.

Article
PubMed
PubMed Central

Google Scholar
Hutchins, E. Cognition in the Wild (MIT Press, 1995).
Lave, J. & Wenger, E. Situated Learning: Legitimate Peripheral Participation (Cambridge Univ. Press, 1991).
Kitcher, P. The division of cognitive labor. J. Philos. 87, 5–22 (1990).

Article

Google Scholar
Hardwig, J. Epistemic dependence. J. Philos. 82, 335–349 (1985).

Article

Google Scholar
Keil, F. in Oxford Studies In Epistemology (eds Gendler, T. S. & Hawthorne, J.) 143–166 (Oxford Academic, 2005).
Weisberg, M. & Muldoon, R. Epistemic landscapes and the division of cognitive labor. Philos. Sci. 76, 225–252 (2009).

Article

Google Scholar
Sloman, S. A. & Rabb, N. Your understanding is my understanding: evidence for a community of knowledge. Psychol. Sci. 27, 1451–1460 (2016).

Article
PubMed

Google Scholar
Wilson, R. A. & Keil, F. The shadows and shallows of explanation. Minds Mach. 8, 137–159 (1998).

Article

Google Scholar
Keil, F. C., Stein, C., Webb, L., Billings, V. D. & Rozenblit, L. Discerning the division of cognitive labor: an emerging understanding of how knowledge is clustered in other minds. Cogn. Sci. 32, 259–300 (2008).

Article
PubMed
PubMed Central

Google Scholar
Sperber, D. et al. Epistemic vigilance. Mind Lang. 25, 359–393 (2010).

Article

Google Scholar
Wilkenfeld, D. A., Plunkett, D. & Lombrozo, T. Depth and deference: when and why we attribute understanding. Philos. Stud. 173, 373–393 (2016).

Article

Google Scholar
Sparrow, B., Liu, J. & Wegner, D. M. Google effects on memory: cognitive consequences of having information at our fingertips. Science 333, 776–778 (2011).

Article
CAS
PubMed

Google Scholar
Fisher, M., Goddu, M. K. & Keil, F. C. Searching for explanations: how the internet inflates estimates of internal knowledge. J. Exp. Psychol. Gen. 144, 674–687 (2015).

Article
PubMed

Google Scholar
De Freitas, J., Agarwal, S., Schmitt, B. & Haslam, N. Psychological factors underlying attitudes toward AI tools. Nat. Hum. Behav. 7, 1845–1854 (2023).

Article
PubMed

Google Scholar
Castelo, N., Bos, M. W. & Lehmann, D. R. Task-dependent algorithm aversion. J. Mark. Res. 56, 809–825 (2019).

Article

Google Scholar
Cadario, R., Longoni, C. & Morewedge, C. K. Understanding, explaining, and utilizing medical artificial intelligence. Nat. Hum. Behav. 5, 1636–1642 (2021).

Article
PubMed

Google Scholar
Oktar, K. & Lombrozo, T. Deciding to be authentic: intuition is favored over deliberation when authenticity matters. Cognition 223, 105021 (2022).

Article
PubMed

Google Scholar
Bigman, Y. E., Yam, K. C., Marciano, D., Reynolds, S. J. & Gray, K. Threat of racial and economic inequality increases preference for algorithm decision-making. Comput. Hum. Behav. 122, 106859 (2021).

Article

Google Scholar
Claudy, M. C., Aquino, K. & Graso, M. Artificial intelligence can’t be charmed: the effects of impartiality on laypeople’s algorithmic preferences. Front. Psychol. 13, 898027 (2022).

Article
PubMed
PubMed Central

Google Scholar
Snyder, C., Keppler, S. & Leider, S. Algorithm reliance under pressure: the effect of customer load on service workers. Preprint at SSRN https://doi.org/10.2139/ssrn.4066823 (2022).
Bogert, E., Schecter, A. & Watson, R. T. Humans rely more on algorithms than social influence as a task becomes more difficult. Sci Rep. 11, 8028 (2021).

Article
CAS
PubMed
PubMed Central

Google Scholar
Raviv, A., Bar‐Tal, D., Raviv, A. & Abin, R. Measuring epistemic authority: studies of politicians and professors. Eur. J. Personal. 7, 119–138 (1993).

Article

Google Scholar
Cummings, L. The “trust” heuristic: arguments from authority in public health. Health Commun. 29, 1043–1056 (2014).

Article
PubMed

Google Scholar
Lee, M. K. Understanding perception of algorithmic decisions: fairness, trust, and emotion in response to algorithmic management. Big Data Soc. 5, https://doi.org/10.1177/2053951718756684 (2018).
Kissinger, H. A., Schmidt, E. & Huttenlocher, D. The Age of A.I. And Our Human Future (Little, Brown, 2021).
Lombrozo, T. Explanatory preferences shape learning and inference. Trends Cogn. Sci. 20, 748–759 (2016). This paper provides an overview of philosophical theories of explanatory virtues and reviews empirical evidence on the sorts of explanations people find satisfying.

Article
PubMed

Google Scholar
Vrantsidis, T. H. & Lombrozo, T. Simplicity as a cue to probability: multiple roles for simplicity in evaluating explanations. Cogn. Sci. 46, e13169 (2022).

Article
PubMed

Google Scholar
Johnson, S. G. B., Johnston, A. M., Toig, A. E. & Keil, F. C. Explanatory scope informs causal strength inferences. In Proc. 36th Annual Meeting of the Cognitive Science Society 2453–2458 (Cognitive Science Society, 2014).
Khemlani, S. S., Sussman, A. B. & Oppenheimer, D. M. Harry Potter and the sorcerer’s scope: latent scope biases in explanatory reasoning. Mem. Cognit. 39, 527–535 (2011).

Article
PubMed

Google Scholar
Liquin, E. G. & Lombrozo, T. Motivated to learn: an account of explanatory satisfaction. Cogn. Psychol. 132, 101453 (2022).

Article
PubMed

Google Scholar
Hopkins, E. J., Weisberg, D. S. & Taylor, J. C. V. The seductive allure is a reductive allure: people prefer scientific explanations that contain logically irrelevant reductive information. Cognition 155, 67–76 (2016).

Article
PubMed

Google Scholar
Weisberg, D. S., Hopkins, E. J. & Taylor, J. C. V. People’s explanatory preferences for scientific phenomena. Cogn. Res. Princ. Implic. 3, 44 (2018).

Article
PubMed
PubMed Central

Google Scholar
Jerez-Fernandez, A., Angulo, A. N. & Oppenheimer, D. M. Show me the numbers: precision as a cue to others’ confidence. Psychol. Sci. 25, 633–635 (2014).

Article
PubMed

Google Scholar
Kim, J., Giroux, M. & Lee, J. C. When do you trust AI? The effect of number presentation detail on consumer trust and acceptance of AI recommendations. Psychol. Mark. 38, 1140–1155 (2021).

Article

Google Scholar
Nguyen, C. T. The seductions of clarity. R. Inst. Philos. Suppl. 89, 227–255 (2021). This article describes how reductive and quantitative explanations can generate a sense of understanding that is not necessarily correlated with actual understanding.

Article

Google Scholar
Fisher, M., Smiley, A. H. & Grillo, T. L. H. Information without knowledge: the effects of internet search on learning. Memory 30, 375–387 (2022).

Article

Google Scholar
Eliseev, E. D. & Marsh, E. J. Understanding why searching the internet inflates confidence in explanatory ability. Appl. Cogn. Psychol. 37, 711–720 (2023).

Article

Google Scholar
Fisher, M. & Oppenheimer, D. M. Who knows what? Knowledge misattribution in the division of cognitive labor. J. Exp. Psychol. Appl. 27, 292–306 (2021).

Article
PubMed

Google Scholar
Chromik, M., Eiband, M., Buchner, F., Krüger, A. & Butz, A. I think I get your point, AI! The illusion of explanatory depth in explainable AI. In 26th International Conference on Intelligent User Interfaces (eds Hammond, T. et al.) 307–317 (Association for Computing Machinery, 2021).
Strevens, M. No understanding without explanation. Stud. Hist. Philos. Sci. A 44, 510–515 (2013).

Article

Google Scholar
Ylikoski, P. in Scientific Understanding: Philosophical Perspectives (eds De Regt, H. et al.) 100–119 (Univ. Pittsburgh Press, 2009).
Giudice, M. D. The prediction–explanation fallacy: a pervasive problem in scientific applications of machine learning. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/4vq8f (2021).
Hofman, J. M. et al. Integrating explanation and prediction in computational social science. Nature 595, 181–188 (2021). This paper highlights the advantages and disadvantages of explanatory versus predictive approaches to modelling, with a focus on applications to computational social science.

Article
CAS
PubMed

Google Scholar
Shmueli, G. To explain or to predict? Stat. Sci. 25, 289–310 (2010).

Article
MathSciNet

Google Scholar
Hofman, J. M., Sharma, A. & Watts, D. J. Prediction and explanation in social systems. Science 355, 486–488 (2017).

Article
CAS
PubMed

Google Scholar
Logg, J. M., Minson, J. A. & Moore, D. A. Algorithm appreciation: people prefer algorithmic to human judgment. Organ. Behav. Hum. Decis. Process. 151, 90–103 (2019).

Article

Google Scholar
Nguyen, C. T. Cognitive islands and runaway echo chambers: problems for epistemic dependence on experts. Synthese 197, 2803–2821 (2020).

Article

Google Scholar
Breiman, L. Statistical modeling: the two cultures. Stat. Sci. 16, 199–215 (2001).

Article
MathSciNet

Google Scholar
Gao, J. & Wang, D. Quantifying the benefit of artificial intelligence for scientific research. Preprint at arxiv.org/abs/2304.10578 (2023).
Hanson, B. et al. Garbage in, garbage out: mitigating risks and maximizing benefits of AI in research. Nature 623, 28–31 (2023).

Article
CAS
PubMed

Google Scholar
Kleinberg, J. & Raghavan, M. Algorithmic monoculture and social welfare. Proc. Natl Acad. Sci. USA 118, e2018340118 (2021). This paper uses formal modelling methods to demonstrate that when companies all rely on the same algorithm to make decisions (an algorithmic monoculture), the overall quality of those decisions is reduced because valuable options can slip through the cracks, even when the algorithm performs accurately for individual companies.

Article
MathSciNet
CAS
PubMed
PubMed Central

Google Scholar
Hofstra, B. et al. The diversity–innovation paradox in science. Proc. Natl Acad. Sci. USA 117, 9284–9291 (2020).

Article
CAS
PubMed
PubMed Central

Google Scholar
Hong, L. & Page, S. E. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proc. Natl Acad. Sci. USA 101, 16385–16389 (2004).

Article
CAS
PubMed
PubMed Central

Google Scholar
Page, S. E. Where diversity comes from and why it matters? Eur. J. Soc. Psychol. 44, 267–279 (2014). This article reviews research demonstrating the benefits of cognitive diversity and diversity in methodological approaches for problem solving and innovation.

Article

Google Scholar
Clarke, A. E. & Fujimura, J. H. (eds) The Right Tools for the Job: At Work in Twentieth-Century Life Sciences (Princeton Univ. Press, 2014).
Silva, V. J., Bonacelli, M. B. M. & Pacheco, C. A. Framing the effects of machine learning on science. AI Soc. https://doi.org/10.1007/s00146-022-01515-x (2022).
Sassenberg, K. & Ditrich, L. Research in social psychology changed between 2011 and 2016: larger sample sizes, more self-report measures, and more online studies. Adv. Methods Pract. Psychol. Sci. 2, 107–114 (2019).

Article

Google Scholar
Simon, A. F. & Wilder, D. Methods and measures in social and personality psychology: a comparison of JPSP publications in 1982 and 2016. J. Soc. Psychol. https://doi.org/10.1080/00224545.2022.2135088 (2022).

Article
PubMed

Google Scholar
Anderson, C. A. et al. The MTurkification of social and personality psychology. Pers. Soc. Psychol. Bull. 45, 842–850 (2019).

Article
PubMed

Google Scholar
Latour, B. in The Social After Gabriel Tarde: Debates and Assessments (ed. Candea, M.) 145–162 (Routledge, 2010).
Porter, T. M. Trust in Numbers: The Pursuit of Objectivity in Science and Public Life (Princeton Univ. Press, 1996).
Lazer, D. et al. Meaningful measures of human society in the twenty-first century. Nature 595, 189–196 (2021).

Article
CAS
PubMed

Google Scholar
Knox, D., Lucas, C. & Cho, W. K. T. Testing causal theories with learned proxies. Annu. Rev. Polit. Sci. 25, 419–441 (2022).

Article

Google Scholar
Barberá, P. Birds of the same feather tweet together: Bayesian ideal point estimation using Twitter data. Polit. Anal. 23, 76–91 (2015).

Article

Google Scholar
Brady, W. J., McLoughlin, K., Doan, T. N. & Crockett, M. J. How social learning amplifies moral outrage expression in online social networks. Sci. Adv. 7, eabe5641 (2021).

Article
PubMed
PubMed Central

Google Scholar
Barnes, J., Klinger, R. & im Walde, S. S. Assessing state-of-the-art sentiment models on state-of-the-art sentiment datasets. In Proc. 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (eds Balahur, A. et al.) 2–12 (Association for Computational Linguistics, 2017).
Gitelman, L. (ed.) “Raw Data” is an Oxymoron (MIT Press, 2013).
Breznau, N. et al. Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty. Proc. Natl Acad. Sci. USA 119, e2203150119 (2022). This study demonstrates how 73 research teams analysing the same dataset reached different conclusions about the relationship between immigration and public support for social policies, highlighting the subjectivity and uncertainty involved in analysing complex datasets.

Article
CAS
PubMed
PubMed Central

Google Scholar
Gillespie, T. in Media Technologies: Essays on Communication, Materiality, and Society (eds Gillespie, T. et al.) 167–194 (MIT Press, 2014).
Leonelli, S. Data-Centric Biology: A Philosophical Study (Univ. Chicago Press, 2016).
Wang, A., Kapoor, S., Barocas, S. & Narayanan, A. Against predictive optimization: on the legitimacy of decision-making algorithms that optimize predictive accuracy. ACM J. Responsib. Comput., https://doi.org/10.1145/3636509 (2023).
Athey, S. Beyond prediction: using big data for policy problems. Science 355, 483–485 (2017).

Article
CAS
PubMed

Google Scholar
del Rosario Martínez-Ordaz, R. Scientific understanding through big data: from ignorance to insights to understanding. Possibility Stud. Soc. 1, 279–299 (2023).

Article

Google Scholar
Nussberger, A.-M., Luo, L., Celis, L. E. & Crockett, M. J. Public attitudes value interpretability but prioritize accuracy in artificial intelligence. Nat. Commun. 13, 5821 (2022).

Article
CAS
PubMed
PubMed Central

Google Scholar
Zittrain, J. in The Cambridge Handbook of Responsible Artificial Intelligence: Interdisciplinary Perspectives (eds. Voeneky, S. et al.) 176–184 (Cambridge Univ. Press, 2022). This article articulates the epistemic risks of prioritizing predictive accuracy over explanatory understanding when AI tools are interacting in complex systems.
Shumailov, I. et al. The curse of recursion: training on generated data makes models forget. Preprint at arxiv.org/abs/2305.17493 (2023).
Latour, B. Science In Action: How to Follow Scientists and Engineers Through Society (Harvard Univ. Press, 1987). This book provides strategies and approaches for thinking about science as a social endeavour.
Franklin, S. Science as culture, cultures of science. Annu. Rev. Anthropol. 24, 163–184 (1995).

Article

Google Scholar
Haraway, D. Situated knowledges: the science question in feminism and the privilege of partial perspective. Fem. Stud. 14, 575–599 (1988). This article acknowledges that the objective ‘view from nowhere’ is unobtainable: knowledge, it argues, is always situated.

Article

Google Scholar
Harding, S. Objectivity and Diversity: Another Logic of Scientific Research (Univ. Chicago Press, 2015).
Longino, H. E. Science as Social Knowledge: Values and Objectivity in Scientific Inquiry (Princeton Univ. Press, 1990).
Daston, L. & Galison, P. Objectivity (Princeton Univ. Press, 2007). This book is a historical analysis of the shifting modes of ‘objectivity’ that scientists have pursued, arguing that objectivity is not a universal concept but that it shifts alongside scientific techniques and ambitions.
Prescod-Weinstein, C. Making Black women scientists under white empiricism: the racialization of epistemology in physics. Signs J. Women Cult. Soc. 45, 421–447 (2020).

Article

Google Scholar
Mavhunga, C. What Do Science, Technology, and Innovation Mean From Africa? (MIT Press, 2017).
Schiebinger, L. The Mind Has No Sex? Women in the Origins of Modern Science (Harvard Univ. Press, 1991).
Martin, E. The egg and the sperm: how science has constructed a romance based on stereotypical male–female roles. Signs J. Women Cult. Soc. 16, 485–501 (1991). This case study shows how assumptions about gender affect scientific theories, sometimes delaying the articulation of what might be considered to be more accurate descriptions of scientific phenomena.

Article

Google Scholar
Harding, S. Rethinking standpoint epistemology: What is “strong objectivity”? Centen. Rev. 36, 437–470 (1992). In this article, Harding outlines her position on ‘strong objectivity’, by which clearly articulating one’s standpoint can lead to more robust knowledge claims.

Google Scholar
Oreskes, N. Why Trust Science? (Princeton Univ. Press, 2019). This book introduces the reader to 20 years of scholarship in science and technology studies, arguing that the tools the discipline has for understanding science can help to reinstate public trust in the institution.
Rolin, K., Koskinen, I., Kuorikoski, J. & Reijula, S. Social and cognitive diversity in science: introduction. Synthese 202, 36 (2023).

Article

Google Scholar
Hong, L. & Page, S. E. Problem solving by heterogeneous agents. J. Econ. Theory 97, 123–163 (2001).

Article
MathSciNet

Google Scholar
Sulik, J., Bahrami, B. & Deroy, O. The diversity gap: when diversity matters for knowledge. Perspect. Psychol. Sci. 17, 752–767 (2022).

Article
PubMed

Google Scholar
Lungeanu, A., Whalen, R., Wu, Y. J., DeChurch, L. A. & Contractor, N. S. Diversity, networks, and innovation: a text analytic approach to measuring expertise diversity. Netw. Sci. 11, 36–64 (2023).

Article

Google Scholar
AlShebli, B. K., Rahwan, T. & Woon, W. L. The preeminence of ethnic diversity in scientific collaboration. Nat. Commun. 9, 5163 (2018).

Article
PubMed
PubMed Central

Google Scholar
Campbell, L. G., Mehtani, S., Dozier, M. E. & Rinehart, J. Gender-heterogeneous working groups produce higher quality science. PLoS ONE 8, e79147 (2013).

Article
CAS
PubMed
PubMed Central

Google Scholar
Nielsen, M. W., Bloch, C. W. & Schiebinger, L. Making gender diversity work for scientific discovery and innovation. Nat. Hum. Behav. 2, 726–734 (2018).

Article
PubMed

Google Scholar
Yang, Y., Tian, T. Y., Woodruff, T. K., Jones, B. F. & Uzzi, B. Gender-diverse teams produce more novel and higher-impact scientific ideas. Proc. Natl Acad. Sci. USA 119, e2200841119 (2022).

Article
CAS
PubMed
PubMed Central

Google Scholar
Kozlowski, D., Larivière, V., Sugimoto, C. R. & Monroe-White, T. Intersectional inequalities in science. Proc. Natl Acad. Sci. USA 119, e2113067119 (2022).

Article
CAS
PubMed
PubMed Central

Google Scholar
Fehr, C. & Jones, J. M. Culture, exploitation, and epistemic approaches to diversity. Synthese 200, 465 (2022).

Article
MathSciNet

Google Scholar
Nakadai, R., Nakawake, Y. & Shibasaki, S. AI language tools risk scientific diversity and innovation. Nat. Hum. Behav. 7, 1804–1805 (2023).

Article
PubMed

Google Scholar
National Academies of Sciences, Engineering, and Medicine et al. Advancing Antiracism, Diversity, Equity, and Inclusion in STEMM Organizations: Beyond Broadening Participation (National Academies Press, 2023).
Winner, L. Do artifacts have politics? Daedalus 109, 121–136 (1980).

Google Scholar
Eubanks, V. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (St. Martin’s Press, 2018).
Littmann, M. et al. Validity of machine learning in biology and medicine increased through collaborations across fields of expertise. Nat. Mach. Intell. 2, 18–24 (2020).

Article

Google Scholar
Carusi, A. et al. Medical artificial intelligence is as much social as it is technological. Nat. Mach. Intell. 5, 98–100 (2023).

Article

Google Scholar
Raghu, M. & Schmidt, E. A survey of deep learning for scientific discovery. Preprint at arxiv.org/abs/2003.11755 (2020).
Bishop, C. AI4Science to empower the fifth paradigm of scientific discovery. Microsoft Research Blog www.microsoft.com/en-us/research/blog/ai4science-to-empower-the-fifth-paradigm-of-scientific-discovery/ (2022).
Whittaker, M. The steep cost of capture. Interactions 28, 50–55 (2021).

Article

Google Scholar
Liesenfeld, A., Lopez, A. & Dingemanse, M. Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators. In Proc. 5th International Conference on Conversational User Interfaces 1–6 (Association for Computing Machinery, 2023).
Chu, J. S. G. & Evans, J. A. Slowed canonical progress in large fields of science. Proc. Natl Acad. Sci. USA 118, e2021636118 (2021).

Article
CAS
PubMed
PubMed Central

Google Scholar
Park, M., Leahey, E. & Funk, R. J. Papers and patents are becoming less disruptive over time. Nature 613, 138–144 (2023).

Article
CAS
PubMed

Google Scholar
Frith, U. Fast lane to slow science. Trends Cogn. Sci. 24, 1–2 (2020). This article explains the epistemic risks of a hyperfocus on scientific productivity and explores possible avenues for incentivizing the production of higher-quality science on a slower timescale.

Article
PubMed

Google Scholar
Stengers, I. Another Science is Possible: A Manifesto for Slow Science (Wiley, 2018).
Lake, B. M. & Baroni, M. Human-like systematic generalization through a meta-learning neural network. Nature 623, 115–121 (2023).

Article
CAS
PubMed
PubMed Central

Google Scholar
Feinman, R. & Lake, B. M. Learning task-general representations with generative neuro-symbolic modeling. Preprint at arxiv.org/abs/2006.14448 (2021).
Schölkopf, B. et al. Toward causal representation learning. Proc. IEEE 109, 612–634 (2021).

Article

Google Scholar
Mitchell, M. AI’s challenge of understanding the world. Science 382, eadm8175 (2023).

Article
PubMed

Google Scholar
Sartori, L. & Bocca, G. Minding the gap(s): public perceptions of AI and socio-technical imaginaries. AI Soc. 38, 443–458 (2023).

Article

Google Scholar

[ad_1]

Certain aspects of scientific life do not lend themselves to working from home. Archaeologist Adrià Breu, who studies neolithic pottery at the Autonomous University of Barcelona, Spain, can’t dig for artefacts in his kitchen, and Claudia Sala’s experiments in molecular microbiology at the Toscana Life Sciences Foundation in Siena, Italy, oblige her to commute to her laboratory most days. But both these researchers also get to work from home — when they write up papers, for example, or analyse data.

It’s a familiar story. The COVID-19 pandemic accelerated a move towards hybrid working in science, as in many other professions, with many universities and institutes formally allowing staff to divide their time between working in the office or lab and working at home. Millions of people altered their work patterns almost overnight, and the changes have stuck.

But the impact of this sweeping shift is less clear. Remote workers claim that they are happier and more productive. But some studies suggest that teams that work in close proximity, including academic research groups, produce higher-grade, more innovative results.

As hybrid working becomes established, researchers are racing to understand the full implications — for science and for everything else. Drawing on economics, psychology and communication theory, they are investigating many aspects of hybrid work, from the way people respond to e-mails and video calls, to how teams that are working remotely collaborate and transfer knowledge.

They are also exploring what science can offer to bridge the divide between office-based and remote teams, and thus make hybrid work a success.

Remote possibilities

Working remotely was an option for some people before the COVID-19 pandemic, but not for many. In 2016, just 4% of full paid days in the United States were worked from home. That proportion rose to as high as 60% in May 2020, and has since levelled out at about 25%. It’s a similar story in other countries. In UK government figures from 2022–23, almost half of workers reported spending some time working from home.

Researchers across the sciences have been ahead of the trend when it comes to working in geographically distant teams. As technology and policies have encouraged the exchange of ideas, data and materials, and as expertise has become more specialized, so the geographical spread of collaborating research teams has increased. A 2011 analysis¹ looked at the addresses of some 39 million authors of research papers, and found that the average collaboration distance had increased more or less linearly from 334 kilometres in 1980 to 1,553 km in 2009. This indicates that remote collaboration was well established by this point and that teams were becoming more international.

Another reason to love remote working: it’s good for the planet

Members of these remote research teams were generally not working from home. But the challenges of collaboration at a distance, and its reliance on technology rather than in-person communication, have much in common with how organizations and companies in all sectors are trying to build successful hybrid structures, says Ágnes Horvát, a communication and computer-science researcher who studies the impact of remote-working practices at Northwestern University in Evanston, Illinois.

In terms of the ways in which scientists work, says Horvát, “the problems we are facing are quite general”. This suggests that researchers can look to studies of remote or hybrid work at insurance firms and in other workplaces and apply the lessons to science, she adds.

There were plenty of studies to draw on even before the pandemic. Firms, researchers and business scholars have been tracking and predicting the consequences of remote working for decades.

In the 1980s, the US banking corporation American Express ran a successful pilot called Project Homebound, which was trialling a home-based alternative office system for people with disabilities. The project was hailed as a success, and the firm boasted of cost savings and increased productivity. But union officials were worried about exploitation, and called for a ban on “electronic home workstations”.

More recently, a series of small studies on specific groups such as call-centre workers and IT professionals have shown that fully remote workers tend to be less productive — by about 10–20%. They handle fewer calls, enter less data and take longer to perform the same tasks. This runs counter to claims in the early days of the pandemic that people who are based at home do more work than do those who are in the office².

In theory, hybrid work balances workers’ desire to be flexible with concerns from bosses about output. And a 2022 study of 1,612 engineers and marketing and finance employees at the global travel agent Trip.com seemed to back that up³. The company assigned people to work from the office either full time or for two days a week. Staff working the hybrid pattern were happier and less likely to leave the company than were those who worked from the office full time. The results, posted as a working paper and not yet peer reviewed, suggested that, although the team members who were assigned to the hybrid group worked different hours and patterns from those who were office-based, the overall productivity of the groups was the same. Workers with longer commutes were more likely to report the benefits of being remote.

Permanently remote

Although such post-pandemic analyses are providing useful data, say researchers, they need longer-term studies to fully assess the rise of remote work.

“The pandemic showed us the effect of working from home in a rather short run, but we need much more evidence of what’s going to happen if we really keep on working remotely for years,” says Marina Schröder, an innovation economist at Leibniz University Hannover in Germany. She studies the effects of remote working on creativity, and has shown, for example, that communicating through chat software results in less innovation when compared with face-to-face conversation.

Late last year, one such long-term study led by Carl Frey, an economist at the University of Oxford, UK, produced the strongest evidence yet that remote work can alter the nature and quality of what researchers collectively produce⁴.

Science-ing from home

Those based at the same site make more breakthrough discoveries, the team found. Although remote collaborators benefit from greater collective knowledge, such teams are less likely to be creative, and are better suited to making incremental progress.

“We showed in the paper that remote teams are more likely to collaborate in technical tasks,” says Frey, “whereas on-site teams are much more likely to collaborate actually in the conceptualization of new ideas.”

The study analysed 20 million research articles published between 1960 and 2020, and 4 million patent applications submitted between 1976 and 2020, around the globe. The researchers looked at the affiliations and geographical range of the contributors, and used citation analysis to assess how ‘disruptive’ publications were.

When the collaboration distance increased from 0 kilometres to more than 600 kilometres, the probability of disruption fell by about 20%. “Remote teams are less likely to create breakthrough findings,” Frey says.

Horvát says that the study provides a valuable note of caution amid the rush to embrace remote working. “This is not the way we want science to evolve. So, I think we absolutely need to take this very seriously.”

Innovation decline

What could be contributing to this trend? “Somehow the ideation process is more difficult when it’s mediated with technology. I think that’s as close to a mechanism as we have,” says Horvát. “That’s an important lack of knowledge on our part, because how are we going to fix it if we don’t know what’s causing it?”

Frey says that there could be several explanations for the decline in innovation. One is the value of sporadic encounters, which are much more likely when people work in the same place.

Meeting in person also exposes people to more knowledge. “If you go for lunch together and things like that, you get more ideas that are sort of filtered down to you because other people have read a lot of stuff themselves.”

A third possibility is what Frey calls collaboration intensity, which drives innovation by bringing together existing ideas from different fields.

“Fusing ideas takes time and effort,” he says. “It can click sometimes, but usually it’s a process. And it’s harder if you’re not in the same place and if you’re not communicating on a very regular basis.”

The nature of online communication, with appointments and priorities, is quite structured and hierarchical, adds Lingfei Wu, an information scientist at the University of Pittsburgh in Pennsylvania, who worked with Frey on the study. This can impede informal conversations and the casual generation of ideas, and might make it harder, for example, for early-career scientists to communicate with more senior colleagues.

“Those who went through junior stage, we all understand how hard it is to get a senior professor to respond to an e-mail,” says Wu. “But if you actually run into a senior professor in the hallway, then it’s easier to say you have a couple of ideas.”

An employee works on a laptop with colourful sticky notes taped to the windows behind — An employee at the headquarters of an online marketplace in Singapore.Credit: Ore Huiying/Bloomberg/Getty

He points to this effect in the data that were collected for the remote-collaboration study. By assessing the relative status (based on numbers of citations) of co-authors on published papers, the analysis showed that collaborations between researchers of markedly different status were much more common when the two individuals in question shared an office or building than when they worked remotely from each other.

A lack of collaboration could have negative consequences for scientists of any status: in a preprint that has not yet been peer reviewed⁵, Wu’s team shows that younger scientists can help older scientists to produce more innovative work.

The group performed an analysis of 241 million articles published by more than 244 million researchers over the past two centuries, and examined the related patterns of citations. It found that the longer that scientists work in a field, the less often their research is classified as disruptive. That trend has become more pronounced in recent decades. In the 1960s, researchers with 20 years of experience produced more than 2% of the most disruptive work. By the 1990s, that had slipped to less than 0.5%.

In a finding that will surprise few early-career researchers, the analysis of publications and how they were cited showed that older scientists were much more likely to criticize emerging work than they were to produce innovative research themselves.

Remote collaboration and the lack of sporadic in-person encounters could be reinforcing hierarchies and exacerbating the trend, Wu says.

Water-cooler effect

The value of spontaneous in-person encounters for generating ideas — known as the water-cooler effect – is especially associated with creativity. And a 2022 study⁶ from two US social scientists demonstrated that communicating through screens can’t replicate this personal touch.

Melanie Brucks at Columbia University in New York City and Jonathan Levav at Stanford University in California asked pairs of volunteers to think of alternative uses for objects such as a frisbee disc and bubble wrap. Half of the creating couples worked in the same room, whereas the other half communicated by video call using laptops. The researchers also set up a similar study among pairs of engineers working on product design in five office locations around the world.

The remote collaborations created fewer ideas than the in-person teams did. But, in follow-up tests once the ideas had been generated, the remote pairs were just as effective as were the in-person pairs — or more so — at analysing the options and deciding which they should pursue.

How hybrid working took hold in science

How did the screens limit their creativity? Eye-tracking technology showed that the virtual couples paid more attention to each other — and the screens did not seem to stop the pairs from generating feelings of connection and trust, or to prevent them from mimicking each other’s language or facial expressions. Instead, the researchers argue that concentrating on a relatively small screen narrows cognitive focus. In turn, this switches off the mental ability to associate and combine concepts, which underlies ideation.

Face-to-face meetings could also boost creativity because they enable teams to fully exploit collective knowledge, in a way that remote collaboration does not.

“If my teammate is really good and I’m seeing what they’re producing, that’s kind of impactful to me,” says Glenn Dutcher, an economist at Ohio University in Athens, who has studied the effect.

Zoom fatigue

Similarly to other industries, some labs have seen the value of in-person meetings and have moved to restore them. “We met on site for the first time after nearly two years last December and were all surprised by how good it felt to be all back in the same room,” says Viktor von Wyl, an epidemiologist at the University of Zurich, Switzerland, who runs a lab of ten people. “We have now decided to go back to at least one team meeting per month in person.”

Although videoconferencing might not be as effective as meeting in person, it still conveys higher-grade communication than do tools such as e-mail and instant messaging. That’s because psychologists see phone and video calls as ‘synchronous’ media, in which communicating in real time helps participants converge on the meaning of complex information. E-mails and messaging, by contrast, are asynchronous channels that are better suited to simply conveying that information. And when people work remotely, they tend to send e-mails.

That effect was shown by the computer giant Microsoft, which used the enforced shift to remote working as a natural experiment⁷ to assess how the company’s 61,000 staff members in the United States responded in the first half of 2020. The analysis showed that remote working actually decreased the number of video or phone calls across the company, as staff switched to e-mail and messaging.

Something similar showed up in the data from the Trip.com analysis³. Hybrid workers, the study found, were more likely to message colleagues than they were to use the phone or speak to them in person, even when they were all in the office.

Horvát argues that ongoing improvements in technology could fix some – but not all – of the issues with remote work, including its impact on creativity. Experiments with virtual reality, for example, have shown that participants can use and pick up on gestures and body language, which is a crucial part of in-person communication. And file and data sharing through the cloud have streamlined the way in which remote teams carry out joint projects.

“Technology looks very different now, especially post-COVID,” she says.

There are certainly reasons to be cheerful about the future of at least some remote collaborations.

In a 2022 working paper⁸ (not yet peer reviewed and published), Frey and his colleagues at the University of Oxford looked at remote collaboration and scientific innovation from 1961 to 2020, and found a surprising twist. After 2010, scientific papers written by remote collaborators were more likely to contain breakthroughs than were papers written by single-location teams.

Unlike their 2023 study⁴, which found fewer breakthroughs over time, this analysis looks only at the output of existing teams that start on site and switch to remote working; it does not capture the impact of teams that have always been remote.

The switch after 2010 makes sense, says Nick Bloom, an economist at Stanford University, because that’s when file-sharing technologies such as Dropbox emerged. (Bloom studies remote working and has co-authored two papers on the subject²^,³.) Frey adds that the trend after 2010 could be due to what economists call knowledge spillovers — each collaborator exposes others in their home institution to the ideas.

Researchers who study work patterns say that there’s no single solution that optimizes everything about jobs, especially in science. Although breakthroughs are important in research, says Dutcher, they often require major investments, such as getting people together. “We need the big discoveries, and for those maybe we need face-to-face meetings,” he says. “But we also need the small advancements.”

[ad_2]

Source link