Tag: Publishing

COVID’s preprint bump set to have lasting effect on research publishing

[ad_1]

Researchers in Nantes, France, working on a COVID-19 vaccine in 2021. The use of preprints to disseminate research findings saw a major uptick during the pandemic.Credit: Loic Venance/AFP/Getty

The COVID-19 pandemic saw an explosion in publication of preprint articles, many by authors who had never produced one before. Now it seems a high proportion of these scientists are likely to continue the practice.

A survey published in PeerJ¹ questioned researchers who had posted preprints relating to COVID-19 or the virus SARS-CoV-2 in 2020, across four preprint servers: arXiv, bioRxiv, medRxiv and ChemRxiv. Of the 673 people who completed the survey, just under 58% had posted their preprints on the biomedical server medRxiv; around 18% on arXiv, which focuses on mathematics and physical sciences; 14% on the life-sciences server bioRxiv; and 7% on ChemRxiv, a chemistry repository.

For two-thirds of respondents, this was the first time they had published a preprint. Almost 80% of these said they intended to post preprints of at least some of their papers going forward.

AI writes summaries of preprints in bioRxiv trial

One of the most intriguing findings is the number of respondents who received feedback on their preprints, says study co-author Narmin Rzayeva, a scientometrics researchers at Leiden University in the Netherlands. Fifty-three per cent received comments from peers, more than half of which were delivered privately through closed channels such as by e-mail or during meetings. Around 20% of respondents received comments on the preprint platforms, which are publicly accessible.

“We expected much lower numbers,” Rzayeva says, because preprint papers don’t typically receive much feedback.

Previous work² found that by the end of December 2021, just 8% of preprints posted on medRxiv since it launched in mid-2019 had received comments online. But that study considered only publicly posted comments.

The impact of feedback

Preprint feedback is having an effect, albeit unevenly. Of all survey respondents, just 1.9% reported making major changes to the results section of their preprints as a result of feedback. By contrast, 10.1% received such changes in response to peer review conducted as part of conventional journal publication. Rzayeva suspects that this is partly because authors feel obliged to make changes after receiving feedback from journal peer reviewers.

Of the survey respondents who reported receiving feedback on their preprints, 21.2% said they had made substantial changes to their discussion and conclusions sections. “I find it pretty exciting and encouraging that authors are making the amount of changes to their preprints that they do in response to preprint commentary,” says Jessica Polka, executive director of ASAPbio, a non-profit organization in San Francisco, California, that promotes innovation in the life sciences.

Polka notes that preprint feedback tends not to be as thorough as a review commissioned by a journal. An analysis of comments left on bioRxiv preprints posted between May 2015 and September 2019 found that only around 12% of non-author comments resembled those from conventional peer review³.

Australian funder backflips on controversial preprint ban

Polka encourages researchers to strike up discussions over preprints. “By conducting peer review in the open, you integrate many more perspectives than you would by doing it behind closed doors,” she says.

The preprint experience seems to have been positive for the survey respondents, 87% of whom said they had later submitted their paper to a peer-reviewed journal. Preprints shouldn’t replace journal articles, Rzayeva says, but should complement them and become an integral part of the publishing system.

Taking AI into account

Rzayeva acknowledges that the survey covered only 4 servers, which accounted for around 55% of all COVID-19 preprints published in 2020. As with most surveys, there was also a self-selection bias, meaning that the proportion of individuals with certain views could be overestimated.

Anita Bandrowski, an information scientist at the University of California, San Diego, says the survey is important, but notes that it did not consider artificial intelligence (AI) tools that are giving automated feedback on preprints. Bandrowski was part of a group of biologists and software specialists who developed a set of automated tools that measure the rigour and reproducibility of COVID-19 preprints and post the results on the social-media platform X.

Similar tools could become common as researchers consider ways to assess the rapidly growing number of preprints, and it will be important to find ways to track the results, says Bandrowski. She predicts that there will be “much more adoption of preprints in the future among biologists” as a result of researchers dipping their toes in during the pandemic.

Polka agrees. “The pandemic gave us a window into what is possible with preprints. It’s just a matter of tweaking policies in order to make use of that potential.”

[ad_2]

Source link

February 9, 2024
Book, Movie, and Product Reviews Are Being Bought and Paid For

[ad_1]

Anyone who writes reviews for a living has heard it before, and plenty: “How much did you get paid to write this?”

I’ve been a critic of many things over the years: movies, wine and spirits, and all manner of tech gear, for WIRED and other publications. And no matter what it is that I’m writing about, there’s always that one guy who pipes up in the comments suggesting that my opinions were bought and paid for.

It was invariably easy to dismiss these comments, but things got more complicated in September, when Vulture published a story that revealed the untold scale of the paid reviews industry. The story showed, among other things, how publicists were paying some independent film critics to review indie films and non-mainstream releases. These reviews, which were often published on independent film review websites, were then getting grabbed by Rotten Tomatoes. This meant, the story suggested, that a coveted Certified Fresh score on the hallowed Tomatometer could potentially be bought, and not earned.

The story caused chaos in the film industry.

Cast an eye beyond the world of art houses and streaming services, and you soon realize that this practice is commonplace. Reviews of everything—from gadgets to books, apparel, hotels, booze, you name it—are all potentially compromised, depending on your definition of that word. And the more you dig, the weirder things get.

In the wake of Vulture’s story, Rotten Tomatoes took action and began to boot movie reviewers who it believed had taken payments off the platform. In doing so, the company upended the lives of many film reviewers and blew a hole in a common tactic employed by indie titles to get visibility. Defenders of the practice argued that those smaller films would have gone unnoticed by critics absent a financial incentive to watch them.

The scenario points to a fundamental paradox in online reviews. Indie films—heck, indie anything—make the creative industry a better place, and boosting their signal above the noise is a net win for anyone with tastes outside of the mainstream. The practice of amplifying these independent voices by paying for coverage can be seen as deceitful, dishonest, and mercenary by readers who aren’t aware of the bigger picture.

That bigger picture is in fact a blockbuster. No matter what you produce, there’s probably a way to buy a review for it. A network of platforms exists to connect filmmakers, authors, and product manufacturers with writers, blogs, and publications who can boost their brand for a fee. My inbox is inundated by overseas manufacturers of white-label tech products who are desperate to pay me to write a review if I can get it published in WIRED or another outlet. I politely declined, and for decades I never accepted outside payment to write a review of a product.

Until, one day, I did.

The Trouble With Bunker 15

Lane Brown’s piece in Vulture, “The Decomposition of Rotten Tomatoes,” claimed that the popular movie review site could be “easily hacked.” At the core of the article is a publicity company called Bunker 15. It’s one of many businesses that help independent filmmakers get reviews for their movies that can count toward the all-important Rotten Tomatoes Tomatometer rating. For the service, it pays some reviewers $50 per review.

Brown emailed me before his story was published to ask if I’d been paid by Bunker 15 for my review of the film Ophelia–also central to his piece–and, honestly, I didn’t know if I had or not. I published my review at Film Racket, an independent film website that I’ve run since 2013, more than five years ago, and I don’t have records going back that far. I told Brown it was possible, and that we did work with Bunker 15 on other films over the years. After the story was published I did more digging and discovered that, yes, I was one of the critics who was paid $50 to write a review of the movie, and that it was probably the first film the company ever submitted to Film Racket for proposed coverage. It’s not a great movie, but I gave it three stars out of five, which Rotten Tomatoes marked as “fresh.” It remains the only review I have ever personally written of a Bunker 15 film or for which I’ve been paid by a third party; other writers did the rest.

[ad_2]

Source link

February 7, 2024
Fake research papers flagged by analysing authorship trends

[ad_1]

A new method searches the scholarly literature for trends in authorship that indicate paper-mill activity.Credit: Zoonar GmbH/Alamy

A research-technology firm has developed a new approach to help identify journal articles that originate from paper mills — companies that churn out fake or poor-quality studies and sell authorships.

The technique, described in a preprint posted on arXiv last month¹, uses factors such as the combination of a paper’s authors to flag suspicious studies. Its developers at London-based firm Digital Science say it can help to identify cases in which researchers might have bought their way onto a paper.

Science’s fake-paper problem: high-profile effort will tackle paper mills

Previous efforts to detect the products of paper mills have tended to focus on analysing the content of the manuscripts. One online tool, for example, searches papers for tortured phrases — strange alternative turns of phrase for existing terminology produced by software designed to avoid plagiarism detection. Another tool, being piloted by the International Association of Scientific, Technical, and Medical Publishers (STM), flags when identical manuscripts are submitted to several journals or publishers at the same time.

An approach that instead analyses the relationships between authors could be valuable as paper mills become better at producing convincing text, says Hylke Koers, chief information officer at the STM, who is based in Utrecht, the Netherlands. “This is the kind of signal that is much more difficult to work around or outcompete by clever use of generative AI.”

Unusual patterns

Paper mills are a growing problem for publishers — according to one estimate, around 2% of all published papers in 2022 resembled studies produced by paper mills — and in recent years publishers have stepped up efforts to tackle them.

As well as being of poor quality, often containing made-up data and nonsensical text, the articles that paper mills churn out are frequently padded with researchers who buy authorship on manuscripts already accepted for publication. Some paper mills claim to have brokered tens of thousands of authorships — including in journals that are indexed in respected databases, such as Web of Science and Scopus.

This can create unusual patterns of co-authorship and networks of researchers that are different from those in legitimate research, says Simon Porter, vice-president for research futures at Digital Science.

Multimillion-dollar trade in paper authorships alarms publishers

Under normal circumstances, “you would expect to find behaviour where a young researcher is publishing with their supervisor, and starts to branch out a little later and publish with other people”, Porter says. “You can see an evolution; it’s not a random network.”

This is not the case with paper-mill works. The technology that Porter developed, together with Leslie McIntosh, vice-president for research integrity at Digital Science, searches for trends that indicate paper-mill activity. These include co-author networks composed of early-career researchers who suddenly have a spike in publications, and papers featuring several authors who have no publication history or a collection of collaborators who are unlikely to have worked together, such as authors from several locations or unrelated disciplines.

When they compared the new technique’s results with those of the Problematic Paper Screener, a tool that searches for tortured phrases and other red flags, Porter and McIntosh identified a significant overlap. Around 10% of authors were directly flagged by both tools, their study found, and 72% of authors in the ‘author networks’ data set can be linked through co-authorship to those in the ‘tortured phrases’ data set.

Technology tricks

Although paper mills have quickly evolved so that fewer papers with tortured phrases are being published, Porter thinks the companies will find it difficult to circumvent flagging by these tools while keeping their current business model.

Digital Science has posted the code underlying the technique online, and Porter says that publishers could begin using it straight away.

Joris Van Rossum, programme director at STM Solutions in Amsterdam, says his organization will consider adding the new technology to the STM Integrity Hub — a collection of resources and tools designed to help publishers to detect fraudulent papers.

AI intensifies fight against ‘paper mills’ that churn out fake research

Chris Graf, research-integrity director at Springer Nature in London, says that obstacles remain, particularly in distinguishing between researchers who share a name and weeding out authors who are flagged erroneously. “We have found that there can be some challenges with data consistency in this context that mean this is not straightforward,” Graf says. “Very brilliant young researchers with a low cluster coefficient could show up as false positives, which is clearly far from ideal.” But he adds: “Having said that, we are exploring a lot of different options, and nothing is off the table.” (Nature’s news team is independent of Springer Nature, its publisher.)

Anna Abalkina, a sociologist at the Free University of Berlin who has been tracking paper-mill studies for years, says it’s a good idea to scrutinize author networks. “Paper mills definitely do have collaboration anomalies,” she says.

Abalkina warns, however, that our knowledge of paper mills’ business models and processes is limited. It is also difficult to prove that a published study is definitely the product of a paper mill, she notes, which makes it hard to use that as a reason for retraction.

Ultimately, “it’s going to take every trick in the book to be able to provide a convincing filter for paper mills”, Porter says. “It won’t just be one technique.”

[ad_2]

Source link

February 7, 2024
Open science — embrace it before it’s too late

[ad_1]

Biodiversity science is benefiting from volunteer researchers (seen here working at Chicago’s Fields Museum).Credit: Nancy Stone/Chicago Tribune/Tribune News Service/Getty

The ‘open science’ concept is gaining more followers, not least through the efforts of the cultural organization UNESCO. Over the past several years, the organization has been consulting on how science can become more collaborative, transparent, accessible, equitable and inclusive, which are all attributes of open science. And in 2021, it published a framework for what a genuinely open science could look like.

At the end of last year, UNESCO, which is headquartered in Paris, published a report on the current status of this endeavour. The report makes it clear that, although there are instances of good practice, there is still much work to do to fulfil the potential of open science globally.

In 2021, UNESCO’s member states agreed on a definition of open science that includes open access to scientific knowledge (including the humanities and social sciences); open access to research infrastructure; open collaboration between scientists and ‘societal actors’ (essentially, all those who are not scientists); and open dialogue between different knowledge systems, including between scientific knowledge and Indigenous knowledge.

The world’s goals to save humanity are hugely ambitious — but they are still the best option

Member states also pledged to incorporate the concept into their research systems, including using open-science principles in publicly funded research; supporting non-profit and community-driven publishing; encouraging the publication of research in more languages; and incentivizing the private sector to join discussions about achieving open-science goals.

UNESCO’s report describes several examples of positive initiatives, such as in research collaboration, open-access scientific publishing and public engagement in science. For example, in 2020, the Brazilian government launched the National Platform of Research Infrastructure, a digital platform in which scientific institutions can register their available infrastructure, and make it available to researchers outside their organization. This is an excellent way to spread access to expensive equipment across the research community.

Meanwhile, South African policymakers are consulting researchers to help to create a national open-science policy for the whole country. The aim here is to build more transparency, scrutiny and reproducibility into the country’s research system. The policy will also include measures to monitor progress.

The European Commission, based in Brussels, was an early proponent of open science. Between 2002 and 2020, it increased its funding for ‘societal engagement’ projects from €88 million to €462 million — an amount that is now equivalent to US$500 million. Moreover, a decade ago, all scientific publications arising from the European Union’s €80 billion Horizon 2020 programme needed to be published open-access. Citizen science is another growing area in open science with much promise, UNESCO notes. By 2018, half of all records in the Global Biodiversity Information Facility — an international open-access data repository based in Copenhagen — were from citizen scientists, up from around 10% in around 2007.

Other indicators are less rosy, however. Around three-quarters (73%) of publications in open-access repositories are in just six languages — with nearly half (46%) being in English alone. And in spite of some of the progress mentioned, overall the report finds that scientific institutions, such as universities, national science academies and journals, are struggling to include communities, in all their diversity, in the process of creating scientific knowledge itself.

Open science, done wrong, will compound inequities

Open science aligns with UNESCO’s founding mission for science and education to benefit all of humanity; and with the idea that access to science is a human right. But the organization’s interest in open science goes beyond these broad founding principles.

The UN’s Sustainable Development Goals (SDGs), adopted in 2015, are humanity’s best attempt to map a pathway towards a better future — and a more open approach to science could have a larger part to play in achieving them.

That effort needs as much help as it can get: only about 12% of the SDG targets are likely to be met by the 2030 deadline. Monitoring SDG indicators is one obvious way that citizen scientists can help. Some of the largest gaps in the collection of relevant SDG data are in low- and middle-income countries, which is where citizen research can really make a difference. In 2020, Dilek Fraisl, a data researcher at the International Institute for Applied Systems Analysis in Laxenburg, Austria, and her colleagues found that citizen-science projects were already helping to monitor at least five SDG indicators (D. Fraisl et al. Sustain. Sci. 15, 1735–1751; 2020). At the time, more than half of the data collected on indicators for sustainable cities, good health and well-being, and clean water and sanitation were provided by citizen scientists.

There’s scope for citizen scientists to do more. UN agencies have also recognized the potential of connecting citizen scientists with official data bodies. The UN Statistical Commission and UN Women are working with researchers in civil society organizations to produce resources, such as toolkits for producers of citizen-generated data.

The UNESCO report shines a much-needed light on some promising developments in open science. The challenge will be how to accumulate individual examples of good practice into something similar to a critical mass, so that, in cases such as monitoring the SDGs, they can be harnessed to get the world to where it needs to be.

[ad_2]

Source link

February 6, 2024
Innovative funding systems are key to fighting inequities in African science

[ad_1]

With African investment in research and development (R&D) still well below the global average, African higher-education and research institutions rely on grants from outside the continent. This is not ideal, but it will be inevitable until African countries follow through on their promises to spend more on research.

Most research grants are merit-based — intended to support the best ideas with the greatest potential for success — but this risks funnelling most of the funds to a few researchers in rich countries and at institutions that have built prestige and reputations. Poorer countries and newly established institutions are struggling to compete. A more innovative and equitable funding system is needed to ensure that they don’t get left behind.

Grant makers — foundations, corporations and government agencies that fund research grants — are exploring new funding models to meet the needs of African researchers. A promising example is the hub-and-spoke model, which aims to distribute resources and knowledge in ways that balance merit with equity. The system features a centralized hub that receives funding and allocates it to each of its spokes, which are sprawled out around the wheel.

Africa’s future depends on government-funded R&D

In practice, the central hub is usually an African research centre or university that receives funding from grant makers and manages all of the procedures surrounding the award. Auxiliary institutions receive sub-grants from the hub to conduct defined research projects on behalf of the group. These spokes can be anywhere in the world, but the Developing Excellence in Leadership, Training, and Science in Africa (DELTAS Africa) initiative, which uses the hub-and-spoke model, has guidelines recommending that at least 60% of the spokes are African institutions.

Hubs will generally select spokes with which their ideas align, as well as those that have good potential to deliver quality research. Spokes are judged on aspects such as their methodologies, training programmes or facilities and track record of grant management.

Hubs are expected to assess the performance of their spokes, while maintaining communications and conflict-resolution protocols to ensure effective collaboration.

Achieving wider reach

Along with a focus on merit, diversity guidelines are integral to the successful running of a hub-and-spoke model. These guidelines aim to open up opportunities to more individuals and institutions than before, to ensure that there is a varied application pool to begin with.

As a result, the model also encourages the forging of mutually beneficial collaborations between rich countries in the global north and less well-off ones in the global south as well as south–south collaborations. The total budget that can be allocated to spokes in countries outside Africa is capped to ensure that African institutions receive the lion’s share of funding. This is to prioritize Africa’s research opportunities and priorities and ensure a balance of power between global-north and global-south participants.

DELTAS Africa’s hub-and-spoke model is being implemented by the Science for Africa Foundation, a non-profit organization based in Nairobi, with support from the London-based biomedical funder Wellcome and the UK Foreign, Commonwealth and Development Office. DELTAS Africa has established the model in all geographical regions of the African Union and has covered more areas of research than any other programme of its type. It has attained near 50:50 gender parity at all levels of the organization, including in directorship, authorship of publications, fellowships and research-management support. This improves on the rates across the African continent, where women form about 20–30% of the scientific workforce.

Susan Gichoga is a grants officer at the Science for Africa Foundation, based in Nairobi, Kenya.Credit: SFA Foundation

No funding model is without shortcomings or challenges. The scope and complexity of the various programmes, and the potential for cultural differences, for example, mean that set strategies are needed to ensure that groups are managed effectively.

Yet, the hub-and-spoke model offers distinct advantages for grant makers by increasing the quality of proposals during the application stage and ensuring richer intellectual capital during the implementation stage. Funders can be assured that their R&D resources are having a wide reach, and are furthering the equity, impact and research output of the programmes.

As outlined by the African Union’s Science Technology and Innovation Strategy for Africa — a framework established in 2015 to accelerate the transition to an innovation-led, knowledge-based economy — investing in R&D is crucial to address the continent’s unique public-health challenges and the looming effects of climate change. As funders accelerate R&D investments, they must ensure that diversity, equity and historical structural realities are factored into their grant-making approaches.

[ad_2]

Source link

February 5, 2024