After identifying antibody validation data manipulation in Thermo Fisher Scientific’s catalogue in May, the science sleuths and researchers behind the finding have now uncovered doctored validation data on a much larger scale, with over 18,000 images relating to over 17,000 antibodies from at least 15 different supplier companies.
Reese Richardson, Elisabeth Bik and Sholto David – who spotted the first doctored blot – have led the investigation. Community reports have also come in from across the world, with the annotated images continuing to be collated in an online repository. The team has traced some suspected image manipulation back over a decade ago.
The companies identified span major suppliers as well as more specialist firms, including MilliporeSigma, G-Biosciences, Proteogenix, Abcam and Origene. Western blots – which show how an antibody binds to a given protein – and other validation images have been identified as manipulated, calling into question not just transparency but antibody performance. This includes ‘painting’ of immunofluorescence images used to visualise the distribution or localisation in cells.
I think that everybody should be demanding change. It’s a massive insult from for-profit companies to the research community.
Reese Richardson
Image integrity expert Bik says Western blot manipulation is easier to uncover, but immunofluorescence manipulation takes more digging. With increasing use of artificial intelligence (AI), identifying doctored images is becoming more difficult. While she wasn’t surprised to discover image manipulation from more companies, Bik says the extent is ‘shocking’ to see. ‘I’m very worried about AI,’ she adds. ‘It may be already happening that images are AI-generated, and how will we detect that?’
Until now, she suggests, there has been a level of trust and expectation of quality – given the high prices of antibody reagents. When experiments fail, or results are not as clean as expected, researchers tend to attribute that to their own errors, rather than ineffective reagents.
Bik says the impact on research is unmeasurable but ‘it must be millions or billions in wasted research funding’. It also raises ethical questions for researchers who use these antibodies as part of animal studies. Bik adds there must be ‘some degree of regulation’ introduced.
Richardson says the contributions and support from scientists gives him hope for change: ‘I think that everybody should be demanding change. It’s an immense waste of [researchers’] time [and resources] that people have been buying antibodies based off fabricated data. It’s a massive insult from for-profit companies to the research community. What we need now is transparency.’
Following the pattern
Richardson explains that with image background patterns being replicated for antibodies across different companies, it’s likely for many that ‘no experiment took place’. He adds that many companies are buying antibodies from the same manufacturer, then selling them under their own brands at different prices (and with different quality guarantees) without clearly disclosing the source.
However, some companies’ validation data show ‘unique styles’ of image manipulation – like Proteogenix and Santa Cruz Biotechnology – with individual patterns reused extensively within their own antibody catalogues. This leads Richardson to suspect ‘fudging the validation data’ may be common practice across the industry.
The companies with the largest number of doctored images are G-Biosciences and Thermo, each with more than 5000 images identified, and LSBio and Origene with more than 2000 images each. However, without running a complete audit of all the companies’ catalogues, the full extent of the problem remains unclear. Thermo and Abcam have begun their own internal reviews. Thermo says it has so far found ‘no evidence of impact to antibody performance’ and is updating and replacing some Western blot images, as well as adding labelling to distinguish raw, unedited images from those that may have been ‘edited, optimised or adjusted’.
Tracing the changes
Richardson explains that forensic image analysis tools can reveal metadata showing when images were opened in Photoshop and other editing software. With this, science sleuths tracked some of the alterations made over time. However, he is clear that this alone is ‘not a smoking gun’, since Photoshop can be used to make legitimate changes such as cropping and adding labels. The incriminating evidence, he says, is traces of multiple modifications that go beyond what would be required for simple labelling.
This includes some images first logged as far back as 2011: ‘Many don’t just preserve the first time an image was created and when it was last modified, but session data for every modification made along the way,’ he explains.
Taking the images that share one specific background pattern, for instance, the team can reconstruct an evolutionary tree from the original image, showing every time it was modified thereafter. Modifications have unique session identifiers but do not show how the image was altered. ‘You can see if the first 20 session IDs match the whole 20 of a different image, so you can see where an image was branched off and manipulated from the same template,’ he says.
One example has up to 80 different session IDs, suggesting modification goes far beyond acceptable changes for presentation.
I hope there will be some lawsuit, I’m not a fan of suing people in general but this is a big scandal
Elisabeth Bik
While the original data is needed for a clearer picture, the thousands of images identified so far represents the proportion uncovered after hours of work from the science sleuths – often alongside other work. Bik says there’s little individual scientists can do, but companies should now be checking their supply chains, identifying fault and taking responsibility. However, she has ‘little hope’ change will come soon.
‘I don’t know how they’ll rebuild trust. I hope there will be some lawsuit, I’m not a fan of suing people in general but this is a big scandal.’ Few scientists will feel they have the time or means for such a lawsuit, however, and pressure to keep producing results means they are unlikely to boycott major antibody suppliers. She suggests that research funders like the US National Institutes of Health could potentially pursue lawsuits for wasted funding, but most likely, she suggests ‘nothing will happen’.
Richardson sees this as a ‘cautionary tale’ of the privatisation of essential research services in general and specifically the for-profit model of antibody production. Independent organisation YCharOS validating antibodies with gold standard protocols estimates these tests can cost up to $25,000 per reagent. He says this cost, combined with the lack of regulatory oversight, provides an incentive to either modify or fabricate validation data. He agrees with Bik that regulation is necessary. In the meantime, Richardson says researchers should run their own validation, since ‘clearly, we cannot count on the industry to do it for us’.
