As I mentioned in my post double-blind violins: towards a more objective selection there are problems that arise when one lowers the entry bar for judges of quality in a system. One of them is cheating.

The surface of attack for cheating is broad, yet it can be sub-divided into two main categories: a) Fake events and b) excessive events. Here I'll cover the first category.

In any interaction, you can either have cooperation or not. The first bottleneck for this is communication.

As the rewards for something increase, systems of reputation signaling appear to select based on skill to avoid putting noobs against advanced players as it would be a waste of time for both parties.

This creates another problem known as power creep. This is when early adopters accumulate an almost insurmountable advantage over newcomers. This is the pyramid dilemma of markets.

Skill imbalance is the main reason for cheating, but not necessary or sufficient. People cheat if they either can or want to get away with it. I'll proceed to elaborate on this and give you FOO.

An image is a graphical representation of an original source for a defined period of observation. The limitations of perception and size of this representations can be extremely big by modern standards, so in order to manipulate them, we use different types of compression.

Due to the way our photoreceptors and brain process light, we perceive better changes in intensity of light (mesopic vision) than changes in color. Our resolution discernment scale goes: Scale of Grey > green 2x > blue and red x.

^{Color Spaces [Mod. composite images, labeled for reuse]}

Also, We don't see high-frequency changes in image intensity, and this pieces of information we see as a blur due to the contrast sensitivity function.¹ This means several combinations of light representation can have a similar perception effect for us. These representations are called a color space.

Depending on the medium and the part of the process - e.g. capture, manipulation _{(HueSaturationLightness)}, display _{(RedGreenBlue)}, high definition _{(Y'lumaCblue differenceCred difference)}, WEB _(HEX) or printing _{(CyanMagentaYellowKey)} - the color space is different.

Taking advantage of this we downsample the color in order to reduce the size of the image. Then we can use lossless or lossy compression.

In lossless compression, we use statistic modeling to find the frequency of redundancies in the data and change them for a representation of their position (chunking), when we uncompress it is exactly the same before compression. Examples of these formats are .BMP and .PNG. Some people argue that images for science purposes -especially if the image contains relevant edges like text- should use lossless compression.²

In lossy compression, as the name hints, we downsample the color and we also use a mathematical technique known as Discrete Cosine Transform³ (DCT) to weight and eliminate by intereference highfrequency information, then we use quantification of the image and encode it, destroys data in the process. The big advantage is it reduces size massively when compared to lossless compression and changes are almost imperceptible for most uses by humans. The most important example here is JPEG.

JPEG (Joint Photographic Experts Group) is the standard in pictures. JPEG is not the file format but the compression codec, we actually normally use the format .JFIF and more recently other derived new standards like .Exif.

Let's use an image in the format .BNP with the YCbCr color space. With YCbCr the luminance we separated the luminance from the color by a factor of 10 and thus we can eliminate x100 information without most of us noticing. Most of the time we only downsample by a factor of 2.

A cosine function is a function that goes between 1 and -1 on the Y-axis. If we represent our data in the form of many cosine waves, due to the fact we can't perceive well changes in high-frequency information, we can get rid of high frequency data and our perception of it won't change much depending on the scale.

Suppose you have an image composed of pixels in a scale of grey. By modern standards, this means it goes from 0 - 255 shades. The JPEG algorithm divides the image in 8 by 8 blocks. Each of the groups can be represented by 8 by 8 cosine waves.

Each one of the waves has a weight that contributes to the resulting final combined wave.

Since the grayscale in our color space goes from 0 - 255, we find the center that is 128 (as the function goes from 1 to -1 and our center is zero) and after applying the cosine transform we multiply the values of weight by each one of the waves to obtain the final sum

Despite JPEG being an standard there's no concensus for the parameter that of the transform and quantization tables. Each program or camera brand has individual quantization tables. that depending on the quality divide the values depending on weight and round the result to the nearest integer , the effect is that almost all of the values get converted into 0, except for the ones on the top left that have the more weight and lower frequency information. This can be compressed even more by serialization of the entropy (Huffman coding) of most of the zeros.

Some people might try to take advantage of this to create images that could pose a big threat to our perception of reality and we need to know if they are fake or if we must start fearing for our lives.

An approach to this is metadata. You have a trusted device and does some kind of encryption to the original to certify it has not been tampered with. An example to do this with your phone is with image hosting sites like izitru, that perform basic forensic analysis to pictures to determine if they are original, the problem is that they can be oversensitive with false positives and difficult to use.

The problem with this approach is that it requires specific hardware and the sites are vulnerable to being hacked like it happened to Nikon and Canon that tried to mass do this a couple of years ago and making their hardware obsolete, costly.⁴

^{Photoforensics seminar [brand specific quantiation tables]}

The problem, once you have an image that is not an original, is what has changed.

• FOO tools: Shadows and Reflections

^{Photoforensics seminar [gif: altered shadow test ]}

^{Fake Ducks [Comparison by rendering spheres]}

^{Photoforensics seminar [vanishing points example]}

• More advanced, yet not that difficult skills

I'll be explaining this using the free online forensic software AMAZING tool by Jonas Wagner Forensically^beta

As you might remember, every time we make a save a JPEG file we compress it again. This is the principle of error level analysis (ELA). Is a technique that saves the picture and compresses it at 95% and compares it to the original, so one can look for places where there are conspicuous areas where compression is not uniform. Sugesting a mix of different pictures

As a rule of thumb, you compare surface to surface and edges to edges in order to find alterations in an image. Performing ELA is no different.

Lenses have imperfections and dust, they introduce the same or more artifacts everytime they are used. They can be identified like a bullet can be traced to the barrel of a gun.

Another cool feature is the cloning detection tool. It finds patterns that are similar, possibly the result of copy-paste or the cloning tool in photoshop or similar programs.

Going from the highest quality to less tolerance down shows places that are suspicious of modification. The tool doesn't host the images, it can take some time as it's beta to show the changes depending on the resolution of the image.

After giving this tools we can bite the meat of the problem I want to discuss.

The particular way of communicating the results of the scientific process (test and evidence) is called peer review. Contrary to popular belief is not experts exchanging opinions based on their knowledge, but a reputation based inquiry of methods and consistency inside the provided piece of communication known as papers.

Not all science is equal and the limiting factor is the testability. Some fields like mathematics have near to 100% testability inside the paper. Some others like biology, have close to 0%.

Is closer to a legal system. A central actor, the editor (judge), gives the summary of the evidence to a couple of witnesses and asks:

The editor then picks based on the different reviews and decides what to do. Accept it after some corrections or flat out reject it.

What peer review offers is a system of proof. A proof of work by the reviewers. Some rely on this as enough and start building on top of it. For others is the starting point of the paper. Just like with a legal system, the defense attorney and the prosecutor still have not made their entrance on the scene. So the groundbreaking pieces go into peer review journals that reject >90% of the submissions or open access dissemination. After getting the best peer review they can convince or fool.

An example of this going wrong is the now retracted article "Chopstick Nanorods: Tuning the Angle between Pairs with High Yield"⁷ in 2013, about a particular orientation in gold nanorods after being synthesized in solution and getting a 90° orientation on a surface. After being published in Nano letter A journal with good reputation, peer review and impact factor at the time.

Attention was brought to it after Mitch at Chemistry-Blog did a post on some suspicious TEM images. As you can see in the close-up, suspicious indeed... MS Paint?

Just as interesting as the fact that the writers lied, was that they lied so badly. What if they had done a better job at faking it. In fact, it didn't take long for someone to show just how much better it could have been at Chembark's blog.

I performed photoforensic basic analysis on the improved version, with good results. The ELA and noise analysis didn't help much as there's a lot of noise in TEM images and the source of the modification probably don't come from a different picture, but the luminance gradient shows the abnormal orientation of illumination, there was some clone modification and the metadata shows photoshop was used on it. A truly dedicated liar that beforehand knows this analysis is going to be performed can do an even better job. It's a cat and mouse race.

This illustrates a current string of problems. There are more and more scientists, there are more publications and papers and the pool of funds for science research is not growing at the same pace. This breeds a scenario where cheating is a viable alternative. Scientists are not experts at not being cheated just for being scientists. It also means they are not great at lying but that is changing fast.

Cases of researchers attacking the vulnerabilities in the computerized systems of publishers have made possible for them to become their own reviewers by faking their identity and creating review and citing rings.⁸ Alternatives like stopping the suggestion of possible reviewers by the author have been made. Others like double-blind review are far more common in the humanities where unconscious bias are a more common worry.⁹

The paradigm of cybersecurity is that access equals ownership. So once systems become more technical and automated there are more ways to game them by hackers. Anonymity and automation are where hackers thrive.

Despite most of the important discoveries in modern history being the result of side projects that had close to no apparent immediate use or support, like the transistor. Patrons of big research (most of the time the government) need to ration the projects they consider will have a bigger impact on society.

The problem keeps on being the finite pool of money, the proposed solution to this is apparently predictions markets.

The layer of crowdsourcing can be surprisingly accurate in optimization problems.¹⁰ Most likely you can't gamble on what's gonna be the next big breakthrough in science, but you can bet on who is gonna make it or who can find the person who can make it. Finding the dark horse is the most profitable thing as the odds are against it. Of course gathering the wisdom of the crowds is an argumentum ad populum. That's what governments and private institutions are doing.

References

Images referenced, sourced or modified from google images, labeled for reuse