RE: Can we trust peer-reviewed papers?
Very interesting video. As a budding young researcher myself I know the importance of understanding a quality journal.
Impact factor (IF)
As mentioned in the video I go off the Impact factor (IF). But one thing to note is that the range of IF that is acceptable varies greatly between disciplines. For example Maths type Journals typically have lower IF than those of say some Engineering fields because there is just less research that fits into that specific field.
How peer review works
When I finish a paper and submit it to a Journal which I have chosen based on this criteria. It will then go to the editor. The editor will probably have a quick read but nothing in depth at this stage. Instead he will have a list of researchers who have already proven themselves in their respective areas. Usually 2-3 'reviewers' will be involved (its not just any random). They will get the paper have a thorougher read through and decide if it is quality or not. They can then choose to accept, accept with changes or decline. All of which comes with comments. If the paper is declined it is usually because either it doesn't fit the journal or it is not good. The paper is passed back to the writer who can make changes and submit again for another review process. This can take months to publish a single paper.
Just because you pay doesn't mean its a bad journal
Many journals make you pay to publish a picture in colour. This is because they have to print it and send it to people and colour just costs more. This incentivises images that can stand in black and white. Also a lot of new journals which are usually for new fields of research, which are good, struggle to get enough funds from sales alone. So again paying doesn't always equal bad quality.
Always check content.
Even if I find a journal article that is from a reliable source I never trust it and neither do other researchers. If I need to reference an article I would usually spend a good chunk of time going down the rabbit hole of its references. Does it reference good quality work. Does the findings match up and so forth. Finally on top of this common errors in papers usually boil down to bad stats so that always needs to be checked
Open access
By this I mean anyone can view the papers (don't confuse this with not peer reviewed). This one is a balancing game. Many funding agency's require you to publish in open source so they can keep up to date with your work. It makes for open communication with the world. However, any research institute and library I have ever been in contact with usually have a subscription to all the reliable journals. This makes it easy. I enter in my details and instant available content. Open access is starting to become more reliable you just need to still do all the checks above.
Any questions just ask!
Today, open access is very very important IMO. It is actually one of the most important point.
All my articles are now published only in open access journals and are available for free from the arxiv (that is spread towards more and more fields). One very interesting platform that I have not yet taken the time to publish with is SciPost. I don't know if you have ever heard about it.
And to answer @lukestokes question, the answer is probably field-dependent and journal-dependent.
PS: I don't have the time to watch the video now as I am at work, but I will do it later.
It's a good video, and yes, "it depends" is usually a safe answer.
Scipost is really cool, although I don't read up on as much physics as I used to.
One thing to remember about things like arxiv is they can be published there with out any peer review. Which means material put there can, be crap. It doesn't mean that it is, but peer review is still, at this point, essential for the reinforcement of research quality.
Open access publishing is essential IMO for the quick effective dissemination of scientific knowledge to all edges of the earth. However these journals must be held to the very highest level of editorial scrutiny and a robust peer review process is essential for this. Should the peer review responses also be made available for all to see upon publication of a paper? I believe it should.
There is actually one protection on the arxiv as you must be endorsed to be able to post. This does not prevent from crap, but it at least imposes new users to have one reasonable article to start with.
Me too. Which is what I like with SciPost (which I should try one day :p ).
Great comment, thank you!
Yeah, I think that was mentioned in the video. Good thing to keep in mind.
That's also a good point about payment. I think there has to be mutually aligned incentives on all sides for market regulation to work. Otherwise, we're playing with other people's money, resources, or reputation. Checking the content is key, but many (including myself) barely read the abstract, let alone the full paper, let alone the references. It's a time-consuming process which takes a lot of expertise to do well.
Open access definitely sounds great, but what regulates it? That, to me, is the tricky part. Reviews can be gamed, references can be faked... it's a tricky game.
Thanks again for sharing your insight and experience. Followed.
Thanks! I forget sometimes that I actually know stuff haha. You're right he did mention it. I just thought it needed more emphasis. I would consider submitting in a journal with impact factor of 1.6 and other times a 3.8 because they are completely different fields. In this case just looking at the number is irrelevant. This is why I don't stray too far out of my research field (Bio-Mechanical).
I admit the first 3 times I read a paper I barely read past the abstract because you have to sort through first is it related to what you are doing. In my field though if you miss-quote a paper you get hounded so if I use it I better well understand it.
Open access is great just as long as the journal is still peer reviewed. Wikipedia has a good introduction to what open access means. All the quality open access journals must be peer reviewed in some way.
Glad you enjoyed my comment. :) I found it hard to read some of the others and not want to bang my head against a brick wall. :/
I know. Unfortunately there are quite a few "free thinkers" on Steemit who frustrate me with their thinking. They love all things conspiracy theory related, and they make connections to everything. I'm sure my own thinking frustrates many as well when I talk about government as a monopoly on the use of force in a geographic region and how we might be able to thrive without a Hobbesian Leviathan. :)
We all have our stories we tell ourselves which make sense until a better one comes along. I remember how silly and ignorant I was in the past and I'm sure I'll chuckle in the future at myself today. At least, I hope so, because that means I've hopefully learned something along the way.
Great summary of the important factors in publishing. I think the problems occur when shortcuts are taken with reasoning rather than examining the evidence for themselves. For example, it's published and it fits my beliefs, therefore it must be true. It's peer-reviewed, it must be true. It's open-access and paid to be published, it must be questionable. It's in a quality journal, it must be legit.
I think there is no avoiding taking some of these shortcuts because of limitation of time or expertise to examine further evidence, which is where some appeals to authority can save time, so long as checks are in place.
Publication in a quality, high impact journal can be seen as a stamp of quality. The journal has staked its reputation of the quality of what is published.
http://sci-hub.bz
Who cares about open access :P