Science!! Fucking magnets, how do they work?

Melvin

Blackwing Lair Raider
1,399
1,168
yet a vast majority of scientific papers/experiments are not able to be reproduced...

Your understanding of that article is fundamentally flawed, which isn't surprising given the misleading headline. The article says very specifically that "more than 70% of researchers have tried and failed to reproduce another scientist's experiments," which is absolutely not the same thing as saying that more than 70% of experiments are not reproducible. The article says nothing about the number of experiments that have been successfully reproduced. The only statistic in the article is the number of scientists who have failed to reproduce someone else's experiment at least once.

People that graduate from college with a C gpa are perfectly qualified to be employed as real scientists that make valuable contributions while researching scientific innovations. They still make mistakes though, like the vast majority of people tend to do from time to time. You know, like the simple mistake you made about what the 70% figure meant.

The more I think about it, the less I like that article, to be honest. It seems "What we see in the published literature is a highly curated version of what's actually happened," can just as easily be applied to the article itself. The important part, the fact that scientists very regularly attempt to reproduce interesting results, is minimized by the article because 100% of scientists are not 100% successful. There is no attempt in the article to quantify what the success/failure rate is, and that's just fucking terrible science right there.
 

Melvin

Blackwing Lair Raider
1,399
1,168
And I'm gonna double post about this because this is important.

It's also really important to remember that some really interesting findings can happen because of failures. If two different teams do 100% the same procedures in 100% similar environment on two 99.999% genetically similar populations of organisms, and they get different results, that "failure" could be the breakthrough that cures cancer. Hypothetically.
 

Melvin

Blackwing Lair Raider
1,399
1,168
Jesus is actually a pretty chill dude. He's usually pretty cool with science even when it's like, not very good. If you want to believe the Earth is flat and 9000 years old and riding on the back of a turtle, that's all good as long as you pay your tithes. He's not down with Unit 731 kinds of stuff, but that kind of science is pretty rare outside of Hollywood scripts.

He says you're going to burn in the eternal fires of Hell if you don't stop touching yourself though.
 

Swagdaddy

There is a war going on over control of your mind
1,960
1,870
anyone else think it's weird that we went to the moon in the 60s (multiple times) and today they struggle to get anything sizable through the radioactive layer of the atmosphere at all

really starting to feel like a wink wink nudge nudge kinda thing
 

Azrayne

Irenicus did nothing wrong
2,161
786
anyone else think it's weird that we went to the moon in the 60s (multiple times) and today they struggle to get anything sizable through the radioactive layer of the atmosphere at all

really starting to feel like a wink wink nudge nudge kinda thing

Here's a hypothesis worth testing, the correlation between lifetime THC consumption and belief that the moon landing was faked.
 

Gesk

Trakanon Raider
11
2
anyone else think it's weird that we went to the moon in the 60s (multiple times) and today they struggle to get anything sizable through the radioactive layer of the atmosphere at all

really starting to feel like a wink wink nudge nudge kinda thing

A lot of know-how has been lost since then which can explain why it would cost so much today to do it again. And there is no longer any motivation to beat Russia and be the first there.

Anyways, if the moon landing were fake, the first ones to say that would have been the Russians back then.
 

Ukerric

Bearded Ape
<Silver Donator>
7,917
9,564
The article says nothing about the number of experiments that have been successfully reproduced.
That's because most studies are NOT reproduced. Ever.

We have a major reproducibility problem because of that. There's been a few studies on reproducibility. One biotech company decided to take 53 major studies in cancer. The kind of studies that will probably dictate how we treat cancer in the years to come. The kind that would probably make that company work on new treatments.

Only 6 of those 53 were reproduced. The rest didn't show the same results, or, at best, showed results that looked the same, but with a probability of not occurring by random chance below normal threshold for publication.

Bayer did the same kind of stats on 67 studies for new molecules that they might have tried to turn into new drugs. Only 14 studies yielded the same level of result as the original publication (with 10 similar results, but not as conclusive).

So, not every study is reproduced, but those that were yielded between 10 and 20% reproduction rates. And no, the reproduction were not done by half-trained PhD students trying to reproduce bad studies from a no-name scientific paper.
 

reavor

I'm With HER ♀
<Bronze Donator>
4,780
16,553
the lack of reproduction of studies has many explanations. in part I believe generally there's simply no glory in reproducing others results, it takes a lot of time and won't get you any publication to speak of, which is really all research groups are about. today it's all about "publish or perish". there's basically no economic or reputable incentive to waste time reproducing other peoples results, when the experiments themselves can cost 10s of thousands of dollars, disregarding the man hours. even if you prove them wrong, a negative study hardly gets any traction. all the attention goes to new innovative studies that build upon previous data, even if it is faulty, which normally takes way too much time to go back and prove, when the whip is to publish.
 

Tuco

I got Tuco'd!
<Gold Donor>
45,420
73,489
Re: reproducibility in the autonomous vehicle research field, studies are frequently performed that examine a handful of the top and recently published approaches to solve a given problem, ex: a specific way of planning in a certain problem set.

Generally all the approaches are made to work, but often the findings of accuracy and speed relative to competing algorithms of the original authors aren't reproduced. This is because the original authors that propose an idea will tweak and tune and calibrate their original system to its fullest extent and give a competing system a once over and call it good.

This isn't really a problem and has scant relationship to the topic being discussed, but I'd figure I'd mention my perspective on it.
 

Tuco

I got Tuco'd!
<Gold Donor>
45,420
73,489
anyone else think it's weird that we went to the moon in the 60s (multiple times) and today they struggle to get anything sizable through the radioactive layer of the atmosphere at all

really starting to feel like a wink wink nudge nudge kinda thing
No.

If we wanted to go to the moon we could.

Landing our rover on mars is more difficult than landing a person on the moon anyway.
 

Pescador

Trakanon Raider
234
239
the lack of reproduction of studies has many explanations. in part I believe generally there's simply no glory in reproducing others results, it takes a lot of time and won't get you any publication to speak of, which is really all research groups are about. today it's all about "publish or perish". there's basically no economic or reputable incentive to waste time reproducing other peoples results, when the experiments themselves can cost 10s of thousands of dollars, disregarding the man hours. even if you prove them wrong, a negative study hardly gets any traction. all the attention goes to new innovative studies that build upon previous data, even if it is faulty, which normally takes way too much time to go back and prove, when the whip is to publish.
As a grad student, I tried to reproduce some interesting studies I had read about, with the intent of adapting them to a system I was developing. My attempts utterly failed 2-3 times until I got in touch with the authors, who gave me some very important details about their methodology that were critical for the system to work properly. Once I implemented those methods, I was able to reproduce their work (though it was never quite as good as what their publication showed!).

What this means is that many papers will have a methods section that reads "A and B were reacted for 48 hours at 60C" when in reality, you need to add B dropwise to A while stirring at exactly 600 RPM under nitrogen purge. In the end, it's almost impossible to write a "readable" methods section that also has step-by-step instructions for reproducibility, so you usually end up with something in between. I know that one of the tenets of scientific publication is that it should be reproducible, but it's actually quite difficult to implement in practice. If I included truly reproducible methods in my papers, the methods section would be 15 pages long and nobody would ever read it!

That being said, there is also a lot of built-in bias (no motivation to publish negative results, getting lucky on your first try and publishing it without reproducing it, improper experimental design) and some amount of outright dishonesty (playing with your stats analysis to achieve significance, running an experiment 100 times and only publishing the positive data, or straight up falsifying data).

And I completely agree with the issue that there's often no incentive to reproduce data UNLESS you have a reason to doubt their data in the first place ("it's too good to be true!") or unless there is follow-up work that requires the published methods.

However, I think it's a difficult problem to solve, because I view most scientific publications as "hypothesis-generating", and if an idea has enough merit, it is bound to be reproduced and thoroughly tested anyways. In this way, publication requirements are essentially filters that try to limit the amount of "bad" research that gets published (ie, tries to reduce type II error), without filtering out potentially good ideas (type I error). Again, I think most scientists view publications as a bunch of "potentially good ideas" but they are also trained to evaluate them with a critical eye, whereas the public, and especially the media, view publications as "statements of fact", and that is where the danger lies.
 
  • 2Like
Reactions: 1 users

Lendarios

Trump's Staff
<Gold Donor>
19,360
-17,424
anyone else think it's weird that we went to the moon in the 60s (multiple times) and today they struggle to get anything sizable through the radioactive layer of the atmosphere at all

really starting to feel like a wink wink nudge nudge kinda thing

We just landed on a comet yoo.

on a fucking comet..
 

khorum

Murder Apologist
24,338
81,363
LOL so word is this is why Google got antsy about BD. They love their moonshot projects so long as they stayed moonshots, but starting the countdown towards the obsolescence of human labor felt a bit much.


Mod an m4 onto it, plug in some goatfucker-hunting algorithms and let's get some autonomous Deus Vulting going.
 
  • 4Like
Reactions: 3 users

khorum

Murder Apologist
24,338
81,363
Yeah they're creepy. But that prototype runs for something like 15 mins untethered and tops out at 22 miles per hour. Kinda curious how it would do in the DARPA grand challenge suite.
 

Lendarios

Trump's Staff
<Gold Donor>
19,360
-17,424
Amazon can replace most of their warehouse employees with this guy, just fetching stuff for your cart.

Right now they use humans and they count their steps to make sure they don't slack. It is a bit inhumane.

I can see a very automated warehouse fulfillment center coming in the near future.
 
  • 1Salty
Reactions: 1 user

Tuco

I got Tuco'd!
<Gold Donor>
45,420
73,489
Replacing Amazon humans with a BD robot would be the most expensive and awesome automation move in history.
 

khorum

Murder Apologist
24,338
81,363
Amazon can replace most of their warehouse employees with this guy, just fetching stuff for your cart.

Right now they use humans and they count their steps to make sure they don't slack. It is a bit inhumane.

I can see a very automated warehouse fulfillment center coming in the near future.

Huh? Amazon bought Kiva years ago, they now have 45,000 robots running their warehouses and even more planned this year. They don't NEED bipedal robots for their warehouses, they've been doing that for years and there's nothing inhumane about having a fucking job at all.


Bipedal robots are best for elder care and exterminating brown people.
 

Sentagur

Low and to the left
<Silver Donator>
3,825
7,937
LOL so word is this is why Google got antsy about BD. They love their moonshot projects so long as they stayed moonshots, but starting the countdown towards the obsolescence of human labor felt a bit much.


Mod an m4 onto it, plug in some goatfucker-hunting algorithms and let's get some autonomous Deus Vulting going.
How much does a The M134 and 180k rounds of ammo weigh again?
 
  • 1Like
Reactions: 1 user