To understand the fundamental problem with this paper and the "CD" measure of "disruptiveness", one need only look at Figure 1a, where three Nobel Prize winning papers and three patents are ranked on the "CD5" scale. Apparently, there are more disruptive discoveries than Watson and Crick's discovery of the double-stranded structure of DNA (CD5 0.62 on a scale of -1.0 to 1.0), and Baltimore's discovery of reverse transcription is -0.55. One might argue the ranks of these two discoveries should be reversed, since DNA had already been shown to be the genetic material, but no one imagined that RNA could be converted to DNA. Likewise, the Wigler patent for transformation into eukaryotic cells (CD5=0.70) has certainly has had less effect on the world economy than Monsanto's patent on glyphosphate resistant plants (CD= -0.85).
It's easy to make up a measure and then develop a story that explains why that measure is useful. Was Newton's theory of gravity disruptive, or consolidating. I'm thinking consolidating, since it allowed a lot of other things to make sense. Likewise, perhaps quantum mechanics was disruptive -- it sounds disruptive -- but it also consolidated a lot of confusing observations.
(And lets not even mention the problem that when there are 1,000-times as many papers between 1940 and 2010, we might not expect 1,000-times as many disruptions.)
This paper has a measure that sounds useful, and is certainly good for headlines, but it is very unclear that it provides any insight into the progress of science.
You misunderstand why W&C was important (although, given your username, I assume you've already sat through the lectures): in elucidating (not discovering) the structure of DNA, they had a biophysical mechanism that explained heredity (specifically, that it was an antiparallel double helix with complementary bases, which immediately suggests a mechanism for replication).
That recognition immediately led to a flurry of follow-on work confirming the mechanism and birthing molecular biology as we know it, which opened up vast new areas of science.
Yes, I know why it was disruptive. The question (for this paper) is, what other biological papers were more disruptive (since, based on the range of the scale, they barely made the first quintile).
I think the more important question is, how do we validate the usefulness of the CD5 metric. And, why is it better to be "disruptive" than "consolidating"?
Based on the examples shown (and others that I can imagine), it is unclear that it measures anything related to scientific significance.
I had already discounted the measure as being too prone to sampling error to be useful. I don't think these things are really measurable, and even if you did measure it, "disruption" and "consolidation" are such subjective terms it would be arguable to have a single-dimensional slider that went between those extremes.
Probably best to just say some of the greatest discoveries are both disruptive, and consolidating.
The trend over time could equally be capturing changes in referencing practice (either that really important, influential papers/inventions in the past didn't reference other really important, influential papers/inventions nearly as often as equally novel papers/inventions today, or that modern referencing is much more exhaustive about referencing minor and tangentially relevant papers/inventions. There is, of course, also more significant prior art in journals and patent indices to actually cite in later years...). They carry out some robustness checks, but I'm not sure any of them really dent that hypothesis. It's also a conclusion based on proportions (ie. the effect mostly comes from by there being a lot more "non-disruptive" papers published rather than fewer "disruptive" papers) and there's actually a trend rise in the raw number of "most disruptive" academic papers from the 1950s through to the 1990s, at which point the numbers drop so sharply it looks like a discontinuity in the data (but would probably coincide with lots of journal content becoming easily accessible online...)
Boring title: "Scientific authors follow more citations when reading modern papers".
When I read a modern paper, I skim a lot more of the references. Getting to a cited paper is often just two clicks: there's a reference to the bibliography, and from there a link to arXiv (or some other public repository). As papers get older the bibliography might not link to arXiv, or might not link to a digitized paper at all. Those older papers are more time consuming to use as a reference. Scientists aren't lazy but they also don't have infinite time.
Put another way: papers form a graph, this result suggests that older nodes are traversed less frequently. I think the metric they propose is useful, but we could rephrase this all as measure of how useful papers are as a node (where CD = -1 is most useful, CD = 1 is least useful).
In short, this result might just mean paper discovery is getting easier. That's a good thing. We can wring our hands endlessly about how there's less low hanging fruit (there is) or about how academia is rotting away (as though it was ever different), but those conclusions are big jumps from what this paper says.
When I was in grad school I had done measurements of the magnetic anisotropy of amorphous compounds. I had found an interesting link between the structure and the magnetic properties which is surprising since the material is amorphous, the material properties should be isotropic yet these had a preferred magnetization axis. During the discussions of these results with my advisor I was told in very clear terms that he would not let me publish results that contradicted a paper of his from the 1980s.
I didn't push back, because I had no power and I just wanted to get the fuck out of there and start making money after being scammed out of 5 years of my life where I poured a lot of effort and got shit pay. I lost respect for my advisor and I was stunned by how un-scientific his response was when it came to correcting his 1980s results. My measurements were more accurate and were strongly supported by structural measurements he never did and couldn't do back then. But he was more interested in protecting his "legacy" of his paper with 13 citations.
So petty and small. I dropped those efforts and was able to get out completely obviating those results, never mentioned them in my thesis even.
Anyway, my point is that I'm not surprised that scientific results are lame, vanilla, uninspired. When the committees that decide on what gets funded are all 65+ dinosaurs stuck in their ways, very few disruptive ideas will be founded because you're disrupting the very research that got the committee members clout and recognition.
It used to come a funeral at a time, but now that the next generation is forced to build their careers on the mistakes of their advisors, mistakes can easily outlive their creators.
I see this kind of stuff all the time and I'm convinced that Academia is corrupt. While working against a Grad Student doesn't seem particularly evil, rather just a bit backstabby/political, I suggest that it is 'really bad' because it contravenes the basic organizing principle of science itself.
Kind of like a Judge ruling one way or another for personal reasons, or ruling with a heavy conflict of interest.
There needs to be much stronger ethical principles around this and an advisor hinting at obfuscating work for such reasons should be up for review.
That timeline correlates well with the corporatization of academic research timeline, which got started in the late 1970s, boomed in the 1980s and has steadily risen ever since. The emphasis in applied science is now on patents and return-on-investment, aka safe bets. This is certainly the direction that much of NIH fundings, as well as DOE funding, has taken. Part of this is that entrenched industries (natural gas and coal power, say), don't want federal funding going to disruptive competition (monocrystalline silicon PV development, say). This is reflected in how federal funds are distributed as well as what research programs are supported at the university level. For example, count the number of renewable energy departments in the USA academic system: miniscule funding -> no programs.
It can't all be blamed on corporatization, however. The control of funding agencies at the federal level by vested interests seems to be a major problem, i.e. innovative lines of research that threaten the status quo could threaten those cabals that control fund disbursement, so such efforts don't get funded. The stagnation of Alzheimer's research and the focus on the amyloid hypothesis seems to fall into this category. This kind of bureaucratic ossification is not new; the classic example is how Trofim Lysenko in the Soviet Union controlled the direction of agricultural research for about three decades, much to the detriment of the understanding of plant genetics and crop development research in the USSR over that period.
So it can be attributed to the rise of monopolistic corporatization on university campuses on one hand, and the growth of entrenched bureaucracies at the federal funding level on the other. Neither group wants the rug pulled out from under them by 'disruptive innovation'. There are many historical antecedents for this kind of situation, incidentally.
> The control of funding agencies at the federal level by vested interests seems to be a major problem
This is a fundamental problem with the nature of science and the nation state in that modern science is constituted by the nation state, not merely imposing into or upon it.
Given that Tech Transfer offices don't make much money at all, and patents are mostly worthless, I doubtful of this theory. There simply is no 'return on investment' on most of the work.
I wonder if this is to do with major vs minor breakthrough.
Take physics. As I read it, modern fundamental physics is mostly some combination of general relativity and quantum mechanics, which are both early 20th century inventions.
They weren't necessarily genius in particular in-depth analysis, rather they were genius new concepts that created a whole new blank canvas to fill in.
What did the field look like just before them? I always forget the details, but was it lord Kelvin who said, physics just needs to figure out the ultraviolet catastrophe and it's done? I wonder if it looked similarly uninnovative. The canvas they had was full, and they needed a new canvas for true innovation.
Now the thing is, I think true disruption, like GR and quantum physics, is not limited by the number of postdocs, but by truly genius insights. Einstein wasn't even working as an academic when he came up with Special Relativity. So the number of papers written isn't a proxy for pace of revolution.
Not to mention, since number of published papers has become a target, it has become useless as a measure of progress.
I agree, I think we're discounting genius insight as the primary driver of innovation and ground breaking science, and we can't expect genius insight to occur in some predictable time frame or a time frame even within a lifespan.
I think perhaps we've also partially lost the meaning of ground breaking research given the constant news articles on "ground breaking science" and the constant overinflation and exaggeration of one's papers and achievements.
A lot of the obvious questions on the table were answered, even without an answer we at least have the question. Today true, groundbreaking innovation needs someone to identify the new questions. It's like doing science in a bubble and you need someone to take a massive new one and merge it into our own. Right now we're not opening wide doors, but just refining and adding almost imperceptibly to an already huge bubble. We're grinding in the game because we can't find the boss on the map or can't beat it yet.
Besides the current structure of research and the known weaknesses (misaligned incentives, funding issues, etc.) science in general today ran out of low hanging fruit. The last unexplored or unanswered riddles may need prerequisites that are outside of the current boundaries so someone needs to open the door do a vast new bubble that has more relatively low hanging fruit.
Also humans probably (don't quote me on that :) ) can't come up with any new questions or answers unless they are already at the edge of that bubble. And they each see and understand a smaller patch of that surface as it expanded. Stumbling onto new thinks is exponentially harder.
This is what those revolutionary discoveries did for us in the past: Opened the door to a new world where the hard answers were suddenly a bit easier, they made sense.
> we can't expect genius insight to occur in some predictable time frame
Yes, and we can’t expect it to occur in a socially confined over-structured environment either. Academia has changed a lot since then, with peer review hell, publishing pressure, etc. I mean, are we really pretending to be surprised that a system of incrementalist incentives produces incrementalist results?
We get around this in part by not understanding (or at least remembering our understanding of) many of the basics. Such as this guy who looked things up in a textbook: https://news.ycombinator.com/item?id=34236889
It also seems unlikely that a patent clerk, even one with a PhD in physics, would be allowed to publish in a physics journal today. The gatekeeping has gotten a lot more strict.
I’m not sure it filters out kook theories, given the replication crisis.
Rather, journals promote well-connected kooks at the expense of less well-connected kooks and happen to occasionally publish some science as a byproduct.
So "peer review" isn't holding back kook theories at all on its own. At best, journal prestige is bolstered by it, which the journal can then translate into holding them back. The track record is a bit unclear, though.
I view peer review as respecting the time of the journal subscriber. Someone filters out the things that are obviously flawed before wasting everyone else's time.
Journal aren't just a magic upload into scientific consciousness; it takes time and energy to read and process them.
It's doing a rather bad job at filtering out obvious flaws, given the reproducibility crisis we've got on our hands. Everyone says peer review is very important, but when it comes to reviewers putting in the hard and boring work of actually checking the raw data, almost none do, so in truth, peer review is NOT important to most scientists.
And we nevertheless pay the price of adhering closer to orthodoxy for it, since effort being put in is a rather superficial "this paper sounds reasonable to me" filter, perhaps with some boxes that must be checked like p-value or sample size.
A peer review young earth creationist journal would still gatekeep anything that did not come from a young earth perspective.
>Papers can be in any scientific, or social scientific, field, but must be from a young-earth perspective and aim to assist the development of the creation model of origins
The problem is that inherently peer review is designed to act as a brake:
>Peer review, in which a scientist's paper is scrutinized by a group of colleagues, is designed to find errors and weed out half-baked ideas.
But in blocking you are going to reject ideas.
>And although some have criticized peer review for rejecting new ideas just because they're too radical for the establishment to stomach
How much of a problem and how many revolutionary ideas are stopped, I don't know. I am more concerned about the reproducibility problem in science.
Sorry, bad phrasing on my part. I didn't mean that peer reviewing doesn't lead to gatekeeping, but that simply making something peer reviewed isn't enough, because ludicrously bad attempts at science can be peer reviewed.
Peer review is also obviously a bad solution for the reproducibility problem, given that we've got the problem now, concurrent with generations of peer reviewed manuscripts.
This article mentions Einstein being offended that his paper was submitted to peer review by Physical Review, and submitting it elsewhere instead:
"We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorised you to show it to specialists before it is printed. I see no reason to address the – in any case erroneous – comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere."
which argues, not unconvincingly IMO, that peer review is a failed experiment which tries to stop bad science at the price of also stopping brilliant, unconventional science, where instead we would profit more from letting the scientific public do the sorting instead of hand-picked specialists. The author phrases it as science being as strong as its strongest links, not as weak as the weakest one, and we should do everything to get the brilliant crazy ideas.
In brief, London 'man off the street' solves 20k year Ice Age drawings mystery, published in Cambridge Archaeological Journal via contact with Durham professor. (And he is first author.)
I don't say that to disagree with you by the way - it's undoubtedly a rare example.
These days Einstein could just put his paper on arXiv and worry about publication later. The physics community goes nuts about arXiv results all the time. There's zero review if you have a university email (or national lab etc).
I tend to agree that peer review has flaws. But given the ease with which someone can publish anything with no review whatsoever it seems reasonable that peer review might shift to fill a niche that is slower and less disruptive.
I disagree, at least in math there have been many instances recently where laymen (with a PhD) found some new and exciting results[0] and got to publish them, and I don't see why it would be any other way in physics.
> but was it lord Kelvin who said, physics just needs to figure out the ultraviolet catastrophe and it's done?
This is apparently a common misconception regarding a lecture given by Lord Kelvin in 1900 concerning "two clouds" in physics.[1] In fact, Max Planck solved the ultraviolet catastrophe, also in 1900, by assuming electromagnetic radiation can only be emitted or absorbed in discrete packets of energy called quanta. Albert Einstein also solved the ultraviolet catastrophe in a paper he published in 1905, for which he was awarded the Nobel Prize in 1922, by, as always, standing on the shoulders of giants, hypothesizing that Planck's quanta were actual particles. These particles are known today by a word coined by physical chemist Gilbert Newton Lewis in a letter to Nature in 1926, namely, photons.
A big part is that science has become very “sanitized”. Work on the wrong issues and you won’t get funding or will even get ridiculed out of a career.
It’s not even just going against headline consensus issues that will get you in trouble. Any findings that overturn what a large group of scientists have spent their careers working on won’t be well received.
More than that though. Modern science research is a highly hierarchal and managed enterprise. It's led, or at least its funding is controlled by, the risk averse managerial class who can only think it terms of returns on investment.
9/11 (believe it or not) had a lot to do with that in IT/computer engineering research (and likely other fields) with government funding. After the attacks, the US Federal government moved to stressing development more than blue sky research. That drew all the funding away from fundamental science to projects that could be made into a product in 3-5 years and focused on addressing some issue of the War on Terror.
Would you say this is a natural consecuence of human emotions (e.g ego), economic interests, powerful players that have corrupted the game etc? I'm genuintely curious. As an ignorant on science and its processes in academia or markets, I cannot feel nothing other than dissapointment and loss of hope everytime I heard this kind of things about a matter I've always considered quite rational and focused on pursuing the truth over anything else. Naive me, I guess...
It’s always been a problem but I’d say these factors make it worse:
1. It’s harder for individuals or small groups to make breakthroughs. We need larger groups and more expertise as well as expensive equipment to continue making discoveries.
More hands in the pie make more gatekeepers and less risk taking.
2. In general our entire society is “circling the wagons”. I’m not sure why but we’re more tribal than normal. Science isn’t immune from the trend.
This is a very natural consequence of the higher difficulty of maintaining one's career in academia. What is a natural consequence of the increased competition for those jobs.
> Any findings that overturn what a large group of scientists have spent their careers working on won’t be well received.
Hasn't that always been the case?
It's said science advances one grave at a time.
I suspect that there isn't a single cause but a cluster of them.
One possible issue is that high level science needs more and more energy (like LHC) and more and more intelligence to crack (most science prizes are teams).
Alternative theory: we've pretty much nailed the basics and it takes a PhD before people can even understand the frontier of most scientific fields. These frontiers are manifold, highly technical, and extremely boring to the layperson. Number theory is a very clear example of this: go read the Birch and Swinnerton-Dyer conjecture. Come back in a decade after you understand the Tate–Shafarevich group. (hahahaha just kidding, if you understood that you'd have proved the conjecture)
I’ve heard this too about number theory. Perhaps in that field, the paradigm shift lies in reducing incidental complexity? Or even managing it in a way that is better for humans?
> it takes a PhD before people can even understand the frontier of most scientific fields
It’s interesting that we have this implicit view that hyper-specialization is the only way to advance. I always thought so too, but why really? If you think about it, specialization is very close to incrementalism. Groundbreaking paradigm-shift stuff can often be written down on a napkin, and rarely does it require a PhD.
That said, to find treasure you have to go down a LOT of wrong paths before you (or someone else) ends up in the right one.
This was considered in the paper: "Some point to a dearth of ‘low-hanging fruit’ as the readily available productivity-enhancing innovations have already been made19,27. Others emphasize the increasing burden of knowledge; scientists and inventors require ever more training to reach the frontiers of their fields, leaving less time to push those frontiers forward18,28. "
I blame the grant funding process. What gets published is what gets funded when written up as a grant proposal. What gets written up as a grant proposal is what is called for by grant funding agencies for their own research initiatives. If you want disruptive science, fund it then. That's all everyone wants to do in science, but the bills need to be payed somehow, so you play the game the funding agencies want you to play.
Plus, there is a large degree of "orthodoxy" in these grants, even most of them proclaim that they are "high-risk, high-gain". In the end, it seems that if you are too much on the "risk" scale, you will not get the funding since committees prefer "a little more of the same, please" to more wild ideas. I realise that such ideas are of course hard to classify in terms of their feasibility, but that's the main purpose of science, right? Venturing boldly into the unknown and all that; sometimes you come back with the treasure, sometimes you come back with lessons learned.
The current small-independent-grant culture is flat. This means that all projects are small and there are is almost no hierarchical 'higher level' science. PIs are happy to publish minor descriptions of the clouds, and journals are happy to publish them , in lack of something more impressive. So much value is lost in creating processes and bureaucracy.
The focus is clearly on "getting the grant". And the grant acts as an entity in itself, as it becomes a line in the CV of scientists, which will help them "get the next grant". The outcome of the grant becomes thus irrelevant. Maybe competition is missing?
It almost seems as a new model would take the world by storm, but it doesnt seem to arise. Even private research often falls for the same pitfalls, like how Deepmind keeps seeking to publish proprietary research in Nature, instead of creating an open ecosystem that will drive reinforcement-learning science forward.
That's an apt way of describing the voter base :). Or the market at scale.
That's the major problem humanity faces. Individuals wielding concentrated power are dangerous, but they're also at least capable of thinking about something other than their immediate short-term interest. The masses, as aggregated through votes or purchasing decisions? They're structurally incapable of it.
If what you are saying was true then NIMBYism wouldn't be a problem, and we also wouldn't have any large "open spaces" left.
The problem is more parochialism than it is short-term thinking. And both sets of entities (powerful individuals and aggregate masses) are capable of parochialism.
> The masses, as aggregated through votes or purchasing decisions? They're structurally incapable of it.
I think this is a dangerous delusion:
Historically, centralized control has been a disaster because those individuals wielding power believed they could think of something greater… but actually lacked the skill. So we ended up with simplistic concepts that failed at scale, ideas that never could work because they missed key details, etc. By contrast, society gets on with the business of building the future in spite of and not due to that meddling by centralized power.
> The focus is clearly on "getting the grant". And the grant acts as an entity in itself, as it becomes a line in the CV of scientists, which will help them "get the next grant". The outcome of the grant becomes thus irrelevant. Maybe competition is missing?
I think there's plenty enough of competition in academia. And competition alone doesn't help, because the metrics over which one loses or wins the race are not correlated with short or long-term worthiness. And while we can describe, even if somewhat vaguely, what makes worthy science, there's no way I know of to have a stable system in which that metric drives funding.
> Deepmind keeps seeking to publish proprietary research in Nature, instead of creating an open ecosystem that will drive reinforcement-learning science forward
This research at least has some feedback: Google funds it because it expects to make money off using its results in practice, so the research has to be at least somewhat correlated with reality. Unfortunately, this structure also means that "open ecosystem" isn't pursued.
I think that the expectation that "ground-breaking" discoveries in science should follow some linear or predictable timetable is not reasonable. Is the time period between "ground-breaking" discoveries supposed to be every 5 years, every 10 years?
The volume of papers has increased significantly, and publish or perish kinda stinks, but it's not just an issue of funding, but some (many?) researchers publish and exaggerate about the importance and difficulty of their research to receive an "I am smart" badge on social media. Although, despite this increase in volume of crap tier papers, the article seems to think it's not correlated:
"Declines in disruptiveness are also not attributable to changing publication, citation or authorship practices..."
What you're pointing to is far more insidious than I think is usually recognized.
People can point to incentive structures like grants etc, and to the implications in terms of the meaning of metrics etc but the real harm is in how it shifts mental focus and attention collectively.
It's not ok to let minor things or things you think are obvious go anymore to focus on bigger picture issues, that might be "riskier" but nonobvious. The way that plays out too is incredibly dependent on your institutional social environment etc.
So much of this is difficult to easily quantify in ways that can be easily studied. I loved this paper for example, but it's easy for me to think of papers in my own field that would look "disruptive" in terms of citation networks but are really the same content. There's these weird shifts I've seen happen in reading old literature and during my career, where big shifts in who is being cited will happen for quixotic social reasons. Usually it's basically politics or ignorance of big parts of a field, who will get introduced to an idea by a particular author or group. Lots of chaotic citation patterns and feedback loops.
Add in shifts in what's motivating grant and paper topics and it really damages authentic scientific discourse. So much is driven by looking like a brilliant scientist and not by scientific progress. People aren't dumb either and they are very very good at looking like brilliant scientists.
Higgs, who hypothesized the Higgs boson back in the 70s, has only published 2-3 papers in the decades since. Before it was announced that he won the Nobel prize, his University was deciding between forcing him out or gambling on him potentially winning the Nobel prize which would bring more reknown to the department. He has commented recently saying if he were a young, freshly minted PhD today he would not have gotten tenure with his publication record. That seems insane to me and a bitter critique of today's incentive structures created by the funding system we have that rewards incremental work over real breakthroughs.
This reminds me of a comment from an academic nearing retirement that I heard second-hand. It was something to the effect that applicants for Assistant Professor (or Lecturer) positions have CVs that are as good as or better than what applications for tenure looked like 20-30 years ago.
Alan Kay wrote an essay on how large scientific breakthroughs where done [1], further discussed in [2]. The funders and the systems of funding of science are to blame for the deciline, for example they won't fund problem finding anymore.
I get that in this context "patents" should be seen as the announcement of a new innovation, a surrogate of the innovation itself if you will, but setting that aside for a moment I'm struggling to imagine how patents could ever do anything but slow down disruption.
Take any revolutionary discovery. Compare making it freely available to anyone, or limiting the availability to those holding the patents. EDIT: And perhaps more importantly give people the freedom to innovate on top versus stopping anyone who does. Shouldn't the former always be more disruptive of the status quo? I can imagine this is an oversimplification though so I'm open to hearing counterintuitive examples where the opposite is true.
And to be clear, either way I'm not saying this is good or bad in and of itself - in some cases it might slow down essential improvements, in others it might be good to have speed limits for change so that things don't break.
You could argue that there is more economic incentive to invent new things as discoveries are worth more if you alone have the right to use them. This in turn could lead to more investments in research that lead to more inventions that increase disruption.
I know that that has been one argument in favor of patents since forever. But that ignores the part where competing parties are not freely allowed to build on another person's patent, which is essential for making good improvements.
And you can see this play out in the history of invention. For example, Edison stopped other people from doing research on the light bulb. Meanwhile the Netherlands was a country without IP laws at the time so Phillips quickly ended up producing the superior and cheaper product. No patent laws, bigger disruption. Similarly the Wright brothers were notorious for holding back aviation with their patents.
James Watt and his patents on steam engine are also an interesting case. Efficiency gains basically stopped for duration of the main patent patent and exploded later.
On the other hand it is also case for more innovation due to necessity, e.g. his patent on beam connection to a shaft (linear-to-rotary movement) required others to come up with a different solutions (e.g. sun and planet gear).
- There are as many revolutionary discoveries with and without patents
- Without patents, discoveries would be freely avalaible
As far as I know, two (related) arguments are generally made for patents:
- Patents create an indirect (by preventing the competition from using your invention) or direct (by licensing it) monetary return to innovation, potentially leading to more innovation
- If a company wants competitors not to copy their innovation, they can 1. keep it secret or 2. disclose it and patent it; without patent the choice is between 1. keep it secret or 2. disclose it and have everybody copy them. In this case, patents lead to more innovation being made freely avalaible (with a delay!).
Whether patents lead to more or less innovation is, as far as I know, contentious.
> There are as many revolutionary discoveries with and without patents
I explicitly don't: I mentioned the patent stops others from innovating on top of the patent. The necessity of patents to be public is one I had not considered though, being a force against trade secrets is a good counterpoint.
> Whether patents lead to more or less innovation is, as far as I know, contentious.
Honestly, I really have trouble accepting this. Not that I have an answer, but with the idea that researchers haven't been able to find some sensible way of measuring this by now. I already mentioned the example of light bulbs. Surely there are enough similar historical scenarios available to analyze where one can make use of global differences in IP laws and other variables to simulate control groups?
"Take any revolutionary discovery. Compare making it freely available to anyone, or limiting the availability to those holding the patents. [...]"
You take the situation where there _is_ a revolutionary discovery, with or without patents, and then wonder about the effect of patents on the next innovations. In doing that you do not consider that may be a revolutionary discovery with patents, and none without.
Extruded plastic took off before resin printers because the patents expired a few years earlier.
As far as I can tell, we could have had the 3D printing revolution in the 90s or early 2000s, but instead we locked the technology away to only be used in a few esoteric commercial applications until the patents expired.
Given the abundance of cheap eink book readers, price tags etc on the market, cost doesn't seem to be a problem. OTOH the limitations of the technology - in particular, the contrast and the refresh rate - and how expensive large screens get, prevents it from expanding into more niches.
Perhaps if there were no patent, there'd be more third-party research to improve all these things faster. But as far as adoption of the existing stuff goes, I don't see how license costs are a direct blocker.
The original idea of a patent is that it must reveal the "secrets" behind an invention, in return for a "temporary monopoly". It's supposed to help diffusion of technology by publishing trade secrets, not to "incentivise" research.
I think what is actually happening is that patents are made obtuse/opaque containing little meaningful information. They are then used as playing cards in patent licensing deals between big corporations.
There is no longer any hope for patent clerks to do meaningful inspection of patents to determine if they are legitimate. Instead, it's left to lawyers/attorneys to play something out in court.
Patents have always been written in precise but vague terms. If you take that as the language and understand the structure of them, they can be quite readable, although you will often need to use outside knowledge and critical thinking to determine how to actually replicate the invention.
I lost a lot of faith in scientific research when I had to spend time reviewing patents in an early job.
The fact that I was looking into battery materials (a notoriously over-hyped sector) certainly didn't help, but some of the approved parents were laughable. They included core materials which had been historically known to be useful in batteries (and hence shouldn't qualify for protection), but then also every possible element-wise substitution of the same structure - many of the resulting materials would not even have been stable, never mind viable as a battery component. There were also instances of multiple patents covering the same material, suggesting that any prior-art searches were not being done rigorously.
> EDIT: And perhaps more importantly give people the freedom to innovate on top versus stopping anyone who does.
Everyone is free to innovate on top of any pre-existing, current patent. They simply can't profit on their new innovation without licensing the patent. However what they can do with their new innovation is patent the innovation itself. This newly patented innovation prevents the original patent holder from profiting off of the new innovation, and thus incentivizes the original patent holder to license their patent to the new patent holder (or vice versa).
Ok, so the average has declined - has the raw # of disruptive studies increased? The definition of 'disruptive' here is based on the ratio of citations of [study] vs citations of [study's references]. If the number of papers increases, do we want a proportional increase in studies that "send a field in a new direction?" Or do we want incremental studies to be improving rigor, comprehensiveness, and clarity?
IMO, it'd be much better to look at the Avg. CD of the top N% of CD scores over time - that'd give me a much better idea of if disruptiveness is changing, as disruptiveness is a "best paper" phenomenon anyway.
I'm left a bit conflicted by what conclusions to draw from this effort.
How much of this is due to how much easier it is to publish something these days? Every statistics package spits out publication-ready charts and tables, Latex/Word has automated much of the typesetter's art, drafts can be communicated instantly via email, there are instantly searchable and linked databases of papers, etc, etc.
Think about what it was like in the 1950's, when many scientists were hiring typists to convert their longhand manuscripts into something an editor could read, attaching hand drawn figures, then heading down to the post office to mail the draft in. That draft then had to be copied and mailed to reviewers, who then mailed their opinions back, every step taking at least a few days of communication overhead. It was a different world and it gradually changed with widespread adoption of different technologies.
Also how does this interact with another pair of common suggestions for improving science: publishing negative results and doing more replications? Is this a sign that those two good practices are already occurring? Is it possible that the higher rate of disruption was due to shoddier work in the past?
> Think about what it was like in the 1950's, when many scientists were hiring typists to convert their longhand manuscripts into something an editor could read
From my training as a computer scientist, I am used to the idea that some problems are easy and some are plainly intractable. Perhaps that's also true in science and technology, and we are starting to exhaust all the easy directions.
Still, many less techy problems remain, at least climate change and wealth inequality come to mind. Perhaps we'll see the biggest innovations in solving those in the future.
As a software developer, I’d want to see a rewrite to clean up the legacy bodges before we conclude we can’t add more.
Also, wealth inequality isn’t a problem: how wealthy the people at the bottom are is. Wealth inequality is associated with fewer poor people and a higher standard of living because wealth inequality requires a wealthy society.
I'm not suggesting we can't add more, rather, that I find it conceivable that there is a barrier of technological progress that we can approach, but never reach. So the pace of progress will slow down.
Concerning wealth inequality, I do think it is a fundamental problem of society if a fraction of a percentage controls majority of the available resources even if the lowest half has their immediate needs met, but we can disagree on that all day and arrive nowhere.
It's indeed not an easy issue, which is why I think there could still be lots of innovations to made in that space in terms of economics, politics, social sciences etc.
The GINI index in absolute monarchies is close to 1.
Wealth inequality implies that some people disproportionately control the life production of other people. If you're in favor of a "great man" idea of progress this is a benefit. If you're in favor of individual power to direct production and innovation (along the lines of de Tocqueville or it's left-wing equivalent) this is a detraction.
Climate change seems like one of the techiest of problems.
I think wealth inequality is too ill defined to even be a problem. If anything i would argue it is only a problem if class mobility is low and that is where the focus should be.
- A "Pain point" focus would tend to be more incremental than truly disruptive
On the good side there may be some decrease of disruptive inventions and discoveries because we are also focusing on eliminating the externalities of current and previous inventions and discoveries. It's a lot easier to make wildly new things if you aren't worried about their side effects.
This is a big problem. PARC, Bell Labs, IBM Research are the first that come to mind. Same is true for mostly government-funded initiatives like the one where Marc Andreessen invented Mosaic browser, that eventually became Netscape Navigator. I don't recall the project name. Do we even do that anymore?
Well you get the funding before you do the experiment and thus before you know whether it's going to disagree with existing research. If you have the results before you do the experiment, you're not doing science.
No, grant proposals normally expect you to provide preliminary results. The more controversial your proposal, the better your preliminaries have to be. Especially you have to convince the funding deciders that you did the appropriate positive and negative controls.
Preliminary results are results from a previous experiment that already had to be funded. And yeah, you don't want to throw funding at researchers who are only getting weird results because they are bad at conducting research.
Literally every preliminary result I got was from an experiment that was explicitly not funded for the purpose it was requested for. That is, by year 2 of my grant, I was spending my monies on stuff I hadn't proposed, to prepare for the next grant proposal.
I guess part of the reason I exited academia was because I was bad at conducting research :)
The chicken-and-egg problem illustrated in this thread seemed like the most appropriate place to highlight:
So (see disclaimer at the end) there is a publication I appreciate called Seeds of Science. My understanding is that it is (in part) intended to offer a pathway to work around structural failure in comventional peer review; As its web page describes:
" What is a 'Seed of Science'? It is a speculation, an idea for an experiment, a novel observation, a thought-provoking question and discussion, the highlighting of an under-appreciated problem, or an unorthodox research study"
They got an ACX Grant last year from AstralCodexTen to pursue this.
He was asked to address some questions and make revisions. He was so aggravated that his paper was subject to peer review at all that he withdrew it. But his subsequent submission to another journal did incorporate some of the referees concerns. Einstein was a great scientist but he too was wrong sometimes. [1]
His 1905 seminal papers were not peer reviewed per-se but the journal's editor who accepted the paper was Max Planck! So it was reviewed by another legendary physicist.
These days science has branched off into such specialized areas that another specialist in the field is needed for review. Whether that should be an anonymous referee, or someone else it is still valuable to catch errors and determine the soundness of the techniques. Also the selections of those peers is something for debate too.
If you're really smart, enough to make disruptive scientific discoveries, why bother doing that when you could learn programming and work at a FAANG for $400k, or work in the finance sector and make possibly more? If you stay in science, you'll earn peanuts and not even have any career stability since the academic track is so bad these days.
"why bother doing that when you could learn programming and work at a FAANG for $400k, or work in the finance sector and make possibly more?"
Because it's boring and depressing. And one thing money can't buy, unless you're solely interested in accumulating money, is satisfaction with your life.
When I worked at IBM, we were given generous cash incentives for filing patents. I am named as an inventor on about 40 patents. To be honest, only one of them was a genuine invention in my opinion.
Doesn't it just stand to reason that there's less to discover, that what there is mostly isn't the low-hanging fruit (and unlike actual fruit, it's not regrowing), over time?
That's what I was thinking, too. Perhaps we already know majority of what there is to know, when it comes to fundamental concepts? Probably plenty of work left in improving our tools and engineering solutions, like machine learning-based software, as well as in understanding the intricates of biology. Maybe mathematics, too. Maybe improvements in these will cause another golden age of discovery. Like, understanding biology enough to gain significantly extended lifespan, which means much more expertise can be built by one individual.
I think the same processes that turn nimble and innovative startups into large and heavy enterprises apply to countries and institutions as well. As academia and research institutions develop, they become more complex, valuing process over result and consensus over conflict. Increasingly, disruption of consensus is perceived as disruption of the institution itself and risk-aversion and incrementalism are rewarded.
I hate to be a doomsayer, but this seems to be a general trajectory that most of the western world is on.
There’s an intuition that recent products and inventions are more incremental and that historical innovations are more revolutionary- in the sense of lacking precedence.
But this is an effect of the simplification of history, and when you read the detailed technical histories you can see how remarkably incremental the old innovations were.
We can easily see the precedents of recent innovations but not for older ones.
This is pointed out in the book I am reading on the development of the turbojet, which only looks more unprecedented than it really was.
> The intuition is that if a paper or patent is disruptive, the subsequent work that cites it is less likely to also cite its predecessors
This intuition seems to overlook the culture of citations in the patent space. There, the incentive is for applicants to be over inclusive with citations. This is combined with the increasing ease with which references can be found.
So, as fastaguy88 already described in another comment, this paper is a fairly arbitrary quantification of “disruptive”. (Also, can we just stop using that word? Such a though-terminating cliche at this point.)
But let’s accept that we are seeing less fundamental breakthroughs in science. Is that a failure of the system…or, depressingly, what if we really are hitting the limits of what we can model and describe?
In a roughly 200 year timespan, we figured out the at least the rough outlines of…wow, where even to start…electromagnetism, general and special relativity, quantum mechanics as it relates to field theories and chemistry, genetics, and computability. We really picked a lot of the low-hanging fruit, and what if there just isn’t any fruit left to easily get at? There’s certainly things like dark energy that we don’t have particularly satisfying theories for at all, but, the basics? We seem to have good models.
The physical realm seems largely picked clean, but there is bountiful fruits available in the metaphysical realm - shame that ~no one can see it, despite complaining about it constantly.
It's a great meme, but memes are only good for estimating the future, and while the future can indeed be strongly influenced by the fact that much of human cognition is meme-based, there are also many other influences on it.
This and many other pieces of very useful knowledge are there for the taking in the metaphysical realm, just as was the case hundreds of years ago in the physical realm, prior to the rise of science (a metaphysical phenomenon itself).
Its just the evolution vs revolution periods...
In any field 99% of the time advancements are slow and incremental, than a new idea sparks a revolution and you get a lot of advancements in a short period of time. This is just the nature of scientific progress.
In the early 20 century new ideas in physics and mathematics sparked a revolution and we are slowly unraveling the benefits of it ever since.
For 40 years we had evolution in the AI research field, now we have a revolution, I do not know when it will die down but after a while we will just start to build over the ideas we discovered and a long period will pass before we do another jump.
Same with psychology, economics, biology...
And yet I've been trying to find $$ for one of the most disruptive agriculture technologies that could propel to sustainable ag to acreage unheard of. Nobody wants to take a chance.
The science knowledge base grows monotonically but not necessarily at a steady rate. Actually far from it.
There was a time in the 19th century when people thought physics discoveries were complete. A few decades later all hell broke lose.
The problem we have right now is that we are stymied on the complexity front (materials, chemistry, biology, medicine etc)
Our mental / mathematical models that we use to understand the world and even enable us to ask original scientific questions are losing their edge when things get complex. These tools are 19th century stuff (including all the AI junk) and we have milked them for all they are worth
I would hypothesise that a new period of discovery will correlate with some breakthrough in the mathematics of complexity. We had a few false dawns but not the real thing
> These tools are 19th century stuff (including all the AI junk)
I'm not really sure what you're looking for here. I mean, you could claim that modern cryptography is based on ideas from the ancient Greeks (factoring numbers), and is therefore old and busted. Is every idea just derivative unless it uses a completely new kind of math?
Software developers (maybe not you, but a large percentage of the users of this site) tend to automatically think older ideas are worse. But that's not true in math. Mathematical ideas don't expire. The idea of the number 7 is aeons old; the number 0 over a thousand years; the complex number i 450 years. But the numbers 7 and 0 and i are still extremely useful, in infinite combinations. If you think we have milked those numbers for all they're worth, you have an extremely limited imagination.
Sure, most of the "AI junk" is a combination of old ideas like linear algebra and gradient descent, but used at a scale orders of magnitude larger than their creators ever imagined. (And quantity has a quality all its own.) It's that combination of ideas that is so powerful. Besides, backpropagation is a pretty damned novel idea. If you're looking for the one truly new idea from AI, it's backpropagation.
You are vastly underestimating the volume of the idea space available to explore simply by combining the ideas and materials we already have. Crossover is a better source of variety than mutation is. You may be right that truly new concepts are the "raw materials" of invention, but we never run out of the raw ideas we have, and they prove to be useful in their combination up to the limits of our imaginations. It's actually surprising how likely you are to come up with a reasonably good idea by simply considering a few randomly-selected ideas together.
> we are stymied on the complexity front (materials, chemistry, biology, medicine etc)
Again you seem like your standard is: if it's not an entirely new thing where nothing like it has ever existed before, then it's boring!. If you think the volume of the idea-space from combining the materials, chemistry, biology, and medicine we already have/understand is X, it's almost certainly at least 2^X, or even X! (factorial).
Besides, we are coming up with brand new materials all the time! Materials science is one of the most exciting frontiers of human technology.
Dont get me wrong, there is plenty of scope for interesting / useful applications of software and pattern matching at scale but those are more a form of "tech assisted social evolution" rather than the steady march of fundamental inventions and discoveries across the entire spectrum of knowledge that characterised the last few centuries.
Old math can still have an impact at the margin (eg speeding up protein folding calculations etc) but its not the cognitive leap that would allow us to steer discoveries in more effective manner. Materials, medicine etc all very exciting and very active but also hampered by their intrinsic complexity.
Good article. Interesting finding at the very end:
>Although the proportion of disruptive research dropped significantly between 1945 and 2010, the number of highly disruptive studies has remained about the same.
So... maybe this is because having more people doing scientific research doesn't yield more breakthroughs. More people doing scientific research DOES yield more incremental achievements, as the study found, i.e. word frequency analysis showing more occurrences of "improve", "expand", "extend".
The number of highly disruptive studies remaining the same seems impressive enough to me. I expect diminishing returns to be the long-term norm in science. The low hanging fruit get picked first, and every breakthrough means one more concept that the next generation of researchers have to learn and incorporate into their work.
Studying the genetic basis for various characteristics is not eugenics, anymore than studying the atom is building a nuclear weapon. Abuse of knowledge should be controlled, but banning the pursuit of knowledge itself?
That's just intentionally crippling our own understanding of the world, for fear that we might not like what we discover. That's hardly an unprecedented behavior, to say the least, but the cultures that have gone that direction are probably not the sort we might want to emulate.
The first half of the 20th century had a lot of "low hanging fruit" in physics and chemistry that basically came out of the enlightenment. Things like electromagnetism, quantum mechanics. I wouldn't guess that's always the steady state for research.
The war may have had something to do with it too, I know a lot of advances (like radio / microwave engineering) have roots in the war efforts. But I suspect it's more of just a dry spell because we pushed forward in the 18th-20th century and ran out of ideas
Wouldn't we be presented with another bout of "low hanging fruits" in the second half of 21st century when tectonic shifts happen in quantum computing, superconductivity, ai, ar/vr/mr etc.?
Nice to see numbers put to this. I remember thinking through a bunch of tech/science that I was familiar with, and realizing that most (not all) of the fundamentals in every case were known by the early 60's, and we've been working out the (very important) details ever since. Nuclear, electronics, chemistry, biology, etc.
plenty of counter-examples, I'm sure, but like once you know the structure of DNA and it's fundamental operations, the remaining discoveries will have a smaller impact.
Maybe it's wrong. At least in the chart, they are missing the Bitcoin invention in 2009 (technology). There was the last "disruptive" science in the area.
While there is no doubt that 'CD Index' has been declining over time, it is debatable whether it truly indicates a drop in 'disruption'.
Let's take AI for example. Research papers that were published in 2019-20 have become obsolete within a year or less, and consequently get cited less and less, as more recent and derivative papers introduced better models. Does that make the older papers any less 'disruptive'? No!
On the paper front, I have a number of published scientific papers with citations. My 3 most important papers—that will go down as turning points in the history of computer science—have a combined 0 citations, and 2 of them not only were not in a journal, but they were even flagged and removed from arxiv.
On the patent front, well patents are for parasites.
Patents went from real innovation to Legalese BS and market protection in the past decades.
Also I suspect most innovation are being done outside of papers, especially when you have "reviewer 2" ready to be nitpicky at every minor detail, so you make it more bland and maybe skip one juicy part or another
I've heard another interesting argument, at least for disruptive science/research from industry.
Lower corporate taxation incentivizes the hoarding of cash. Prior, corporations were incentivized to allocate more funding towards their industry research (IBM, Bell Labs, etc).
"People talk glibly about science. What is science? People coming out of a university with a master's degree or a PhD, you take them into the field and they literally don't believe anything unless it's a peer-reviewed paper. It's the only thing they accept and you say to them 'But let's observe. Let's think. Let's discuss.' They don't do it. It's just, 'Is it in a peer-reviewed paper or not?' That's their view of science. I think it's pathetic.
Gone into universities as bright young people. They come out of them brain dead, not even knowing what science means. They think it means peer-reviewed papers etc. No, that's academia. And if a paper is peer-reviewed it means everybody thought the same therefore they approved it.
An unintended consequence is that when new knowledge emerges, new scientific insights, they can never ever be peer-reviewed. So we're blocking all new advances in science that are big advances. If you look at the breakthroughs in science almost always they don't come from the center of that profession, they come from the fringe. The finest candle makers in the world couldn't even think of electric lights. They don't come from within they often come from outside the bricks. We're going to kill ourselves because of stupidity."
— Allan Savory, Zimbabwean ecologist, research biologist, livestock farmer, and president and co-founder of the Savory Institute. He originated the Holistic management of agriculture to fight desertification.
Yea, sadly I can relate to this quote. Modern science has lost it voice and purpose it seems and it nothing more than a glorified printing press of poor research and glad-handing consortiums.
While I appreciate the sentiment, nearly all new knowledge goes through peer review because for every important problem in academia, there's a competitor who is the best person to point out problems with the theory.
"Holistic management of agriculture to fight desertification" is a non-quantitative science and falls under a very different regime from, say, people working at the state of the art in particle physics or biology.
I'm not defending peer review; I never enjoyed it when I published. It certainly never made my papers any better.
Newton and others all published and then discussed the details in person, a lot of the peer review was "off-paper" and "post-publication".
Also, the vast majority of scientific breakthroughs, by count, occurred after peer review- mainly due to the unbelievable pace of breakthroughs due to technological advancement.
That sounded interesting so I looked into it. It seems to instead support peer review.
According to this [1], Einstein's paper that didn't pass peer review turned out to be wrong (it said gravitational waves didn't exist), and was later published elsewhere with different conclusions and fixing problems that the referee's report had already pointed out.
> The irony, of course, is that Einstein could have found that escape route months earlier, simply by reading the referee’s report that he had dismissed so hastily. The referee had also observed that casting the Einstein–Rosen metric (as we now call this solution of the Einstein equations) in cylindrical coordinates removes the apparent difficulty.
Every time I read one of these editorials on the topic of lack of available research, I just look at the huge list of significant problems that still exist for life on earth and wonder: “what are they thinking?”
Hardly surprising. Besides fraud, in science you're not overturning past experimental data, you are adding to it, forming an ever more complete and accurate view of reality. Disruption really just means stumbling upon something unexpected. Once you know there is something there, it opens up a whole world of new avenues for studying in that space, but it doesn't really make it any easier to find something unexpected again.
Consider for example deep sea archaeology. When you find a shipwreck, there's going to be a flurry of activity as people study the shipwreck, and the more they answer the questions they started with, the more new questions they are going to want to ask, so even long after the initial discovery there's going to be a continuous string of studies refining knowledge about that wreck. And this is true of every wreck. But finding one shipwreck doesn't generally make it easier to find the next, assuming the same resources, new wrecks are going to be discovered at a pretty steady rate. So the number of discovered wrecks grows linearly, but with each being revisited regularly, the cumulative amount of research done on all wrecks grows geometrically.
Similarly discovering a new chemical opens up many possibilities for studying that chemical, likewise for a new phylum of bacteria, a new type of star, you can only see something for the first time once, but you can take closer looks forever. We should expect the ratio of new things to look like a graph of 1/x.
Perhaps it's weird that the absolute number of disruptive papers is holding steady. While there are competing forces at work - on the one hand we have more scientists with better instruments, but on the other hand the low hanging fruit is gone and they must go ever further to find something new - there's no fundamental reason these two forces should perfectly counterbalance. I might posit our society has a certain appetite for discovery and we adjust our allocation of resources to science and innovation to maintain this continuous stream of disruptions, but I at least feel that scientific breakthroughs tend to excite people about science and desire to double down on science, not pull back. Perhaps more realistically we just move the goal posts as to what we consider disruptive such that the top X discoveries of the year are always "groundbreaking." Regardless of the why, this definitely the more interesting part of the story.
There are just too many gatekeepers and too many people who are used to doing things a certain way. The world is just more populous now. We also need to make sure that people can speak their minds freely and write freely without worrying about getting cancelled.
Too much talent going into trivial tech ventures.
The shift towards rentier capitalism in the US, especially in terms of cheap money/QE being used for share buybacks instead of R&D.
The colossal cost of higher education in the US, with grad stipends barely budging until literally this month.
Fewer people willing to tolerate the (mostly) toxic culture of academic advisors overseeing Phds and post-grads. Google Scholar skewing papers to exist more for popularity/citation prevalence. The Internet as the biggest distraction in the history of humanity.
etc. etc.
To understand the fundamental problem with this paper and the "CD" measure of "disruptiveness", one need only look at Figure 1a, where three Nobel Prize winning papers and three patents are ranked on the "CD5" scale. Apparently, there are more disruptive discoveries than Watson and Crick's discovery of the double-stranded structure of DNA (CD5 0.62 on a scale of -1.0 to 1.0), and Baltimore's discovery of reverse transcription is -0.55. One might argue the ranks of these two discoveries should be reversed, since DNA had already been shown to be the genetic material, but no one imagined that RNA could be converted to DNA. Likewise, the Wigler patent for transformation into eukaryotic cells (CD5=0.70) has certainly has had less effect on the world economy than Monsanto's patent on glyphosphate resistant plants (CD= -0.85).
It's easy to make up a measure and then develop a story that explains why that measure is useful. Was Newton's theory of gravity disruptive, or consolidating. I'm thinking consolidating, since it allowed a lot of other things to make sense. Likewise, perhaps quantum mechanics was disruptive -- it sounds disruptive -- but it also consolidated a lot of confusing observations.
(And lets not even mention the problem that when there are 1,000-times as many papers between 1940 and 2010, we might not expect 1,000-times as many disruptions.)
This paper has a measure that sounds useful, and is certainly good for headlines, but it is very unclear that it provides any insight into the progress of science.
Agreed. I think this is fundamentally a sampling rate observation.
Pace of publishing and observations have dramatically as you rightly point out.
You misunderstand why W&C was important (although, given your username, I assume you've already sat through the lectures): in elucidating (not discovering) the structure of DNA, they had a biophysical mechanism that explained heredity (specifically, that it was an antiparallel double helix with complementary bases, which immediately suggests a mechanism for replication).
That recognition immediately led to a flurry of follow-on work confirming the mechanism and birthing molecular biology as we know it, which opened up vast new areas of science.
Yes, I know why it was disruptive. The question (for this paper) is, what other biological papers were more disruptive (since, based on the range of the scale, they barely made the first quintile).
I think the more important question is, how do we validate the usefulness of the CD5 metric. And, why is it better to be "disruptive" than "consolidating"?
Based on the examples shown (and others that I can imagine), it is unclear that it measures anything related to scientific significance.
I had already discounted the measure as being too prone to sampling error to be useful. I don't think these things are really measurable, and even if you did measure it, "disruption" and "consolidation" are such subjective terms it would be arguable to have a single-dimensional slider that went between those extremes.
Probably best to just say some of the greatest discoveries are both disruptive, and consolidating.
Yep.
The trend over time could equally be capturing changes in referencing practice (either that really important, influential papers/inventions in the past didn't reference other really important, influential papers/inventions nearly as often as equally novel papers/inventions today, or that modern referencing is much more exhaustive about referencing minor and tangentially relevant papers/inventions. There is, of course, also more significant prior art in journals and patent indices to actually cite in later years...). They carry out some robustness checks, but I'm not sure any of them really dent that hypothesis. It's also a conclusion based on proportions (ie. the effect mostly comes from by there being a lot more "non-disruptive" papers published rather than fewer "disruptive" papers) and there's actually a trend rise in the raw number of "most disruptive" academic papers from the 1950s through to the 1990s, at which point the numbers drop so sharply it looks like a discontinuity in the data (but would probably coincide with lots of journal content becoming easily accessible online...)
Whatever CD is measuring, is going down as a trend. So one cannot blame the obviously ethereal nature of CD for this.
Boring title: "Scientific authors follow more citations when reading modern papers".
When I read a modern paper, I skim a lot more of the references. Getting to a cited paper is often just two clicks: there's a reference to the bibliography, and from there a link to arXiv (or some other public repository). As papers get older the bibliography might not link to arXiv, or might not link to a digitized paper at all. Those older papers are more time consuming to use as a reference. Scientists aren't lazy but they also don't have infinite time.
Put another way: papers form a graph, this result suggests that older nodes are traversed less frequently. I think the metric they propose is useful, but we could rephrase this all as measure of how useful papers are as a node (where CD = -1 is most useful, CD = 1 is least useful).
In short, this result might just mean paper discovery is getting easier. That's a good thing. We can wring our hands endlessly about how there's less low hanging fruit (there is) or about how academia is rotting away (as though it was ever different), but those conclusions are big jumps from what this paper says.
When I was in grad school I had done measurements of the magnetic anisotropy of amorphous compounds. I had found an interesting link between the structure and the magnetic properties which is surprising since the material is amorphous, the material properties should be isotropic yet these had a preferred magnetization axis. During the discussions of these results with my advisor I was told in very clear terms that he would not let me publish results that contradicted a paper of his from the 1980s.
I didn't push back, because I had no power and I just wanted to get the fuck out of there and start making money after being scammed out of 5 years of my life where I poured a lot of effort and got shit pay. I lost respect for my advisor and I was stunned by how un-scientific his response was when it came to correcting his 1980s results. My measurements were more accurate and were strongly supported by structural measurements he never did and couldn't do back then. But he was more interested in protecting his "legacy" of his paper with 13 citations.
So petty and small. I dropped those efforts and was able to get out completely obviating those results, never mentioned them in my thesis even.
Anyway, my point is that I'm not surprised that scientific results are lame, vanilla, uninspired. When the committees that decide on what gets funded are all 65+ dinosaurs stuck in their ways, very few disruptive ideas will be founded because you're disrupting the very research that got the committee members clout and recognition.
As they say, change comes a funeral at a time.
It used to come a funeral at a time, but now that the next generation is forced to build their careers on the mistakes of their advisors, mistakes can easily outlive their creators.
Did you push at all? From experience, this is not an uncommon story. Perhaps this was your advisor's way of saying 'convince me'.
I see this kind of stuff all the time and I'm convinced that Academia is corrupt. While working against a Grad Student doesn't seem particularly evil, rather just a bit backstabby/political, I suggest that it is 'really bad' because it contravenes the basic organizing principle of science itself.
Kind of like a Judge ruling one way or another for personal reasons, or ruling with a heavy conflict of interest.
There needs to be much stronger ethical principles around this and an advisor hinting at obfuscating work for such reasons should be up for review.
That timeline correlates well with the corporatization of academic research timeline, which got started in the late 1970s, boomed in the 1980s and has steadily risen ever since. The emphasis in applied science is now on patents and return-on-investment, aka safe bets. This is certainly the direction that much of NIH fundings, as well as DOE funding, has taken. Part of this is that entrenched industries (natural gas and coal power, say), don't want federal funding going to disruptive competition (monocrystalline silicon PV development, say). This is reflected in how federal funds are distributed as well as what research programs are supported at the university level. For example, count the number of renewable energy departments in the USA academic system: miniscule funding -> no programs.
It can't all be blamed on corporatization, however. The control of funding agencies at the federal level by vested interests seems to be a major problem, i.e. innovative lines of research that threaten the status quo could threaten those cabals that control fund disbursement, so such efforts don't get funded. The stagnation of Alzheimer's research and the focus on the amyloid hypothesis seems to fall into this category. This kind of bureaucratic ossification is not new; the classic example is how Trofim Lysenko in the Soviet Union controlled the direction of agricultural research for about three decades, much to the detriment of the understanding of plant genetics and crop development research in the USSR over that period.
So it can be attributed to the rise of monopolistic corporatization on university campuses on one hand, and the growth of entrenched bureaucracies at the federal funding level on the other. Neither group wants the rug pulled out from under them by 'disruptive innovation'. There are many historical antecedents for this kind of situation, incidentally.
> The control of funding agencies at the federal level by vested interests seems to be a major problem
This is a fundamental problem with the nature of science and the nation state in that modern science is constituted by the nation state, not merely imposing into or upon it.
https://www.youtube.com/watch?v=Jvntr0J5eQk
Given that Tech Transfer offices don't make much money at all, and patents are mostly worthless, I doubtful of this theory. There simply is no 'return on investment' on most of the work.
I wonder if this is to do with major vs minor breakthrough.
Take physics. As I read it, modern fundamental physics is mostly some combination of general relativity and quantum mechanics, which are both early 20th century inventions.
They weren't necessarily genius in particular in-depth analysis, rather they were genius new concepts that created a whole new blank canvas to fill in.
What did the field look like just before them? I always forget the details, but was it lord Kelvin who said, physics just needs to figure out the ultraviolet catastrophe and it's done? I wonder if it looked similarly uninnovative. The canvas they had was full, and they needed a new canvas for true innovation.
Now the thing is, I think true disruption, like GR and quantum physics, is not limited by the number of postdocs, but by truly genius insights. Einstein wasn't even working as an academic when he came up with Special Relativity. So the number of papers written isn't a proxy for pace of revolution.
Not to mention, since number of published papers has become a target, it has become useless as a measure of progress.
I agree, I think we're discounting genius insight as the primary driver of innovation and ground breaking science, and we can't expect genius insight to occur in some predictable time frame or a time frame even within a lifespan.
I think perhaps we've also partially lost the meaning of ground breaking research given the constant news articles on "ground breaking science" and the constant overinflation and exaggeration of one's papers and achievements.
A lot of the obvious questions on the table were answered, even without an answer we at least have the question. Today true, groundbreaking innovation needs someone to identify the new questions. It's like doing science in a bubble and you need someone to take a massive new one and merge it into our own. Right now we're not opening wide doors, but just refining and adding almost imperceptibly to an already huge bubble. We're grinding in the game because we can't find the boss on the map or can't beat it yet.
Besides the current structure of research and the known weaknesses (misaligned incentives, funding issues, etc.) science in general today ran out of low hanging fruit. The last unexplored or unanswered riddles may need prerequisites that are outside of the current boundaries so someone needs to open the door do a vast new bubble that has more relatively low hanging fruit.
Also humans probably (don't quote me on that :) ) can't come up with any new questions or answers unless they are already at the edge of that bubble. And they each see and understand a smaller patch of that surface as it expanded. Stumbling onto new thinks is exponentially harder.
This is what those revolutionary discoveries did for us in the past: Opened the door to a new world where the hard answers were suddenly a bit easier, they made sense.
> we can't expect genius insight to occur in some predictable time frame
Yes, and we can’t expect it to occur in a socially confined over-structured environment either. Academia has changed a lot since then, with peer review hell, publishing pressure, etc. I mean, are we really pretending to be surprised that a system of incrementalist incentives produces incrementalist results?
It might be that human brains can only do so much physics, which is a tiny subset of all the physics there is.
It’s amazing upright apes can figure out the speed of light or postulate and find the Higgs Boson in the first place.
How much physics do we expect the smartest dog in the world to do? What if we 10x’d its lifespan?
At some point either:
1. There is no more physics
2. We build beings that can do physics that we can’t
3. We’re stopped dead in our tracks at the limit of our ability to comprehend things that apes weren’t built to comprehend
We get around this in part by not understanding (or at least remembering our understanding of) many of the basics. Such as this guy who looked things up in a textbook: https://news.ycombinator.com/item?id=34236889
A bunch of blind people could theoretically construct a model of an elephant, if they work together well. https://en.wikipedia.org/wiki/Blind_men_and_an_elephant
Looks like time to pivot to bioengineering and get better physics brains. ;)
I wonder if we’d have any luck engineering a brain that can visualize higher dimensions the way we visualize 3d.
It would be cool to have a brain that sees time, just as an incidental consequence of what’s happening in higher dimensions.
My brain did this once. It doesn’t work very well anymore after that.
It also seems unlikely that a patent clerk, even one with a PhD in physics, would be allowed to publish in a physics journal today. The gatekeeping has gotten a lot more strict.
Filters out a lot of kook theories but I'm sure some revolutionary stuff from some Goodwill Hunting genius has been filtered out as well.
I’m not sure it filters out kook theories, given the replication crisis.
Rather, journals promote well-connected kooks at the expense of less well-connected kooks and happen to occasionally publish some science as a byproduct.
Fun fact: peer review isn't gatekept. There are peer reviewed young earth creationist articles out there: https://www.cnet.com/culture/creationists-launch-peer-review...
So "peer review" isn't holding back kook theories at all on its own. At best, journal prestige is bolstered by it, which the journal can then translate into holding them back. The track record is a bit unclear, though.
I view peer review as respecting the time of the journal subscriber. Someone filters out the things that are obviously flawed before wasting everyone else's time.
Journal aren't just a magic upload into scientific consciousness; it takes time and energy to read and process them.
It's doing a rather bad job at filtering out obvious flaws, given the reproducibility crisis we've got on our hands. Everyone says peer review is very important, but when it comes to reviewers putting in the hard and boring work of actually checking the raw data, almost none do, so in truth, peer review is NOT important to most scientists.
And we nevertheless pay the price of adhering closer to orthodoxy for it, since effort being put in is a rather superficial "this paper sounds reasonable to me" filter, perhaps with some boxes that must be checked like p-value or sample size.
A peer review young earth creationist journal would still gatekeep anything that did not come from a young earth perspective. >Papers can be in any scientific, or social scientific, field, but must be from a young-earth perspective and aim to assist the development of the creation model of origins
The problem is that inherently peer review is designed to act as a brake: >Peer review, in which a scientist's paper is scrutinized by a group of colleagues, is designed to find errors and weed out half-baked ideas.
But in blocking you are going to reject ideas. >And although some have criticized peer review for rejecting new ideas just because they're too radical for the establishment to stomach
How much of a problem and how many revolutionary ideas are stopped, I don't know. I am more concerned about the reproducibility problem in science.
Sorry, bad phrasing on my part. I didn't mean that peer reviewing doesn't lead to gatekeeping, but that simply making something peer reviewed isn't enough, because ludicrously bad attempts at science can be peer reviewed.
Peer review is also obviously a bad solution for the reproducibility problem, given that we've got the problem now, concurrent with generations of peer reviewed manuscripts.
https://theconversation.com/hate-the-peer-review-process-ein...
This article mentions Einstein being offended that his paper was submitted to peer review by Physical Review, and submitting it elsewhere instead:
"We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorised you to show it to specialists before it is printed. I see no reason to address the – in any case erroneous – comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere."
I just recently ended up there via this blog: https://experimentalhistory.substack.com/p/the-rise-and-fall...
which argues, not unconvincingly IMO, that peer review is a failed experiment which tries to stop bad science at the price of also stopping brilliant, unconventional science, where instead we would profit more from letting the scientific public do the sorting instead of hand-picked specialists. The author phrases it as science being as strong as its strongest links, not as weak as the weakest one, and we should do everything to get the brilliant crazy ideas.
> the price of also stopping brilliant, unconventional science, where instead we would profit more from letting the scientific public
Also currently on the front page: https://news.ycombinator.com/item?id=34257150
In brief, London 'man off the street' solves 20k year Ice Age drawings mystery, published in Cambridge Archaeological Journal via contact with Durham professor. (And he is first author.)
I don't say that to disagree with you by the way - it's undoubtedly a rare example.
These days Einstein could just put his paper on arXiv and worry about publication later. The physics community goes nuts about arXiv results all the time. There's zero review if you have a university email (or national lab etc).
I tend to agree that peer review has flaws. But given the ease with which someone can publish anything with no review whatsoever it seems reasonable that peer review might shift to fill a niche that is slower and less disruptive.
I disagree, at least in math there have been many instances recently where laymen (with a PhD) found some new and exciting results[0] and got to publish them, and I don't see why it would be any other way in physics.
Most recent example: https://www.quantamagazine.org/long-out-of-math-an-ai-progra...
> but was it lord Kelvin who said, physics just needs to figure out the ultraviolet catastrophe and it's done?
This is apparently a common misconception regarding a lecture given by Lord Kelvin in 1900 concerning "two clouds" in physics.[1] In fact, Max Planck solved the ultraviolet catastrophe, also in 1900, by assuming electromagnetic radiation can only be emitted or absorbed in discrete packets of energy called quanta. Albert Einstein also solved the ultraviolet catastrophe in a paper he published in 1905, for which he was awarded the Nobel Prize in 1922, by, as always, standing on the shoulders of giants, hypothesizing that Planck's quanta were actual particles. These particles are known today by a word coined by physical chemist Gilbert Newton Lewis in a letter to Nature in 1926, namely, photons.
[1] https://arxiv.org/pdf/2106.16033.pdf
TIL; very interesting!
A big part is that science has become very “sanitized”. Work on the wrong issues and you won’t get funding or will even get ridiculed out of a career.
It’s not even just going against headline consensus issues that will get you in trouble. Any findings that overturn what a large group of scientists have spent their careers working on won’t be well received.
More than that though. Modern science research is a highly hierarchal and managed enterprise. It's led, or at least its funding is controlled by, the risk averse managerial class who can only think it terms of returns on investment.
9/11 (believe it or not) had a lot to do with that in IT/computer engineering research (and likely other fields) with government funding. After the attacks, the US Federal government moved to stressing development more than blue sky research. That drew all the funding away from fundamental science to projects that could be made into a product in 3-5 years and focused on addressing some issue of the War on Terror.
Would you say this is a natural consecuence of human emotions (e.g ego), economic interests, powerful players that have corrupted the game etc? I'm genuintely curious. As an ignorant on science and its processes in academia or markets, I cannot feel nothing other than dissapointment and loss of hope everytime I heard this kind of things about a matter I've always considered quite rational and focused on pursuing the truth over anything else. Naive me, I guess...
Good questions.
It’s always been a problem but I’d say these factors make it worse:
1. It’s harder for individuals or small groups to make breakthroughs. We need larger groups and more expertise as well as expensive equipment to continue making discoveries. More hands in the pie make more gatekeepers and less risk taking.
2. In general our entire society is “circling the wagons”. I’m not sure why but we’re more tribal than normal. Science isn’t immune from the trend.
I could go on. It’s a big topic.
This is a very natural consequence of the higher difficulty of maintaining one's career in academia. What is a natural consequence of the increased competition for those jobs.
> Any findings that overturn what a large group of scientists have spent their careers working on won’t be well received.
Hasn't that always been the case? It's said science advances one grave at a time.
I suspect that there isn't a single cause but a cluster of them.
One possible issue is that high level science needs more and more energy (like LHC) and more and more intelligence to crack (most science prizes are teams).
Is there any evidence that this has gotten any worse over time? Planck's principle[0] is nothing new.
[0] https://en.wikipedia.org/wiki/Planck%27s_principle?wprov=sfl...
Alternative theory: we've pretty much nailed the basics and it takes a PhD before people can even understand the frontier of most scientific fields. These frontiers are manifold, highly technical, and extremely boring to the layperson. Number theory is a very clear example of this: go read the Birch and Swinnerton-Dyer conjecture. Come back in a decade after you understand the Tate–Shafarevich group. (hahahaha just kidding, if you understood that you'd have proved the conjecture)
I’ve heard this too about number theory. Perhaps in that field, the paradigm shift lies in reducing incidental complexity? Or even managing it in a way that is better for humans?
> it takes a PhD before people can even understand the frontier of most scientific fields
It’s interesting that we have this implicit view that hyper-specialization is the only way to advance. I always thought so too, but why really? If you think about it, specialization is very close to incrementalism. Groundbreaking paradigm-shift stuff can often be written down on a napkin, and rarely does it require a PhD.
That said, to find treasure you have to go down a LOT of wrong paths before you (or someone else) ends up in the right one.
There’s not a lot left that fits on a napkin. All sequences of tokens below length n have been enumerated.
This was considered in the paper: "Some point to a dearth of ‘low-hanging fruit’ as the readily available productivity-enhancing innovations have already been made19,27. Others emphasize the increasing burden of knowledge; scientists and inventors require ever more training to reach the frontiers of their fields, leaving less time to push those frontiers forward18,28. "
The grant/funding process itself slows down science.
How to slow down scientific progress According to Leo Szilard
https://rootsofprogress.org/szilard-on-slowing-science
Discussed in https://news.ycombinator.com/item?id=34264436
I blame the grant funding process. What gets published is what gets funded when written up as a grant proposal. What gets written up as a grant proposal is what is called for by grant funding agencies for their own research initiatives. If you want disruptive science, fund it then. That's all everyone wants to do in science, but the bills need to be payed somehow, so you play the game the funding agencies want you to play.
Plus, there is a large degree of "orthodoxy" in these grants, even most of them proclaim that they are "high-risk, high-gain". In the end, it seems that if you are too much on the "risk" scale, you will not get the funding since committees prefer "a little more of the same, please" to more wild ideas. I realise that such ideas are of course hard to classify in terms of their feasibility, but that's the main purpose of science, right? Venturing boldly into the unknown and all that; sometimes you come back with the treasure, sometimes you come back with lessons learned.
The current small-independent-grant culture is flat. This means that all projects are small and there are is almost no hierarchical 'higher level' science. PIs are happy to publish minor descriptions of the clouds, and journals are happy to publish them , in lack of something more impressive. So much value is lost in creating processes and bureaucracy.
The focus is clearly on "getting the grant". And the grant acts as an entity in itself, as it becomes a line in the CV of scientists, which will help them "get the next grant". The outcome of the grant becomes thus irrelevant. Maybe competition is missing?
It almost seems as a new model would take the world by storm, but it doesnt seem to arise. Even private research often falls for the same pitfalls, like how Deepmind keeps seeking to publish proprietary research in Nature, instead of creating an open ecosystem that will drive reinforcement-learning science forward.
That’s because our financial system is controlled by power hungry but small minded sociopaths.
That's an apt way of describing the voter base :). Or the market at scale.
That's the major problem humanity faces. Individuals wielding concentrated power are dangerous, but they're also at least capable of thinking about something other than their immediate short-term interest. The masses, as aggregated through votes or purchasing decisions? They're structurally incapable of it.
If what you are saying was true then NIMBYism wouldn't be a problem, and we also wouldn't have any large "open spaces" left.
The problem is more parochialism than it is short-term thinking. And both sets of entities (powerful individuals and aggregate masses) are capable of parochialism.
> The masses, as aggregated through votes or purchasing decisions? They're structurally incapable of it.
I think this is a dangerous delusion:
Historically, centralized control has been a disaster because those individuals wielding power believed they could think of something greater… but actually lacked the skill. So we ended up with simplistic concepts that failed at scale, ideas that never could work because they missed key details, etc. By contrast, society gets on with the business of building the future in spite of and not due to that meddling by centralized power.
> The focus is clearly on "getting the grant". And the grant acts as an entity in itself, as it becomes a line in the CV of scientists, which will help them "get the next grant". The outcome of the grant becomes thus irrelevant. Maybe competition is missing?
I think there's plenty enough of competition in academia. And competition alone doesn't help, because the metrics over which one loses or wins the race are not correlated with short or long-term worthiness. And while we can describe, even if somewhat vaguely, what makes worthy science, there's no way I know of to have a stable system in which that metric drives funding.
> Deepmind keeps seeking to publish proprietary research in Nature, instead of creating an open ecosystem that will drive reinforcement-learning science forward
This research at least has some feedback: Google funds it because it expects to make money off using its results in practice, so the research has to be at least somewhat correlated with reality. Unfortunately, this structure also means that "open ecosystem" isn't pursued.
I think that the expectation that "ground-breaking" discoveries in science should follow some linear or predictable timetable is not reasonable. Is the time period between "ground-breaking" discoveries supposed to be every 5 years, every 10 years?
The volume of papers has increased significantly, and publish or perish kinda stinks, but it's not just an issue of funding, but some (many?) researchers publish and exaggerate about the importance and difficulty of their research to receive an "I am smart" badge on social media. Although, despite this increase in volume of crap tier papers, the article seems to think it's not correlated:
"Declines in disruptiveness are also not attributable to changing publication, citation or authorship practices..."
What you're pointing to is far more insidious than I think is usually recognized.
People can point to incentive structures like grants etc, and to the implications in terms of the meaning of metrics etc but the real harm is in how it shifts mental focus and attention collectively.
It's not ok to let minor things or things you think are obvious go anymore to focus on bigger picture issues, that might be "riskier" but nonobvious. The way that plays out too is incredibly dependent on your institutional social environment etc.
So much of this is difficult to easily quantify in ways that can be easily studied. I loved this paper for example, but it's easy for me to think of papers in my own field that would look "disruptive" in terms of citation networks but are really the same content. There's these weird shifts I've seen happen in reading old literature and during my career, where big shifts in who is being cited will happen for quixotic social reasons. Usually it's basically politics or ignorance of big parts of a field, who will get introduced to an idea by a particular author or group. Lots of chaotic citation patterns and feedback loops.
Add in shifts in what's motivating grant and paper topics and it really damages authentic scientific discourse. So much is driven by looking like a brilliant scientist and not by scientific progress. People aren't dumb either and they are very very good at looking like brilliant scientists.
Higgs, who hypothesized the Higgs boson back in the 70s, has only published 2-3 papers in the decades since. Before it was announced that he won the Nobel prize, his University was deciding between forcing him out or gambling on him potentially winning the Nobel prize which would bring more reknown to the department. He has commented recently saying if he were a young, freshly minted PhD today he would not have gotten tenure with his publication record. That seems insane to me and a bitter critique of today's incentive structures created by the funding system we have that rewards incremental work over real breakthroughs.
This reminds me of a comment from an academic nearing retirement that I heard second-hand. It was something to the effect that applicants for Assistant Professor (or Lecturer) positions have CVs that are as good as or better than what applications for tenure looked like 20-30 years ago.
Alan Kay wrote an essay on how large scientific breakthroughs where done [1], further discussed in [2]. The funders and the systems of funding of science are to blame for the deciline, for example they won't fund problem finding anymore.
[1] http://www.vpri.org/pdf/Kay_How.pdf
[2] https://m.youtube.com/watch?v=j9ZGFaIHegE
Alan Kay's advocacy for funding science is a truly eminent life's work.
I get that in this context "patents" should be seen as the announcement of a new innovation, a surrogate of the innovation itself if you will, but setting that aside for a moment I'm struggling to imagine how patents could ever do anything but slow down disruption.
Take any revolutionary discovery. Compare making it freely available to anyone, or limiting the availability to those holding the patents. EDIT: And perhaps more importantly give people the freedom to innovate on top versus stopping anyone who does. Shouldn't the former always be more disruptive of the status quo? I can imagine this is an oversimplification though so I'm open to hearing counterintuitive examples where the opposite is true.
And to be clear, either way I'm not saying this is good or bad in and of itself - in some cases it might slow down essential improvements, in others it might be good to have speed limits for change so that things don't break.
You could argue that there is more economic incentive to invent new things as discoveries are worth more if you alone have the right to use them. This in turn could lead to more investments in research that lead to more inventions that increase disruption.
I know that that has been one argument in favor of patents since forever. But that ignores the part where competing parties are not freely allowed to build on another person's patent, which is essential for making good improvements.
And you can see this play out in the history of invention. For example, Edison stopped other people from doing research on the light bulb. Meanwhile the Netherlands was a country without IP laws at the time so Phillips quickly ended up producing the superior and cheaper product. No patent laws, bigger disruption. Similarly the Wright brothers were notorious for holding back aviation with their patents.
James Watt and his patents on steam engine are also an interesting case. Efficiency gains basically stopped for duration of the main patent patent and exploded later.
On the other hand it is also case for more innovation due to necessity, e.g. his patent on beam connection to a shaft (linear-to-rotary movement) required others to come up with a different solutions (e.g. sun and planet gear).
You make two assumptions:
- There are as many revolutionary discoveries with and without patents
- Without patents, discoveries would be freely avalaible
As far as I know, two (related) arguments are generally made for patents:
- Patents create an indirect (by preventing the competition from using your invention) or direct (by licensing it) monetary return to innovation, potentially leading to more innovation
- If a company wants competitors not to copy their innovation, they can 1. keep it secret or 2. disclose it and patent it; without patent the choice is between 1. keep it secret or 2. disclose it and have everybody copy them. In this case, patents lead to more innovation being made freely avalaible (with a delay!).
Whether patents lead to more or less innovation is, as far as I know, contentious.
> There are as many revolutionary discoveries with and without patents
I explicitly don't: I mentioned the patent stops others from innovating on top of the patent. The necessity of patents to be public is one I had not considered though, being a force against trade secrets is a good counterpoint.
> Whether patents lead to more or less innovation is, as far as I know, contentious.
Honestly, I really have trouble accepting this. Not that I have an answer, but with the idea that researchers haven't been able to find some sensible way of measuring this by now. I already mentioned the example of light bulbs. Surely there are enough similar historical scenarios available to analyze where one can make use of global differences in IP laws and other variables to simulate control groups?
You do here:
"Take any revolutionary discovery. Compare making it freely available to anyone, or limiting the availability to those holding the patents. [...]"
You take the situation where there _is_ a revolutionary discovery, with or without patents, and then wonder about the effect of patents on the next innovations. In doing that you do not consider that may be a revolutionary discovery with patents, and none without.
3D printing seems a good example:
Extruded plastic took off before resin printers because the patents expired a few years earlier.
As far as I can tell, we could have had the 3D printing revolution in the 90s or early 2000s, but instead we locked the technology away to only be used in a few esoteric commercial applications until the patents expired.
Similar story with e-ink; the license costs for the technology are the main blocker to wider adoption.
> Similar story with e-ink; the license costs for the technology are the main blocker to wider adoption.
How much is the license cost?
Given the abundance of cheap eink book readers, price tags etc on the market, cost doesn't seem to be a problem. OTOH the limitations of the technology - in particular, the contrast and the refresh rate - and how expensive large screens get, prevents it from expanding into more niches.
Perhaps if there were no patent, there'd be more third-party research to improve all these things faster. But as far as adoption of the existing stuff goes, I don't see how license costs are a direct blocker.
> Compare making it freely available to anyone
The original idea of a patent is that it must reveal the "secrets" behind an invention, in return for a "temporary monopoly". It's supposed to help diffusion of technology by publishing trade secrets, not to "incentivise" research.
I think what is actually happening is that patents are made obtuse/opaque containing little meaningful information. They are then used as playing cards in patent licensing deals between big corporations.
There is no longer any hope for patent clerks to do meaningful inspection of patents to determine if they are legitimate. Instead, it's left to lawyers/attorneys to play something out in court.
Patents have always been written in precise but vague terms. If you take that as the language and understand the structure of them, they can be quite readable, although you will often need to use outside knowledge and critical thinking to determine how to actually replicate the invention.
I lost a lot of faith in scientific research when I had to spend time reviewing patents in an early job.
The fact that I was looking into battery materials (a notoriously over-hyped sector) certainly didn't help, but some of the approved parents were laughable. They included core materials which had been historically known to be useful in batteries (and hence shouldn't qualify for protection), but then also every possible element-wise substitution of the same structure - many of the resulting materials would not even have been stable, never mind viable as a battery component. There were also instances of multiple patents covering the same material, suggesting that any prior-art searches were not being done rigorously.
> EDIT: And perhaps more importantly give people the freedom to innovate on top versus stopping anyone who does.
Everyone is free to innovate on top of any pre-existing, current patent. They simply can't profit on their new innovation without licensing the patent. However what they can do with their new innovation is patent the innovation itself. This newly patented innovation prevents the original patent holder from profiting off of the new innovation, and thus incentivizes the original patent holder to license their patent to the new patent holder (or vice versa).
Ok, so the average has declined - has the raw # of disruptive studies increased? The definition of 'disruptive' here is based on the ratio of citations of [study] vs citations of [study's references]. If the number of papers increases, do we want a proportional increase in studies that "send a field in a new direction?" Or do we want incremental studies to be improving rigor, comprehensiveness, and clarity?
IMO, it'd be much better to look at the Avg. CD of the top N% of CD scores over time - that'd give me a much better idea of if disruptiveness is changing, as disruptiveness is a "best paper" phenomenon anyway.
I'm left a bit conflicted by what conclusions to draw from this effort.
How much of this is due to how much easier it is to publish something these days? Every statistics package spits out publication-ready charts and tables, Latex/Word has automated much of the typesetter's art, drafts can be communicated instantly via email, there are instantly searchable and linked databases of papers, etc, etc.
Think about what it was like in the 1950's, when many scientists were hiring typists to convert their longhand manuscripts into something an editor could read, attaching hand drawn figures, then heading down to the post office to mail the draft in. That draft then had to be copied and mailed to reviewers, who then mailed their opinions back, every step taking at least a few days of communication overhead. It was a different world and it gradually changed with widespread adoption of different technologies.
Also how does this interact with another pair of common suggestions for improving science: publishing negative results and doing more replications? Is this a sign that those two good practices are already occurring? Is it possible that the higher rate of disruption was due to shoddier work in the past?
> Think about what it was like in the 1950's, when many scientists were hiring typists to convert their longhand manuscripts into something an editor could read
This was done well into the 1980s, if not longer.
From my training as a computer scientist, I am used to the idea that some problems are easy and some are plainly intractable. Perhaps that's also true in science and technology, and we are starting to exhaust all the easy directions.
Still, many less techy problems remain, at least climate change and wealth inequality come to mind. Perhaps we'll see the biggest innovations in solving those in the future.
As a software developer, I’d want to see a rewrite to clean up the legacy bodges before we conclude we can’t add more.
Also, wealth inequality isn’t a problem: how wealthy the people at the bottom are is. Wealth inequality is associated with fewer poor people and a higher standard of living because wealth inequality requires a wealthy society.
What exactly are you “fixing”?
I'm not suggesting we can't add more, rather, that I find it conceivable that there is a barrier of technological progress that we can approach, but never reach. So the pace of progress will slow down.
Concerning wealth inequality, I do think it is a fundamental problem of society if a fraction of a percentage controls majority of the available resources even if the lowest half has their immediate needs met, but we can disagree on that all day and arrive nowhere.
The Pareto principle is incredibly harsh, but such hierarchies seem to form naturally.
My concern with your view is that in trying to “squash the gap”, you end up doing so by making everyone poorer — and society worse overall.
Happy to agree to disagree; hopefully we can consider each other’s perspectives in crafting policies.
It's indeed not an easy issue, which is why I think there could still be lots of innovations to made in that space in terms of economics, politics, social sciences etc.
The GINI index in absolute monarchies is close to 1.
Wealth inequality implies that some people disproportionately control the life production of other people. If you're in favor of a "great man" idea of progress this is a benefit. If you're in favor of individual power to direct production and innovation (along the lines of de Tocqueville or it's left-wing equivalent) this is a detraction.
Wealth inequality is not a problem, it's a symptom. Economic rent is the problem.
Climate change seems like one of the techiest of problems.
I think wealth inequality is too ill defined to even be a problem. If anything i would argue it is only a problem if class mobility is low and that is where the focus should be.
- Grant funding
- Tenure requirements
- Job duties (both requirements and limitations)
- Elimination of corporate skunk works like PARC
- Credentialism
- A "Pain point" focus would tend to be more incremental than truly disruptive
On the good side there may be some decrease of disruptive inventions and discoveries because we are also focusing on eliminating the externalities of current and previous inventions and discoveries. It's a lot easier to make wildly new things if you aren't worried about their side effects.
>Elimination of corporate skunk works like PARC
This is a big problem. PARC, Bell Labs, IBM Research are the first that come to mind. Same is true for mostly government-funded initiatives like the one where Marc Andreessen invented Mosaic browser, that eventually became Netscape Navigator. I don't recall the project name. Do we even do that anymore?
We still have DARPA. We also have ARPA-E, ARPA-H, and HSARPA.
A variety of DOE national labs and core facilities (including NCSA where Mosaic was developed) still exist and are funded.
We may need more to take up the slack of the corporate research facilities.
Remind me, in the history of the physical sciences, how many "groundbreaking breakthroughs," came from academics teaching undergraduates?
My mind is genuinely struggling...
Certainly not Curie, Einstein, Fleming worked in a hospital.
Maybe the problem is...
Undergraduates!
You can't get funding for studies that disagree with existing research.
So instead of being branded a heretic and chased out of town you just starve. The more things change the more they stay the same.
> So instead of being branded a heretic and chased out of town you just starve.
Your income level is probably higher after leaving academia and going to work in industry. So you probably starve more by staying in academia.
Disruptive work doesn't necessarily need to disagree with previous results.
Well you get the funding before you do the experiment and thus before you know whether it's going to disagree with existing research. If you have the results before you do the experiment, you're not doing science.
No, grant proposals normally expect you to provide preliminary results. The more controversial your proposal, the better your preliminaries have to be. Especially you have to convince the funding deciders that you did the appropriate positive and negative controls.
Preliminary results are results from a previous experiment that already had to be funded. And yeah, you don't want to throw funding at researchers who are only getting weird results because they are bad at conducting research.
Literally every preliminary result I got was from an experiment that was explicitly not funded for the purpose it was requested for. That is, by year 2 of my grant, I was spending my monies on stuff I hadn't proposed, to prepare for the next grant proposal.
I guess part of the reason I exited academia was because I was bad at conducting research :)
The chicken-and-egg problem illustrated in this thread seemed like the most appropriate place to highlight: So (see disclaimer at the end) there is a publication I appreciate called Seeds of Science. My understanding is that it is (in part) intended to offer a pathway to work around structural failure in comventional peer review; As its web page describes:
" What is a 'Seed of Science'? It is a speculation, an idea for an experiment, a novel observation, a thought-provoking question and discussion, the highlighting of an under-appreciated problem, or an unorthodox research study"
They got an ACX Grant last year from AstralCodexTen to pursue this.
---
CoI Disclaimer: My work is published in SoS.
And you can't get through peer review if you did. That process maintains consensus.
Albert Einstein failed peer review the one time he was subjected to it. Newton was not subject to it at all.
He was asked to address some questions and make revisions. He was so aggravated that his paper was subject to peer review at all that he withdrew it. But his subsequent submission to another journal did incorporate some of the referees concerns. Einstein was a great scientist but he too was wrong sometimes. [1]
His 1905 seminal papers were not peer reviewed per-se but the journal's editor who accepted the paper was Max Planck! So it was reviewed by another legendary physicist.
These days science has branched off into such specialized areas that another specialist in the field is needed for review. Whether that should be an anonymous referee, or someone else it is still valuable to catch errors and determine the soundness of the techniques. Also the selections of those peers is something for debate too.
[1] https://physicstoday.scitation.org/doi/full/10.1063/1.211782...
If you're really smart, enough to make disruptive scientific discoveries, why bother doing that when you could learn programming and work at a FAANG for $400k, or work in the finance sector and make possibly more? If you stay in science, you'll earn peanuts and not even have any career stability since the academic track is so bad these days.
That's arguably an even greater point than anyone realises.
All these geniuses... people are giving them jobs!
"why bother doing that when you could learn programming and work at a FAANG for $400k, or work in the finance sector and make possibly more?"
Because it's boring and depressing. And one thing money can't buy, unless you're solely interested in accumulating money, is satisfaction with your life.
Obviously not true for everyone, but we've got 8.8 million "scientists" (I'm not sure what they mean by the word). https://sciencebusiness.net/news/number-scientists-worldwide...
Most scientists don't work in academia. Some of those who do have stable jobs as staff instead of faculty positions.
And this guy did it in his spare time while working at FAANG: https://news.ycombinator.com/item?id=34236889
When I worked at IBM, we were given generous cash incentives for filing patents. I am named as an inventor on about 40 patents. To be honest, only one of them was a genuine invention in my opinion.
Doesn't it just stand to reason that there's less to discover, that what there is mostly isn't the low-hanging fruit (and unlike actual fruit, it's not regrowing), over time?
That's what I was thinking, too. Perhaps we already know majority of what there is to know, when it comes to fundamental concepts? Probably plenty of work left in improving our tools and engineering solutions, like machine learning-based software, as well as in understanding the intricates of biology. Maybe mathematics, too. Maybe improvements in these will cause another golden age of discovery. Like, understanding biology enough to gain significantly extended lifespan, which means much more expertise can be built by one individual.
Currently running a research lab and this fits with my anecdotal experience of what ideas and approaches are considered fundable by the NIH and NSF.
I think the same processes that turn nimble and innovative startups into large and heavy enterprises apply to countries and institutions as well. As academia and research institutions develop, they become more complex, valuing process over result and consensus over conflict. Increasingly, disruption of consensus is perceived as disruption of the institution itself and risk-aversion and incrementalism are rewarded.
I hate to be a doomsayer, but this seems to be a general trajectory that most of the western world is on.
There’s an intuition that recent products and inventions are more incremental and that historical innovations are more revolutionary- in the sense of lacking precedence.
But this is an effect of the simplification of history, and when you read the detailed technical histories you can see how remarkably incremental the old innovations were.
We can easily see the precedents of recent innovations but not for older ones.
This is pointed out in the book I am reading on the development of the turbojet, which only looks more unprecedented than it really was.
> The intuition is that if a paper or patent is disruptive, the subsequent work that cites it is less likely to also cite its predecessors
This intuition seems to overlook the culture of citations in the patent space. There, the incentive is for applicants to be over inclusive with citations. This is combined with the increasing ease with which references can be found.
So, as fastaguy88 already described in another comment, this paper is a fairly arbitrary quantification of “disruptive”. (Also, can we just stop using that word? Such a though-terminating cliche at this point.)
But let’s accept that we are seeing less fundamental breakthroughs in science. Is that a failure of the system…or, depressingly, what if we really are hitting the limits of what we can model and describe?
In a roughly 200 year timespan, we figured out the at least the rough outlines of…wow, where even to start…electromagnetism, general and special relativity, quantum mechanics as it relates to field theories and chemistry, genetics, and computability. We really picked a lot of the low-hanging fruit, and what if there just isn’t any fruit left to easily get at? There’s certainly things like dark energy that we don’t have particularly satisfying theories for at all, but, the basics? We seem to have good models.
The physical realm seems largely picked clean, but there is bountiful fruits available in the metaphysical realm - shame that ~no one can see it, despite complaining about it constantly.
> The physical realm seems largely picked clean,
Famous Last Words
It's a great meme, but memes are only good for estimating the future, and while the future can indeed be strongly influenced by the fact that much of human cognition is meme-based, there are also many other influences on it.
This and many other pieces of very useful knowledge are there for the taking in the metaphysical realm, just as was the case hundreds of years ago in the physical realm, prior to the rise of science (a metaphysical phenomenon itself).
Its just the evolution vs revolution periods... In any field 99% of the time advancements are slow and incremental, than a new idea sparks a revolution and you get a lot of advancements in a short period of time. This is just the nature of scientific progress.
In the early 20 century new ideas in physics and mathematics sparked a revolution and we are slowly unraveling the benefits of it ever since. For 40 years we had evolution in the AI research field, now we have a revolution, I do not know when it will die down but after a while we will just start to build over the ideas we discovered and a long period will pass before we do another jump. Same with psychology, economics, biology...
And yet I've been trying to find $$ for one of the most disruptive agriculture technologies that could propel to sustainable ag to acreage unheard of. Nobody wants to take a chance.
shrug
The science knowledge base grows monotonically but not necessarily at a steady rate. Actually far from it.
There was a time in the 19th century when people thought physics discoveries were complete. A few decades later all hell broke lose.
The problem we have right now is that we are stymied on the complexity front (materials, chemistry, biology, medicine etc)
Our mental / mathematical models that we use to understand the world and even enable us to ask original scientific questions are losing their edge when things get complex. These tools are 19th century stuff (including all the AI junk) and we have milked them for all they are worth
I would hypothesise that a new period of discovery will correlate with some breakthrough in the mathematics of complexity. We had a few false dawns but not the real thing
> These tools are 19th century stuff (including all the AI junk)
I'm not really sure what you're looking for here. I mean, you could claim that modern cryptography is based on ideas from the ancient Greeks (factoring numbers), and is therefore old and busted. Is every idea just derivative unless it uses a completely new kind of math?
Software developers (maybe not you, but a large percentage of the users of this site) tend to automatically think older ideas are worse. But that's not true in math. Mathematical ideas don't expire. The idea of the number 7 is aeons old; the number 0 over a thousand years; the complex number i 450 years. But the numbers 7 and 0 and i are still extremely useful, in infinite combinations. If you think we have milked those numbers for all they're worth, you have an extremely limited imagination.
Sure, most of the "AI junk" is a combination of old ideas like linear algebra and gradient descent, but used at a scale orders of magnitude larger than their creators ever imagined. (And quantity has a quality all its own.) It's that combination of ideas that is so powerful. Besides, backpropagation is a pretty damned novel idea. If you're looking for the one truly new idea from AI, it's backpropagation.
You are vastly underestimating the volume of the idea space available to explore simply by combining the ideas and materials we already have. Crossover is a better source of variety than mutation is. You may be right that truly new concepts are the "raw materials" of invention, but we never run out of the raw ideas we have, and they prove to be useful in their combination up to the limits of our imaginations. It's actually surprising how likely you are to come up with a reasonably good idea by simply considering a few randomly-selected ideas together.
> we are stymied on the complexity front (materials, chemistry, biology, medicine etc)
Again you seem like your standard is: if it's not an entirely new thing where nothing like it has ever existed before, then it's boring!. If you think the volume of the idea-space from combining the materials, chemistry, biology, and medicine we already have/understand is X, it's almost certainly at least 2^X, or even X! (factorial).
Besides, we are coming up with brand new materials all the time! Materials science is one of the most exciting frontiers of human technology.
Dont get me wrong, there is plenty of scope for interesting / useful applications of software and pattern matching at scale but those are more a form of "tech assisted social evolution" rather than the steady march of fundamental inventions and discoveries across the entire spectrum of knowledge that characterised the last few centuries.
Old math can still have an impact at the margin (eg speeding up protein folding calculations etc) but its not the cognitive leap that would allow us to steer discoveries in more effective manner. Materials, medicine etc all very exciting and very active but also hampered by their intrinsic complexity.
All apparent exponential growth is just an S-curve we haven't found the inflection point of yet.
I know the danger of thinking this, but maybe we're just less wrong than we used to be?
‘Disruptive’ science has declined — and no one knows why
https://www.nature.com/articles/d41586-022-04577-5
Good article. Interesting finding at the very end:
>Although the proportion of disruptive research dropped significantly between 1945 and 2010, the number of highly disruptive studies has remained about the same.
So... maybe this is because having more people doing scientific research doesn't yield more breakthroughs. More people doing scientific research DOES yield more incremental achievements, as the study found, i.e. word frequency analysis showing more occurrences of "improve", "expand", "extend".
The number of highly disruptive studies remaining the same seems impressive enough to me. I expect diminishing returns to be the long-term norm in science. The low hanging fruit get picked first, and every breakthrough means one more concept that the next generation of researchers have to learn and incorporate into their work.
May be relevant: https://www.city-journal.org/nih-blocks-access-to-genetics-d...
Eugenics research is disruptive?
Why not phrenology?
Studying the genetic basis for various characteristics is not eugenics, anymore than studying the atom is building a nuclear weapon. Abuse of knowledge should be controlled, but banning the pursuit of knowledge itself?
That's just intentionally crippling our own understanding of the world, for fear that we might not like what we discover. That's hardly an unprecedented behavior, to say the least, but the cultures that have gone that direction are probably not the sort we might want to emulate.
I too am a socialist. But I believe people are different: hereditarily and culturally.
I assume only people like me and maybe a few like you will know what this means, but doesn't it upset you that I can freely say it?
The first half of the 20th century had a lot of "low hanging fruit" in physics and chemistry that basically came out of the enlightenment. Things like electromagnetism, quantum mechanics. I wouldn't guess that's always the steady state for research.
The war may have had something to do with it too, I know a lot of advances (like radio / microwave engineering) have roots in the war efforts. But I suspect it's more of just a dry spell because we pushed forward in the 18th-20th century and ran out of ideas
Wouldn't we be presented with another bout of "low hanging fruits" in the second half of 21st century when tectonic shifts happen in quantum computing, superconductivity, ai, ar/vr/mr etc.?
Nice to see numbers put to this. I remember thinking through a bunch of tech/science that I was familiar with, and realizing that most (not all) of the fundamentals in every case were known by the early 60's, and we've been working out the (very important) details ever since. Nuclear, electronics, chemistry, biology, etc.
plenty of counter-examples, I'm sure, but like once you know the structure of DNA and it's fundamental operations, the remaining discoveries will have a smaller impact.
Maybe it's wrong. At least in the chart, they are missing the Bitcoin invention in 2009 (technology). There was the last "disruptive" science in the area.
While there is no doubt that 'CD Index' has been declining over time, it is debatable whether it truly indicates a drop in 'disruption'.
Let's take AI for example. Research papers that were published in 2019-20 have become obsolete within a year or less, and consequently get cited less and less, as more recent and derivative papers introduced better models. Does that make the older papers any less 'disruptive'? No!
On the paper front, I have a number of published scientific papers with citations. My 3 most important papers—that will go down as turning points in the history of computer science—have a combined 0 citations, and 2 of them not only were not in a journal, but they were even flagged and removed from arxiv.
On the patent front, well patents are for parasites.
> Wang and his colleagues have found that big teams are more likely to produce incremental than disruptive science
Interesting observation.
Patents went from real innovation to Legalese BS and market protection in the past decades.
Also I suspect most innovation are being done outside of papers, especially when you have "reviewer 2" ready to be nitpicky at every minor detail, so you make it more bland and maybe skip one juicy part or another
I've heard another interesting argument, at least for disruptive science/research from industry.
Lower corporate taxation incentivizes the hoarding of cash. Prior, corporations were incentivized to allocate more funding towards their industry research (IBM, Bell Labs, etc).
Would love some counterarguments, here!
Related ongoing thread:
How to slow down scientific progress, according to Leo Szilard - https://news.ycombinator.com/item?id=34264436 - Jan 2023 (20 comments)
"People talk glibly about science. What is science? People coming out of a university with a master's degree or a PhD, you take them into the field and they literally don't believe anything unless it's a peer-reviewed paper. It's the only thing they accept and you say to them 'But let's observe. Let's think. Let's discuss.' They don't do it. It's just, 'Is it in a peer-reviewed paper or not?' That's their view of science. I think it's pathetic.
Gone into universities as bright young people. They come out of them brain dead, not even knowing what science means. They think it means peer-reviewed papers etc. No, that's academia. And if a paper is peer-reviewed it means everybody thought the same therefore they approved it.
An unintended consequence is that when new knowledge emerges, new scientific insights, they can never ever be peer-reviewed. So we're blocking all new advances in science that are big advances. If you look at the breakthroughs in science almost always they don't come from the center of that profession, they come from the fringe. The finest candle makers in the world couldn't even think of electric lights. They don't come from within they often come from outside the bricks. We're going to kill ourselves because of stupidity."
— Allan Savory, Zimbabwean ecologist, research biologist, livestock farmer, and president and co-founder of the Savory Institute. He originated the Holistic management of agriculture to fight desertification.
https://www.youtube.com/watch?v=qkXZ3_ZmKzw
Yea, sadly I can relate to this quote. Modern science has lost it voice and purpose it seems and it nothing more than a glorified printing press of poor research and glad-handing consortiums.
While I appreciate the sentiment, nearly all new knowledge goes through peer review because for every important problem in academia, there's a competitor who is the best person to point out problems with the theory.
"Holistic management of agriculture to fight desertification" is a non-quantitative science and falls under a very different regime from, say, people working at the state of the art in particle physics or biology.
Most scientific breakthroughs in history did not go through peer review. That's a new consensus mechanism that was introduced not that long ago.
Einstein was only subject to it once and didn't pass. Newton, Galileo and the rest were not subject to it at all.
I'm not defending peer review; I never enjoyed it when I published. It certainly never made my papers any better.
Newton and others all published and then discussed the details in person, a lot of the peer review was "off-paper" and "post-publication".
Also, the vast majority of scientific breakthroughs, by count, occurred after peer review- mainly due to the unbelievable pace of breakthroughs due to technological advancement.
That sounded interesting so I looked into it. It seems to instead support peer review.
According to this [1], Einstein's paper that didn't pass peer review turned out to be wrong (it said gravitational waves didn't exist), and was later published elsewhere with different conclusions and fixing problems that the referee's report had already pointed out.
> The irony, of course, is that Einstein could have found that escape route months earlier, simply by reading the referee’s report that he had dismissed so hastily. The referee had also observed that casting the Einstein–Rosen metric (as we now call this solution of the Einstein equations) in cylindrical coordinates removes the apparent difficulty.
[1] https://physicstoday.scitation.org/doi/10.1063/1.2117822
> While I appreciate the sentiment, nearly all new knowledge goes through peer review...
Only knowledge that originates in academia, and even then only a portion of it.
> because for every important problem in academia, there's a competitor who is the best person to point out problems with the theory.
While this may be generally true, it is not absolutely true.
> "Holistic management of agriculture to fight desertification" is a non-quantitative science...
Partially non-quantitative.
Also: it'a (partially) true, but so what?
> ...and falls under a very different regime from, say, people working at the state of the art in particle physics or biology.
Right: one is known to be important, and the others are (at this point in time) hoped to be important (as they have been in the past).
"all metrics seem to decline as we keep adding more metrics"
it's called bureaucracy
Every time I read one of these editorials on the topic of lack of available research, I just look at the huge list of significant problems that still exist for life on earth and wonder: “what are they thinking?”
Another pertinent paper from 2021: https://arxiv.org/abs/2106.11184
This is an earlier publication of the same paper before it was published in Nature.
Theory of Scientific Revolution by Thomas Kuhn.
Major breakthrough that disrupts the paradime. Followed by normal science with diminishing returns until the next paradime.
Hardly surprising. Besides fraud, in science you're not overturning past experimental data, you are adding to it, forming an ever more complete and accurate view of reality. Disruption really just means stumbling upon something unexpected. Once you know there is something there, it opens up a whole world of new avenues for studying in that space, but it doesn't really make it any easier to find something unexpected again.
Consider for example deep sea archaeology. When you find a shipwreck, there's going to be a flurry of activity as people study the shipwreck, and the more they answer the questions they started with, the more new questions they are going to want to ask, so even long after the initial discovery there's going to be a continuous string of studies refining knowledge about that wreck. And this is true of every wreck. But finding one shipwreck doesn't generally make it easier to find the next, assuming the same resources, new wrecks are going to be discovered at a pretty steady rate. So the number of discovered wrecks grows linearly, but with each being revisited regularly, the cumulative amount of research done on all wrecks grows geometrically.
Similarly discovering a new chemical opens up many possibilities for studying that chemical, likewise for a new phylum of bacteria, a new type of star, you can only see something for the first time once, but you can take closer looks forever. We should expect the ratio of new things to look like a graph of 1/x.
Perhaps it's weird that the absolute number of disruptive papers is holding steady. While there are competing forces at work - on the one hand we have more scientists with better instruments, but on the other hand the low hanging fruit is gone and they must go ever further to find something new - there's no fundamental reason these two forces should perfectly counterbalance. I might posit our society has a certain appetite for discovery and we adjust our allocation of resources to science and innovation to maintain this continuous stream of disruptions, but I at least feel that scientific breakthroughs tend to excite people about science and desire to double down on science, not pull back. Perhaps more realistically we just move the goal posts as to what we consider disruptive such that the top X discoveries of the year are always "groundbreaking." Regardless of the why, this definitely the more interesting part of the story.
I had worked in academe long enough to see many papers submitted simply out of pressure to finish the degree.
I know this is OT, but I keep reading this as "paupers and parents"...
Low hanging fruit are picked.
There are just too many gatekeepers and too many people who are used to doing things a certain way. The world is just more populous now. We also need to make sure that people can speak their minds freely and write freely without worrying about getting cancelled.
>and no one knows why
Really?
Too much talent going into trivial tech ventures. The shift towards rentier capitalism in the US, especially in terms of cheap money/QE being used for share buybacks instead of R&D. The colossal cost of higher education in the US, with grad stipends barely budging until literally this month. Fewer people willing to tolerate the (mostly) toxic culture of academic advisors overseeing Phds and post-grads. Google Scholar skewing papers to exist more for popularity/citation prevalence. The Internet as the biggest distraction in the history of humanity. etc. etc.
I blame: the currency and inflationary environment