It is a niche that needs filling for sure, but I'm concerned that the code analysis and transformation talent is going to continue in the current path, namely, spread out thinly across many environments, languages, and toolkits. What we need is not only a wonderful tool for static code analysis, generation, and transformation, but also an energetic juggernaut of a foundation to evangelize and drive adoption of it. Otherwise, talent will end up corralled in to the Eclipse SDKs of the world.
Goals are often similar, but methods very different. Epsilon and Rascal share many design decisions. Rascal is based fully on immutable data, including relations, while the EMF is based on objects.
I have programmed in both Rascal and Racket (for university courses). I really prefer Racket.
Though most of that might be because Rascal isn't really polished for mainstream usage. Rascal doesn't have a lot of good error messages at the moment (the most common error message you might see, is CallFailed, with just the bare minimum of information where it occured).
Another issue is how to run Rascal. Currently, you can only install it as an Eclipse plugin, and that seems to be the only way to run it.
Edit: I see that last statement is false. There is a commandline REPL. But I do not know if it works well.
I don't really know Racket. But from what i've seen, it's mostly sold as a language to create new languages in. Rascal on the other was designed to analyse existing languages. They wanted to have a single framework to analyse both Java and C++ for example. The goal was to be able to easily add new syntax and reuse semantic analysis, I believe.
I’d like to ask the same. Most PL research is done with Racket; I imagine this might have a steep uphill battle if it wants to gain traction. But maybe it’s target application is sufficiently specialized… ¯\_(ツ)_/¯
Reminds me of Coccinelle - a language for writing semantic patches for C (like eg: add close_foo(fooid) in every function that has open_foo for each fooid etc...) , but I see Rascal can do some more.
Like many things from the Netherlands, it uses SGLR parsing (scannerless GLR, where GLR = generalized LR), which actually handles a class slightly more general than CFG s.
That was version 0.1 bootstrapped off SDF2. Since 2009 Rascal is based on a topdown general algorithm calked GLL. It's still Scannerless with declarative disambiguation, like SGLR. Currently it's evolving to data-dependent GLL to be able to model the offside rule, the C pre-processor and symbol table, and other "interesting" stuff we encounter in programming languages. See thesis by Afroozeh and Izmaylova.
GLR, GLL and Earley's parser (with the tree construction) are labeled "context-free general" parsing algorithms because they can handle _any_ context-free grammar while the other algorithms associated with sub-classes of CFGs, such as LR, LALR and LL do not implement all CFGs. In particular this comes in handy for languages which are unambiguous yet require a lot of lookahead or context to avoid parser action conflicts. The context-free general algorithms are popular among people who have to rapidly prototype languages or implement many different language processing tools without the opportunity to spend months on a parser. We pay in speed. GLR and GLL typically run nearly linearly on normal files, while Earley has a bad average case behavior which is often quadratic.
Another benefit of context-free general implementations is composability of grammars. Since CFGs are closed under composition you can implement modular grammar formalisms using a context-free general parser.
Another main drawback is that (un)ambiguity of context-free grammars is undecidable, and so prototypes generated using GLR, GLL and Earley may produce more than one tree, and in release mode you migh report an unexpected static error to your user in that case. We deal with this drawback using random testing/fuzzing and some static analysis.
I'm getting the sense that this project is building something that appears to be complete gibberish to me right now but that in 3 months time it will become absolutely important that I learn it quickly.
Bookmarked so that future me doesn't get angry at losing this.
I wish they made their licensing clearer. Their website has nothing to say about this. There is no "LICENSE" file at the top-level of their main source repo, nor any discussion of licensing in their README.
Many of their actual source files have a copyright notice at the top including an EPL (Eclipse Public License) grant, so it looks like this is licensed under EPL. But they could make that fact clearer. (Also, I haven't actually checked every single source file, so there is some risk some of them might be under different license terms.)
> In principle Rascal is a research laboratory and research vehicle that is not intended for production use (yet).
No offense and nothing personal, but the following remark struck me as rather entitled and self-serving to your commercial interests:
> But they could make that fact clearer. (Also, I haven't actually checked every single source file, so there is some risk some of them might be under different license terms.)
This perspective may be slightly biased with recent personal experiences in dealing with an aquisition team who habitually tries to weasel its way into having its cake and eating it too.
> No offense and nothing personal, but the following remark struck me as rather entitled and self-serving to your commercial interests:
I have no commercial interests here. I have no plans on using this software commercially (in fact, I have no plans on using this software at all.) I don't want to speak for my employer, but I'm not aware of any plans to use this software at my place of work, and I personally have no plans to propose that they do.
Why do I speak up about it then? Simply because I think clarity in open-source licensing is important.
> Why do I speak up about it then? Simply because I think clarity in open-source licensing is important.
Fair enough and thanks for clarifying.
To be sure, using the term "risk" in the context of licensing speaks quite clearly in my mind of interests aligned with commercialization (or the prohibition thereof).
> To be sure, using the term "risk" in the context of licensing speaks quite clearly in my mind of interests aligned with commercialization (or the prohibition thereof).
In principle at least, unclear licensing is a legal risk even to non-commercial users. If someone posts stuff on GitHub without any clear license, and I then fork it and makes some modifications as part of some non-commercial hobby project, and then the author of the original project sends a DMCA takedown notice against my non-commercial hobby fork, then legally I would not be in a very strong position. (Of course, in practice, this is unlikely to happen very often, but I suspect something like this probably does on rare occasion.)
> ...and I then fork it and makes some modifications as part of some non-commercial hobby project, and then the author of the original project sends a DMCA takedown notice against my non-commercial hobby fork, then legally I would not be in a very strong position.
In agreement with the pragmatic implications, but I still genuinely struggle to see the point. The financial constraints of legal defense is what strikes me as the killer of non-commercial hobby projects if the hypothetical scenario would arise; with or without appropriate licensing terms, there's really nothing to stop an original project with the financial means and intent from issuing a DMCA takedown and outspending an unfunded project in legal maneuvering. IANAL but at least it sure does feel that way in my limited experience.
IANAL either, but launching totally bogus lawsuits, without a shred of justification, in the hope of overwhelming defendants without the financial resources to defend them, is a legally risky strategy – there is a risk of disbarment, or even criminal prosecution, for lawyers who engage in such behaviour (see the Prenda Law case). A lawyer can get away with making flimsy arguments, but a lawyer who knowingly makes arguments totally ungrounded in law or fact is risking their own career.
I'm with skissane, I'd be leery of any project without clear licensing as well... regardless of whether or not I had any kind of commercial interest in the project. To me, it's just one of Those Things That Should Be There, ya know? If the license terms aren't clear, I don't really know what I'm allowed to do, or not, with the project, and that just makes me uncomfortable.
But, I'm a bit of a radical ideologue when it comes to F/OSS, so maybe I'm a bit outside the mainstream on this one.
> ...I don't really know what I'm allowed to do, or not, with the project, and that just makes me uncomfortable.
If your interests weren't somehow commercially aligned--be it directly or indirectly--why would it really matter? Perhaps I'm the real radical here who still parades around with a false sense of hacker ethos.
I don't see how commercial interests even factor into this at all. If somebody puts a project out there, and I'm tempted to invest time/energy/attention into studying it, learning it, maybe building stuff with or, or possibly even contributing to it, then I want to know that it's really open-source. If it's not, then it's probably not worth my time/energy/attention because I have an ideological aversion to proprietary software.
As a counterexample to your point, consider RMS: he cares a lot about the details of software licensing, but most people would consider him a "real radical" with lots of old school "hacker ethos" and few would call him "commercially aligned"
> To be sure, using the term "risk" in the context of licensing speaks quite clearly in my mind of interests aligned with commercialization (or the prohibition thereof).
There are other risks, like the risk of wasting time learning something that will one day be very expensive and proprietary, will disappear completely, etc.
Situation: "many different tools, possibly even written in different languages."
Many different languages?! Ridiculous! We need to develop one universal system "for any kind of metaprogramming task: to construct parsers for programming languages, to analyze and transform source code, or to define new DSLs with full IDE support. "
Soon:
Situation: many+1 different tools, definitely written in different languages.
Working with code Analysis and transformation techniques at the same time was always about glueing database and logic progressing tech to functional programming and algebraic specification tech for us. Just having those linguistically integrated before you start thinking about a language progressing tool makes working with Rascal a lot of fun.
It is a niche that needs filling for sure, but I'm concerned that the code analysis and transformation talent is going to continue in the current path, namely, spread out thinly across many environments, languages, and toolkits. What we need is not only a wonderful tool for static code analysis, generation, and transformation, but also an energetic juggernaut of a foundation to evangelize and drive adoption of it. Otherwise, talent will end up corralled in to the Eclipse SDKs of the world.
Interesting! Can anyone comment on how it fits in compared to XTRAN (http://www.xtran-llc.com/), Xtext (http://www.eclipse.org/Xtext/), or Epsilon (http://www.eclipse.org/epsilon/doc/eol/)?
Goals are often similar, but methods very different. Epsilon and Rascal share many design decisions. Rascal is based fully on immutable data, including relations, while the EMF is based on objects.
Thanks! That's exactly what I wanted to know.
Curious how people who use this find it compares to an environment like Racket?
I have programmed in both Rascal and Racket (for university courses). I really prefer Racket.
Though most of that might be because Rascal isn't really polished for mainstream usage. Rascal doesn't have a lot of good error messages at the moment (the most common error message you might see, is CallFailed, with just the bare minimum of information where it occured).
Another issue is how to run Rascal. Currently, you can only install it as an Eclipse plugin, and that seems to be the only way to run it.
Edit: I see that last statement is false. There is a commandline REPL. But I do not know if it works well.
I don't really know Racket. But from what i've seen, it's mostly sold as a language to create new languages in. Rascal on the other was designed to analyse existing languages. They wanted to have a single framework to analyse both Java and C++ for example. The goal was to be able to easily add new syntax and reuse semantic analysis, I believe.
I’d like to ask the same. Most PL research is done with Racket; I imagine this might have a steep uphill battle if it wants to gain traction. But maybe it’s target application is sufficiently specialized… ¯\_(ツ)_/¯
> Most PL research is done with Racket
A lot of PL research is done with Racket, but I doubt it is even close to being most, or even more than 1 or 2%.
As for Rascal vs Racket, they really are going after different language engineering niches.
How would you describe the distribution of languages used in PL research?
Very diverse.
Reminds me of Coccinelle - a language for writing semantic patches for C (like eg: add close_foo(fooid) in every function that has open_foo for each fooid etc...) , but I see Rascal can do some more.
Since Rascal has integrated Context Free Grammars, does this mean it incorporates the Earley Algorithm for parsing ?
Like many things from the Netherlands, it uses SGLR parsing (scannerless GLR, where GLR = generalized LR), which actually handles a class slightly more general than CFG s.
That was version 0.1 bootstrapped off SDF2. Since 2009 Rascal is based on a topdown general algorithm calked GLL. It's still Scannerless with declarative disambiguation, like SGLR. Currently it's evolving to data-dependent GLL to be able to model the offside rule, the C pre-processor and symbol table, and other "interesting" stuff we encounter in programming languages. See thesis by Afroozeh and Izmaylova.
How is GLR more general than CFGs?
GLR, GLL and Earley's parser (with the tree construction) are labeled "context-free general" parsing algorithms because they can handle _any_ context-free grammar while the other algorithms associated with sub-classes of CFGs, such as LR, LALR and LL do not implement all CFGs. In particular this comes in handy for languages which are unambiguous yet require a lot of lookahead or context to avoid parser action conflicts. The context-free general algorithms are popular among people who have to rapidly prototype languages or implement many different language processing tools without the opportunity to spend months on a parser. We pay in speed. GLR and GLL typically run nearly linearly on normal files, while Earley has a bad average case behavior which is often quadratic.
Another benefit of context-free general implementations is composability of grammars. Since CFGs are closed under composition you can implement modular grammar formalisms using a context-free general parser.
Another main drawback is that (un)ambiguity of context-free grammars is undecidable, and so prototypes generated using GLR, GLL and Earley may produce more than one tree, and in release mode you migh report an unexpected static error to your user in that case. We deal with this drawback using random testing/fuzzing and some static analysis.
I'm getting the sense that this project is building something that appears to be complete gibberish to me right now but that in 3 months time it will become absolutely important that I learn it quickly.
Bookmarked so that future me doesn't get angry at losing this.
OMeta was here
I wish they made their licensing clearer. Their website has nothing to say about this. There is no "LICENSE" file at the top-level of their main source repo, nor any discussion of licensing in their README.
Many of their actual source files have a copyright notice at the top including an EPL (Eclipse Public License) grant, so it looks like this is licensed under EPL. But they could make that fact clearer. (Also, I haven't actually checked every single source file, so there is some risk some of them might be under different license terms.)
What was developed in IBM time and what was written as Eclipse plugin is EPL. The rest is BSD2.
From their About page[1]:
> In principle Rascal is a research laboratory and research vehicle that is not intended for production use (yet).
No offense and nothing personal, but the following remark struck me as rather entitled and self-serving to your commercial interests:
> But they could make that fact clearer. (Also, I haven't actually checked every single source file, so there is some risk some of them might be under different license terms.)
This perspective may be slightly biased with recent personal experiences in dealing with an aquisition team who habitually tries to weasel its way into having its cake and eating it too.
[1] https://www.rascal-mpl.org/about/
> No offense and nothing personal, but the following remark struck me as rather entitled and self-serving to your commercial interests:
I have no commercial interests here. I have no plans on using this software commercially (in fact, I have no plans on using this software at all.) I don't want to speak for my employer, but I'm not aware of any plans to use this software at my place of work, and I personally have no plans to propose that they do.
Why do I speak up about it then? Simply because I think clarity in open-source licensing is important.
> Why do I speak up about it then? Simply because I think clarity in open-source licensing is important.
Fair enough and thanks for clarifying.
To be sure, using the term "risk" in the context of licensing speaks quite clearly in my mind of interests aligned with commercialization (or the prohibition thereof).
> To be sure, using the term "risk" in the context of licensing speaks quite clearly in my mind of interests aligned with commercialization (or the prohibition thereof).
In principle at least, unclear licensing is a legal risk even to non-commercial users. If someone posts stuff on GitHub without any clear license, and I then fork it and makes some modifications as part of some non-commercial hobby project, and then the author of the original project sends a DMCA takedown notice against my non-commercial hobby fork, then legally I would not be in a very strong position. (Of course, in practice, this is unlikely to happen very often, but I suspect something like this probably does on rare occasion.)
> ...and I then fork it and makes some modifications as part of some non-commercial hobby project, and then the author of the original project sends a DMCA takedown notice against my non-commercial hobby fork, then legally I would not be in a very strong position.
In agreement with the pragmatic implications, but I still genuinely struggle to see the point. The financial constraints of legal defense is what strikes me as the killer of non-commercial hobby projects if the hypothetical scenario would arise; with or without appropriate licensing terms, there's really nothing to stop an original project with the financial means and intent from issuing a DMCA takedown and outspending an unfunded project in legal maneuvering. IANAL but at least it sure does feel that way in my limited experience.
IANAL either, but launching totally bogus lawsuits, without a shred of justification, in the hope of overwhelming defendants without the financial resources to defend them, is a legally risky strategy – there is a risk of disbarment, or even criminal prosecution, for lawyers who engage in such behaviour (see the Prenda Law case). A lawyer can get away with making flimsy arguments, but a lawyer who knowingly makes arguments totally ungrounded in law or fact is risking their own career.
I'm with skissane, I'd be leery of any project without clear licensing as well... regardless of whether or not I had any kind of commercial interest in the project. To me, it's just one of Those Things That Should Be There, ya know? If the license terms aren't clear, I don't really know what I'm allowed to do, or not, with the project, and that just makes me uncomfortable.
But, I'm a bit of a radical ideologue when it comes to F/OSS, so maybe I'm a bit outside the mainstream on this one.
> ...I don't really know what I'm allowed to do, or not, with the project, and that just makes me uncomfortable.
If your interests weren't somehow commercially aligned--be it directly or indirectly--why would it really matter? Perhaps I'm the real radical here who still parades around with a false sense of hacker ethos.
I don't see how commercial interests even factor into this at all. If somebody puts a project out there, and I'm tempted to invest time/energy/attention into studying it, learning it, maybe building stuff with or, or possibly even contributing to it, then I want to know that it's really open-source. If it's not, then it's probably not worth my time/energy/attention because I have an ideological aversion to proprietary software.
As a counterexample to your point, consider RMS: he cares a lot about the details of software licensing, but most people would consider him a "real radical" with lots of old school "hacker ethos" and few would call him "commercially aligned"
> To be sure, using the term "risk" in the context of licensing speaks quite clearly in my mind of interests aligned with commercialization (or the prohibition thereof).
There are other risks, like the risk of wasting time learning something that will one day be very expensive and proprietary, will disappear completely, etc.
Situation: "many different tools, possibly even written in different languages."
Many different languages?! Ridiculous! We need to develop one universal system "for any kind of metaprogramming task: to construct parsers for programming languages, to analyze and transform source code, or to define new DSLs with full IDE support. "
Soon:
Situation: many+1 different tools, definitely written in different languages.
Working with code Analysis and transformation techniques at the same time was always about glueing database and logic progressing tech to functional programming and algebraic specification tech for us. Just having those linguistically integrated before you start thinking about a language progressing tool makes working with Rascal a lot of fun.