I like this. There is strength in having many different species of operating system. Monocultures are vulnerable to all kinds of diseases, one way to combat that is to have a (much) larger variety of operating systems so that infecting any of them has a very low return-on-investment.
So any new operating system that is far from the beaten path should be very much welcomed, this one is more interesting than most since it is written using a high level language.
Begs the question: is it possible to write a Lisp-based escape from the Lisp VM to the Virtualbox VM, then an escape from the Virtualbox VM to the host OS? Perhaps, but not something a casual attacker is likely to have resources for ;)
The idea of a VM is that it provides an iron box which nothing can break out of, because it's completely invisible. There's nothing a guest can do which could possibly distinguish a VM session from running on the bare hardware. Partly this is for compatibility, partly for ease of development (develop the next VM as a guest in the current VM!), and partly for simple security: You can't escape a prison if you are utterly convinced you're already out.
Maybe that part of the theory never made it into practice.
The idea of a VM is that it provides an iron box which nothing can break out of, because it's completely invisible. There's nothing a guest can do which could possibly distinguish a VM session from running on the bare hardware. Partly this is for compatibility, partly for ease of development (develop the next VM as a guest in the current VM!), and partly for simple security: You can't escape a prison if you are utterly convinced you're already out.
Maybe that part of the theory never made it into practice.
This is true of any dynamic memory management unless very careful constraints are put in place. I would assume anyone writing an OS kernel, regardless of language, would be careful to avoid unbounded consing. (Of course, memory allocation in kernels is tricky even in C: http://lwn.net/Articles/627419/)
The "garbage velocity" is not the most important parameter. Remember that garbage is all the space which remains after we identify what is reachable. The traversal of the graph of what is reachable is mainly where the performance pitfalls lie. When garbage is generated at a high rate, it just means that collection has to be more frequent. However, real-time techniques avoid scanning the entire graph of everything that is reachable.
Under ephemeral garbage collection, if the software generates a lot of garbage fast, it means that it's rapidly making large numbers of "baby" objects (objects in the "nursery" or "generation 0") and immediately losing them. Whenever a generation pass comes along, there are hardly any nursery objects to visit (they almost all been lost due to the "garbage velocity"), and the tenured objects aren't traversed either, so ... it's quick. Quick isn't "free", but it's not "beyond the ability to cope".
Have you ran the OS and seen this actual issue you're describing? If not, then what's your problem? If you're going to armchair quarterback, you should at least have thrown a football a few times first.
You may as well be complaining that the Windows 1.0 press release from 1985 has trouble with multitasking. Duh, this is pre-alpha software we're looking at.
Jon pls, do you even Lisp Machine? Seriously, you can't base your opinion on kernels/OS:s written in Lisp on this guy's hobby project. I know what you think of GC, but history shows what an industrial Lisp OS is like and it's awesome.
>So any new operating system that is far from the beaten path should be very much welcomed, this one is more interesting than most since it is written using a high level language.
I agree. But let's keep in mind that this isn't the first OS kernel written in Common Lisp.
This looks very cool. I am going to wait until the source is posted on github before trying it. I was lucky enough to have received a Xerox 1108 Lisp Machine in the early 1980s, so having an OS written in Lisp seems reasonable :-)
Also, I agree with jacquesm's comment on the advantages of many operating systems.
Are people up voting this because they like the title or they like the link? Because there's literally more information about what this is in the HN title than the link. Is an anonymous paste the closest this thing has to a homepage?
But, as with many Lisp OS images, there is an issue with explorability: how do I discover system commands in the REPL? I know a bit of Common Lisp, but it's not clear to me how to discover what system can and what not, how to explore the environment. Neither Filesystem Viewer provides any clues, nor does REPL.
1) Install VirtualBox from https://www.virtualbox.org/
2) Download and ungzip the disk image from https://dl.dropboxusercontent.com/u/46753018/Mezzanine%20Demo%201-disk1.vmdk.gz
3) Create a new VM in VirtualBox with the following settings:
Type: Other
Version: Other/Unknown 64-bit
Memory size: 512MB
Use an existing virtual hard drive file, the one you downloaded
4) Open the settings for the VM and switch to the Network tab.
5) Open the advanced settings and change the adapter type to virtio-net
6) Click OK. The VM is ready to start.
Interesting that they need the virtio-net adapter (host hands off raw network packets to the guest instead of simulating a network card). How many other low level drivers are missing?
Of course, for a toy OS it doesn't really matter, but it does unfortunately constrain this to the toy OS category.
Why do you think virtualized == toy? Is MirageOS a toy too then? They have no chance of ever getting enough device drivers to be able to support all the hardware out there. Running virtualized makes sense, and it's not a big performance hit.
A possible reason why "virtualized == toy" is that the virtualized system simulates certain specific hardware. The OS provides drivers for that hardware. But it hasn't been validated on the real hardware.
We don't know whether that is the case here. If the authors get a real box that has the same peripherals as the VirtualBox, and bring the OS up on that, then this "toy" argument can no longer be made, in any case.
Heh. If you think "only runs in a VM" and "toy" are synonyms, you need a history lesson.
Back in the 1960s, one of the hot new things from IBM (you know, the company that wins on Jeopardy) was VM/CMS. VM is a VM, hence the imaginative name, and CMS stood for Conversational Monitor System, originally Cambridge Monitor System, for reasons I'll let you figure out.
Conversational meant "has a command line", as opposed to batch, which you might have heard of. CMS was, all told, somewhere between pond scum and MS-DOS on the complexity scale: it provided some APIs for applications, but didn't handle memory protection, multitasking, multiple users, or security policy. A mainframe running CMS on the bare metal would have been a really expensive version of a PC from twenty or so years later; obviously, you can only justify doing that in a Serious University.
However, it was convenient and user-friendly back before the term was coined, when that meant "not seeing raw machine code unless you asked for it", so the gag was to run multiple instances of CMS as guests under VM, which provided everything CMS didn't. Every user got their own copy of the OS, and the single physical mainframe was turned into dozens or hundreds of virtual mainframes by VM.
My punchline: As the decades wore on, CMS was never modified to take newer hardware into account. Why would it be? It never ran on the bare metal. VM gained emulation functionality, CMS grew to depend on that, and now CMS is effectively an appendage of VM, a friendly face to the thing which does all the hard work. You know, like marketing, or upper management.
Out of curiosity, have you written a driver for a real network card, or are you guessing as to its difficulty? One of my college OS class's assignments was to write an e1000 driver, and as I recall, it was way easier than, like, getting interrupt handling right.
Drivers are also pluggable, by nature. Why do you think this constrains the OS to the "toy" category? Or do you mean it constrains only the current version, and if so, do you believe there to be an observable difference between "toy" and "prerelease"?
Now this is looks pretty nice and I'd like it to take off. It can be the next Lisp Machine but unfortunately it raises a red flag with the licensing terms - MIT.
The author distributed the binary without the source code which is fine with MIT. He might release the code or change his mind and never release it. Again this is his choice.
But at this point, the distributed image is not any different than a proprietary solution. What good is it for me if I can't change it?
The .vmdk file you get after running gunzip on the downloaded file is a virtual hard drive.
When you create a new VM in VirtualBox, it asks if you want to create a new virtual hard drive or use an existing one. Select the one downloaded.
Yep, got it. I didn't recognize that funny graphic beside the pop-up menu in VirtualBox was supposed to be a button for opening a file. Creative UI choices like that do confuse me a great deal.
It's in its early days, but it has a working compiler, garbage collection, etc. It was announced on the Freenode #lisp channel; start reading at http://log.irc.tymoon.eu/freenode/lisp?around=2015-01-25T15:...
Screenshot: https://dl.dropboxusercontent.com/u/46753018/Screen%20Shot%2...
Thanks for posting the #lisp chat, it's got some good discussion about the project.
I'm surprised how quiet this guy has been for years, don't ever seem to remember seeing him in IRC!
The correlation between strength of IRC presence and productivity is, shall we say, imperfect...
I'd say there's a pretty strong negative correlation ;)
how about a small video screencast?
No, please no video! Written instructions are much better.
Also available in ircbrowse form: http://ircbrowse.net/browse/lisp?id=8939107&events_per_page=...
I like this. There is strength in having many different species of operating system. Monocultures are vulnerable to all kinds of diseases, one way to combat that is to have a (much) larger variety of operating systems so that infecting any of them has a very low return-on-investment.
So any new operating system that is far from the beaten path should be very much welcomed, this one is more interesting than most since it is written using a high level language.
Looking forward to an eventual release.
Begs the question: is it possible to write a Lisp-based escape from the Lisp VM to the Virtualbox VM, then an escape from the Virtualbox VM to the host OS? Perhaps, but not something a casual attacker is likely to have resources for ;)
Wouldn’t this possibility depend on security holes in Virtualbox rather than in the guest OS?
It would depend on both.
But if there are security holes in Virtualbox, wouldn’t they be exploitable by any OS, regardless of what language it was written in?
Yes. The relevance depends on how likely you are to run a virtualbox image you found on the Internet because somebody told you "run this image".
The idea of a VM is that it provides an iron box which nothing can break out of, because it's completely invisible. There's nothing a guest can do which could possibly distinguish a VM session from running on the bare hardware. Partly this is for compatibility, partly for ease of development (develop the next VM as a guest in the current VM!), and partly for simple security: You can't escape a prison if you are utterly convinced you're already out.
Maybe that part of the theory never made it into practice.
Yeah. Welcome to practice: I type "ifconfig | grep vio" and if it comes back nonblank, I know I'm running in virtualbox.
OK. Is anyone working on x86 VMs which do try to be completely invisible to guests?
That was probably abandoned because it was not performant.
The idea of a VM is that it provides an iron box which nothing can break out of, because it's completely invisible. There's nothing a guest can do which could possibly distinguish a VM session from running on the bare hardware. Partly this is for compatibility, partly for ease of development (develop the next VM as a guest in the current VM!), and partly for simple security: You can't escape a prison if you are utterly convinced you're already out.
Maybe that part of the theory never made it into practice.
I doubt he was working several years to create this OS in Common Lisp to attack these 50 people who tries it :)
When I think of what I want in an operating system, "I wish it would lock up occasionally while everything garbage collects" is not high on the list.
When I think of what I want in an operating system, "I wish it would lock up occasionally while everything garbage collects" is not high on the list.
When you think of what you want in a discussion forum, how high are sarcasm and snarkiness on your list?
Real-time GC is a real thing now. Google it.
No, no it isn't.
There is always some garbage velocity beyond which any given system is not able to cope.
Usually that limit is kinda small compared to what you'd actually like your program to be able to do.
This is true of any dynamic memory management unless very careful constraints are put in place. I would assume anyone writing an OS kernel, regardless of language, would be careful to avoid unbounded consing. (Of course, memory allocation in kernels is tricky even in C: http://lwn.net/Articles/627419/)
In other shocking news resources are discovered to be finite after all.
Could you please define “garbage velocity”?
The "garbage velocity" is not the most important parameter. Remember that garbage is all the space which remains after we identify what is reachable. The traversal of the graph of what is reachable is mainly where the performance pitfalls lie. When garbage is generated at a high rate, it just means that collection has to be more frequent. However, real-time techniques avoid scanning the entire graph of everything that is reachable.
Under ephemeral garbage collection, if the software generates a lot of garbage fast, it means that it's rapidly making large numbers of "baby" objects (objects in the "nursery" or "generation 0") and immediately losing them. Whenever a generation pass comes along, there are hardly any nursery objects to visit (they almost all been lost due to the "garbage velocity"), and the tenured objects aren't traversed either, so ... it's quick. Quick isn't "free", but it's not "beyond the ability to cope".
Have you ran the OS and seen this actual issue you're describing? If not, then what's your problem? If you're going to armchair quarterback, you should at least have thrown a football a few times first.
You may as well be complaining that the Windows 1.0 press release from 1985 has trouble with multitasking. Duh, this is pre-alpha software we're looking at.
Yet, that wish what the mainstream operating systems effectively deliver, at least whenever you use them for real work.
Jon pls, do you even Lisp Machine? Seriously, you can't base your opinion on kernels/OS:s written in Lisp on this guy's hobby project. I know what you think of GC, but history shows what an industrial Lisp OS is like and it's awesome.
>So any new operating system that is far from the beaten path should be very much welcomed, this one is more interesting than most since it is written using a high level language.
I agree. But let's keep in mind that this isn't the first OS kernel written in Common Lisp.
Lisp is also one of the few languages in which commercial operating systems have been written.
OT: nitpick: monocultures are completely invulnerable to some kinds of diseases. ;)
Nice! Please do alert us when it appears on github. I'm sure there will be interest.
EDIT: In fact if you can announce your github id now/soon people can start watching in earnest.
This looks very cool. I am going to wait until the source is posted on github before trying it. I was lucky enough to have received a Xerox 1108 Lisp Machine in the early 1980s, so having an OS written in Lisp seems reasonable :-)
Also, I agree with jacquesm's comment on the advantages of many operating systems.
Are people up voting this because they like the title or they like the link? Because there's literally more information about what this is in the HN title than the link. Is an anonymous paste the closest this thing has to a homepage?
It's funny how little infrastructure some projects can get away with. Many Android ROMs only have a thread on XDA and a file on Google Drive.
apparently the link has been changed. when I clicked it, it went to a website with some information and a few screenshots.
edit: here's the screenshot (from another comment): https://dl.dropboxusercontent.com/u/46753018/Screen%20Shot%2...
There was more content previously (but no install instructions).
The submission reminded me of: https://news.ycombinator.com/item?id=7126818
Here is an IRC log about it:
http://log.irc.tymoon.eu/freenode/lisp?around=2015-01-25T15:...
Awesome! I just downloaded the vm disk and the system is up and running inside VirtualBox. It was good to see the CL-USER> prompt.
I've been pleasantly surprised with its slick UI.
But, as with many Lisp OS images, there is an issue with explorability: how do I discover system commands in the REPL? I know a bit of Common Lisp, but it's not clear to me how to discover what system can and what not, how to explore the environment. Neither Filesystem Viewer provides any clues, nor does REPL.
Copy of install instructions here: http://okturing.com/src/2648/body
Also might as well copy them here too:
Interesting that they need the virtio-net adapter (host hands off raw network packets to the guest instead of simulating a network card). How many other low level drivers are missing?
Of course, for a toy OS it doesn't really matter, but it does unfortunately constrain this to the toy OS category.
Why do you think virtualized == toy? Is MirageOS a toy too then? They have no chance of ever getting enough device drivers to be able to support all the hardware out there. Running virtualized makes sense, and it's not a big performance hit.
A possible reason why "virtualized == toy" is that the virtualized system simulates certain specific hardware. The OS provides drivers for that hardware. But it hasn't been validated on the real hardware.
We don't know whether that is the case here. If the authors get a real box that has the same peripherals as the VirtualBox, and bring the OS up on that, then this "toy" argument can no longer be made, in any case.
Heh. If you think "only runs in a VM" and "toy" are synonyms, you need a history lesson.
Back in the 1960s, one of the hot new things from IBM (you know, the company that wins on Jeopardy) was VM/CMS. VM is a VM, hence the imaginative name, and CMS stood for Conversational Monitor System, originally Cambridge Monitor System, for reasons I'll let you figure out.
Conversational meant "has a command line", as opposed to batch, which you might have heard of. CMS was, all told, somewhere between pond scum and MS-DOS on the complexity scale: it provided some APIs for applications, but didn't handle memory protection, multitasking, multiple users, or security policy. A mainframe running CMS on the bare metal would have been a really expensive version of a PC from twenty or so years later; obviously, you can only justify doing that in a Serious University.
However, it was convenient and user-friendly back before the term was coined, when that meant "not seeing raw machine code unless you asked for it", so the gag was to run multiple instances of CMS as guests under VM, which provided everything CMS didn't. Every user got their own copy of the OS, and the single physical mainframe was turned into dozens or hundreds of virtual mainframes by VM.
My punchline: As the decades wore on, CMS was never modified to take newer hardware into account. Why would it be? It never ran on the bare metal. VM gained emulation functionality, CMS grew to depend on that, and now CMS is effectively an appendage of VM, a friendly face to the thing which does all the hard work. You know, like marketing, or upper management.
> you know, the company that wins on Jeopardy
Hehe. Priceless.
Out of curiosity, have you written a driver for a real network card, or are you guessing as to its difficulty? One of my college OS class's assignments was to write an e1000 driver, and as I recall, it was way easier than, like, getting interrupt handling right.
http://pdosnew.csail.mit.edu/6.828/2014/labs/lab6/
Drivers are also pluggable, by nature. Why do you think this constrains the OS to the "toy" category? Or do you mean it constrains only the current version, and if so, do you believe there to be an observable difference between "toy" and "prerelease"?
So what needs to happen to run this on the bare metal with stumpwm and emacs on top?
Pretty much everything.
No issue with naming? I mean there is already an open-source project named Mezzanine, but it is CMS built with Django. http://mezzanine.jupo.org/
It's up on github (name changed to Mezzano because of the conflict with the CMS project):
https://github.com/froggey/Mezzano
IRC log discussing building: http://log.irc.tymoon.eu/freenode/lisp?from=2015-01-27T19%3A...
Now this is looks pretty nice and I'd like it to take off. It can be the next Lisp Machine but unfortunately it raises a red flag with the licensing terms - MIT. The author distributed the binary without the source code which is fine with MIT. He might release the code or change his mind and never release it. Again this is his choice.
But at this point, the distributed image is not any different than a proprietary solution. What good is it for me if I can't change it?
No flame-wars please.
Sounds cool but I'm kind of reluctant to run a VBox only described by a pastebin-like bit of text. Does anyone else think that way?
I agree :)
This is good work! I seriously _need_ to see the source code. Always wanted to make an OS in a high level language.
I don't know VirtualBox that well, the instructions in the post are unclear to me and don't work. Could someone elaborate this for VB-dummies like me?
Edit: Especially unclear is “Use an existing virtual hard drive file, the one you downloaded”. How do I do that?
The .vmdk file you get after running gunzip on the downloaded file is a virtual hard drive. When you create a new VM in VirtualBox, it asks if you want to create a new virtual hard drive or use an existing one. Select the one downloaded.
Yep, got it. I didn't recognize that funny graphic beside the pop-up menu in VirtualBox was supposed to be a button for opening a file. Creative UI choices like that do confuse me a great deal.
A video screencast is welcome!
A better elaboration on the written instructions would even be much more welcome than a video. Video screen casts are vastly overrated.
I don't mean a screancast how to install it. I just want to avoid installation and see how it looks.
If I could misread that, so could everyone else :)