roblabla 7 years ago

XonSH has been my main shell for a solid two years now. The concept is awesome, but unfortunately it has many flaws.

* Nesting bash and python scripts makes performance terrible compared to using python's subprocess module directly. I have a xonsh script at [0], with an equivalent written in straight bash. The python scripts ends in less than a second, the xonsh script takes ages.

  - As a sub-problem of the above: You can't ctrl+c while in a python function in xonsh. That stinks.

* $(program) returns a string that ends with a \n. In bash, when the program returns a single line, the \n gets stripped.

* It sometimes has problems with certain interactive programs. My last experience with this was sudoedit starting neovim in xonsh caused neovim to crash. XonSH -> Neovim works. Bash -> Sudoedit -> neovim works. but Xonsh -> Sudoedit -> Neovim doesn't.

And that's just a small list of my biggest gripes that I keep running into. I really hope to see the project improve though! Despite all the bugs, it's still my favorite shell by a mile.

[0] https://gist.github.com/roblabla/d82c440908d08c8a232ac483e6b...

  • jijojv 7 years ago

    Filed https://github.com/xonsh/xonsh/issues/2815

    $ yes test | tee /tmp/xonsh.log

    $ wc -l /tmp/xonsh.log

      208076 /tmp/xonsh.log
    

    bummer ctrl+c does eventually work several seconds later (about 10x slower) which is still a deal breaker as it could be disastrous when you need to abort a mistyped command or for scripts that give you a few seconds to abort before it does something the destructive step...

  • austinpray 7 years ago

    I use zsh daily but type `xonsh` any time I do something in the shell that would normally require a google.

    “Uh ok so I have this JSON file. When I pipe this to `jq` I forget: to pivot an array of objects to an object keyed by array[x].id is it like `to_entries` then chain it to `map`? Uhh let’s look at the docs...”

    However in Python I can write that function at the speed of thought. It may be a couple more lines of code but I’m also 100% confident in the output.

    Like I understand some people can legitimately program in bash [0]. But I just can’t bring myself to practice. Why would anyone inflict this language on themselves?

    [0] https://github.com/docker-library/python/blob/master/update....

    • joelthelion 7 years ago

      > Like I understand some people can legitimately program in bash [0]. But I just can’t bring myself to practice. Why would anyone inflict this language on themselves?

      Bash has a lot of pitfalls, but once you learn how to deal with them, it can also be quite superb for some tasks. Piping the output of a complex set of loops into GNU parallel and watching your computer process stuff at blazing speed is a very nice experience. Doing the same in python would be many time more verbose.

    • type0 7 years ago

      Bash combined with utilities is like a swiss knife of scripting, you won't be doing any major woodwork with it but it sure is a heck of a useful tool.

    • pqb 7 years ago

      > I use zsh daily but type `xonsh` any time I do something in the shell that would normally require a google.

      Personally, I stopped using `jq` in favor of python oneliners, they a bit more verbose but also cross-compatible, works out of the box (not everyone has jq, but most of people have python (pre)installed on their machines) and it is easy to implement a oneliner, i.e.:

         curl -X GET "https://httpbin.org/json" -H "Accept: application/json" | python -c 'import sys, json; print(json.load(sys.stdin)["slideshow"]["author"])'
      

      Do xonsh helps you write a tar command without googling [0] or browsing a man?

      [0]: https://xkcd.com/1168/

ilyagr 7 years ago

Sounds neat. However, the fact that `ls -l` either subtracts two numbers or calls a shell command, depending on whether there exist valriables 'ls' and 'l' in the Pythom environment, looks like a security nightmare waiting to happen. What if I create a python object 'ls' deep inside a script that does Evil when subtracted from?

  • scopatz 7 years ago

    Python variables are scoped as normal, so the security story for xonsh is the same as for Python. We are always open to suggestions and PRs to harden the code base as needed.

  • bitwize 7 years ago

    It could be worse. It could be a Wolfram shell, in which case 'ls -l' would return 'l(s - 1)'.

  • Norther 7 years ago

    What's stopping a Bash script from doing something similar? You could alias ls to a malicious actor for example.

    • kbd 7 years ago

      Well, aliases are ignored in scripts usually, but you could prepend something to the PATH so your script/exe gets found before built-ins, no? I agree, I don't see that this is a problem unique to Xonsh.

    • djsumdog 7 years ago

      I mean, that's the underlying concept behind the fork bomb isn't it? With ":() { :|:&}" you're essentially redefining the bash noop to be a function that pipes/forks itself into itself recursively.

      It's the reason './' is not in the default path as well (so you can't place an 'ls' in your home directory and have the admin run your command instead of the real ls).

  • daveFNbuck 7 years ago

    Why not just do the evil directly in the script if you can do that?

    • bmer 7 years ago

      Obfuscation minimizes time to discovery? Indirection is a possible technique for obfuscation?

      • ryanlol 7 years ago

        This is a super silly concern.

JepZ 7 years ago

Sometimes I wonder which languages people use for system tools. I mean, it is hard enough that C programs (with dynamic linking) break sometimes, but building a shell on top of Python seems like a particualar bad choice to me (similar to building a package manager with python).

In my opinion, such few-dependency, high-importance tools should linked statically.

  • munchbunny 7 years ago

    I think that anyone who is trying to imagine "the next shell" owes it to themselves to try PowerShell.

    It's .NET based and very, very different from the sh lineage, but I absolutely love its ability to use objects instead of strings as the underlying transmission medium between "processes". E.g. ls gives you file descriptors, ps gives you process descriptors, etc.

    • JepZ 7 years ago

      I was just about to ask if it is available for Linux, but then I saw that the official page advertises its cross-platform properties [1] :D

      I know about the different concept of Powershell for quite a while now, but since it is MS tech I didn't expect it to be available for Linux. Maybe I will try someday how it feels, but I am sceptical as the thing I like most about POSIX shells is that everything is just text and I wonder how Powershell will integrate with other *nix tools.

      [1]: https://github.com/powershell/powershell

      • munchbunny 7 years ago

        The boundary layer between text and object is definitely a weak spot in the abstraction for PowerShell. That's the main reason why I don't consider PowerShell strictly superior.

        That said, the way the language is designed makes calling into compiled .NET code seamless, which in turn makes doing really sophisticated shell scripting a breeze. When I saw what xonsh was trying to do, I recognized a lot of "PowerShell solves this problem a bit more gracefully".

    • xrd 7 years ago

      I've heard of it but never heard a compelling reason to try it. Until your statement: I'm now very interested.

  • insertnickname 7 years ago

    Gentoo's portage package manager is written in Python.

    https://github.com/gentoo/portage/

    • JepZ 7 years ago

      And that is probably one of the reasons why projects like Paludis were started years ago.

      The funny thing about Portage is though that Python doesn't cause any performance issues as Portage/Emerge is slow by design. So while Paludis is written in C++, it isn't much faster, as it still has to interpret all those thousands of bash-style ebuilds/exheres.

      This might sound as if I wouldn't like Gentoo/Exherbo et al. but in fact, my desktop PC is running Gentoo or Exherbo for the last 10 years at least. But it was exactly that experience which made me conclude that system tools like package managers, shells, etc. should be written in statically linked languages.

  • marmaduke 7 years ago

    > building a shell on top of Python seems like a particualar bad choice to me

    can you say why? Python's stdlib handles a lot of cross-platforminess, Python is mostly easy to write, and is available as or more easily than a C compiler on many platforms.

    • JepZ 7 years ago

      It's about stability. It is much easier to break a dynamically linked or interpreted script than a statically linked binary. The worst case of fixing a system with a software failure is booting a live-cd and replace the files which cause trouble, but a broken shell and sometimes also a broken package manager are precisely those cases when you have to boot a live-cd.

      • marmaduke 7 years ago

        > It is much easier to break a dynamically linked or interpreted script than a statically linked binary

        If we’re talking about new code, why is this the case?

pyedpiper 7 years ago

Love xonsh!! And gets more and more stable with every release.

it's the shell I always wanted. And cross platform to boot.

Alias ls -l?

aliases['ls'] = "ls -l"

Oops no I don't want that any more:

del aliases['ls']

Add dir to path:

$PATH.append('/foo/bar')

Just the awesome python syntax sugar and readbility right in the

May I never right a bash script again.

  • stochastic_monk 7 years ago

    del aliases[‘ls’] seems like a step back from unalias ls.

    • scopatz 7 years ago

      Sure, but aliases is a real mapping object that you can access and manipulate

    • BeetleB 7 years ago

      Only if you're coming from a different shell.

      For me, it's easier to remember that aliases acts like a dictionary, so I can treat it like any other dictionary. With "unalias" I have to remember the command. As I rarely use unalias (a few times over a period of years), I typically have to search the web on how to remove an alias.

      And if it really bothers you, I'm pretty sure it's trivial to define an unalias function in Xonsh that will act as you expect (without needing parentheses).

      It's part of the beauty of Xonsh - many such commands you're used to can be brought into it by writing fairly simple Python code.

    • marmaduke 7 years ago

      no the point is that if you know Python, you know how to do it without looking it up

  • Cyph0n 7 years ago

    > May I never right a bash script again.

    Bash is not going away any time soon, for better or worse..

protomikron 7 years ago

So I just tried it and it's surprisingly usable (I am always somehow skeptical about non-Bash shells). It even takes into account my ~/.inputrc, so kudos.

However, I struggled a little bit with

$ time ls

where the output of `time` is mangled in-between the output of `ls` - this may be related to STDOUT and STDERR buffering?

h1d 7 years ago

I've always been thinking there can be a better shell like having a split pane to show the list of command options with descriptions as you start typing in the command or show snippets of a preset of command options that does task X as in search by use case instead of going to stackoverflow every time.

I think the way people handle shell, as in typing commands on a dumb shell, has not improved for decades and I don't know why. The only thing they help without explicitly asking is with autocompletion.

  • djsumdog 7 years ago

    I really like fish. It doesn't open a split pane, but it can show lists of command switches in tab complete based on man pages.

    I'm really tempted to try xnosh again as my daily driver for a few weeks though. I like the idea of typing out arbitrary Python code without spawning up IPython.

    The problems people seem to be stating about xonsh were similar to a lot of the early bugs with Fish, but those things tend to go away once more people start using the shell and reporting issues.

    I hope we got a whole new generation of usable shells with xonsh, oil and fish.

pen2l 7 years ago

I love the concept, and I am generally happy with how it works. I think it should catch on.

Except the damned name. Come on guys, there are already enough crazily named things out there, you didn't have to be another one :(

  • programmarchy 7 years ago

    You know it's a pun on conch shell, right? ;-)

    • dguaraglia 7 years ago

      Wouldn't it make more sense if it was called "Ponsh" then? Heck, now that I think of it, "posh" would be a perfect name for a Python-powered shell.

    • pen2l 7 years ago

      That makes it worse. Too many names are based on insider-jokes, too many of it just scares away noobies.

      posh, I agree, with the other poster, is a fabulous name for something like this

      • itomato 7 years ago

        It doesn't even make sense, since "conch" is pronounced "konk".

  • djsumdog 7 years ago

    ehh.. I keep wanting to call it "nosh" or spell it "xnosh" but I'm sure if it catches on people will remember. Plus it's easy to use with a search engine, unlike "Go" or "Rust."

    I will say I prefer names that can be searched for on their own (e.g. with Go people typically search for "Go lang").

    And as much as I love Void Linux, it's better named than all the xbps-* package manager commands.

Sir_Cmpwn 7 years ago

Note, while you should feel free to use any interactive shell you like, you should only write scripts with POSIX sh. A friend of mine is working on another shell, mrsh, which aims to be POSIX-sh-as-a-library and will serve as the basis for building more sophisticated interactive shells on a POSIX base, if anyone is interested in that:

https://git.sr.ht/~emersion/mrsh

  • roblabla 7 years ago

    I have a better idea: don't write scripts in shell language at all. They're terrible at basically everything. Instead, use a scripting language, like Python, Ruby, or Perl. They have control flow that make sense, are not riddled with subtle bugs due to legacy and compatibility reasons, and are simply easier to wield.

    Shell scripts are a maintenance nightmare - even when used for personal scripts. They should be avoided at all cost.

    • Sir_Cmpwn 7 years ago

      Nonsense, shell scripts are an incredibly useful tool for a huge variety of use-cases. Shell can be learned like anything else, and it mostly makes sense when it does. More so than I can say for Ruby and Perl, that's for sure!

      You should know when to stop and reach for a different tool, but that point isn't before you use shell scripts at all.

    • cookiecaper 7 years ago

      Ruby and Python are pretty unwieldy for the kind of situations that are well-suited to shell scripts. It's a pain to spawn a command, a pain to make sure you get the process's output in the right place, and just far more overhead than necessary.

      I agree, however, that classic shells are worse, because they lack many of the niceties expected in modern languages and have opaque, sensitive syntax.

      fish shell has made me pretty happy. It's a great middle ground. It makes no pretense at POSIX compatibility. Truly a shell for the 90s.

      • h1d 7 years ago

        Agreed.

        There's definitely a good demand for a modern shell scripting language but fish is an existing solution that is saner than bash/zsh.

        I don't use fish as a shell but for shell scripts only as it acted a bit weird on me, though it's possible I didn't search enough but zsh can cover pretty much all I want from fish with plugins.

    • ekianjo 7 years ago

      shell is incredibly powerful when you interact with tools that use piping. Doing the same in python is much more verbose.

  • joshumax 7 years ago

    While that's true for portability (there's obviously reasons why nearly all distros ship with a POSIX sh compatible shell at /bin/sh), for simple tasks on my own system, oftentimes I simply want to get the job done without banging my head against a wall dealing with all the idiosyncrasies of sh. For internal use, I think this project excels for automation and administration tasks. Of course, I probably wouldn't ship anything written in xonsh-python, but I'd totally use it to, say, back up my dotfiles to a git repo automatically.

    • Sir_Cmpwn 7 years ago

      Because many distros ship with /bin/sh linked as bash, I think using arbitrary shells for your personal scripts encourages bad habits that can easily lead to mistakenly writing non-portable scripts elsewhere without realizing it. This has happened to me many times!

      POSIX shell is not really worse than bash or any of the other choices imo. For tasks to which it's not suited, bash et al probably isn't either, and you should just use Python.

meowface 7 years ago

Sounds amazing. If it ever gets a relatively active plugin ecosystem like zsh or fish, I'd definitely switch. Being able to write shell functions in Python would be very refreshing.

If they want bigger adoption, dedicating a bunch of people who do nothing but port zsh and fish plugins would probably go a long way.

  • scopatz 7 years ago

    We have a couple of dozen very handy plugins available already (powerline, conda support, etc) and are always looking for more!

baddash 7 years ago

what exactly does "Unix-gazing" mean?

hathawsh 7 years ago

This looks very promising:

  >>> ls `a(a+|b+)a`
  aaa  aba  abba

Regex filename matching triggered by backticks. I like that design choice!

  • h1d 7 years ago

    Good that they didn't choose slash delimiters.

amasad 7 years ago

Awesome that it uses repl.it embed. But leaves so much to be desired. I'm the repl.it ceo and have a few things in the works to make this experience so much better:

1. caching pypi installs between instances 2. using a real terminal emulator which means you'll get to use xon.sh (or any shell) in it's full glory

Still a nice surprise to see repl.it embedded in the wild.

fiatjaf 7 years ago

Your best option for designing scripts that must talk to other programs and also do what you want (as bash won't).

di 7 years ago

Xonsh is awesome! Execute Python directly on the command line?! Sign me up!

Sadly it doesn't play well with all of my three thousand various vim plugins, so I still haven't been able to use it as my daily shell. Hopefully someday!

  • devnonymous 7 years ago

    If vim plugins are the only thing holding you back you might want to consider just adding

       set shell=/path/to/bash
    

    to your vimrc and see how you get along.

    • djsumdog 7 years ago

      I current have that set in vim when using fish shell.

  • mlevental 7 years ago

    lol do you really have 3000 vim plugins?

j2kun 7 years ago

The demo on the front page raises an error when running "ls"

xiaq 7 years ago

Shameless plug: if you like modern shells that you can use to do real programming, try Elvish too: https://elv.sh

therealmarv 7 years ago

Anybody switched from fish shell to this? What is your experience? I like fish shell because the preconfiguration is pretty good. Same on xon.sh or config hell like zsh?

  • anreekoh 7 years ago

    Ugh. I have so many conflicting opinions on this question.

    So, Xonsh is superior in the manipulation of shell objects. I felt super at home changing the shell prompt and configuring/extending the default Xonsh functionality. This is all because it's literally a superset of Python.

    Fish however, blows Xonsh out of the water when it comes to autocomplete and navigation. Xonsh's autocomplete / suggestions are very very weak in comparison to fish's. You feel bogged down when trying to navigate directories quickly with Xonsh. I wrote a small blog post about it here [0]

    There was a piece of software I was trying to write that was meant to be Xonsh but with Fish's autocomplete [1], but really didn't succeed and have slightly abandoned it for the time being. But maybe I'll pick it back up soon.

    [0] https://ezb.io/thoughts/programming/mollusk/mollusk_1.html

    [1] https://github.com/enricozb/mollusk

    • faho 7 years ago

      >I wrote a small blog post about it here [0]

      I work on fish, and one of the things that the upcoming fish 3.0 contains is a new `math` builtin (previously we shipped a function of that name that just wrapped `bc`).

      So your math expressions here are interesting to me, because they work in math (with the caveat that `` is still the symbol for globbing, so it needs to be quoted or escaped):

          $ math 0x113e8b3
          18081971
          $ math '719 * 114679'
          82454201
      

      Adding them to the core syntax carries backward-compatibility questions and also isn't really what fish is about.

      • anreekoh 7 years ago

        The benefits of xonsh are more than just the quick math, but I'm glad fish is getting something like this. I've had a Python script that does math for me, something like this [0]

        I do think something like a Python shell is the best solution (for me), however, simply because of the ease of use with modifying the current state of the shell (environment variables, etc).

        [0] https://gist.github.com/wackywendell/919042f19ab6932447df

brazzledazzle 7 years ago

This seems really cool. Trying it out with repl.it was really neat. One thing I'd do is add PowerShell to the comparison table.

blt 7 years ago

what does the "-gazing" suffix mean?

  • scopatz 7 years ago

    It means that it is not fully sh-lang compatible (i.e. a true unix shell), but is as compatible as reasonably achievable while maintaining full compatibility with Python 3.

    • bmer 7 years ago

      What sort of capabilities does sh-lang provide that Python cannot? Does sh-lang have some sort of privileged access to the OS not available to Python?

      • TheDong 7 years ago

        posix sh mandates certain constructs work. For example:

           x=foo; x=${x%o}; echo $x # prints 'fo'
        

        Supporting that is non-trivial in python. There's no quotes around the strings, '%' is kinda weird there, etc.

        Of course, in python you can just say:

           x="foo"; x=x[0:2]
        

        ... but those are not compatible with each other. colon does different things in posix sh than python, and it would be quite complicated to reconcile it.

        I'll also point out the parent poster, when they wrote "sh-lang", meant "the superset of posix-sh that zsh/bash users expect", because at this point quite a few bash features are expected to work in most sh-like shells.

        Naturally, both xonsh and sh (and all other shells pretty much) are turing complete and can execute programs, which makes them all equally capable.

        Having equivalent capabilities does not mean things are compatible though. I recommend reading up on what a POSIX shell is, and then reading fish and xonsh's arguments why they're not POSIX compliant (spread through various locations)

      • t-3 7 years ago

        He means it's not POSIX-compliant. You can't replace /bin/sh with it and expect stuff to work.

dec0dedab0de 7 years ago

I have wanted something like this for a while. I can't wait to try it out

0x8BADF00D 7 years ago

I tested it out on mobile, I like the ability to script with it in a HLL like Python, will give it a try for sure on desktop.

mastrsushi 7 years ago

A shell scripting language written in a shell scripting language. What could go wrong?